Latent Structure Linear Regression

Size: px
Start display at page:

Download "Latent Structure Linear Regression"

Transcription

1 Applied Mthemtics, 2014, 5, Published Online Mrch 2014 in SciRes. Ltent Structure Liner Regression Agnr Höskuldsson Centre for Advnced Dt Anlysis, Lyngby, Denmrk Emil: Received 19 November 2013; revised 19 December 2013; ccepted 6 Jnury 2014 Copyright 2014 by uthor nd Scientific Reserch Publishing Inc. his work is licensed under the Cretive Commons Attribution Interntionl License (CC BY). Abstrct A short review is given of stndrd regression nlysis. It is shown tht the results presented by progrm pckges re not lwys relible. Here is presented generl frmework for liner regression tht includes most liner regression methods bsed on liner lgebr. he H-principle of mthemticl modelling is presented. It uses the nlogy between the modelling tsk nd mesurement sitution in quntum mechnics. he principle sttes tht the modelling tsk should be crried out in steps where t ech step n optiml blnce should be determined between the vlue of the objective function, the fit, nd the ssocited precision. H-methods re different methods to crry out the modelling tsk bsed on recommendtions of the H-principle. hey hve been pplied to different types of dt. In generl, they provide better predictions thn liner regression methods in the literture. Keywords H-Principle of Mthemticl Modelling; H-Methods; PLS Regression; Ltent Structure Regression 1. Introduction Regression methods re mong the most studied methods within theoreticl sttistics. Numerous books hve been published on the topic. hey re stndrd in mthemticl clsses in most universities in the world. Advnced progrm pckges like SAS nd SPSS hve been used by students since the 1970s. here re still gret interest nd developments in regression methods. he reson is tht mny modern mesurement equipments require specil methods in order to crry out regression nlysis. In pplied sciences nd industry there re three fundmentl issues. 1) Dt hve ltent structure. Dt in pplied sciences nd industry hve typiclly ltent structure. his mens tht vlues of the vribles re geometriclly locted in low-dimensionl spce. he loction is often low dimensionl ellipsoid. Vribles re often correlted nd therefore, my or should not be treted s independent. Mthemticl models nd methods tht ssume tht X-dt re of full rnk re often incorrect. 2) It is trdition to compute optiml or unbised solutions. he optiml solution hs often interesting proper- How to cite this pper: Höskuldsson, A. (2014) Ltent Structure Liner Regression. Applied Mthemtics, 5,

2 ties, when dt stisfy the given model. But in prctice the optiml solution hs often bd or no prediction bility. It mens tht the model cnnot be used for predictions t new smples. By relxing on the optimlity or unbisedness better solution cn often be obtined. 3) Sttisticl significnce testing. Sometimes the significnce testing in liner regression nlysis my not be relible. his is serious problem tht is being discussed t professionl orgniztions. See Section 2.3. In Section 2, brief review is given of liner lest squres methods for stndrd liner regression. he bsic properties of the modelling tsk re discussed. In Section 3, generl frmework for stndrd liner regression is presented. he bsic ide is to seprte how one should view the regression tsk nd the ssocited numericl procedure. he numericl prt is the sme for ll types of liner regressions within this frmework. It llows flexibility in developing new methods tht fit different types of dt nd objectives. In Section 4, the H-principle is formulted. It uses the nlogy between the modeling tsk nd mesurement sitution in quntum mechnics. It ssumes tht dt re centered nd scled. It suggests tht one should use the covrinces s the weight vectors in the generl lgorithm. Section 5 considers some methods to vlidte ltent structure results. In Section 6, brief introduction is given to H-methods. hey focus on finding wht prt of dt should be used, when covrinces re used s weight vectors. In Section 7, the use of n H-method is compred to Ridge Regression. he results obtined look similr, but the H-method uses much smller dimension. In Section 8, the use of ltent structure liner regression is discussed. It is shown tht the ltent structure methods much more quickly rrive t the finl solution thn forwrd vrible selection. Nottion. Mtrices re denoted by upper-cse bold letters nd vectors by lower-cse bold letters. he letters i, j nd k denote indices reltive to vectors nd mtrices, while nd b re relted to the steps in the methods. he regression dt re (X,y), X is N K mtrix nd y n N-vector. X is the instrumentl dt nd y the response dt. Only one y-vrible is considered here in order to simplify the nottion. Dt re ssumed to be centred, i.e. men vlues subtrcted from columns of X nd y. 2. Liner Lest Squres Methods 2.1. he Liner Model nd Solution It is ssumed tht there re given instrumentl dt X nd response dt y. A liner regression model is given by y = β x + β x + + β x + ε (1) K K he x-vribles re clled independent vribles nd the y-vrible the dependent one. When the prmeters β hve been estimted s b, the estimted model is now y= bx + bx + + b x (2) K K When there is given new smple = ( x x x ) y = bx + + b x K K0 x0 10, 20,, K 0, it gives the estimted or predicted vlue. Greek letters re used for the theoreticl prmeters nd romn letters for the estimted vlues, becuse there cn be mny estimtes for β. Liner lest squres method is usully used for estimting the prmeters. It is bsed on minimizing the residuls, (y Xb) (y Xb) minimum. In cse the dt follow norml distribution, the procedure coincides with the Mximum Likelihood method. he solution is given by ( ) 1 b = X X X y. (3) he ssumption of normlity is often written s y N(Xβ, σ 2 ). his mens tht the expected vlues of y is E(y) = Xβ nd the vrince is Vr(y) = σ 2 I, I is the identity mtrix. he prmeters b hve the vrince given by, Here σ 2 is estimted by 2 ( ) = σ ( ) 1 Vr b X X. (4) 2 N 2 1 i ( ) σ e N K (5) where the residuls, e i s, re computed from e = y Xb. he estimtes (3) hve some desirble properties compred with other possible estimtes: ) he estimte b is unbised, E(b) = β. 809

3 b) he estimte b hs the minimum vrince mong ll liner estimtes of β, which re unbised. It mens tht if nother unbised estimte of β hs vrince mtrix E, then E Vr(b) is semi-positive definite mtrix. he mtrix (X X) 1 is clled the precision mtrix, nd it shows how precise the estimtes b re. We would like the vrince mtrix Vr(b) to be s smll s possible. ) nd b) stte tht ssuming the liner model (1), the solution (3) is in fct theoreticlly the best possible one. It hppens often tht the model (1) is too detiled in the sense tht it contins mny vribles. In this cse significnce test cn be crried out to evlute if some vribles in the model (1) re not significnt, nd thus my be excluded from the model. Interprettion of the estimtion results is n importnt prt of the sttisticl nlysis. A prmeter vlue is evluted by t-test. A t-test of prmeter b i is given by bi t = Vr b 2 ii ( bi ) ( s s ) Here s 2 is given by (5) nd s ii is the i th digonl element of (X X) he Liner Lest Squres Solution in Prctice he model (1) is so obviously proper tht sttisticins do not doubt its vlidity. In the clssroom it is emphsised tht n importnt prt of the sttisticl nlysis is the interprettion of the resulting Eqution (2). Chnges in y cn be relted to chnges in x i, for instnce, y = b i x i. But the model (1) is incorrect in most cses in pplied sciences nd industry (prt from simple lbortory experiments) nd the nice theory bove is not pplicble. Why is this the cse? We cn not utomticlly insert new smples into (2) nd compute the y-vlue. It must be secured tht the smple used is locted long the smples used for nlysis, the rows of X. A correct model is y = ατ + α τ + + α τ + ε (7) A A. Here (τ ) re ltent vribles nd (α ) the regression coefficients on the ltent vribles. Ltent vribles re liner combintions of the originl vribles. hus the regression tsk is both to find the ltent vribles, τ s, nd the ssocited regression coefficients. he resulting Eqution (7) is then converted into (2). he model (7) is clled ltent structure liner regression. he dt vlues ssocited with ltent vrible re clled score vlues. he problem in using the lest squres solution ppers, when the precision mtrix becomes close to singulr. Progrm pckges like SAS nd SPSS inform the user when it hs become close to being singulr. But the informtion is concerned with determining the precision mtrix by the numericl precision of the computer. here re prcticl problems long before the lgorithm informs tht the precision mtrix is close to be singulr nd estimtes, b s, my not be determined precisely. Mny sttisticins do not like ltent structure regression. Singulrity is lso well understood mong sttisticins. When singulrity occurs mny sttisticins recommend Ridge Regression, Principl Component Regression (PCR), vrible selection methods or some other well understood methods. As mentioned bove, sttisticins emphsize the interprettion of prmeters of the results (2). But this is difficult, if not impossible, in most ltent structure regressions. he mesurement dt often represent system, where vribles re dependent on ech other from chemicl equilibrium, ction-rection in physics nd physicl nd technicl blnce. In pplied sciences nd industry the emphsis is on prediction. he qulity of models is to be judged by their predictions. A good exmple is the compny Foss Anlytic, Denmrk, which produces mesurement instruments for the food nd oil industry nd hs sles of round 200 mio euros pr yer. he instruments re clibrted by regression methods nd the compny is only interested in how well the instruments mesure new smples Significnce esting Progrm pckges, SAS, SPSS nd others, introduce the results of regression nlysis by presenting the regression coefficient, b i, its stndrd devition, t-test vlue for its significnce, (6), nd the significnce probbility. hese results re typiclly copied into the scientific report nd dded * s to indicte how significnt the prmeters re (One * is 5% significnce, etc). his report is then bsis for further publictions. But this nlysis my not be relible. Consider the t-test closer nd tke the lst vrible, x K. Suppose tht we for the dt cn write i (6) 810

4 xk = z, K + t K (8) where t K is orthogonl to the other vribles dt, x1, x2,, xk 1. his cn be chieved by Grm-Schmidt orthogonlistion of X. hen the t-vlue (6) for the prmeter b K cn be written s t = y t K ( s tk ) his shows tht the t-vlue is invrint to the size of t K, which is the mrginl vlue of x K given the other vribles. he size of t K cn be smll nd without ny prcticl importnce, while being significnt by (9). When there re mny vribles, there will be few, where the mrginl effect is smll but sttisticlly significnt. herefore, significnce testing in the experimentl studies will usully declre few vribles significnt, lthough they hve no prcticl importnce. his hs mde the Americn Psychologicl Assocition introduce the rule tht ppers tht only bse their results on significnce testing re no longer ccepted in scientific psychologicl journls. Insted guidelines for experimenters hve been prepred Some Aspects of the Modelling sk It is instructive to study closer the vrince ssocited with new smples in the liner lest squres model. Suppose tht x 0 is new smple nd the ssocited response vlue y 0 is computed by (2). Using (4) nd (5) the vrince of y 0 cn be written s ( ) ( 1 1 ) ( Vr y0 y y y X X X X y 1 + x 0 X X) x 0 ( N K ) (10) It cn be stted tht the primry objective of the modelling tsk is to keep the vlue of (10) s smll s possible. here re two spects of (10) tht re importnt. When new vrible (or component) is dded to the model, it cn be shown tht 1) the error of fit, ( ) 1 y y y X X X X y, lwys decreses 2) the model vrition, 1+ ( ) x X X x, lwys increses (In theory these mesures cn be unchnged, but in prctice these chnges lwys occur). If the dt follow norml distribution, it cn be shown tht ): the error of fit, ( ) 1 y y y X X X X y, b): the precision mtrix, ( ) 1 X X, re stochsticlly independent. It mens tht the knowledge of () does not give ny informtion on (b). In order to know the qulity of predictions, (b) must be computed. Progrm pckges use the t-test or some equivlent mesure to test the dimension. No informtion is given on the precision mtrix. It must be computed seprtely in order to find out how well the model is performing. But modelling procedure must include both () nd (b) in order to secure smll vlue of (10). It must hve procedure of how to hndle the decrese of 1) nd the increse of 2) during the model estimtion. 3. Frmework for Liner Regression Anlysis 3.1. Views on the Regression Anlysis Here is presented generl frmework for crrying out liner regression. he bsic ide is to seprte the computtions into two prts. One prt is concerned how it is preferble to look t dt nd the other prt the numericl lgorithm to compute the solution vector nd ssocited mesures. At the first prt it is ssumed tht weight mtrix W (,, K = w1 wk ) is given. Ech weight vector should be of length one, lthough this is not necessry. he weight vectors (w ) reflect how one wnts to look t the dt. If w = ( 0,,0,1,0,,0), vrible is selected ccording to some criterion. For w equl the th eigenvector of X X we get PCA. If w is the left singulr vector of X Y, the result will be PLS regression. Some progrm pckges hve the option of defining priorities for 1 1 (9) 811

5 the vribles, where higher priority vribles re treted first. his cn be chieved by defining elements of w to be zeros for vribles tht re not used t this step. Different options cn be mixed in order to define the weight vectors tht prescribe how the regression nlysis should be crried out. he role of the weight vectors is only to compute X-score vectors, when instrumentl dt re given nd to compute loding vectors when covrinces re given. he only requirement to the weight vectors is tht the resulting score, or loding vectors, my not be zero. he other prt of the computtions is numericl lgorithm, which is the sme for ll dmissible choices of W K A Numeric Algorithm In most cses in regression it is ssumed tht there re given instrumentl dt X nd response vlues y. In Ridge regression nd engineering pplictions like Klmn filtering nd some others, the strting point is covrince mtrix S. In Ridge Regression S = X X + ki, where k is constnt determined by dt. Here the strt of the lgorithm cn be of two kinds, which re treted seprtely. Instrumentl dt he equtions re written for severl y-vectors. Initilly X 0 = X, Y 0 = Y, S 0 = S = X X, C 0 = C = X Y, S + = 0 nd B = 0. For = 1,, K : Score vector : t = X 1w Loding vector : p = X 1t 2 Scling constnt : d = 1 t 1 Y -loding vector : q = Y t Loding weight vector : v he loding weight vectors (v ) re defined by the eqution = XV, or t = Xv. X nd Y re djusted t ech step by the score vector t, Vrince nd covrinces re given Initilly S 0 = S nd C 0 = C. Loding vector : p = S 1w Scling constnt : d = 1 w p ( ) X = X d t p, 1 Y = Y d tq 1 Y -loding vector : q = C 1w Loding weight vector : v he loding weight vectors re defined by the eqution P = SV, or p = Sv. S is djusted by the loding vector p 3.3. Continution of the Algorithm S = S d p p 1 In either cse the continution is s follows. he loding weight vectors re computed by v1 = w 1, v = w d1( p1 w) v1 d 1( p 1w) v 1 = 2,3, he covrince mtrix C is djusted nd regression coefficients nd precision mtrix estimted, C = C d pq, 1 B = B + d vq, 1 he expnsions tht re being crried out by the lgorithm re: S = S d vv X = d t p + + d t p + + d t p = DP 11 1 A A A K K K Yˆ = d t q + + d t q + + d t q = DQ 11 1 A A A K K K X Y = d1p1q 1+ + dapaqa + + dk pkqk = PDQ 812

6 Yˆ Yˆ = d q q + + d q q + + d q q = QDQ A A A K K K S = d p p + + d p p + + d p p = PDP, A A A K K K B = S X Y = d v q + + d v q + + d v q = VDQ, A A A K K K S = d v v + + d v v + + d v v = VDV A A A K K K Normlly, only A terms of the expnsions re used becuse it is verified tht the modelling tsk cnnot be improved beyond A terms. he numericl procedure hs the properties: he djustments of X or S re rnk one reductions. In the cse of instrumentl dt the score vectors (t ) re orthogonl. (v ) nd (p ) re conjugte sets, i.e., v p = 0 for b, nd b v p = 1 d. In mtrix term V P = D 1. For proof of these sttements nd some more properties, which cn be stted in generl for dmissible weight vectors (w ), see [1]. Numericl precision is importnt, when writing generl computing progrms. he djustments cn be numericlly unstble. Also, inner products cn be unstble if the number of multiplictions is lrge. herefore, the score vector t should be scled to unit length before being used. If one is working with test set, X t, the estimted y-vlues cn be computed s Ŷ t = X t B. he score vectors for the test set re computed s t = X t V. We cn plot columns of Ŷ t ginst the score vectors of the test set, the columns of t. Note, tht the weight vectors (w ) do not pper in the expnsions. hey re only used to determine the score or loding vectors. he lgorithm is independent of the size of the weight vectors. But in order to secure numericl precision it is recommended to scle the vectors to be of unit length. he number of terms, A, in the expnsions will depend on the set of w s being used. 4. he H-Principle of Mthemticl Modelling In the 1920s W. Heisenberg pointed out tht in quntum mechnics there re certin mgnitudes tht re conjugte nd cnnot be determined exctly t the sme time. He formulted his fmous uncertinty inequlity, which sttes tht there is lower limit to how well the conjugte mgnitudes cn be determined t the sme time. he lower limit is relted to the Plnck s constnt of light. An exmple of such conjugte mgnitudes is the position nd momentum (speed) of n elementry prticle. he inequlity is formulted s ( position) ( momentum) constnt, N. Bohr pointed out tht there re mny complementry mgnitudes, which cnnot be determined exctly t the sme time. In prctice it mens tht the experimenter must be wre of tht the instrument nd the phenomenon in question plce some restrictions on the outcome of the experiment. It is only by the ppliction of the instrument tht the restrictions re detected. he uncertinty inequlity is kind of guidnce of wht cn be expected he H-Principle When modelling dt we hve nlogous sitution. Insted of mesurement instrument we hve mthemticl method. ΔFit nd ΔPrecision re conjugte mgnitudes tht cnnot be controlled t the sme time. It is necessry to crry out the modelling in steps nd t ech step evlute the sitution s prescribed by the uncertinty inequlity. he recommendtions of the H-principle re: 1) Crry out the modelling in steps. You specify how you wnt to look t the dt t this step by formulting how the weights re computed. 2) At ech step compute ) expression for improvement in fit, Δ(Fit) b) nd the ssocited prediction, Δ(Precision) 3) Compute the solution tht minimizes the product ( Fit) ( Precision) 4) In cse the computed solution improves the prediction bilities of the model, the solution is ccepted. If the solution does not provide this improvement, the modelling is stopped. 813

7 5) he dt is djusted for wht hs been selected; restrt t 1. Consider closer how this pplies to liner regression. he tsk is to determine weight vector w ccording to this principle. For the score vector t = Xw we hve 2 ) Improvement in fit: Y t ( t t ) b) Associted vrince: Σ ( t t) reting Σ s constnt the tsk is to mximize 2 Y t 1 = t t 1 t t w X YY Xw his is mximiztion tsk becuse improvement in fit is negtive. he mximiztion is crried out under the restriction tht w is of length 1, w = 1. Using Lgrnge multiplier method it cn be shown tht the tsk is n eigenvlue tsk, X YY Xw = λ w In cse there is only one y-vrible, the eigenvlue tsk hs direct solution w = X y X y hese re the solutions used in PLS regression. hus we cn stte tht PLS regression is consistent with the H-principle. hus, the H-principle in cse of generl liner regression suggests: ) Crry out the modelling tsk in steps. b) It recommends using s weight vectors in the lgorithm in Section 3 the covrinces of the reduced dt, w X y (11) 1 1 In cse of instrumentl dt (X,y) it is recommended to use (11), which will give stndrd PLS regression. In cse some vribles re not to be included, the indices in w ssocited with these vribles re zero. In cse tht vrince mtrix S is used s strting point, like t Ridge Regression, the solution is found by using (11) s weight vectors Interprettion in erms of Prediction Vrince Suppose tht score vector t = Xw hs been computed. he explined fit by the score vector is (y t) 2 /(t t). If x 0 is new smple, the vrince of the estimted y-vlue would be ( y t) 2 2 t 0 Vr ( y( x0 )) y y 1+ 1 t t t t ( N ) Here t 0 is the score vlue ssocited with x 0, t0 = x0w. he properties of the norml distribution re being used here. he ltent vrible ssocited with t will be normlly distributed, becuse it is liner combintion of the originl vribles. he expression for the vrince is conditionl vrince given the score vector. Define now the function ( y t) 2 ( ) 2 t 0 f w = y y 1+ t t t t he function f hs the property tht it does not chnge when w is scled by positive constnt, c > 0. Choose c = c 0 so tht t = 1. hen we cn write f ( w) = f ( c ) ( ) 2 2 0w = y y y t 1+ t 0 reting t 0 s constnt it is cler tht mximizing the covrince is importnt. 814

8 Note tht the rgumenttion used here is heuristic, becuse we use the scle invrince of f(w) to ssume t = Scling of Dt he H-principle nd ssocited H-methods ssume tht the columns of X nd Y hve equl weight in the nlysis. herefore, it my be necessry to scle the dt. Scling of columns of X nd Y cn be chieved by multiplying the mtrices from the right by digonl mtrix. he liner lest squres solution is given by ( ) 1 B = X X X Y. If X nd Y re scled column-wise (by vribles), it mounts to the trnsformtions, X (XC 1 ) nd Y (YC 2 ), where C 1 nd C 2 re digonl mtrices. he solution B cn be obtined from the solution for scled dt, B 1, s follows, 1 {( ) ( )} ( ) ( ) B = C XC XC XC YC C = C B C his eqution shows tht if we re computing or pproximting the liner lest squres solution, we cn work with scled dt. When we wnt the solution for the originl dt, we scle bck s shown in the eqution. his property is lso used, when the pproximte solution is being computed. he effect of scling is better numericl precision. In the cse of spectrl dt scling is usully not necessry. If it is desired to work with the differentil or the second differentil of the spectrl curve, it my be necessry to scle the dt in order to obtin better precision. If originl dt follow norml distribution, the scled ones will not. here will be slight chnge due to scling. But properties due to normlity re only used s here s guidnce. Slight devitions from normlity will not ffect the nlysis. Scling of dt is difficult subject. e.g., if some vribles hve mny vlues below the noise level, it my be risky to scle dt. he problem of scling is not restricted to specific methods of regression nlysis. his topic is not considered closer here H-Principle in Other Ares he H-principle hs been extended to mny res of pplied mthemtics. In mny cses it hs opened up for new mthemtics. In [2] it hs been extended to pth modelling, giving new methods to crry out modelling of dt pths. In [3] it hs been pplied to non-liner modelling tht my give good low rnk solutions, where full rnk regulrized solutions do not give convergence. In [4] it is extended to multi-liner lgebr, where there re mny indices in dt, e.g., X = (x ijklm ) nd y = (y ij ). he use of directionl inverse mkes it possible to extend methods of ordinry mtrix nlysis. It hs lso been pplied to dynmic systems (process control nd time series), pttern recognition nd discriminnt nlysis, optimistion (qudrtic nd liner progrmming) nd severl other res. By working with ltent vribles they cn be tilored to hndle the mthemticl objective in question. e.g., dynmic systems re typiclly estimted by liner lest squres methods. But the H-principle suggests determining score vectors with optiml forecsting properties resulting in better methods thn trditionl ones. Numericl nlysts tlk bout n ill-posed or ill-conditioned mthemticl model, when the precision mtrix is singulr or close to singulr. But there cn be stble low rnk solution, which is fine to use. his is especilly importnt, where inter- nd extrpoltion is needed for the solution obtined (e.g. t prtil differentil eqution). 5. Model Vlidtion Mny methods in sttisticl nlysis re bsed on serch or some optimistion procedure. But the results re evluted s if they re bsed on rndom smpling. For instnce, in (forwrd) stepwise regression serch is crried out to find the vrible tht hs the lrgest squred correltion coefficient with the response vrible. he significnce of the finding is then evluted by t-test s if the vrible ws given beforehnd. But in rndom dt we cn expect 5% of the vrible to be significnt by sttisticl test. In PLS regression weight vector is determined by mximizing the size of the resulting y-loding vector, q. In these cses it is importnt to vlidte the obtined results. here re mny criteri vilble to check the dimension of model. Exmples re Mllows C p vlue nd Akike s informtion criterion. But they re developed for rndom smpling from distribution. In prctice they suggest too high number of (ltent) vribles. herefore, especilly in chemometrics, the prctice 815

9 hs become to use cross-vlidtion nd test set. Experience suggests tht both should be used. It is recommended t ech step to look t how well the score vectors re doing. his holds for both clibrtion dt, which re used for estimting the prmeters, for the test set nd for cross-vlidtion. hus, we plot y ginst the columns of, the test set y t ginst t nd cross-vlidted y-vlues, y c, ginst c. If the lst two types of plots do not show ny correltion, it is time to consider stopping modelling of dt. In most cses these procedures work well. But there my be cses when they do not work well. his my hppen, when there is non-linerity in dt, some extreme y-vlues, gps in dt or grouping in dt. Crossvlidtion or the use of test set my be ffected by these specil fetures in dt. It cn be recommended to check the Y-t (y ginst the columns of ) plots to see if there re these specil fetures in dt. Simple sttisticl tests cn lso be used to inform of these specil fetures. An importnt test is test of rndomness of residuls. But mny other cn be recommended Cross-Vlidtion At cross-vlidtion the smples re divided into groups, sy 10 groups. One group is put side nd the other 9 re used to estimte the prmeters of the model. his is repeted for ll groups. he result is n estimtion of the y-vlues, ŷ c, where ech is estimted by using 90% of the smples. Similrly, score vlues, c, re estimted t the sme time s the y-vlues similrly s t test set, see Section 3.2. In sttistics it is common to use leve-one-out cross-vlidtion, where the number of groups is equl to the number of smples. his procedure is importnt to use in order to check for outliers, which my spoil the modelling tsk. But it is not considered very efficient s vlidtion procedure. In chemometrics it is common to use 7 fold cross-vlidtion where smples re rndomly divided into 7 groups. he rgument is tht the model developed needs strong evlution test. here my be problems, when using the cross-vlidtion pproch. In the glucose dt considered here, round 80% of the glucose vlues re in the rnge he histogrm of the lrger vlues flls smoothly until Even the histogrm of the logrithmic vlues is skew. Different cross-vlidtions give different results depending on how the lrge y-vlues re locted in the groups. herefore, it my be better to use ordered cross-vlidtion. Here the y-vlues re ordered nd sy, every 10 th of the ordered y-vlues define the groups. In the exmples in Section 7 this type of ordered cross-vlidtion is used (first group 1 st, 11 th,, etc) est Set Industry stndrds [5] require tht new mesurement method should be tested by 40 new smples in blind test. In blind test the 40 smples hve not been used in ny wy in the modelling tsk. Idelly, sufficient mount of smples should be obtined so tht 40 smples cn be put side. est set is often selected by ordering the y-vlues like t ordered cross-vlidtion. However, it my be better to use the Kennrd-Stone procedure, [6], tht orders the smples ccording to their mutul distnce. he first two re the ones tht re most prt, next smple the furthermost wy from the first two, etc. In the exmples in this pper test set is determined by selecting every 5 th of the ordered smples by the Kennrd-Stone procedure. Sometimes, different types of test sets should be selected. As n exmple one cn mention the sitution, where most y-vlues re very smll nd only few re lrge. Here it my be better to order the smples ccording to the first PCA- or PLS score vectors Mesures of Importnce Modelling of dt should continue s long s there is covrince. he dimension of model is determined by finding, when X y 0 for reduced mtrices. here re two wys to look t this. One is to study the individul terms, ( xi y ), nd the other is totl vlue, ( ) 2 i y XXy = i x y. Assume tht dt cn be described by multivrite norml distribution with covrince mtrix Σ xy. hen, it is shown in [7] tht the smple covri- nces ( xi y ) ( N 3) re pproximtely normlly distributed. If σ xi,y = 0, it is shown in [7] tht pproximte 95% limits for the residul covrince, ( x y ) ( ) i N 3, re given by ( ) ( ) ± 1.96 Nσ σ N 3 ± 1.96 Ns s N 3 xi y xi y 816

10 hus, when modelling stops, it is required tht ll residul covrinces should be within these limits. If σ xi,y = 0, the distribution of the residul covrince pproches quickly the norml distribution by the centrl limit theorem. herefore, it is relible mesure to judge, if the covrinces hve become zero or close to zero. If the covrince Σ xy is zero, Σ xy = 0, the men, E{(y XX y)}, nd vrince, Vr{(y XX y)}, cn be computed, see [1]. he upper 95% limit of norml distribution N(µ,σ 2 ) is µ σ. his is used for men nd vrince. In the nlysis it is checked if y XX y is below the upper 95% limit ( one-sided test), ( )( ) N tr ( )( ) 2 y XX y < tr X X y y + X XX X y y 2 tr is the trce function. When this inequlity is stisfied, there is n indiction tht modelling should stop. his nlysis hs been found useful, when nlysing spectrl dt. 6. H-Methods he H-principle recommends using s weights in the regression lgorithm the covrinces t ech step, w X 1y 1. his holds both in the cse where there re given instrumentl dt (X,y) nd vrince/ covrince (S, X y). H-methods study the sitution, when the weight vectors re the (residul) covrinces. hey re of two types. One type is concerning finding the dt, X1 = ( xi 1, xi2,, xin ), tht should be used. he other type re studies of the weights, (w ). A brief review of some of the methods is presented Determining Subset of Vribles It is importnt to find good subset of vribles for use in regression nlysis. Some of the most used methods in progrm pckges re vrible selection methods. In forwrd selection method vribles re selected one by one. he vrible tht is dded to the pool of selected vribles is the one the gives the highest vlue of some mesure mong the vribles tht hve not been selected Forwrd nd Bckwrd Selection of Vribles In ltent structure regression the following pproch hs been found efficient. Crry out forwrd selection of vribles. A vrible is dded to the pool of selected vribles, if it, together with the vribles lredy selected, gives the best cross-vlidtion mong the unselected vribles. Bckwrd selection of vribles to be deleted is then strted, where the best results of the forwrd selection were obtined. One vrible is deleted from the pool of vribles. he vrible is deleted, which gives the highest cross-vlidtion vlues mong the vribles in the pool. he number of vribles chosen for nlysis is the one tht gives the highest vlue of cross-vlidtion t the bckwrds deletion. his procedure cn be time consuming. Altogether K(K + 1)/2 forwrd regression nlysis with cross-vlidtion re crried out. If K = 1000 more thn hlf million nlysis is crried out Anlysis Bsed on Principl Vribles Principl vribles re ones tht hve mximl covrince. he first one is the one t mx ( x i y ). In the lgorithm = ( 0,0,,1,0, ) N w, where the index of 1 is t the principl vrible, [1]. he next principl vrible is the one hving mximl covrince, now for the djusted dt. he importnt property is tht if the covrince is x y x y. Principl vribles re used to replce the not smll, the size of the reduced x i is not smll, ( i ) originl ones with smller set of vribles. e.g. spectrum my represent 1000 vribles. Computing the principl vribles, one my suffice to nlyse, sy 150 principl vribles insted of the originl Forwrd nd bckwrd selection of vribles cn be crried out now for the principl vribles. It is known tht it is not efficient to bse the modelling tsk on the first few principl vribles, lthough this is stted in [8] Modelling long Ordering of Vribles here re now vilble spectroscopic sensors tht cn be instlled t different loctions in compny. Especilly chemicl process compnies hve instlled mny; 200 loctions is not lrge. here is need for simple method tht cn be progrmmed esily nd is not time consuming to run. his cn be obtined by computing the simple correltions coefficients, r yxi, or the covrinces, i x y, nd sort them ccording to their numericl vlue i 817

11 with lrgest first. PLS regression cn be crried out long this ordering of vribles. his simple method, which is described in [9], hs been implemented t severl chemicl process compnies Study of the Residul Covrinces Smll vlues in weight vectors indicte tht corresponding vribles hve smll influence on the results. For spectroscopic dt there lwys re collection of vribles tht should not be used. here cn lso be question of the focus of the regression nlysis tht my suggest some other weight vlues then (11) Cov Proc Methods In bio-ssy studies there cn be vribles or even more. he experimenter is eqully interested in importnt vribles nd s in prediction. his is done by the CovProc method, [10]. he ide is to combine the pproch of theoreticl sttistics, where focus is on R 2, nd of PLS regression, where score vector my not be too smll. When the weight vector, w, of PLS regression hs been computed, it is sorted ccording to size, w (s), the lrgest first. A score vector is chosen tht gives the mximum of R 2, ( s) ( s) ( s) ( s) 2 = w, w,, w,0,0,,0, mx R 2 = mx y t t t, i = 1, 2,, K. ( 1 2 i ) ti X i ( i) ( i i) X (s) is the sme sorting of columns of X. CovProc method hs the dvntge tht the first few score vectors, typiclly one or two or three, re bsed on very few vribles but they explin lrge vrition of X nd Y. his implies tht plots of the y-t 1, y-t 2, y-t 3 nd t 1 -t 2 -t 3 cn be trnsferred to few vribles, which mkes the interprettion useful. Lrge experimentl studies hve found this method efficient, [11]. Different comprisons to other methods in the experimentl re hve been crried out, see [12] [13], where CovProc method is the preferred one Smll Vlues of Residul Covrinces When there re mny vribles, it is necessry to delete vribles tht hve no or smll covrince with the response vrible. A useful procedure is to sort the vlues of the weight vectors, w, nd study the smll vlues found. One procedure is to delete one by one of the smllest vlues nd mesure the effects. his is continued until no more should be deleted. hen we strt over with the deleted vlues nd investigte if some of the deleted ones should be included gin. his procedure is fine tuning of the weights. If no other procedure is used, it often gives significnt improvement compred to the originl weights. 7. Comprison of Methods 7.1. he Dt 210 blood plsm smples were mesured on Bruker ensor 27 FIR instrument, [14]. he mesurement unit tht receives the plsm smples is AquSpec from Micro-Biolytics, [15]. he direct mesurements of glucose in blood plsm smples ws crried out on Cobs C 501 instrument by stff members t Deprtment of Clinicl Biochemistry, Holbæk Hospitl, Denmrk. Most of the smples were from dibetes ptients giving high glucose vlues. he producer of the Cobs instrument tht mesured glucose, Roche Dignostics D Mnnheim, informs tht the coefficient of vrition, CV, is less thn 1.5%. he verge glucose vlue is 8.17 mmol/l. his gives stndrd devition of s = hus, we my expect tht the lbortory vlues of glucose hve uncertinties with this stndrd devition. Ech spectrum from the Bruker instrument consists of 1115 vlues. he differentil spectr were used. Cross-vlidtion showed tht better results were obtined by using the differentil spectr compred to the originl ones. he instrumentl dt X re thus nd the y-dt vector of 210 glucose vlues. Leve-one-out nlysis reveled tht one smple ws n outlier nd it ws removed. Forwrd selection of vribles ws crried out followed by bckwrds deletion of vribles, Section A mesure of cross-vlidtion ws used in selection nd deletion of vribles. he bckwrd nlysis strted t the number of vribles, where the cross-vlidtion t the forwrd nlysis ws t its highest. he result ws tht 50 vribles of the originl 1115 were optiml to use. hus, in the following nlysis X is nd y is vector of 209 elements. In ech nlysis, both forwrd nd bckwrd, PLS Regressions re used with dimension A t most PLS Regression he dt were centred nd scled to unit vrince. Considerble improvement ws obtined by using scling of 818

12 spectr. he methods of section 5.3 suggested tht the covrinces hd become zeros, when the 12 th dimension hd been completed. Anlysis of cross-vlidtions nd test sets lso suggested dimension 12. In Figure 1() is shown sctter plot between the mesured glucose vlues on y-xis nd the estimted y-vlues ŷ on the x-xis by PLS regression hving dimension 12. he squred correltion coefficient of the sctter of points is R 2 = nd stndrd devition of the residul devition, (y ŷ), equls s = mmol/l. In Figure 1(b) is shown the sctter plot of glucose vlues on y-xis nd the cross-vlidted y-vlues, ŷ c, on x-xis. Ordered cross-vlidtion using the y-vlues ws used, see Section 5.1. Here the squred correltion of the sctter of points is R 2 = nd the stndrd devition of (y ŷ c ) equls s = mmol/l. Figures 1(c) nd (d) re the results of pplying test set. 40 smples derived by the Kennrd-Stone procedure, see Section 5.2, were used s test set. PLS regression ws crried out for the 169 smples nd the results re presented in Figure 1(c). Here we get the squred correltion of R 2 = nd the stndrd devition of the residul devition of s = mmol/l. he results of this nlysis were pplied to the 40 excluded smples nd the results re shown in Figure 1d. Here the squred correltion is R 2 = nd the stndrd devition of the residuls equl s = mmol/l. Note, tht completely new PLS regressions were crried out for ech of the 10 groups t cross-vlidtion nd t the test set. he use of test set here is not blind test s recommended by [5], becuse ll 209 smples were used in finding the 50 vribles. But seprte blind test gve close to the sme results. he results of the nlysis re reported by the results from the ordered cross-vlidtion. hus, we expect to be ble to mesure glucose by the FIR instrument with stndrd devition of 0.21 mmol/l. his is stisfctory, becuse we my expect stndrd devition of 0.12 mmol/l by the lbortory nd, s rule of thumb, one my expect uncertinty of 1% - 1.5% by the FIR instrument. his result of determining glucose vlues from n FIR instrument is better thn seen in the literture; see [16] nd the references there Ridge Regression When there re mny vribles, it is common to pply Ridge Regression, RR. here is not problem in determining the inverse for the present dt. he condition number is λ mx /λ min = 396, where the λ s re the singulr vlues of Singulr Vlue Decomposition of scled X. he regulrized solution of RR is given by ( k ) 1 b = X X + I X y, where k is some smll constnt nd I the identity mtrix. he theory sttes tht there is constnt k tht improves the men squred error of prediction compred to the lest squres solution. here re some theoreticl considertions of how to determine k. But they re rrely pplied, becuse the vlue of k obtined is usully bd. Insted it is common to use the leve-one-out pproch. he vlue of k is used tht gives the lowest leve-one-out vlue, N ( y ˆ( ) ) 2 1 i y, where ŷ i (i) is the estimted y i -vlue, when the i th smple is excluded. An esy nd numericlly precise wy to crry out this nlysis is to use the SVD decomposition of X (i), where i th row is excluded from X. he vlue of k cn be determined in two steps. At first the intervl (0,1) is divided into 100 vlues, 0, 0.01, 0.02, he vlue of k is determined tht gives the smllest vlue. Suppose it is hen, secondly, the intervl 0 to 0.02 is divided into 200 prts nd 200 leve-one-out nlyses re crried out. When these hve been crried out for the 50 vribles, the result is k = RR nlysis ws repeted for the 169 smples, where the 40 smples from the Kennrd-Stone procedure were excluded. Here the vlue of k is k = he RR nlysis is be crried out by the lgorithm described in Section 3.2 with the weights (11). he vlue of k in both cses is so smll tht there is prcticlly no difference from the stndrd cse, where k = 0, which corresponds to PLS regression. In the RR nlysis the result is tht dimension should be 12. Model control is crried out in the sme wys s for PLS regression. Appliction of the methods in Section 5.3 shows tht there re no significnt covrinces left fter dimension 12. he score vectors cn be computed by the eqution = XV. Also for the test set. Here the score vectors re not orthogonl. Score vectors from ordered cross-vlidtion, c, cn lso be computed. All plots of score vectors,, t, c, ginst the respective y-vlues show tht there re no correltions fter dimension 12. herefore, one should not compute the full rnk solution, but suffice with the solution t dimension 12. In ble 1 is shown the results of the full rnk RR solution. he differences between the results re so smll tht they hve no prcticl vlue. herefore, Figure 1 is not shown for RR results. here is property of RR, which mkes RR unppeling to use for scientists. Suppose tht there re more sm- ples thn vribles, N > K. hen, see [1], for given k there is mtrix Z so tht ( X + Z) ( X + Z) = X X +ki. hus, RR cn be viewed s dding some smll rndom numbers to X. his chnge of X is not necessry, becuse 819

13 Figure 1. Sctter plots of mesured versus computed glucose vlues. ble 1. Results of PLS regression nd Ridge regression. All dt Ord. Cr-vl Reduced est Figure 1 1b 1c 1d PLS regression R Dimension 12 s Ridge Regression R k = , k = s there will be low rnk solution for X, which is eqully good or better. In conclusion, only dimension 12 should be used for the RR nlysis. In this cse there is no prcticl difference between PLS regression nd RR. Full rnk RR solution is lso lmost the sme s the PLS one he Vrince Vr(b) here is common greement tht Vr(b) is n importnt mesure. herefore, it is useful to look t how it increses by the number of dimensions used. his is shown in Figure 2. he trce function, tr, gives the sum of digonl elements of mtrix. hus, tr(vr(b)) is the sum of the vrince of the regression coefficients. In plot ( ) 12 it is esier to use the squre root to mke the lrge vlues smller, Std( ) = tr ( Vr ( )) it is Std d N A 1 A ( b) = y 2 Σ( y t ) t Σ 2 v ( ) b b. For PLS regression It is shown in Figure 2 for A = 1 to 50. At A = 12 the vlue is Std(b) = 2.8. In full rnk solution the vlue is Std(b) = 350. A high price is being pid, if full rnk solution is chosen. It should be emphsized tht Std(b) A is only guidnce mesure. he vrince of the regression coefficients cn be derived by bootstrpping methods. But the vlues give guidnce for wht cn be expected. Vr(b) is slightly different for RR nd the curve is lower (highest vlue is 115). 8. Vribles nd Ltent Vribles Ltent vribles re estimtes of the min xis in the hyper-ellipsoid, where dt typiclly re locted. Ltent vribles usully pick up the totl vrition much quicker thn the vribles. his is illustrted in Figure 3. he 820

14 Figure 2. Plot of Std(b) versus dimension, A. Figure 3. Explined vrition, y-xis, versus the number of vribles in the model, x-xis. 821

15 ltent vribles strt t explined vrition of nd t the 12 th the totl vrition hs reched he vribles strt t nd it is first t the 33 th vrible tht the explined vrition of is reched. Mllows C p theory is vlid for regression nlysis both for vribles nd ltent vribles. It sttes tht, from modelling point of view, it is importnt to keep the size of dimension s low s possible. ypiclly, the number is much smller for ltent structure regression thn for regression methods bsed on vribles. Scientists/experimenters often tend to sk the question: Which vribles re most importnt? When working with ltent structure regression it is difficult to give precise nswer. For the FIR dt we cn sy tht there re 1115 vribles. Forwrd nd bckwrd nlysis reduces this number to 50. Ltent structure regression uses 12 dimensions in the regression nlysis. All 50 vribles contribute to the regression nlysis. It is possible to explin the cumultive effects like t Figure Conclusions A short review of stndrd regression nlysis used in the populr progrm pckges hs been presented. It is pointed out tht it is serious problem in industry nd pplied sciences tht the obtined results nd presented in ppers nd reports my not be relible. A generl frmework is presented for liner regression, where the score vectors re orthogonl. Any set of weight vectors cn be used tht do not give score/loding vectors of zero size. Using the frmework, different types of regression nlysis cn be tilored to different situtions. he sme type of grphic nlysis cn be crried out for ny type of regression nlysis within this frmework; lso when the strt of the computtions is vrince nd covrince mtrices. he lgorithm cn be viewed s n pproximtion to the full rnk solution. Modelling stops, if further steps re not supported by dt. Dimension mesures, cross-vlidtion nd test sets re used to identify the dimension properly. he H-principle is formulted in close nlogy to the Heisenberg Uncertinty Inequlity. It suggests tht modelling should be crried out in steps, where t ech step n optiml blnce between the fit nd ssocited precision should be obtined. A collection of methods, the H-methods, hve been developed to implement the H-principle in different contexts. H-methods hve been pplied for number of yers to different types of dt within pplied sciences nd industry. he generl experience is tht they provide better predictions thn other methods in the literture. Acknowledgements he Sime project is crried out in coopertion with Clinicl Biochemistry, Holbæk Hospitl, Denmrk, on determining substnces in blood plsm by FIR spectroscopy. I pprecite the use of the results from the glucose nlysis in the present rticle. References [1] Höskuldsson, A. (1996) Prediction Methods in Science nd echnology. Vol. 1, hor Publishing, Copenhgen. [2] Höskuldsson, A. (2009) Modelling Procedures for Directed Network of Dt Blocks. Chemometrics nd Intelligent Lbortory Systems, 97, [3] Höskuldsson, A. (2008) H-Methods in Applied Sciences. Journl of Chemometrics, 22, [4] Höskuldsson, A. (1994) Dt Anlysis, Mtrix Decompositions nd Generlised Inverse. SIAM Journl on Scientific Computing, 15, [5] Clinicl nd Lbortory Stndrds Institute. [6] Kennrd, R.W. nd Stone, L.A. (1969) Computer Aided Design of Experiment. echnometrics, 11, [7] Siotni, M., Hykw,. nd Fujikoshi, Y. (1985) Modern Multivrite Anlysis: A Grdute Course nd Hndbook. Americn Science Press, Columbus. [8] Roger, J.M., Plgos, B., Bertrnd, D. nd Fernndez-Ahumd, E. (2011) CovSel: Vrible Selection for Highly Multivrite nd Multi-Response Clibrtion. Appliction to IR Spectroscopy. Chemometrics nd Intelligent Lbortory Systems, 106,

16 [9] Höskuldsson, A. (2001) Vrible nd Subset Selection in PLS Regression. Chemometrics nd Intelligent Lbortory Systems, 557, [10] Reinikinen, S.-P. nd Höskuldsson, A. (2003) COVPROC Method: Strtegy in Modeling Dynmic Systems. Journl of Chemometrics, 17, [11] Grove, H., et l. (2008) Combintion of Sttisticl Approches for Anlysis of 2-DE Dt Gives Complementry Results. Proteome Reserch, 7, [12] McLeod, G., et l. (2009) A Comprison of Vrite Pre-Selection Methods for Use in Prtil Lest Squres Regression: A Cse Study on NIR Spectroscopy Applied to Monitoring Beer Fermenttion. Journl of Food Engineering, 90, [13] pp, H.S., et l. (2012) Evlution of Multiple Vrite Methods from Biologicl Perspective: A Nutrigenomics Cse Study. Genes & Nutrition, 7, [14] Bruker Optics, Germny. [15] Micro-Biolytics, Germny. [16] Perez-Guit, D. et l. (2013) Modified Loclly Weighted Prtil Lest Squres Regression, Improving Clinicl Predictions from Infrred Spectr of Humn Serum Smples. lnt, 170,

New Expansion and Infinite Series

New Expansion and Infinite Series Interntionl Mthemticl Forum, Vol. 9, 204, no. 22, 06-073 HIKARI Ltd, www.m-hikri.com http://dx.doi.org/0.2988/imf.204.4502 New Expnsion nd Infinite Series Diyun Zhng College of Computer Nnjing University

More information

Student Activity 3: Single Factor ANOVA

Student Activity 3: Single Factor ANOVA MATH 40 Student Activity 3: Single Fctor ANOVA Some Bsic Concepts In designed experiment, two or more tretments, or combintions of tretments, is pplied to experimentl units The number of tretments, whether

More information

Monte Carlo method in solving numerical integration and differential equation

Monte Carlo method in solving numerical integration and differential equation Monte Crlo method in solving numericl integrtion nd differentil eqution Ye Jin Chemistry Deprtment Duke University yj66@duke.edu Abstrct: Monte Crlo method is commonly used in rel physics problem. The

More information

Data Assimilation. Alan O Neill Data Assimilation Research Centre University of Reading

Data Assimilation. Alan O Neill Data Assimilation Research Centre University of Reading Dt Assimiltion Aln O Neill Dt Assimiltion Reserch Centre University of Reding Contents Motivtion Univrite sclr dt ssimiltion Multivrite vector dt ssimiltion Optiml Interpoltion BLUE 3d-Vritionl Method

More information

Math 1B, lecture 4: Error bounds for numerical methods

Math 1B, lecture 4: Error bounds for numerical methods Mth B, lecture 4: Error bounds for numericl methods Nthn Pflueger 4 September 0 Introduction The five numericl methods descried in the previous lecture ll operte by the sme principle: they pproximte the

More information

How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?

How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique? XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk bout solving systems of liner equtions. These re problems tht give couple of equtions with couple of unknowns, like: 6 2 3 7 4

More information

1B40 Practical Skills

1B40 Practical Skills B40 Prcticl Skills Comining uncertinties from severl quntities error propgtion We usully encounter situtions where the result of n experiment is given in terms of two (or more) quntities. We then need

More information

7.2 The Definite Integral

7.2 The Definite Integral 7.2 The Definite Integrl the definite integrl In the previous section, it ws found tht if function f is continuous nd nonnegtive, then the re under the grph of f on [, b] is given by F (b) F (), where

More information

NUMERICAL INTEGRATION. The inverse process to differentiation in calculus is integration. Mathematically, integration is represented by.

NUMERICAL INTEGRATION. The inverse process to differentiation in calculus is integration. Mathematically, integration is represented by. NUMERICAL INTEGRATION 1 Introduction The inverse process to differentition in clculus is integrtion. Mthemticlly, integrtion is represented by f(x) dx which stnds for the integrl of the function f(x) with

More information

Chapter 5 : Continuous Random Variables

Chapter 5 : Continuous Random Variables STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 216 Néhémy Lim Chpter 5 : Continuous Rndom Vribles Nottions. N {, 1, 2,...}, set of nturl numbers (i.e. ll nonnegtive integers); N {1, 2,...}, set of ll

More information

p-adic Egyptian Fractions

p-adic Egyptian Fractions p-adic Egyptin Frctions Contents 1 Introduction 1 2 Trditionl Egyptin Frctions nd Greedy Algorithm 2 3 Set-up 3 4 p-greedy Algorithm 5 5 p-egyptin Trditionl 10 6 Conclusion 1 Introduction An Egyptin frction

More information

SUMMER KNOWHOW STUDY AND LEARNING CENTRE

SUMMER KNOWHOW STUDY AND LEARNING CENTRE SUMMER KNOWHOW STUDY AND LEARNING CENTRE Indices & Logrithms 2 Contents Indices.2 Frctionl Indices.4 Logrithms 6 Exponentil equtions. Simplifying Surds 13 Opertions on Surds..16 Scientific Nottion..18

More information

Math 520 Final Exam Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Math 520 Final Exam Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Mth 520 Finl Exm Topic Outline Sections 1 3 (Xio/Dums/Liw) Spring 2008 The finl exm will be held on Tuesdy, My 13, 2-5pm in 117 McMilln Wht will be covered The finl exm will cover the mteril from ll of

More information

Chapter 14. Matrix Representations of Linear Transformations

Chapter 14. Matrix Representations of Linear Transformations Chpter 4 Mtrix Representtions of Liner Trnsformtions When considering the Het Stte Evolution, we found tht we could describe this process using multipliction by mtrix. This ws nice becuse computers cn

More information

Chapter 3 Polynomials

Chapter 3 Polynomials Dr M DRAIEF As described in the introduction of Chpter 1, pplictions of solving liner equtions rise in number of different settings In prticulr, we will in this chpter focus on the problem of modelling

More information

Continuous Random Variables

Continuous Random Variables STAT/MATH 395 A - PROBABILITY II UW Winter Qurter 217 Néhémy Lim Continuous Rndom Vribles Nottion. The indictor function of set S is rel-vlued function defined by : { 1 if x S 1 S (x) if x S Suppose tht

More information

Numerical integration

Numerical integration 2 Numericl integrtion This is pge i Printer: Opque this 2. Introduction Numericl integrtion is problem tht is prt of mny problems in the economics nd econometrics literture. The orgniztion of this chpter

More information

ARITHMETIC OPERATIONS. The real numbers have the following properties: a b c ab ac

ARITHMETIC OPERATIONS. The real numbers have the following properties: a b c ab ac REVIEW OF ALGEBRA Here we review the bsic rules nd procedures of lgebr tht you need to know in order to be successful in clculus. ARITHMETIC OPERATIONS The rel numbers hve the following properties: b b

More information

Non-Linear & Logistic Regression

Non-Linear & Logistic Regression Non-Liner & Logistic Regression If the sttistics re boring, then you've got the wrong numbers. Edwrd R. Tufte (Sttistics Professor, Yle University) Regression Anlyses When do we use these? PART 1: find

More information

Lecture 14: Quadrature

Lecture 14: Quadrature Lecture 14: Qudrture This lecture is concerned with the evlution of integrls fx)dx 1) over finite intervl [, b] The integrnd fx) is ssumed to be rel-vlues nd smooth The pproximtion of n integrl by numericl

More information

1.9 C 2 inner variations

1.9 C 2 inner variations 46 CHAPTER 1. INDIRECT METHODS 1.9 C 2 inner vritions So fr, we hve restricted ttention to liner vritions. These re vritions of the form vx; ǫ = ux + ǫφx where φ is in some liner perturbtion clss P, for

More information

Properties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives

Properties of Integrals, Indefinite Integrals. Goals: Definition of the Definite Integral Integral Calculations using Antiderivatives Block #6: Properties of Integrls, Indefinite Integrls Gols: Definition of the Definite Integrl Integrl Clcultions using Antiderivtives Properties of Integrls The Indefinite Integrl 1 Riemnn Sums - 1 Riemnn

More information

The First Fundamental Theorem of Calculus. If f(x) is continuous on [a, b] and F (x) is any antiderivative. f(x) dx = F (b) F (a).

The First Fundamental Theorem of Calculus. If f(x) is continuous on [a, b] and F (x) is any antiderivative. f(x) dx = F (b) F (a). The Fundmentl Theorems of Clculus Mth 4, Section 0, Spring 009 We now know enough bout definite integrls to give precise formultions of the Fundmentl Theorems of Clculus. We will lso look t some bsic emples

More information

W. We shall do so one by one, starting with I 1, and we shall do it greedily, trying

W. We shall do so one by one, starting with I 1, and we shall do it greedily, trying Vitli covers 1 Definition. A Vitli cover of set E R is set V of closed intervls with positive length so tht, for every δ > 0 nd every x E, there is some I V with λ(i ) < δ nd x I. 2 Lemm (Vitli covering)

More information

Lecture 19: Continuous Least Squares Approximation

Lecture 19: Continuous Least Squares Approximation Lecture 19: Continuous Lest Squres Approximtion 33 Continuous lest squres pproximtion We begn 31 with the problem of pproximting some f C[, b] with polynomil p P n t the discrete points x, x 1,, x m for

More information

Chapter 0. What is the Lebesgue integral about?

Chapter 0. What is the Lebesgue integral about? Chpter 0. Wht is the Lebesgue integrl bout? The pln is to hve tutoril sheet ech week, most often on Fridy, (to be done during the clss) where you will try to get used to the ides introduced in the previous

More information

The steps of the hypothesis test

The steps of the hypothesis test ttisticl Methods I (EXT 7005) Pge 78 Mosquito species Time of dy A B C Mid morning 0.0088 5.4900 5.5000 Mid Afternoon.3400 0.0300 0.8700 Dusk 0.600 5.400 3.000 The Chi squre test sttistic is the sum of

More information

Math& 152 Section Integration by Parts

Math& 152 Section Integration by Parts Mth& 5 Section 7. - Integrtion by Prts Integrtion by prts is rule tht trnsforms the integrl of the product of two functions into other (idelly simpler) integrls. Recll from Clculus I tht given two differentible

More information

1 Online Learning and Regret Minimization

1 Online Learning and Regret Minimization 2.997 Decision-Mking in Lrge-Scle Systems My 10 MIT, Spring 2004 Hndout #29 Lecture Note 24 1 Online Lerning nd Regret Minimiztion In this lecture, we consider the problem of sequentil decision mking in

More information

Recitation 3: More Applications of the Derivative

Recitation 3: More Applications of the Derivative Mth 1c TA: Pdric Brtlett Recittion 3: More Applictions of the Derivtive Week 3 Cltech 2012 1 Rndom Question Question 1 A grph consists of the following: A set V of vertices. A set E of edges where ech

More information

Week 10: Line Integrals

Week 10: Line Integrals Week 10: Line Integrls Introduction In this finl week we return to prmetrised curves nd consider integrtion long such curves. We lredy sw this in Week 2 when we integrted long curve to find its length.

More information

Unit #9 : Definite Integral Properties; Fundamental Theorem of Calculus

Unit #9 : Definite Integral Properties; Fundamental Theorem of Calculus Unit #9 : Definite Integrl Properties; Fundmentl Theorem of Clculus Gols: Identify properties of definite integrls Define odd nd even functions, nd reltionship to integrl vlues Introduce the Fundmentl

More information

Math 8 Winter 2015 Applications of Integration

Math 8 Winter 2015 Applications of Integration Mth 8 Winter 205 Applictions of Integrtion Here re few importnt pplictions of integrtion. The pplictions you my see on n exm in this course include only the Net Chnge Theorem (which is relly just the Fundmentl

More information

A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H. Thomas Shores Department of Mathematics University of Nebraska Spring 2007

A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H. Thomas Shores Department of Mathematics University of Nebraska Spring 2007 A REVIEW OF CALCULUS CONCEPTS FOR JDEP 384H Thoms Shores Deprtment of Mthemtics University of Nebrsk Spring 2007 Contents Rtes of Chnge nd Derivtives 1 Dierentils 4 Are nd Integrls 5 Multivrite Clculus

More information

ECO 317 Economics of Uncertainty Fall Term 2007 Notes for lectures 4. Stochastic Dominance

ECO 317 Economics of Uncertainty Fall Term 2007 Notes for lectures 4. Stochastic Dominance Generl structure ECO 37 Economics of Uncertinty Fll Term 007 Notes for lectures 4. Stochstic Dominnce Here we suppose tht the consequences re welth mounts denoted by W, which cn tke on ny vlue between

More information

Best Approximation. Chapter The General Case

Best Approximation. Chapter The General Case Chpter 4 Best Approximtion 4.1 The Generl Cse In the previous chpter, we hve seen how n interpolting polynomil cn be used s n pproximtion to given function. We now wnt to find the best pproximtion to given

More information

Exam 2, Mathematics 4701, Section ETY6 6:05 pm 7:40 pm, March 31, 2016, IH-1105 Instructor: Attila Máté 1

Exam 2, Mathematics 4701, Section ETY6 6:05 pm 7:40 pm, March 31, 2016, IH-1105 Instructor: Attila Máté 1 Exm, Mthemtics 471, Section ETY6 6:5 pm 7:4 pm, Mrch 1, 16, IH-115 Instructor: Attil Máté 1 17 copies 1. ) Stte the usul sufficient condition for the fixed-point itertion to converge when solving the eqution

More information

MATH34032: Green s Functions, Integral Equations and the Calculus of Variations 1

MATH34032: Green s Functions, Integral Equations and the Calculus of Variations 1 MATH34032: Green s Functions, Integrl Equtions nd the Clculus of Vritions 1 Section 1 Function spces nd opertors Here we gives some brief detils nd definitions, prticulrly relting to opertors. For further

More information

Chapter 4 Contravariance, Covariance, and Spacetime Diagrams

Chapter 4 Contravariance, Covariance, and Spacetime Diagrams Chpter 4 Contrvrince, Covrince, nd Spcetime Digrms 4. The Components of Vector in Skewed Coordintes We hve seen in Chpter 3; figure 3.9, tht in order to show inertil motion tht is consistent with the Lorentz

More information

221B Lecture Notes WKB Method

221B Lecture Notes WKB Method Clssicl Limit B Lecture Notes WKB Method Hmilton Jcobi Eqution We strt from the Schrödinger eqution for single prticle in potentil i h t ψ x, t = [ ] h m + V x ψ x, t. We cn rewrite this eqution by using

More information

Credibility Hypothesis Testing of Fuzzy Triangular Distributions

Credibility Hypothesis Testing of Fuzzy Triangular Distributions 666663 Journl of Uncertin Systems Vol.9, No., pp.6-74, 5 Online t: www.jus.org.uk Credibility Hypothesis Testing of Fuzzy Tringulr Distributions S. Smpth, B. Rmy Received April 3; Revised 4 April 4 Abstrct

More information

ODE: Existence and Uniqueness of a Solution

ODE: Existence and Uniqueness of a Solution Mth 22 Fll 213 Jerry Kzdn ODE: Existence nd Uniqueness of Solution The Fundmentl Theorem of Clculus tells us how to solve the ordinry differentil eqution (ODE) du = f(t) dt with initil condition u() =

More information

Physics 116C Solution of inhomogeneous ordinary differential equations using Green s functions

Physics 116C Solution of inhomogeneous ordinary differential equations using Green s functions Physics 6C Solution of inhomogeneous ordinry differentil equtions using Green s functions Peter Young November 5, 29 Homogeneous Equtions We hve studied, especilly in long HW problem, second order liner

More information

The Regulated and Riemann Integrals

The Regulated and Riemann Integrals Chpter 1 The Regulted nd Riemnn Integrls 1.1 Introduction We will consider severl different pproches to defining the definite integrl f(x) dx of function f(x). These definitions will ll ssign the sme vlue

More information

CS667 Lecture 6: Monte Carlo Integration 02/10/05

CS667 Lecture 6: Monte Carlo Integration 02/10/05 CS667 Lecture 6: Monte Crlo Integrtion 02/10/05 Venkt Krishnrj Lecturer: Steve Mrschner 1 Ide The min ide of Monte Crlo Integrtion is tht we cn estimte the vlue of n integrl by looking t lrge number of

More information

Operations with Polynomials

Operations with Polynomials 38 Chpter P Prerequisites P.4 Opertions with Polynomils Wht you should lern: How to identify the leding coefficients nd degrees of polynomils How to dd nd subtrct polynomils How to multiply polynomils

More information

Review of Gaussian Quadrature method

Review of Gaussian Quadrature method Review of Gussin Qudrture method Nsser M. Asi Spring 006 compiled on Sundy Decemer 1, 017 t 09:1 PM 1 The prolem To find numericl vlue for the integrl of rel vlued function of rel vrile over specific rnge

More information

Review of Calculus, cont d

Review of Calculus, cont d Jim Lmbers MAT 460 Fll Semester 2009-10 Lecture 3 Notes These notes correspond to Section 1.1 in the text. Review of Clculus, cont d Riemnn Sums nd the Definite Integrl There re mny cses in which some

More information

Numerical Integration

Numerical Integration Chpter 1 Numericl Integrtion Numericl differentition methods compute pproximtions to the derivtive of function from known vlues of the function. Numericl integrtion uses the sme informtion to compute numericl

More information

Predict Global Earth Temperature using Linier Regression

Predict Global Earth Temperature using Linier Regression Predict Globl Erth Temperture using Linier Regression Edwin Swndi Sijbt (23516012) Progrm Studi Mgister Informtik Sekolh Teknik Elektro dn Informtik ITB Jl. Gnesh 10 Bndung 40132, Indonesi 23516012@std.stei.itb.c.id

More information

State space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies

State space systems analysis (continued) Stability. A. Definitions A system is said to be Asymptotically Stable (AS) when it satisfies Stte spce systems nlysis (continued) Stbility A. Definitions A system is sid to be Asymptoticlly Stble (AS) when it stisfies ut () = 0, t > 0 lim xt () 0. t A system is AS if nd only if the impulse response

More information

UNIFORM CONVERGENCE. Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3

UNIFORM CONVERGENCE. Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3 UNIFORM CONVERGENCE Contents 1. Uniform Convergence 1 2. Properties of uniform convergence 3 Suppose f n : Ω R or f n : Ω C is sequence of rel or complex functions, nd f n f s n in some sense. Furthermore,

More information

4.4 Areas, Integrals and Antiderivatives

4.4 Areas, Integrals and Antiderivatives . res, integrls nd ntiderivtives 333. Ares, Integrls nd Antiderivtives This section explores properties of functions defined s res nd exmines some connections mong res, integrls nd ntiderivtives. In order

More information

and that at t = 0 the object is at position 5. Find the position of the object at t = 2.

and that at t = 0 the object is at position 5. Find the position of the object at t = 2. 7.2 The Fundmentl Theorem of Clculus 49 re mny, mny problems tht pper much different on the surfce but tht turn out to be the sme s these problems, in the sense tht when we try to pproimte solutions we

More information

Tests for the Ratio of Two Poisson Rates

Tests for the Ratio of Two Poisson Rates Chpter 437 Tests for the Rtio of Two Poisson Rtes Introduction The Poisson probbility lw gives the probbility distribution of the number of events occurring in specified intervl of time or spce. The Poisson

More information

Math 270A: Numerical Linear Algebra

Math 270A: Numerical Linear Algebra Mth 70A: Numericl Liner Algebr Instructor: Michel Holst Fll Qurter 014 Homework Assignment #3 Due Give to TA t lest few dys before finl if you wnt feedbck. Exercise 3.1. (The Bsic Liner Method for Liner

More information

First midterm topics Second midterm topics End of quarter topics. Math 3B Review. Steve. 18 March 2009

First midterm topics Second midterm topics End of quarter topics. Math 3B Review. Steve. 18 March 2009 Mth 3B Review Steve 18 Mrch 2009 About the finl Fridy Mrch 20, 3pm-6pm, Lkretz 110 No notes, no book, no clcultor Ten questions Five review questions (Chpters 6,7,8) Five new questions (Chpters 9,10) No

More information

Goals: Determine how to calculate the area described by a function. Define the definite integral. Explore the relationship between the definite

Goals: Determine how to calculate the area described by a function. Define the definite integral. Explore the relationship between the definite Unit #8 : The Integrl Gols: Determine how to clculte the re described by function. Define the definite integrl. Eplore the reltionship between the definite integrl nd re. Eplore wys to estimte the definite

More information

New data structures to reduce data size and search time

New data structures to reduce data size and search time New dt structures to reduce dt size nd serch time Tsuneo Kuwbr Deprtment of Informtion Sciences, Fculty of Science, Kngw University, Hirtsuk-shi, Jpn FIT2018 1D-1, No2, pp1-4 Copyright (c)2018 by The Institute

More information

Pre-Session Review. Part 1: Basic Algebra; Linear Functions and Graphs

Pre-Session Review. Part 1: Basic Algebra; Linear Functions and Graphs Pre-Session Review Prt 1: Bsic Algebr; Liner Functions nd Grphs A. Generl Review nd Introduction to Algebr Hierrchy of Arithmetic Opertions Opertions in ny expression re performed in the following order:

More information

Module 6: LINEAR TRANSFORMATIONS

Module 6: LINEAR TRANSFORMATIONS Module 6: LINEAR TRANSFORMATIONS. Trnsformtions nd mtrices Trnsformtions re generliztions of functions. A vector x in some set S n is mpped into m nother vector y T( x). A trnsformtion is liner if, for

More information

Acceptance Sampling by Attributes

Acceptance Sampling by Attributes Introduction Acceptnce Smpling by Attributes Acceptnce smpling is concerned with inspection nd decision mking regrding products. Three spects of smpling re importnt: o Involves rndom smpling of n entire

More information

Calculus of Variations

Calculus of Variations Clculus of Vritions Com S 477/577 Notes) Yn-Bin Ji Dec 4, 2017 1 Introduction A functionl ssigns rel number to ech function or curve) in some clss. One might sy tht functionl is function of nother function

More information

8 Laplace s Method and Local Limit Theorems

8 Laplace s Method and Local Limit Theorems 8 Lplce s Method nd Locl Limit Theorems 8. Fourier Anlysis in Higher DImensions Most of the theorems of Fourier nlysis tht we hve proved hve nturl generliztions to higher dimensions, nd these cn be proved

More information

Construction and Selection of Single Sampling Quick Switching Variables System for given Control Limits Involving Minimum Sum of Risks

Construction and Selection of Single Sampling Quick Switching Variables System for given Control Limits Involving Minimum Sum of Risks Construction nd Selection of Single Smpling Quick Switching Vribles System for given Control Limits Involving Minimum Sum of Risks Dr. D. SENHILKUMAR *1 R. GANESAN B. ESHA RAFFIE 1 Associte Professor,

More information

CBE 291b - Computation And Optimization For Engineers

CBE 291b - Computation And Optimization For Engineers The University of Western Ontrio Fculty of Engineering Science Deprtment of Chemicl nd Biochemicl Engineering CBE 9b - Computtion And Optimiztion For Engineers Mtlb Project Introduction Prof. A. Jutn Jn

More information

Appendix 3, Rises and runs, slopes and sums: tools from calculus

Appendix 3, Rises and runs, slopes and sums: tools from calculus Appendi 3, Rises nd runs, slopes nd sums: tools from clculus Sometimes we will wnt to eplore how quntity chnges s condition is vried. Clculus ws invented to do just this. We certinly do not need the full

More information

Discrete Mathematics and Probability Theory Spring 2013 Anant Sahai Lecture 17

Discrete Mathematics and Probability Theory Spring 2013 Anant Sahai Lecture 17 EECS 70 Discrete Mthemtics nd Proility Theory Spring 2013 Annt Shi Lecture 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion,

More information

Duality # Second iteration for HW problem. Recall our LP example problem we have been working on, in equality form, is given below.

Duality # Second iteration for HW problem. Recall our LP example problem we have been working on, in equality form, is given below. Dulity #. Second itertion for HW problem Recll our LP emple problem we hve been working on, in equlity form, is given below.,,,, 8 m F which, when written in slightly different form, is 8 F Recll tht we

More information

Multivariate problems and matrix algebra

Multivariate problems and matrix algebra University of Ferrr Stefno Bonnini Multivrite problems nd mtrix lgebr Multivrite problems Multivrite sttisticl nlysis dels with dt contining observtions on two or more chrcteristics (vribles) ech mesured

More information

How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique?

How do we solve these things, especially when they get complicated? How do we know when a system has a solution, and when is it unique? XII. LINEAR ALGEBRA: SOLVING SYSTEMS OF EQUATIONS Tody we re going to tlk out solving systems of liner equtions. These re prolems tht give couple of equtions with couple of unknowns, like: 6= x + x 7=

More information

MIXED MODELS (Sections ) I) In the unrestricted model, interactions are treated as in the random effects model:

MIXED MODELS (Sections ) I) In the unrestricted model, interactions are treated as in the random effects model: 1 2 MIXED MODELS (Sections 17.7 17.8) Exmple: Suppose tht in the fiber breking strength exmple, the four mchines used were the only ones of interest, but the interest ws over wide rnge of opertors, nd

More information

A Matrix Algebra Primer

A Matrix Algebra Primer A Mtrix Algebr Primer Mtrices, Vectors nd Sclr Multipliction he mtrix, D, represents dt orgnized into rows nd columns where the rows represent one vrible, e.g. time, nd the columns represent second vrible,

More information

Math 113 Fall Final Exam Review. 2. Applications of Integration Chapter 6 including sections and section 6.8

Math 113 Fall Final Exam Review. 2. Applications of Integration Chapter 6 including sections and section 6.8 Mth 3 Fll 0 The scope of the finl exm will include: Finl Exm Review. Integrls Chpter 5 including sections 5. 5.7, 5.0. Applictions of Integrtion Chpter 6 including sections 6. 6.5 nd section 6.8 3. Infinite

More information

THERMAL EXPANSION COEFFICIENT OF WATER FOR VOLUMETRIC CALIBRATION

THERMAL EXPANSION COEFFICIENT OF WATER FOR VOLUMETRIC CALIBRATION XX IMEKO World Congress Metrology for Green Growth September 9,, Busn, Republic of Kore THERMAL EXPANSION COEFFICIENT OF WATER FOR OLUMETRIC CALIBRATION Nieves Medin Hed of Mss Division, CEM, Spin, mnmedin@mityc.es

More information

Testing categorized bivariate normality with two-stage. polychoric correlation estimates

Testing categorized bivariate normality with two-stage. polychoric correlation estimates Testing ctegorized bivrite normlity with two-stge polychoric correltion estimtes Albert Mydeu-Olivres Dept. of Psychology University of Brcelon Address correspondence to: Albert Mydeu-Olivres. Fculty of

More information

Classical Mechanics. From Molecular to Con/nuum Physics I WS 11/12 Emiliano Ippoli/ October, 2011

Classical Mechanics. From Molecular to Con/nuum Physics I WS 11/12 Emiliano Ippoli/ October, 2011 Clssicl Mechnics From Moleculr to Con/nuum Physics I WS 11/12 Emilino Ippoli/ October, 2011 Wednesdy, October 12, 2011 Review Mthemtics... Physics Bsic thermodynmics Temperture, idel gs, kinetic gs theory,

More information

19 Optimal behavior: Game theory

19 Optimal behavior: Game theory Intro. to Artificil Intelligence: Dle Schuurmns, Relu Ptrscu 1 19 Optiml behvior: Gme theory Adversril stte dynmics hve to ccount for worst cse Compute policy π : S A tht mximizes minimum rewrd Let S (,

More information

Entropy and Ergodic Theory Notes 10: Large Deviations I

Entropy and Ergodic Theory Notes 10: Large Deviations I Entropy nd Ergodic Theory Notes 10: Lrge Devitions I 1 A chnge of convention This is our first lecture on pplictions of entropy in probbility theory. In probbility theory, the convention is tht ll logrithms

More information

Numerical Integration

Numerical Integration Chpter 5 Numericl Integrtion Numericl integrtion is the study of how the numericl vlue of n integrl cn be found. Methods of function pproximtion discussed in Chpter??, i.e., function pproximtion vi the

More information

THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS.

THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS. THE EXISTENCE-UNIQUENESS THEOREM FOR FIRST-ORDER DIFFERENTIAL EQUATIONS RADON ROSBOROUGH https://intuitiveexplntionscom/picrd-lindelof-theorem/ This document is proof of the existence-uniqueness theorem

More information

MAA 4212 Improper Integrals

MAA 4212 Improper Integrals Notes by Dvid Groisser, Copyright c 1995; revised 2002, 2009, 2014 MAA 4212 Improper Integrls The Riemnn integrl, while perfectly well-defined, is too restrictive for mny purposes; there re functions which

More information

Math Lecture 23

Math Lecture 23 Mth 8 - Lecture 3 Dyln Zwick Fll 3 In our lst lecture we delt with solutions to the system: x = Ax where A is n n n mtrix with n distinct eigenvlues. As promised, tody we will del with the question of

More information

MAC-solutions of the nonexistent solutions of mathematical physics

MAC-solutions of the nonexistent solutions of mathematical physics Proceedings of the 4th WSEAS Interntionl Conference on Finite Differences - Finite Elements - Finite Volumes - Boundry Elements MAC-solutions of the nonexistent solutions of mthemticl physics IGO NEYGEBAUE

More information

Introduction to the Calculus of Variations

Introduction to the Calculus of Variations Introduction to the Clculus of Vritions Jim Fischer Mrch 20, 1999 Abstrct This is self-contined pper which introduces fundmentl problem in the clculus of vritions, the problem of finding extreme vlues

More information

Lecture 1. Functional series. Pointwise and uniform convergence.

Lecture 1. Functional series. Pointwise and uniform convergence. 1 Introduction. Lecture 1. Functionl series. Pointwise nd uniform convergence. In this course we study mongst other things Fourier series. The Fourier series for periodic function f(x) with period 2π is

More information

2.4 Linear Inequalities and Problem Solving

2.4 Linear Inequalities and Problem Solving Section.4 Liner Inequlities nd Problem Solving 77.4 Liner Inequlities nd Problem Solving S 1 Use Intervl Nottion. Solve Liner Inequlities Using the Addition Property of Inequlity. 3 Solve Liner Inequlities

More information

Section 11.5 Estimation of difference of two proportions

Section 11.5 Estimation of difference of two proportions ection.5 Estimtion of difference of two proportions As seen in estimtion of difference of two mens for nonnorml popultion bsed on lrge smple sizes, one cn use CLT in the pproximtion of the distribution

More information

Riemann Sums and Riemann Integrals

Riemann Sums and Riemann Integrals Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 203 Outline Riemnn Sums Riemnn Integrls Properties Abstrct

More information

221A Lecture Notes WKB Method

221A Lecture Notes WKB Method A Lecture Notes WKB Method Hmilton Jcobi Eqution We strt from the Schrödinger eqution for single prticle in potentil i h t ψ x, t = [ ] h m + V x ψ x, t. We cn rewrite this eqution by using ψ x, t = e

More information

How can we approximate the area of a region in the plane? What is an interpretation of the area under the graph of a velocity function?

How can we approximate the area of a region in the plane? What is an interpretation of the area under the graph of a velocity function? Mth 125 Summry Here re some thoughts I ws hving while considering wht to put on the first midterm. The core of your studying should be the ssigned homework problems: mke sure you relly understnd those

More information

Orthogonal Polynomials and Least-Squares Approximations to Functions

Orthogonal Polynomials and Least-Squares Approximations to Functions Chpter Orthogonl Polynomils nd Lest-Squres Approximtions to Functions **4/5/3 ET. Discrete Lest-Squres Approximtions Given set of dt points (x,y ), (x,y ),..., (x m,y m ), norml nd useful prctice in mny

More information

Abstract inner product spaces

Abstract inner product spaces WEEK 4 Abstrct inner product spces Definition An inner product spce is vector spce V over the rel field R equipped with rule for multiplying vectors, such tht the product of two vectors is sclr, nd the

More information

Estimation of Binomial Distribution in the Light of Future Data

Estimation of Binomial Distribution in the Light of Future Data British Journl of Mthemtics & Computer Science 102: 1-7, 2015, Article no.bjmcs.19191 ISSN: 2231-0851 SCIENCEDOMAIN interntionl www.sciencedomin.org Estimtion of Binomil Distribution in the Light of Future

More information

Chapter 9: Inferences based on Two samples: Confidence intervals and tests of hypotheses

Chapter 9: Inferences based on Two samples: Confidence intervals and tests of hypotheses Chpter 9: Inferences bsed on Two smples: Confidence intervls nd tests of hypotheses 9.1 The trget prmeter : difference between two popultion mens : difference between two popultion proportions : rtio of

More information

Riemann Sums and Riemann Integrals

Riemann Sums and Riemann Integrals Riemnn Sums nd Riemnn Integrls Jmes K. Peterson Deprtment of Biologicl Sciences nd Deprtment of Mthemticl Sciences Clemson University August 26, 2013 Outline 1 Riemnn Sums 2 Riemnn Integrls 3 Properties

More information

Review of basic calculus

Review of basic calculus Review of bsic clculus This brief review reclls some of the most importnt concepts, definitions, nd theorems from bsic clculus. It is not intended to tech bsic clculus from scrtch. If ny of the items below

More information

The Algebra (al-jabr) of Matrices

The Algebra (al-jabr) of Matrices Section : Mtri lgebr nd Clculus Wshkewicz College of Engineering he lgebr (l-jbr) of Mtrices lgebr s brnch of mthemtics is much broder thn elementry lgebr ll of us studied in our high school dys. In sense

More information

Probability Distributions for Gradient Directions in Uncertain 3D Scalar Fields

Probability Distributions for Gradient Directions in Uncertain 3D Scalar Fields Technicl Report 7.8. Technische Universität München Probbility Distributions for Grdient Directions in Uncertin 3D Sclr Fields Tobis Pfffelmoser, Mihel Mihi, nd Rüdiger Westermnn Computer Grphics nd Visuliztion

More information

Discrete Mathematics and Probability Theory Summer 2014 James Cook Note 17

Discrete Mathematics and Probability Theory Summer 2014 James Cook Note 17 CS 70 Discrete Mthemtics nd Proility Theory Summer 2014 Jmes Cook Note 17 I.I.D. Rndom Vriles Estimting the is of coin Question: We wnt to estimte the proportion p of Democrts in the US popultion, y tking

More information