CISE 3: umercl Methods Lecture 5 Topc 4 Lest Squres Curve Fttng Dr. Amr Khouh Term Red Chpter 7 of the tetoo c Khouh CISE3_Topc4_Lest Squre
Motvton Gven set of epermentl dt 3 5. 5.9 6.3 The reltonshp etween nd m not e cler we wnt to fnd n epresson for f 3 CISE3_Topc4_Le st Squre c Khouh
Motvton Model uldng : In engneerng two tpes of pplctons re encountered: Trend nlss. Predctng vlues of dependent vrle m nclude etrpolton eond dt ponts or nterpolton etween dt ponts. Hpothess testng. Comprng estng mthemtcl model wth mesured dt. Wht s the est mthemtcl model functon f tht represents the dtset? Wht s the est crteron to ssess the fttng of the functon to the dt? 3 c Khouh CISE3_Topc4_Lest Squre
Motvton Curve Fttng : Gven set of tulted dt fnd curve or functon tht est represents the dt. Gven:.The tulted dt.the form of the functon 3.The curve fttng crter Fnd the unnown coeffcents 4 c Khouh CISE3_Topc4_Lest Squre
Crter for Best Ft Lner model f o Mnmze the sum of the resdul errors for ll vlle dt: So s the sum of the solute vlues n n e e n n o Dtset of pts n lne pssng throu md pt fts? Dtset of 4 pts An lne Fllng n dshed lnes fts? Mnmum of the mmum error for n Indvdul pont. Ill-suted for regresson As t gves undue nfluence to n outler CISE3_Topc4_Le st Squre c Khouh 5
Lest Squres Regresson Lner Regresson Fttng strght lne to set of pred oservtons: n n. = + +e - slope - ntercept e- error or resdul etween the model nd the oservtons 6 c Khouh CISE3_Topc4_Lest Squre
Selecton of the functons CISE3_Topc4_Lest Squre c Khouh 7 re nown g g f Generl f Polnoml c f Qudrtc f Lner m n
Decde on the crteron. Lest Squres mn.ect n M tchngnterplton f f f f Chpter 7 Chpter 8 8 c Khouh CISE3_Topc4_Lest Squre
Lest Squres Gven. n. n The form of the functon s ssumed to e nown ut the coeffcents re unnown f e The dfference s ssumed to e the result of epermentl error 9 c Khouh CISE3_Topc4_Lest Squre
Determne the Unnowns We wnt tofnd to mnmze n f Howdo we otn nd tomnmze? c Khouh CISE3_Topc4_Lest Squre
Determne the Unnowns ecessr condton for the mnmum c Khouh CISE3_Topc4_Lest Squre
Emple Assume f 3 5. 5.9 6.3 ecessr condton for the mnmum CISE3_Topc4_Le st Squre c Khouh
Rememer CISE3_Topc4_Lest Squre c Khouh 3 n n n n g g d d
Emple orml Equtons CISE3_Topc4_Le st Squre c Khouh 4
Emple the orml Equtons gves Solvng CISE3_Topc4_Le st Squre c Khouh 5
Emple 3 sum 3 6 5. 5.9 6.3 7.3 4 9 4 5..8 8.9 35.8 6 c Khouh CISE3_Topc4_Lest Squre
Emple.6 4.5667 35.8 4 6 7.3 6 3 orml Equtons Solvng CISE3_Topc4_Le st Squre c Khouh 7
Fttng wth onlner Functons Emple.4.65.95.4.73..3.5.3 -.3 -. -.45.7. -.9.4 It f s requred tofnd functon of ln cos c e ths form to ft thedt. 8 c Khouh CISE3_Topc4_Lest Squre
Emple e f CISE3_Topc4_Le st Squre c Khouh 9 n ce cos ln e c f cos ln mnmum condton for the ecessr Equtons orml c c c c
Emple equtons Evlute thesums nd solve the norml cos ln cos cos cos cos ln ln ln cos ln ln 8 8 8 8 8 8 8 8 8 8 8 8 e e c e e e c e c CISE3_Topc4_Le st Squre c Khouh
How do ou judge performnce? Gven twoor more functons to ft thedt How do ou select the est? Answer : Determne the prmetersfor ech functon then compute for ech one.thefuncton resultngn smller s the est n the lest squre sense. c Khouh CISE3_Topc4_Lest Squre
Multple Regresson Emple: Gven the followng dt t 3..4.. ft 3 It s requred to determne functon of two vrles ft = + + c t to epln the dt tht s est n the lest squre sense. CISE3_Topc4_Le st Squre c Khouh
Soluton of Multple Regresson Construct the sum of the squre of the error nd derve the necessr condtons equtng the prtl dervtves wth respect to the unnown prmeters to zero then solve the equtons. t 3..4.. ft 3 CISE3_Topc4_Le st Squre c Khouh 3
Soluton of Multple Regresson condtons ecessr 4 4 4 4 t f ct c c f ct c f ct c f ct c ct t f CISE3_Topc4_Le st Squre c Khouh 4
5 Generl Lner Lest Squres z z Y Z Y A E Z A E mtr of t themesured vlues of oserved vlued of z z m z unnown coeffcents resduls c Khouh n m... m j z e re m ss functons theclculted vlues of m z m thendependent vrle thedependent vrle j z j the ss functons Mnmzed tng ts prtl dervtve w.r.t. ech of the coeffcents nd settng the resultng equton equl to zero CISE3_Topc4_Lest Squre
SE3: umercl Methods Lecture 3: onlner lest squres prolems + More Emples of nonlner lest squres Soluton of nconsstent equtons Contnuous lest squre prolems 6 c Khouh CISE3_Topc4_Lest Squre
Outlnes Emples of nonlner lest squres Soluton of nconsstent equtons Contnuous lest squre prolems CISE3_Topc4_Le st Squre c Khouh 7
onlner Prolem fnd Gven fucton of 3 orml CISE3_Topc4_Le st Squre e the form e Equtons 3 3 3.4 5. 9 e e e re e tht otned usng est ft thedt. c Khouh 8
Alterntve Soluton Lnerzton Method Gven 3.4 5. 9 fnd fucton of the form e tht est ft thedt. Defne z ln Let ln nd Instedof usng We wll use ln z ln 3 3 e z eser tosolve CISE3_Topc4_Le st Squre c Khouh 9
Emples Lnerzton Method Gven 3.3..4 fnd fucton of f Defne z theform / 3 tht est ft thedt. Let z CISE3_Topc4_Le st Squre We wll use 3 z c Khouh 3
Inconsstent Sstem of Equtons Prolem: Solve thefollowng sstemof equtons 3 Ths s 4. osoluton 4 6 nconsstent sstemof Redundnt Sstems Over-determned: m>n Equtons CISE3_Topc4_Le st Squre c Khouh 3
Inconsstent Sstem of Equtons Resons Inconsstent equtons m occur ecuse of errors n formultng the prolem errors n collectng the dt or computtonl errors. Soluton f ll lnes ntersect t one pont CISE3_Topc4_Le st Squre c Khouh 3
to mnmze the lest squres error nd Fnd 6 4 4. 3 We cn vew theequtons s 3 CISE3_Topc4_Le st Squre c Khouh 33 Inconsstent Sstem of Equtons Formulton s lest squres prolem
Soluton CISE3_Topc4_Lest Squre c Khouh 34 49.6 36.6 8 4 6 33.6 8 8 4.6 8 4 4. 8.3 6 4 4 4 9 36.6 8 6 4 8 4.6 8 4 8 8 4. 63 6 4 4 to mnmze nd Fnd 4. 3 6 4
Soluton ormlequtons: 8 9 36.6 49.6 Soluton :.48.9799 36.6 35 c Khouh CISE3_Topc4_Lest Squre
HW Chec WeCT for HW prolems nd due dte 36 c Khouh CISE3_Topc4_Lest Squre