Density estimation III. Linear regression.

Transcription

1 Lecure 6 Mlos Hauskrec mlos@cs.p.eu 539 Seo Square Des esmao III. Lear regresso. Daa: Des esmao D { D D.. D} D a vecor of arbue values Obecve: r o esmae e uerlg rue probabl srbuo over varables X px usg eamples D rue srbuo samples p X D D D.. D } { esmae pˆ X Saar assumpos: Samples are epee of eac oer come from e same ecal srbuo fe px

2 Epoeal faml Epoeal faml: all probabl mass / es fucos a ca be re e oeal ormal form f [ ] Z a vecor of aural or caocal parameers a fuco referre o as a suffce sasc a fuco of s less mpora Z a ormalzao cosa a paro fuco Z { } Oer commo form: [ A ] f log Z A Epoeal faml: eamples Beroull srbuo p π π π π log log π π π { log π } log π Epoeal faml f [ ] Z Parameers?? Z??

3 Epoeal faml: eamples Beroull srbuo p π π π π log log π π π { log π } log π Epoeal faml f [ ] Z Parameers π log oe π π e Z e π Epoeal faml: eamples Uvarae Gaussa srbuo p µ [ µ ] π µ µ log π Epoeal faml f Z Parameers?? Z?? [ ]

4 Epoeal faml: eamples Uvarae Gaussa srbuo Epoeal faml Parameers log µ µ π / / µ π / log 4 log µ Z [ ] Z f ] [ µ π µ p Epoeal faml For samples e lkeloo of aa s Impora: e mesoal of e suffce sasc remas e same e umber of samples [ ] A p D P A A

5 Epoeal faml e log lkeloo of aa s Opmzg e loglkeloo For e ML esmae mus ol log A D l log A A D l A Epoeal faml Rerg e grae:

6 Rerg e grae: Epoeal faml A log Z log { } A { } { A } A A E Resul: E For e ML esmae e parameers soul be ause suc a e ecao of e sasc s equal o e observe sample sascs { } Momes of e srbuo For e oeal faml e k- mome of e sasc correspos o e k- ervave of A If s a compoe of e e ge e momes of e srbuo b ffereag s correspog aural parameer Eample: Beroull π p π log log π π A log log e π Dervaves: A e log e π e e A π π e

7 Oule Lear Regresso Lear moel Error fuco base o e leas squares f Parameer esmao. Supervse learg Daa: D { D D.. D} a se of eamples D < > L s a pu vecor of sze s e esre oupu gve b a eacer Obecve: lear e mappg f : X Y s.. f for all.. Regresso: Y s couous Eample: eargs prouc orers compa sock prce Classfcao: Y s scree Eample: are g bar form g label

8 Lear regresso Fuco f : X Y s a lear combao of pu compoes f k - parameers egs Bas erm f Ipu vecor Lear regresso Sorer vecor efo of e moel Iclue bas cosa e pu vecor L f k - parameers egs Ipu vecor f

9 Lear regresso. Error. Daa: D < > Fuco: f We oul lke o ave f for all.. Error fuco measures o muc our precos evae from e esre asers Mea-square error.. Learg: We a o f e egs mmzg e error! f Lear regresso. Eample mesoal pu

10 Lear regresso. Eample. mesoal pu Lear regresso. Opmzao. We a e egs mmzg e error f.. For e opmal se of parameers ervaves of e error respec o eac parameer mus be Vecor of ervaves:.. gra

11 Lear regresso. Opmzao. efes a se of equaos gra Solvg lear regresso B rearragg e erms e ge a ssem of lear equaos ukos A b

12 Solvg lear regresso e opmal se of egs sasfes: Leas o a ssem of lear equaos SLE ukos of e form Soluo o SLE:? A b Solvg lear regresso e opmal se of egs sasfes: Leas o a ssem of lear equaos SLE ukos of e form Soluo o SLE: mar verso A b b A