Single Neuron. Hung-yi Lee

Size: px

Start display at page:

Download "Single Neuron. Hung-yi Lee"

Erica Tucker
5 years ago
Views:

1 Singl Nuron Hung-yi L

2 Singl Nuron 2 w w 2 Activation function a N w N ias

3 Larning to say ys/no Binary Classification

4 Larning to say ys/no Sam filtring Is an -mail sam or not? Rcommndation systms rcommnd th roduct to th customr or not? Malwar dtction Is th softwar malicious or not? Stock rdiction Will th futur valu of a stock incras or not with rsct to its currnt valu? Binary Classification

5 Eaml Alication: Sam filtring f : X Y { ys, no} Sam Not sam 2 f ys 2 (htt://sam-filtr-rviw.totnrviws.com/) f no

6 Eaml Alication: Sam filtring f : X Y { ys, no} What dos th function f look lik? y f ys no P P ys ys How to stimat P(ys )?

7 Eaml Alication: Sam filtring To stimat P(ys ), collct amls first.. Earn fr fr Ys (Sam) Som words frquntly aar in th sam.g., fr 2 Win fr Ys (Sam) Us th frquncy of fr to dcid if an -mail is sam 3. Talk Mting. No (Not Sam) Estimat P(ys fr = k) fr is th numr of fr in -mail

8 Rgrssion In training data, thr is no - mail containing 3 fr. (ys fr ) (ys fr = ) = 0.4 (ys fr = 0 ) = 0. Frquncy of Fr ( fr ) in an -mail Prolm: What if on day you rciv an -mail with 3 fr.

9 Rgrssion f( fr ) = w fr + (f( fr ) is an stimat of (ys fr ) ) Stor w and (ys fr ) Rgrssion Frquncy of Fr ( fr ) in an -mail

10 Rgrssion f( fr ) = w fr + Th outut of f is not twn 0 and (ys fr ) Rgrssion Frquncy of Fr ( fr ) in an -mail Prolm: What if on day you rciv an -mail with 6 fr.

11 Logit ln fr vrtical lin: Proaility to sam (ys fr ) () is always twn 0 and vrtical lin: logit() logit ln

12 Logit f ( fr ) = w fr + (f ( fr ) is an stimat of logit() ) fr fr vrtical lin: Proaility to sam (ys fr ) () is always twn 0 and vrtical lin: logit() logit ln

13 Logit Stor w and fr f 3 fr w 3.5 logit ln > 0.5, so ys f ( fr ) = w fr + (f ( fr ) is an stimat of logit() ) f fr w fr ln ys 0 (rctron) vrtical lin: logit() logit fr ln

14 Multil Varials Considr two words fr and hllo comut (ys fr, hllo ) () logit ln hllo fr

15 Multil Varials Considr two words fr and hllo comut (ys fr, hllo ) () logit ln f fr, hllo Rgrssion w fr w2 hllo hllo fr

16 Multil Varials Of cours, w can considr all words {t, t 2, t N } in a dictionary w w w f N N t N t t t t t 2 2 2, w w N w w w 2 t N t t 2 is to aroimat logit() t N t t ys 2, P :

17 Logistic Rgrssion w aroimat logit ln : P ys If th roaility = or 0, ln(/-) = +infinity or infinity Can not do rgrssion t, t 2 t N Th roaility to sam is always or 0. t aars 3 tims t 2 aars 0 tim t N aars tim P ys t t 2 t N 3 0

18 Logistic Rgrssion w w w ln Sigmoid Function

19 Logistic Rgrssion Ys (Sam) t N t t w clos to w No (not Sam) 2 2 w 2 clos to 0

20 Logistic Rgrssion Ys No t t2 t N fatur w w 2 w N w ias w 0

21 Logistic Rgrssion Ys No t t2 t N w w 2 w N w w 0

22 Rfrnc for Linar Classifir htt://grgor.chruala.m/ars/ml4nl/linarclassifirs.df

23 Mor than saying ys/no Multiclass Classification

24 Mor than saying ys/no Handwriting digit classification This is Multiclass Classification

25 Mor than saying ys/no Handwriting digit classification Simlify th qustion: whthr an imag is 2 or not Dscri th charactristics of inut ojct 2 Each il corrsonds to on dimnsion in N th fatur fatur of an imag

26 Mor than saying ys/no Handwriting digit classification Simlify th qustion: whthr an imag is 2 or not 2 2 or not N

27 Mor than saying ys/no Handwriting digit classification Binary classification of, 2, 3 If y 2 is th ma, thn th imag is 2. y or not 2 y 2 2 or not N y 3 3 or not

28 Limitation of singl nuron Motivation of cascading nurons

29 Limitation of Logistic Rgrssion w w 2 2 a ys no a a ys no w w Inut Outut No 0 Ys 0 Ys No Ys No

30 Limitation of Logistic Rgrssion w w 2 2 Inut Outut No 0 Ys 0 Ys a 0 0 XNOR gat NOT AND AND 0 0 OR No

31 Nural Ntwork NOT XNOR gat AND Nural Ntwork AND OR a 2 2 a 2 a Hiddn Nurons

32 a =0.73 a =0.27 a 2 a =0.27 a = a 2 =0.05 a 2 = a 2 2 a 2 =0.27 a 2 =0.73

33 2 a =0.73 a =0.27 a w a a =0.27 a =0.05 a 2 w 2 a 2 =0.05 a 2 =0.27 (0.73, 0.05) 2 a 2 a 2 =0.27 a 2 =0.73 (0.27, 0.27) (0.05,0.73) a

34 Thank you for your listning!

35 Andi

36 Mor rfrnc htt:// /2_GD_REG_ton_NN/lctur_nots/logistic_rgr ssion_loss_function/logistic_rgrssion_loss.df htt://mathgotchas.logsot.tw/20/0/why-isrror-function-minimid-in.html htts://cs.nyu.du/~yann/talks/lcun nonconv.df htt:// lms.df

37 Logistic Rgrssion t N t t 2? P ys w ln w

First Lecture of Machine Learning. Hung-yi Lee

First Lecture of Machine Learning. Hung-yi Lee Firs Lcur of Machin Larning Hung-yi L Larning o say ys/no Binary Classificaion Larning o say ys/no Sam filring Is an -mail sam or no? Rcommndaion sysms rcommnd h roduc o h cusomr or no? Malwar dcion Is