Singl Nuron Hung-yi L
Singl Nuron 2 w w 2 Activation function a N w N ias
Larning to say ys/no Binary Classification
Larning to say ys/no Sam filtring Is an -mail sam or not? Rcommndation systms rcommnd th roduct to th customr or not? Malwar dtction Is th softwar malicious or not? Stock rdiction Will th futur valu of a stock incras or not with rsct to its currnt valu? Binary Classification
Eaml Alication: Sam filtring f : X Y { ys, no} E-mail Sam Not sam 2 f ys 2 (htt://sam-filtr-rviw.totnrviws.com/) f no
Eaml Alication: Sam filtring f : X Y { ys, no} What dos th function f look lik? y f ys no P P ys ys 0.5 0.5 How to stimat P(ys )?
Eaml Alication: Sam filtring To stimat P(ys ), collct amls first.. Earn fr fr Ys (Sam) Som words frquntly aar in th sam.g., fr 2 Win fr Ys (Sam) Us th frquncy of fr to dcid if an -mail is sam 3. Talk Mting. No (Not Sam) Estimat P(ys fr = k) fr is th numr of fr in -mail
Rgrssion In training data, thr is no - mail containing 3 fr. (ys fr ) (ys fr = ) = 0.4 (ys fr = 0 ) = 0. Frquncy of Fr ( fr ) in an -mail Prolm: What if on day you rciv an -mail with 3 fr.
Rgrssion f( fr ) = w fr + (f( fr ) is an stimat of (ys fr ) ) Stor w and (ys fr ) Rgrssion Frquncy of Fr ( fr ) in an -mail
Rgrssion f( fr ) = w fr + Th outut of f is not twn 0 and (ys fr ) Rgrssion Frquncy of Fr ( fr ) in an -mail Prolm: What if on day you rciv an -mail with 6 fr.
Logit ln fr vrtical lin: Proaility to sam (ys fr ) () is always twn 0 and vrtical lin: logit() logit ln
Logit f ( fr ) = w fr + (f ( fr ) is an stimat of logit() ) fr fr vrtical lin: Proaility to sam (ys fr ) () is always twn 0 and vrtical lin: logit() logit ln
Logit Stor w and fr f 3 fr w 3.5 logit ln.5 0.87 > 0.5, so ys f ( fr ) = w fr + (f ( fr ) is an stimat of logit() ) f fr w fr ln 0 0.5 ys 0 (rctron) vrtical lin: logit() logit fr ln
Multil Varials Considr two words fr and hllo comut (ys fr, hllo ) () logit ln hllo fr
Multil Varials Considr two words fr and hllo comut (ys fr, hllo ) () logit ln f fr, hllo Rgrssion w fr w2 hllo hllo fr
Multil Varials Of cours, w can considr all words {t, t 2, t N } in a dictionary w w w f N N t N t t t t t 2 2 2, w w N w w w 2 t N t t 2 is to aroimat logit() t N t t ys 2, P :
Logistic Rgrssion w aroimat logit ln : P ys If th roaility = or 0, ln(/-) = +infinity or infinity Can not do rgrssion t, t 2 t N Th roaility to sam is always or 0. t aars 3 tims t 2 aars 0 tim t N aars tim P ys t t 2 t N 3 0
Logistic Rgrssion w w w ln Sigmoid Function
Logistic Rgrssion Ys (Sam) 7 0 3 2 t N t t w clos to w No (not Sam) 2 2 w 2 clos to 0
Logistic Rgrssion Ys No t t2 t N fatur w w 2 w N w ias w 0
Logistic Rgrssion Ys No t t2 t N w w 2 w N w w 0
Rfrnc for Linar Classifir htt://grgor.chruala.m/ars/ml4nl/linarclassifirs.df
Mor than saying ys/no Multiclass Classification
Mor than saying ys/no Handwriting digit classification This is Multiclass Classification
Mor than saying ys/no Handwriting digit classification Simlify th qustion: whthr an imag is 2 or not Dscri th charactristics of inut ojct 2 Each il corrsonds to on dimnsion in N th fatur fatur of an imag
Mor than saying ys/no Handwriting digit classification Simlify th qustion: whthr an imag is 2 or not 2 2 or not N
Mor than saying ys/no Handwriting digit classification Binary classification of, 2, 3 If y 2 is th ma, thn th imag is 2. y or not 2 y 2 2 or not N y 3 3 or not
Limitation of singl nuron Motivation of cascading nurons
Limitation of Logistic Rgrssion w w 2 2 a ys no a a 0.5 0.5 ys no w w2 2 0 0 Inut Outut 2 0 0 No 0 Ys 0 Ys No Ys No
Limitation of Logistic Rgrssion w w 2 2 Inut Outut 2 0 0 No 0 Ys 0 Ys a 0 0 XNOR gat NOT 0 0 0 0 AND AND 0 0 OR No
Nural Ntwork NOT XNOR gat AND Nural Ntwork AND OR a 2 2 a 2 a Hiddn Nurons
a =0.73 a =0.27 a 2 a =0.27 a =0.05 2 a 2 =0.05 a 2 =0.27 2 a 2 2 a 2 =0.27 a 2 =0.73
2 a =0.73 a =0.27 a w a a =0.27 a =0.05 a 2 w 2 a 2 =0.05 a 2 =0.27 (0.73, 0.05) 2 a 2 a 2 =0.27 a 2 =0.73 (0.27, 0.27) (0.05,0.73) a
Thank you for your listning!
Andi
Mor rfrnc htt://www.ccs.nu.du/hom/vi/tach/mlcours /2_GD_REG_ton_NN/lctur_nots/logistic_rgr ssion_loss_function/logistic_rgrssion_loss.df htt://mathgotchas.logsot.tw/20/0/why-isrror-function-minimid-in.html htts://cs.nyu.du/~yann/talks/lcun-2007207- nonconv.df htt://www.cs.columia.du/~li/fogm/lcturs/g lms.df
Logistic Rgrssion t N t t 2? P ys w ln w