1 Widrow-Hoff Algorithm

Transcription

1 COS 511: heoreical Machine Learning Lecurer: Rob Schapire Lecure # 18 Scribe: Shaoqing Yang April 10, Widrow-Hoff Algorih Firs le s review he Widrow-Hoff algorih ha was covered fro las lecure: Algorih 1: Widrow-Hoff Algorih Iniialize paraeer > 0, w 1 = 0 for = 1... ge x R n predic ŷ = w x R observe y R updae w +1 = w (w x y ) x And we define he loss funcions as L A = (ŷ y ). And L u = (u x y ). Wha we wan is L A in L u + sall u here are goals in choosing he updae funcion o be w +1 = w (w x y ) x : (1) Wan loss of w +1 on x, y o be sall. his eans we wan o iniize (w +1 x y ) () Wan w +1 close o w so ha we do no forge everyhing we learn so far. And his eans we wan o iniize w +1 w. herefore o su up, we wan o iniize (w +1 x y ) + w +1 w If we ake he derivaive of he above equaion and se i o zero, we have w +1 = w (w +1 x y ) x Insead of solving w +1, we approxiae he er w +1 inside he parenhesis and change i o w. he reason we can do his is because w +1 does no change uch fro w. herefore we have w +1 = w (w x y ) x which is he updae funcion saed in he algorih. Now le s sae a heore: heore 1.1 If we assue on every round, x 1, hen: L W H in u R n[ L u 1 + u ]

2 Fro his heore, we have u: If we divide on boh side, we have: L W H 1 1 L u + u L W H 1 1 Lu + u he er u goes o 0 when ges large. And we can choose sall enough o ake 1 1 o be close o 1. herefore we have he rae ha he algorih is suffering loss is close o rae ha L u is suffering loss. Now le s prove he heore: Proof: Pick any u R n. Firs le s define soe ers: Φ = w u (easure of progess) l = w x y = ŷ y (noice l is he loss of WH on round ) g = u x y (g is he loss of u on round ) = (w x y ) x = l x w +1 = w Our ain clai is ha he change of poenial is: Φ +1 Φ l + 1 g (1) his shows ha l ends o drive poenial down while g ends o drive poenial up. Now assue (1) holds. Noice ha oal change in poenial should be non-negaive. And also we iniialize w 1 = 0. So we have he following inequaliy: u = Φ 1 Φ +1 Φ 1 Now we solve for L W H, we ge = Φ +1 Φ + Φ Φ Φ Φ 1 = (Φ +1 Φ ) [ l + 1 g ] = l + 1 = L W H + 1 L u g L W H 1 1 L u + u

3 And since his inequaliy holds for all u, we have: which is he heore. Now le s go back o prove (1): Φ +1 Φ = w +1 u w u = w u w u L W H in [ 1 u R 1 L u + u ] = (w u) + w u w u = (w u) = (w u) (dropping subscrip since i doesn affec he proof) = l x lx (w u) = l x l(w x u x y + y) = l x l[(w x y) (u x y)] = l x l[l g] = l x l + lg l l + lg ( x 1) ( )l + g [ 1 + l (1 )] g = ( )l + [ 1 + l (1 )] = l + 1 g Failies of Online Algorih (ab a +b ) he wo goals of he learning algorih are iniizing he loss of w +1 on x and y, and iniizing he disance beween w +1 and w. So o generalize, we are rying o iniize L(w +1, x, y ) + d(w +1, w ) So if we use he Euclidean nor as our disance easureen, hen he above funcion becoes: L(w +1, x, y ) + w w +1 So if we ry o opiize he above funcion, we have he updae equaion: w +1 = w w L(w +1, x, y ) w w L(w, x, y ) Noice ha we use w o approxiae w +1 when we calculae w +1. his is called he Gradien Descen Algorih. 3

4 Alernaively, we can use relaive enropy as a easure of disance. So d(w, w +1 ) = RE(w w +1 ). Now we can have he updae funcion as w +1,i = w,i exp( L(w +1,x,y )) w i ) Z his is called he Exponeniaed Gradien Algorih, or EG algorih. We need o change he nor: x 1 and u 1 = 1. I s also possible o prove a bound on his updae equaion, bu we skip i in his class. 3 Online Algorih in a Bach Seing We can odify he online algorihs slighly so ha we can use he in he bach learning seings. Le s ake a look a one exaple in a linear regression seing. In a linear regression seing, raining and es saples are drawn i.i.d fro a fixed disribuion D. So we have S = (x 1, y 1 )... (x, y ) where (x i, y i ) D. Our goal is o find v wih low risk, where risk is defined o be R v = E (x,y) D [(v x y) ] We wan o find v such ha R v is sall copared o in u R u. Now we can apply WH algorih o he daa as follows: (1) run WH on (x 1, y 1 ),..., (x, y ), and calculae w 1, w,..., w. () Cobine he vecors: v = 1 and oupu v. We choose o oupu he average of all he w s because we can prove soehing heoreically good abou i, which is no necessarily he case for he las vecor w. Now le s sae anoher heore: heore 3.1 w E S [R v ] in u R n[ R u 1 + u ] If we divide on boh side of he equaion above and if is chosen o be sall, we can see ha Rv Ru will be close o when is large. Proof: here are hree observaions needed in he proof: (1): Le x, y be a rando es exaple fro D. hen we have (v x y) 1 (w x y) 4

5 Proof for (1): (v x y) = [( 1 w ) x y] = [( 1 w x) y] = [ 1 (w x y)] 1 (w x y) (convexiy of f(x) = x ) (): E[(u x y ) ] = E[(u x y) ] he above expecaion is wih respec o he rando choice of (x 1, y 1 ),..., (x, y ) and (x, y). his is because (x, y )and (x, y) are fro he sae disribuion. (3): E[(w x y ) ] = E[(w x y) ] his is because w only depends on he firs 1 saples bu doesn depend on (x, y ). Now le s sar he proof: E S [R v ] = E S,(x,y) [(v x y) ] E[ 1 (w x y) ] (using observaion (1)) E[(w x y) ] E[(w x y ) ] (observaion (3)) = 1 E[ (w x y ) ] 1 E[ (u x y ) 1 [ E[(u x y ) ] + u 1 ] [ E[(u x y) ] 1 = R u 1 + u and we have copleed he proof. + u ] (by WH bound) ] + u (by observaion ()) 5