Chapter 8: Temporal Analysis

Transcription

1 Lecure Noes for Maagg ad Mg Mulplayer Ole Games Summer semeser 7 Chaper 8: Temporal Aalyss Lecure Noes Mahas Schuber hp://

2 Chaper Overvew Behavor ad Sequeces Comparg Sequeces Fdg freque subsequeces Markov chas Hdde Markov-Chas Tme seres ad feaure-rasformaos Comparg me seres Posso-Processes

3 player behavor examples for player behavor sequece of moves chess sequece of moveme, aco ad eraco a MMORPG sequece of orders o us RTS Games cocepoally behavor cosss of a sequece of possble acos Smples models for behavor are srgs or sequeces. Defo: Le A{A,, A } be a fe alphabe of possble player acos, he he l-tuple (a,, a l ) A A s a sequece of l legh over A. Remark: Model descrbes oly observaos ad does o dffereae bewee possble ad mpossble sequeces. Model eglecs he me bewee acos. 3

4 Example: SC II Zerg Rushes 4

5 Subsequeces ad Parog Whch player s observed a a gve me ad for how log? The loger a player s observed, he less lkely becomes ha aoher player behaves smlarly To fd ypcal behavoral paers a sequece s usually dvded o subsequeces. Wdowg (paros a sequece) Slde a wdow of legh k over he sequece ad cosder all subsequeces. ( here k 3) A B C V B W E E E R Q A F A B C V B W E E E R Q A B C V B W E E E R Q A F C V B W E E E R Q 5

6 Subsequeces ad Parog problem: A sequece of legh l has l (k-) k-elemeal subsequeces ad may of hose are rreleva. dea: Oly sequeces appearg wh a cera frequecy are of eres. Freque Subsequece Mg Fd all subsequeces a sequece daabase appearg more frequely ha msup. (cf. Freque Iemse Mg) legh of he sequece s arbrary. search space s larger ha he search space of emse mg. (several occurreces of elemes ad orders) 6

7 Freque Subsequece Mg frequecy fr(s,g) of S sequece G: cou occurrece of S G relave frequecy of S: ϕ( S, G) sequece descrpo of G: δ (G) ϕ fr( S, G) G S {(S, (S,G)) S G} mg sequeal paers s well explored > may approaches ad algorhms 7

8 Suffx Trees Properes of a Suffx Tree ST for he alphabe A wh sequece G where G : o rule ou ambvalece, words are padded wh a ermal symbol (A), commoly $. ST has exacly + leaf odes umbered from o, o he way from he roo o he leaf he suffx of legh - s fled. Edges represe elemes of A{$} (ucompressed form), oempy paral-sequeces of A{$} respecvely Edges, emaag from he same sarg ode, mus beg wh dffere elemes of A. Creao O( pu srg ), Search O( query srg ) 8

9 Suffx Trees example: alphabe A {ea, hu, seek, flee, defed} ser: S (seek, hu, ea, seek) S (seek, flee,hu) 9

10 Suffx Trees example: Alphabe A {ea, hu, seek, flee, defed} ser: S (hu, seek, ea, seek) (hu, seek, ea, seek, $) S (seek, flee, hu) (seek, flee, hu, $) 4 3 $ roo $ seek ea ea seek seek $ $ hu seek ea seek $ ucompressed vara: every edge s labeled wh a eleme of A{$} compressed vara: combe sub-pahs whou braches o oe edge

11 Suffx Trees example: Alphabe A {ea, hu, seek, flee, defed} ser: S (hu, seek, ea, seek) (hu, seek, ea, seek, $) S (seek, flee, hu) (seek, flee, hu, $) 4 3 roo $ seek $ ea,seek,$ ea,seek,$ hu,seek,ea,seek,$ ucompressed vara: Every edge s labeled wh a eleme of A{$} compressed vara: combe sub-pahs whou braches o oe edge

12 Suffx Trees example: Alphabe A {ea, hu, seek, flee, defed} ser: S (hu, seek, ea, seek) (hu, seek, ea, seek, $) S (seek, flee, hu) (seek, flee, hu, $) 4 3 $ roo $ seek ea ea seek seek $ $ hu seek ea seek $ $ 4,3 $ 3 flee hu $ roo seek ea seek $ flee hu ea hu seek $ $ $ seek ea seek $

13 Suffx Trees example: Alphabe A {ea, hu, seek, flee, defed} sample queres: Is q a Suffx? Is q a Subsrg? How ofe occurs q? 4,3 3 $ $ flee hu $ roo seek ea seek $ flee ea seek $ hu hu $ seek ea seek $ $ 3

14 Suffx Trees Example: Alphabe A {ea, hu, seek, flee, defed} Sample reques: Is q a Suffx? > follow pah (q$) sarg a roo, If reachg a leaf, he s a Suffx roo flee Is q a Subsrg? 4,3 $ seek ea > follow pah (q) sarg a roo, $ If processg possble, ea seek 3 flee he Subsrg seek $ How ofe occurs q? hu > follow pah (q) sarg a roo, $ #leaves below ermal odes $ #Occureces hu hu $ seek ea seek $ $ 4

15 Ieresgess of Subsequeces eresg freque commo sequece: selec droes, collec crysals, ra droe, bu: he frs acos SC II are almos always decal. umber of freque subsequeces ca be very large. mos of whch descrbe sadard game plays. eresgess should be evaluaed relao o aoher arbue: Map (Relag o a place) Player (Relag o a dvdual) Sraegy (Relag o suao) Combao of mulple relaos (Map ad Sraegy ) 5

16 Measures for Ieresgess use correlao measures: fd a arge varable: e.g. player_d fd eresg eves: e.g. boss-fghs, flag bearer, fd places rggerg smlar behavor: spawg pos, flag delvery locaos, boss ecouer se,... example calculaos: Muual Iformao MI(S, Player_ID) Odds Rao oddsr S (G,G ) P Players S {S,S} ϕ(s,g ϕ(s,g ) ) Pr[S,P] Pr[ S, P] log Pr[S] Pr[ P ] 6

17 Use of freque subsequeces player defcao: use he occurrece of he k- mos eresg paral sequeces as vecor space dmesos. (here eresg hghes MI wh player_d) > descrbe players as vecors of observed subsequeces. search locaos specfc behavor: compare he cdece of acos o he map o he amou of acos a gve locao. (Odds-Rao) 7

18 Comparg wo Sequeces gve: Alphabe A ad a sequece daabase DB {(x,, x k ) k IN x A for k}. ask: compue he smlary of S, S DB. Hammg Dsace: umber of dffere eres over all posos. For sequeces wh S S k: k f s, s, DsHam( S, S) else Remark: For sequeces of dffere legh, he shorer sequece s flled wh he gap symbol -. example: S (A,B,B,A,B) ud S (A,A,A,A,A) (A,B,B,A,B) (A,A,A,A,A) Ds Ham (S,S)3 8

19 Leveshe Dsace Hammg Dsace: Compug he mmum cos o rasform S o S. Oly subsuos of sgle elemes are allowed dog so. (Tur B o A.) Hammg Smlary: Cous he umber of smlar elemes. dea: Exed he allowed rasformaos o clude deleo ad sero of symbols. Leveshe Dsace: Mmum expese o rasform S o S usg 3 operaos Delee, Iser ad Subsue. (A,B,B,A,B) (A,A,B) (A,B,B,A,B) (A,-,-,A,B) Sm Lev (S,S)3 9

20 Calculag Leveshe Dsace gve: Two sequeces S, S over he alphabe A wh S ad S m. ask: Ds Lev (S,S) Calculag Leveshe Dsace wh dyamc programmg: Le D be a m-marx over IN wh: Afer cosruco of marx D, D,m coas he Leveshe-dsace bewee boh pu sequeces.

21 Example Leveshe Dsace S auo, S ue - a u o - a u o - a u o u u u 3 e 3 e e a u o - u (a,u,,o) (-, u,,e) Ds Lev (S,S) e 3 3 3

22 Ed Dsaces geeralzao of Leveshe-Dsaces: dffere cos marx: subsuo coss 4, deleo, sero.. more operaos: rasposg order (A,B,B,A,B) (A,B,A,B,B) rasposo duplcag, (A,B,B,B,B) (A,B,) 3 duplcaes of B coss may dffer for dffere values: Subs.(A,B) Subs.(A,Z) works for sequeces based o real-valued alphabes, for example: For A IR: Subs(5,) 5-

23 Markov Chas ad Sequeces sequeces of acos are subec o cera rules modelg wh fe auomaos (esg sequece for valdy) Markov chas are probablsc auomaas: allowed sae rasos probably dsrbuos for sae rasos. s order Markov assumpo : The sae a me + depeds solely o he sae a me. he order of a Markov cha s he umber of predecessor saes o whch he choce of he ex sae mgh deped. 3

24 Frs Order Markov-Chas defo: A Markov cha M s defed for a sae se A ad a sochasc raso-marx A A D. explaaos: A may coa a sar- ad a absorpo-sae (Modelg Sar ad Ed) sochasc Marx: rows add up o. (row coas he dsrbuo of successors for sae ) example: - A B C A A B C B C p( ACBB) P( A ) P( C A) P( B C) P( B B) P( B)

25 Hdde Markov Models rag a Markov cha: break he rag sequece dow o -grams ad deerme he relave frequecy. (How ofe s A followed by B?) P ( B A) fr( AB) fr( A) problem: observaos ofe do o mach he observed behavor: aco log s avalable, bu game-play has o be aalyzed correc execuo obfuscaes acual eos aalyss of a AI sae chages (observed acos may be employed dffere saes) 5

26 Hdde Markov Models Defo: A Hdde Markov Model M s defed by a sae se A, a sochasc raso marx A A D, a observao se B ad a sochasc oupu-marx A B F. Example: A{A,B,C}, B{,,3} 3 B A C D - A B C F 3 P(): defe all possble sae rples, geeraed by : BAA, BAC - A B C P( ) P( BAA) P( BAA) + P( BAC) P( BAC) A B C

27 Use of HMM Evaluao: How lkely s a observao O(o,.., o k ) wh o B for he HMM (A,B,D,F)? (Forward Esmao) Recogo: Gve he observao O(o,, o k ) ad he HMM (A,B,D,F) whch sequece (s,, s k ) wh s A gves he bes explaao for O? (Verb-Algorhm) Trag: Gve he observao O(o,, o k ), how ca we modfy D ad F o maxmze P(O (A,B,D,F))? (Baum-Welch Esmao) 7

28 Evaluao: Forward Varables gve: O(o,, o k ) ad (A,B,D,F) ask: P(O (A,B,D,F)) ave soluo: calculae P(O S) for all k-elemeal sequeces S o A. (umber grows expoeally wh k) mproved soluo: ulze Markov assumpo defe forward-varable α () as α ( ) P( o, o,..., o, s a ( ABDF)) calculao by duco: α ( ) d, f, o, A A α ) α ( ) d, f, o calculag wh A k operaos: ( + + P( O ( A, B, D, F)) A P( O, s a, k ( A, B, D, F)) A α ( k) 8

29 9 Recogo: Verb Algorhm gve: O(o,, o k ), ad Model (A, B,D,F). ask: S(s,, s k ), whch maxmzes P(O S,(A, B,D,F)). defe δ() as he hghes probably of a sequece o A of legh for he observao O. calculao by duco smlar o forward algorhm, bu more effce sce oly he bes soluo s pursued. )),,, (,...,, ( max ) (,.., F D B A O s s P s s δ ( ) ( ), ) ( arg max ) (, (), ) ( max ) (, (),,,,, k d A k f d A f d A o A o δ ψ ψ δ δ δ

30 Backward Varables aalogously o Forward-Varable a Backward-Varable ca be defed, used rag he HMM. defe Backward-Varable β () as β ( ) P( o,..., o s a,( ABDF)) + k Calculao by Iduco: β ( k), A A β ( ) d, f, o β ( ), k a a... a d, d, d, a α () d, f, o β (+) d, d, a a a d,... a

31 3 Trag: Baum-Welch Esmao gve: O(o,, o k ), A ad B. ask: D, F, maxmzg P(O (A,B,D,F)). Locally opmze soluo wh Expecao Maxmzao (EM) Defe ξ, () as he lkelhood of beg sae a a he po me ad beg sae a a he po me + : Defe γ () as he probably of beg sae a a he po me : ,,,,,,, ) ( ) ( ) ( ) ( )),,, ( ( ) ( ) ( )),,,,(, ( ) ( A k A l o l l k k o o f d f d F D B A O P f d F D B A O a s a s P β α β α β α ξ A, ) ( ) ( ξ γ

32 Trag: Baum-Welch Esmao k ξ, ( ) equals he expeced umber of sae rasos from a o a. k γ ( ) equals he expeced umber of sae rasos from a o oher saes. parameer are beg recompued as follows: d k ( ) rag happes alerag seps calculae of γ (), ξ,, () ad P(O (A,B,D,F)) updaes of D ad F (updaes see above) algorhm ermaes whe P(O (A,B,D,F)) grows less ha., { o bl }, a γ ( ), d,, f k, b k ξ γ ( ) l γ ( ) γ ( ) 3

33 Real-Value Sequeces so far: Alphabe s a dscree doma Sequeces ca also be creaed based o real-value domas, for example IR d. Freque Paer Mg o real-value domas s usually mpossble. Comparg real-value sequeces o doma D wh a dsace fuco ds: D D IR +. Aalogous o Hammg Dsace oe ca deerme he sum of dsaces for every poso of he sequece. ds sequ S,,, + ( S S ) ds( s, s ) + ( S S ) ϕ, für S S, ϕ IR Exeso of ed dsace s als possble: Subsuo cos for v ad u correlaes o ds(v,u). (More deals follow laer for Dyamc Tme Warpg) 33

34 Tme seres so far: sequeces model he order of acos, bu o he pos me. bu: real me games mg s esseal. RTS games: buld order are oly effecve f hey ca be realzed mmal me. MMORPGs he damage caused depeds o he umber of acos per me u. chess wh chess clock: a move s also measured by he me eeded o hk. me seres: Le T be a doma o model me ad le F be a obec preseao, he: Z((x, ),.., (x l, l )) (F T).. (F T) s a me seres of legh l o F. 34

35 Examples for Tme Seres SC-Logs: me seres o dscree acos : TSLHyuN Selec Hachery (3) : TSLHyuN Selec Larva x3 (7c,8,84), Deselec all : TSLHyuN Tra Droe : TSLHyuN Tra Droe : TSLHyuN Selec Droe x6 (34,38,3c,4,44,48), Deselec all : TSLHyuN Rgh clck; arge: Meral Feld (4) : TSLHyuN Deselec 6 us : TSLHyuN Rgh clck; arge: Meral Feld (7). Nework-Traffc: used bo deeco esmag game esy 35

36 Preprocessg Tme seres () offse raslao smlar me seres wh dffere offses shfg all me seres aroud he mea MW: o : o o MW(o) ds(q,o)??? q o q q - MW(q) o o - MW (o) ds(q,o)???

37 preprocessg me seres () scalg ampludes me seres wh smlar progresso bu dffere ampludes shfg he me seres aroud he mea (MW) ad ormalzg he amplude by sadard devao (SD): o : o (o MW(o)) / SD(o) q (q - MW (q)) / SD(q) o (o - MW(o)) / SD(o) 37

38 preprocessg me seres (3) lear reds smlar me seres wh dffere reds Iuo: deerme regresso le move me seres by meas of hs le offse raslao + ampludes scalg offse raslao + Ampludes scalg + lear red-removal

39 Preprocessg me seres (4) recfyg ose smlar me seres wh a large amou of ose smoohg: deerme for every value o he mea over all values [o -k,, o,, o +k ] for a gve k

40 Dscree Fourer Trasformao (DFT) dea: descrbe arbrary perodc fucos as weghed sum of perodc base fucos wh dffere frequeces. A me seres urs o a vecor of cosa legh. base fucos: s ad cos 4

41 Dscree Fourer Trasformao (DFT) Fourer s heorem: A perodc fuco (whch s reasoable couous) may be expressed as he sum of a seres of se ad cose erms wh a specfc amplude. properes: rasformao does o chage a fuco, oly he preseao rasformao s reversble > verse DFT aalogy: chage of base vecor calculao [x ] DFT f f [X f ] f f f 4

42 4 Dscree Fourer Trasformao (DFT) formal: gve a me seres of legh : x [x ],,, he DFT of x s a sequece X [X f ] of complex umbers for he frequeces f,, wh where defes he magary u vz.. he real par dcaes he share of he cose fucos, whereas he magary par dcaes he share of se fucos of he frequecy f. Real par Imagary par s cos f f f x f x e x X π π π

43 Dscree Fourer Trasformao (DFT) he verse DFT resores he orgal sgal: x f X f e π f,, (: pos me) [x ] [X f ] descrbes a Fourer-Paar, vz. DFT([x ]) [X f ] ad DFT ([X f ]) [x ]. he DFT s a lear map, vz. from [x ] [X f ] ad [y ] [Y f ] follows: [x + y ] [X f + Y f ] ad [ax ] [ax f ] for a Scalar a IR eergy of a sequece eergy E(c) of c s he square of he amplude: E(c) c. eergy E(x) of a sequece x s he sum of all eerges of he sequece: E( x) x x 43

44 Dscree Fourer Trasformao (DFT) Parseval s heorem: Eergy of a sgal a me rage equals he eergy he frequecy rage. Formal: Le X he DFT of x, he follows: x X f cosequece from Parseval s heorem ad he DFT s leary: The eucldea dsace of wo sgals x ad y correspod me ad frequecy rage: x y X Y DFT f f [x ] [X f ] f f f Tme rage (-space) Frequecy rage (-space) 44

45 Dscree Fourer Trasformao (DFT) Basc Idea of query processg: The eucldea dsace s used as a sequece s smlary fuco: ds( x, y) x y x y parseval s heorem allows for dsaces o be calculaed he frequecy rage sead of he me rage: ds(x,y) ds(x,y) pracce he lowes frequeces are he mos mpora. he frs frequecy coeffces coa he mos mpora formao. for dexg he rasformed sequeces are shoreed, for [X f ], f,,, coeffces oly he frs c coeffces [X f < c], c < are dexed. c dsc ( x, y) x f y f x f y f f f ds( x, y) for he dex a lower boud of he rue dsace ca be calculaed: fler-refeme: fler sep s based o shoreed me seres (dex asssed) refeme sep deermes rue hs o complee me seres 45

46 Dsaces of Tme Seres problems: Whch pos me are o be compared? offse a he begg: S s shfed me o S. 5 clockg of readg: pos me of measurg dffer. legh of me seres: measurg perods dffer. me seres wh he same clockg ad legh ca be compared as vecors. (dmeso po me) T for varable legh, clockg ad offses: adapo of ed-dsace for sequeces > Dyamc Tme Warpg Ds 5 5 meseres ( S, S) dsob ( s, s ) 46

47 Dyamc Tme Warpg Dsaz calculao: gve: me seres q ad o of dffere legh fd mappg of all q o o wh mmal expese o q o m q Search marx q m wk o w 47

48 Dyamc Tme Warpg Dsace Search Marx All possble mappgs q o o ca be erpreed as a warpg pah wh he search marx Of all hese mappgs, we search for he pah wh he lowes cos K DTW ( q, o) m w K k k Dyamc Programmg > Ru-me (. m) (see Ed Dsaces) m Q w k C w 48

49 Approxmae Dyamc Tme Warpg Dsace dea: approxmae he me seres (compressed represeao, Samplg, ) calculae DTW for he approxmaes 49

50 Sasc Models for Tme problem: How s he me of he ex aco modeled? sasc models for he me bewee wo eves s ecessary. me s a couous varable > probably desy fuco ask: compue he probably for he ex eve e occurrg wh he me frame +. he cumulave probably desy fuco descrbes hs probably 5

51 Homogeeous Posso Processes smples process o model me pos me bewee eves are expoeally dsrbued λ probably desy of he expoeal dsrbuo: pλ ( x) λ e egrao yelds he cumulave desy fuco descrbg he probably of he ex aco happeg he me erval bewee x. x λx Pλ ( x) pλ ( ) d e x p λ ( x) λ e λx P ( x) λ e λx Desy fuco of he expoeal dsrbuo Accumulaed desy fuco of he expoeal dsrbuo 5

52 5 Parameer assessme gve: A rag se of pos me X{x,, x }, whch are dsrbued expoeally. ask: The mos lkely value for he esy parameer. Approxmao wh Maxmum Lkelhood > Search he value of λ wh he hghes probably of geerag X. Lkelhood fuco L for Sample X: Dffereae he log-lkelhood for λ ad se he grade o zero: ) ( ) ( X E x x X e e e L λ λ λ λ λ λ λ ) ( ) ( )) ( ) l( ( ) ( l * X E X E X E d d L d d λ λ λ λ λ λ λ x X E m ) (

53 Learg Goals Sequeces ad me seres Freque Subsequece Mg wh Suffx-Trees Dsace measurg sequeces Hammg Dsace Leveshe Dsace Markov-Chas Hdde Markov chas Tme seres ad preprocessg seps Dyamc Tme Warpg Posso processes 53

54 Leraure Kyog J Shm, Jadeep Srvasava: Sequece Algme Based Aalyss of Player Behavor Massvely Mulplayer Ole Role-Playg Games (MMORPGs), Proceedgs of he IEEE Ieraoal Coferece o Daa Mg Workshops,. Be G. Weber, Mchael Maeas: A daa mg approach o sraegy predco, Proceedgs of he 5h Ieraoal Coferece o Compuaoal Iellgece ad Games, 9. K.T. Che, J.W. Jag, P. Huag, H.H. Chu, C.L. Le, W.C. Che: Idefyg MMORPG bos: A raffc aalyss approach, I Proceedgs of he 6 ACM SIGCHI Ieraoal Coferece o Advaces Compuer Eerame Tsechology, 6. 54