ADAPTIVE ROBUSTNESS ANALYSIS

Transcription

1 ROBUSTNESS ANALYSIS 入力 ADAPTIVE OF 中文 Mike Tian-Jian Jiang, Cheng-Wei Lee, Chad Liu, Yung-Chun Chang, Wen-Lian Hsu Institute of Information Science, Academia Sinica

2 入力, Input Method (IM) Text Entry IIS, Sinica 2/30

3 groups/cue/mobileinteraction/ IIS, Sinica 3/30

4 Radical vs. Phonetic Homographs vs. Homophones IIS, Sinica 4/30

5 Disambiguation To predict or not IIS, Sinica 5/30

6 HCI, NLP (, SE) Unified error metrics (Soukoreff and MacKenzie, 2001) Error correction (Arif and Stuerzlinger, 2010) Reused vocabulary (Tanaka-Ishii et al., 2003) Backward compatibility (Suzuki and Gao, 2005) IIS, Sinica 6/30

7 Prediction and spell correction can be very annoying if they are not smart enough. For many applications, user input can be very noisy (imagine voice recognition or typing on a small screen), so the input methods must be robust against such noise. Finally, there is no standard data set or evaluation metric, which is necessary for quantitative analysis of user input experience. WTIM 2011 statements of call for papers IIS, Sinica 7/30

8 Prediction & Adaptation Properties of Chinese Phonetic IM IIS, Sinica 8/30

9 Adaptation via Online Implicit User Feedback (Online Offline Implicit Explicit) user feedback Adaptation procedure extends Tanaka-Ishii et al. (2003) User ambiguous source keystroke string s IM retrieve(s D) candidate chunks c[]; D {db profile} IM sort(c[]) IM compose(c[]) target string t eval(t) User modify(t) t IM adapt(t t) {feedback profile} IIS, Sinica 9/30

10 Dilemma Type long and (either right or wrong things) prosper IIS, Sinica 10/30

11 Amortized Cost Trade-off between benefic and cost of error correction IIS, Sinica 11/30

12 based on Unified Error Metrics Related to minimum string distance (MSD) error and key-stroke per character (KSPC) With Fitts law and Hick s law IIS, Sinica 12/30

13 Notations P: presented text T: transcribed text IS: input stream C: number of correct characters in T F: keystrokes for fixing in IS like editing, modifier, or navigation. INF: number of incorrect yet not fixed errors in T IF: number of incorrect but fixed errors (keystrokes in IS that are not F and not in T) IIS, Sinica 13/30

14 MSD(P,T ) S A 100% P: the quick brown fox T: the quixck brwn fox MSD(P, T) = 2 here Only for T without editing process IIS, Sinica 14/30

15 KSPC IS / T IIS, Sinica 15/30

16 Total Error Rate = MSD Error Rate = KSPC INF + IF C + INF + INF 100% INF C + INF 100% C + INF + IF + F C + INF T: the quixck brwn fox T : the quixck brown fox Total Error Rate(T ) = (2/18)% MSD Error Rate(T ) = (1/17)% KSPC(T ) = 19/17 IIS, Sinica 16/30

17 t F = a + blog 2 ( d w +1) t H = blog 2 (n +1) t: time a, b: empirical constants d: distance to target w: width of target n: number of equal possible choices Index of difficulty (ID): log 2 ((d/w) + 1) IIS, Sinica 17/30

18 Error Correction Conditions None, Forced, or Recommended conditions No relations between typing speed and correction attempt Spectrum of Recommended condition Situation Fixed characters INF IF F S 0 none INF S i some INF i IF i F i S all all 0 IF all F all IIS, Sinica 18/30

19 AC = Wasted Bandwidth Utilized Bandwidth = INF + IF + F C + INF + IF + F C C + INF + IF + F = INF + IF + F C INF 0 C AC i = INF i + IF i + F i C IF all C + F all C = INF 0 C + F all C penalty = t H INF 0 + t F max(d) C + INF 0 reward = C C + INF 0 AC modification = penalty reward = t INF H 0 + t F max(d) C MAC = INF 0 C + AC modification IIS, Sinica 19/30

20 Vocabulary Reuse % vocabulary reused only after a small offset window in KB (such that simulations of typing repeatedly are representative enough) IIS, Sinica 20/30

21 Backward Compatibility Error Ratio = errors by adaptation / corrections by adaptation IIS, Sinica 21/30

22 however, Error correction can be complicated. IIS, Sinica 22/30

23 Process of input Process of Correction Input [User doesn t correct] c γ correction [User corrects] [User doesn t verify] i γ verification [There are errors] [There are no errors] c, s error γ + γ c, h error [User verify] [There are no errors] [There are errors] i, s error γ + γ i, h error [User verify] [User doesn t verify] c γ verification [User doesn t correct] h correction s error [User corrects] i γ correction i verification h error = i γ correction + i, s i, h error + γ error c verification c, s γ error Correction ρ = γ + γ + γ ρ + ρ γ + + γ c correction c, h error

24 Simulation 3 IMs A, B, and C Phonetic method Bopomofo (Zhuyin) Daqian keyboard layout Data Academia Sinica Balanced Corpus 4,000 sentences 39,469 words Independent variables Context length k ρ h correction IIS, Sinica 24/30

25 Comparison of MAC IM-A seems to be different to others? IIS, Sinica 25/30

26 GBC at Context Length 6 Again, IM-A is segregated IIS, Sinica 26/30

27 more aspects wanted Than this V curve IIS, Sinica 27/30

28 Error Tolerance Level Futile Effort (E f ): never adapted chunks Beneficial Effort (E b ): adapted chunks Utility (U): before forgotten E b (IM-A) vs. E b (IM-B) his or her or its? 他 or 她 or 它 /ta1/ Correlation Coefficient to CAR E f avg E b avg U avg IM-A IM-B IIS, Sinica 28/30

29 How about a shared task? Just my humble suggestion :p IIS, Sinica 29/30

30 Thank YOU? Or do we have time for IIS, Sinica 30/30

31 Many MORE Things IIS, Sinica 31/30

32 zhi1- shi4 shi4- wei4 tou2- shi4 shi4- li4 dao4- shi4 shi4- shang4 fang1- shi4 shi4- gu4 chang2- shi4 shi4- yi2 yi4- shi4 shi4- ji1 zhi4- shi4 shi4- zhong1 IIS, Sinica 32/30

33 Reduced n-gram British Rail Enquiries P(Enquiries British, Rail) P(Enquiries Rail) IIS, Sinica 33/30

34 OpenVanilla IIS, Sinica 34/30

35 SearchTyping IIS, Sinica 35/30

36 Thank YOU! Any question or comment? IIS, Sinica 36/30