User Modeling: The Long and Winding Road

Size: px

Start display at page:

Download "User Modeling: The Long and Winding Road"

Jack King
6 years ago
Views:

1 User Modeling: The Long and Winding Road Gerhard Fischer Center for LifeLong Learning and Design (L 3 D), Department of Computer Science and Institute of Cognitive Science University of Colorado, Boulder, CO, USA Abstract. The long and winding road of user modeling is grounded in different epistemological assumptions exploring different dimensions of the problem. User-modeling research has explored different domains, identified important distinctions underlying different approaches within user modeling research, and created a number of challenging research problems. These issues are explored in the context of high-functionality applications and how our research over the last ten years has addressed the problems of making high-functionality applications more usable, more useful, and more learnable with a variety of different user modeling approaches. 1. Introduction User modeling is one of a number of research areas that intuitively seem to be winning propositions and worthwhile investments based on their obvious need and potential payoff. One area comparable to user modeling is software reuse. The approaches seem to be appealing, natural, theoretically justifiable, desirable, and needed (e.g., reuse can be justified by the fact that complex systems develop faster if they can build on stable subsystems). But in reality, progress in these areas has been slow and difficult, and success stories are rare. The research area has explored different domains such as: natural language dialog, human computer interaction, intelligent assistants, information retrieval, and high-functionality applications; identified important distinctions such as: adaptive versus adaptable components, explicit versus implicit modeling techniques, user models versus task models, canonical versus individual models, and long-term versus short-term models; created a number of challenging research problems, such as how to: (1) integrate different modeling techniques; (2) capture the larger (often unarticulated) context and what users are doing (especially beyond the direct interaction with the computer system); (3) identify user goals from low-level interactions; (4) reduce information overload by making information relevant to the task at hand; (5) support differential descriptions by relating new information to known information and concepts; and (6) reach a better balance for task distributions between systems and users. Some of these challenges will be illustrated in the context of the work that we have done to make high-functionality applications more usable, more useful, and more learnable. Gerhard Fischer 1 UM 99

2 2. High-Functionality Applications High-functionality applications (HFAs) (such as Unix, MS-Office, Photoshop, Eudora, etc.) are used to model parts of the world and not just to implement algorithms. They are complex systems because reality is not user friendly. If you ask a 100 different people what features they would like to have in a particular application, you end up with a very large number of features. The design of HFAs must address two problems: (1) the unused functionality must not get in the way and (2) unknown existing functionality must be accessible or delivered at times when it is needed. We have conducted a variety of empirical studies to determine the usage patterns of HFAs, their structure, their associated help and learning mechanisms. All of these studies have led us to the identification of the qualitative relationships between usage patterns of HFAs as illustrated in Figure 1. The ovals represent users' D 3 D2 D 1 D4 knowledge about the system's information space. D1 represents concepts well known, easily employed, and used regularly by a user. D2 contains concepts known vaguely and used only occasionally, often requiring passive help systems. D3 represents concepts users believe to exist in the system, some of which lie outside the actual information space. The rectangle D4 represents the actual information space of a system. As the functionality of HFAs increases to D4', little is gained for users unless there are mechanisms to help them relate the additional functionality to their needs. The area of D4 that is not part of D3 is of specific interest to research in user modeling. This is system functionality, whose existence is unknown to users. For the D4 and not D3 domain, information access (the user-initiated location of information when they perceive a need for an operation) is not sufficient, but information delivery (the system volunteering information that it inferred to be relevant to the users task at hand) is required. Active help systems and critics are required to point out to users functionality that may be useful for their tasks and to help users to avoid to get stuck on suboptimal plateaus. Figure 1 shows usage patterns of HFAs without taking the users tasks into account. There is no reason for users to worry about additional existing functionality in D4, if this functionality is not relevant to their tasks. However, if the system does provide functionality in D4 related to users tasks, it is desirable to avoid having users unable to perform the task or do so in a suboptimal or error-prone way because they do not know about this functionality. In Figure 2 the gray rectangle T represents the information that is relevant to the users task at hand, and the dots represent different pieces of functionality. Passive intelligent support systems supporting information access can help users to explore pieces of functionality that are contained in D3 and T, whereas active intelligent systems supporting information delivery are Gerhard Fischer 2 UM 99 D 4 ' Figure 1: Levels of Users' Knowledge about a System's Information Spaces

3 D 3 D2 D1 T for references, and U4 knows about collaborative writing tools. needed for the functionality contained in T and not in D3. The functionality in D4 outside of T is often offered by push systems such as Did You Know (DYK) systems [Owen, 1986] or Microsoft s Tip of the Day [Horvi t z, 1997], which throw decontextualized concepts at users. Experts and Expertise in HFA. Experts (users who know everything about a system) no longer exist in HFAs. Being an expert is at best an attribute of a specific context, rather than a personal attribute. The different spaces of expertise (determined by individual interest) are illustrated in Figure 3. In this multikernel model, {D1, Ui} means the area of functionality that is well known to a particular user Ui; for example: U1 knows about the equation editor; U2 about mail-merge functionality; U3 uses a bibliography system 3. How Our Research Address the Problems Created by HFAs Our research related to user modeling has attempted to address the challenges created by HFAs. Active help systems [F i s cher et al., 1985] were an early attempt to analyze the behavior of users and infer higher-level goals from low-level operations [Horvi t z, 1997; Nardi et al., 1998]. HFAs, as argued above, came into existence as environments that are useful for a large number of different users (see Figure 3). In order to reduce their complexity, HFAs have often migrated to a collection of domain-oriented subsystems each with their own templates, forms, and their own associated wizards thereby being able to provide additional support for user modeling and assistance not available in more general systems. In our own research, we have taken this approach further by developing domain-oriented design environments [F i s cher et al., 1998]. These are environments which model specific domains (such as computer networks, user interfaces, kitchens, and voice dialog design [S um ner et al., 1997]) by allowing designers to engage in authentic tasks from their own respective work practices. Domain-oriented design environments make computers invisible and enable designers to communicate with domain-specific concepts, representations and tools. The domain orientation of these environments makes HFAs more usable, more useful, and more D4 Figure 2: Functionality and its Relevancy to the Task at Hand D 1, U1 D, U Equation Editor D, U 1 4 Collaborative Writing 1 2 Mail Merge D, U 1 3 Referencing Figure 3: Distributed Expertise in HFAs D4 Gerhard Fischer 3 UM 99

4 learnable by bringing the objects closer to the conceptual world of their users. In support of user modeling, domain-oriented design environments provide specification components [Nakakoj i, 1993] to allow users to enrich the description of their tasks, and critiquing components [F i s cher et al., 1998] to analyze and infer the task at hand in order to be able to detect and identify the potential for a design information need and then present stored knowledge for designers. We have used the artifact (including its partial construction and partial specification) combined with domain knowledge in the design environment as an index to infer high-level goals from simple user actions. The existence of the specification component has allowed us to complement generic critics (which can be defined at design time) with specific critics whose behavior is dependent on the information provided by individual users at use time. Domain-oriented design environments integrate a number of components, and the research built around them has tried to developed principles and arguments for the trade-off between adaptive and adaptable approaches [F i s cher, 1993; Thom as, 1996]. Figure 4 shows a screen image from the Voice Dialog Design Environment [S um ner et al., 1997], which supports adaptation mechanisms to allow users to select specific rule sets for critiquing and to determine the intervention strategy for the intrusiveness of the critics ranging from active behavior (every step is immediately critiqued) through intermediate levels to passive behavior (users have to explicitly invoke the critiquing system). Similar solutions can be found in modern spelling correction programs. The need to customize and tailor critiquing systems to individual users objectives has also been explored in the domain of business graphs and use of color [Gut kauf, 1998]. Domain-oriented design environments pose a number of challenging problems for user modeling. Contrary to Intelligent Tutoring Systems [B urton & Brown, 1982], they model (open-ended) domains in support of self-directed learning and user-directed activities within a domain. They are able to exploit domain models for user modeling [Hol lan, 1990; Mas t agl i o, 1990]. At design time (when the system is developed), domain models including generic critiquing knowledge and support for specification, and end-user adaptation is provided. In these open-ended systems, the task that users engage in cannot be anticipated: they have to be inferred or articulated at use time. Design environments support both adaptable and adaptive components and tune system behavior to the specific needs of individual users and their tasks [Gut kauf, 1998; Nakakoj i, 1993]. Gerhard Fischer 4 UM 99

VM-Residential Figure 4: Adaptation Mechanism to Control Different Critiquing Rule Sets and Different Intervention Strategies HFAs create challenging learning problems that are representative for

5 VM-Residential Figure 4: Adaptation Mechanism to Control Different Critiquing Rule Sets and Different Intervention Strategies HFAs create challenging learning problems that are representative for numerous complex systems. As illustrated with the above figures, nobody learns these systems completely, but users acquire some base functionality and learn additional functionality on demand. Usermodeling techniques can effectively support learning on demand [F i s cher, 1991] by helping users to identify opportunities to learn additional functionality relevant to their task at hand and to avoid people becoming stuck on suboptimal plateaus. User modeling techniques based on logged user data can support the organization-wide learning of HFAs [Li nton et al., 1998]. 4. User Modeling and Human-Computer Collaboration Some of the beginnings of the long and winding road of user modeling were derived from the need and desire to provide better support for human-computer collaboration. Collaboration in this context is defined as a process in which two or more agents work together to achieve shared goals [Terveen, 1995]. Some fundamental issues (such as shared goals, shared context, control, (co)-adaptation, (co)-evolution, and learning) can be derived from this definition. Human-computer collaboration can be approached from two different perspectives: an emulation and a complementing approach. The emulation approach is based on the metaphor that to improve human-computer collaboration is to endow computers with human-like abilities. The complementing approach is based on the fact that computers are not human and that hu- Gerhard Fischer 5 UM 99

6 man-centered design should exploit the asymmetry of human and computer by developing new interaction and collaboration possibilities [S uchm an, 1987]. Historically, the major emphasis in user modeling has focused on the human emulation approach (see for example, [Kobsa & Wahl s ter, 1989], and the section user and discourse modeling in [M aybury & Wahls t er, 1998]). However, based on the limited success of the emulating approach, the interest has shifted more and more to the complementing approach [B obrow, 1991; F i s cher, 1990]. There is growing evidence that the problems of user modeling in the complementing approach are more tractable, more feasible, and more desirable, as evidenced by their increasing influence in the design of commercial high-functionality applications [Horvi t z, 1997]. As the complexity of commercially available HFAs grows, and as we see more computational assistants (such as agents, advisors, coaches, and critics) appear in widely available commercial applications, a detailed, state-of-the-art analysis and understanding of how people learn, work and collaborate with and around HFAs will provide us with new requirements for the design of user modeling components. In the long and winding road that is not only behind us but also before us, our understanding of the trade-offs, the promises, and the pitfalls between adaptive and adaptable systems, between push (information delivery) and pull (information access) technologies, and between contextualized information representation and serendipity will hopefully lead us to new ideas and new insights in the design of future human-centered systems supported by adequate user modeling techniques. Acknowledgments. I would like to thank the members of the Center for LifeLong Learning and Design (L 3 D) who have contributed to the ideas and perspectives expressed in this paper. A special thank you goes to my former students who, in their Ph.D. theses, explored important issues in user modeling as they occur in high-functionality applications, domain-oriented design environments, and critiquing systems. References Bo b r o w, D. G. ( )" Di me n s i on s o f I n t e r a c t i o n," AI Ma g a z i ne, 1 2 ( 3 ), p p Bu r t o n, R. R. & Br o wn, J. S. (1982)" An I n v e s t i g a ti o n o f Co mp u t e r Co a ch i n g f o r I n f o r ma l Lea r n i n g Ac t i v i t i e s." I n D. H. Sl e e ma n & J. S. Br o wn ( Ed s.), I n t e l l i g e nt Tu t o r i n g S y s t e ms, Ac a d e mi c Pr e s s, Lo n d o n - Ne w Yo r k, p p Fi s c h e r, G. ( ) " Co mmu n i c a ti o n Re q u i r e me n t s f o r Co o p e r a t i v e Pr o b l e m So l v i n g Sy s t e ms," I n - f o r ma t i o n S y s t e ms, 1 5 ( 1 ), p p Fi s c h e r, G. ( ) " Su p p o r t i n g Le a r n i n g o n De ma n d wi t h De s i g n En v i r o n me n t s." I n L. Bi r n b a um ( Ed.) I n t e r n a t i on a l Co n f e r e n c e o n t he Le a r n i n g S c i e n c e s ( Ev a n s t o n, I L), As s o c i a t io n f o r t h e Ad - v a n c e me n t o f Co mp u t i n g i n Ed u ca t i o n, p p Fi s c h e r, G. ( ) " Sh a r e d Kn owl e d g e i n Co o p e r a t i ve Pr o b l e m- So l v i n g Sys t e ms - I n t e g r a t i n g Ad a p - t i v e a n d Ad a p t a b l e Co mp o n e n t s." I n M. Sc h n e i d e r - Huf s c h mi d t, T. Ku e h me, & U. Ma l i n o ws k i ( Ed s.), Ad a p t i v e Us e r I n t e r f a c e s - Pr in c i p l e s a n d Pr a c t i c e, El s e v i e r Sc i e n c e Pu b l i s h e r s, Ams t e r - d a m, p p Fi s c h e r, G., Le mk e, A. C., & Sc h wa b, T. ( ) " Kno wl e d g e - Ba s e d He l p Sy s t e ms." I n L. Bo r man & B. Cu r t i s ( Eds.), Pr o c e e d i n gs o f CHI ' 8 5 Co n f e r e nc e o n Hu ma n Fa c t o r s i n Co mp u t i n g S y s t e ms, ACM, Ne w Yo r k, p p Fi s c h e r, G., Na k a k o j i, K., Os twa l d, J., St a h l, G., & Su mn e r, T. ( 1998 ) " Emb e d d i n g Cr i t i c s i n De s i g n En v i r o n me nt s." I n M. T. Ma y b u ry & W. Wa h l s t e r ( Ed s.), Re a d i n g s in I n t e l l i g e n t Us e r In t e r f a c e s, Mo r g a n Ka uf ma n n, Sa n Fr a n c i s c o, p p Gerhard Fischer 6 UM 99

7 Gu t k a u f, B. ( ) I mp r o v i n g De s i g n & Co mmu n i c a t io n o f Bu s i n e s s Gr a p hs t h r o u g h Us e r Ad a p ti v e Cr i t i q u i n g, Ph.D. Dis s e r t a t i o n, Ma t h e ma ti c s a n d Co mp u t e r Sc ie n c e, Un i v e r s i t ä t - GH Pa d e r b o r n, Pa d e r b o r n, Ge r ma n y. Ho l l a n, J. D. ( ) " Us e r Mo de l s a n d Us e r I n t e r f ac e s : A Ca s e f o r Do mai n Mo d e l s, Ta s k Mo d el s, a n d Ta i l o r a b i li t y." I n Pr o c e e d i n gs o f AAAI - 9 0, Ei g h t h Na t i o n a l Co n f e r e n c e o n Ar t i f i c i a l I n t e ll i g e n c e, AAAI Pr e s s/ Th e MI T Pr e s s, Ca mbr i d g e, MA, p Ho r v i t z, E. ( ) " Ag e n t s wi th Be l i e f s : Re f l e c t i on s o n Ba y e s i a n Me t h od s f o r Us e r Mo d e l i n g." I n C. P. A. J a mes o n, C. Ta s s o ( Ed.) Pr o c e e d i n gs o f t h e S i x t h I n t e rn a t i o n a l Co n f e r e n c e o n Us e r Mo d e l - i n g ( UM' 9 7), Sp r i n g e r Wi e n - Ne w Yo r k, Sa r d eg n a, I t a l y, p p Ko b s a, A. & Wa h l s t e r, W. ( Ed s.) ( ) Us e r Mo d e ls i n Di a l o g S y s t e ms, Sp r i n g e r - Ve r l a g, Ne w Yo r k. Li n t o n, F., Ch a r r o n, A., & J o y, D. ( ) OWL: A Re co mme n d e r S y s t e m f o r Or g a n i z a t i o n - Wi d e Lea r n - i n g, a t h t t p : // t r e.o r g / t e c h no l o g y / t e c h _ t a t s / mo d el i n g / o wl / Co a c h i n g _ Sof t wa r e _ Sk i l l s.p d f. Ma s t a g l i o, T. ( ) A Us e r - Mo de l l i n g Ap p r o a c h t o Co mp u t e r - Ba s e d Cr i t i qu i n g, Ph.D. Dis s e r t a t i o n, De p a r t me n t o f Co mp u t e r Sc i e n c e, Un i v e r s i t y o f Co l or a d o a t Bo u l d e r, Bo ul d e r, CO. Ma y b u r y, M. T. & Wa h l s t e r, W. ( ) Re a d i n g s in I n t e l l i g e n t Us e r In t e r f a c e s, Mo r g a n Kau f ma n n, Sa n Fr a n c is c o. Na k a k o j i, K. ( ) I n c r e a s i n g S h a r e d Un d e r s t a n d i ng o f a De s i g n Ta s k Be t we e n De s i g n e r s a n d De s i g n En vi r o n me n t s : Th e Ro l e o f a S p e c i f i c a t i o n Co mp o n e n t, Ph.D. Dis s e r t a t i o n, De p a r t men t o f Co mp u t e r Sc i e n c e, Un i v e r s i t y of Co l o r a d o a t Bo u l d er, Bo u l d e r, CO. Na r d i, B. A., Mi l l e r, J. R., & Wr i g h t, D. J. ( ) " Co l l a b o r a t i v e, Pro g r a mma b l e I n t e l l i g en t Ag e n t s," Co mmu n i c a ti o n s o f t h e ACM, 4 1 ( 3 ), p p Owe n, D. ( ) " An s we r s Fi r s t, Th e n Qu e s t i o n s." In D. A. No r ma n & S. W. Dr a p e r ( Ed s.), Us e r - Ce n t e r e d Sy s t e m De s i g n, Ne w Per s p e c t i v e s o n Hu ma n -Co mp u t e r I n t e r a c t i o n, La wr e n c e Er l b a u m As s o c i a t e s, I n c., Hi l l s d a l e, NJ, p p Su c h ma n, L. A. ( ) Pl a n s a n d S i t u a t e d Ac t i o n s, Ca mb r i d g e Un i v e r s i t y Pr e s s, Ca mb r i d g e, UK. Su mn e r, T., Bo n n a r d e l, N., & Ka l l a k, B. H. ( ) " Th e Co g n i t i v e Er g o no mi c s o f Kn o wl e d g e - Ba s e d De s i g n Su pp o r t Sy s t e ms." I n S. Pe mb e r t o n ( Ed.) Pr o c e e d i n gs o f CHI 9 7 Co n f e r e nc e o n Hu ma n Fa c t o r s i n Co mp u t i n g S y s t e ms, ACM/ Ad d i so n - We s l e y, p p Te r v e e n, L. G. ( ) " An Ov e rv i e w o f Hu ma n - Co mp u te r Co l l a b o r a t i o n," Kn o wl e d g e -Ba s e d S y s t e ms J o u r n a l, Sp e c i a l I s s u e o n Hu man - Co mp u t e r Co l l a b o r at i o n, 8 ( 2-3 ), pp Th o ma s, C. G. ( ) To As s i s t t h e Us e r : On t h e Embe d d i n g o f Ad a p t i v e an d Ag e n t - Ba s e d Me c h a - n i s ms, R. Ol d e n bu r g Ve r l a g, Mü n c h e n /Wi e n. Gerhard Fischer 7 UM 99

A L A BA M A L A W R E V IE W

A L A BA M A L A W R E V IE W Volume 52 Fall 2000 Number 1 B E F O R E D I S A B I L I T Y C I V I L R I G HT S : C I V I L W A R P E N S I O N S A N D TH E P O L I T I C S O F D I S A B I L I T Y I N