|
Abstracts
for Pirolli, Peter
Collaborative Intelligence Analysis with CACHE and its Effects on Information Gathering and Cognitive Bias
CACHE is a system supporting collaborative analysis tasks. We describe CACHE and a study of the effects of its use on reducing confirmation bias in analysis—the tendency to seek information that confirms rather than refutes an initial belief. We compare collaborative vs individual (Nominal) use of CACHE, as well as groups made of members with diverse (Heterogeneous) beliefs vs Homogeneous beliefs. We found reduction in confirmation bias for Heterogeneous collaborative groups using CACHE and working alone, but Homogeneous groups accentuated initial biases, rather than debiasing. Benefits of CACHE, future extensions, and our perspective on design are discussed.
Billman, D., Convertino, G., Shrager, J., Massar, J. P. and Pirolli, P. (2006).
Human Computer Interaction Consortium Workshop, Snow Mountain, CO. [PDF]
|
Social information foraging and collaborative search
In this short paper, we summarized recent work in the understanding of social information foraging, and models of how users socially foraging with diverse hints. In communities of practice that depend on foraging in overly rich information environments, there appears to be pressure to self-organize into a balance of some division of labor, plus some degree of cooperation. One way to bridge between different communities of users is to diversify their information sources. We have been examining the possibility of using a collaborative search engine to achieve this effect.
Chi, E. H. and Pirolli, P. (2006).
HCIC Workshop. [PDF]
|
Generality of Transfer of Navigation Knowledge from Degree-of-Interest Trees
An experiment was conducted to assess the generality of
navigation knowledge learned from interaction with the
Degree-of-Interest (DOI) Tree. The degree of transfer of
learning from the DOI Tree was compared to transfer of
learning from a standard Internet Explorer browser. Twenty
participants worked with either the DOI or Explorer
browser in a Training Phase in which they performed
information-finding or comparison tasks requiring
navigation through a widely used Web directory. Degree of
transfer was measured by effects on task completion time in
a Transfer Phase in situations with varying degrees of
similarity to the original training conditions. An analysis
based on percent transfer scores indicated that participants
trained on navigation of an information structure using the
DOI showed superior transfer across a range of conditions
as compared to participants trained using the Explorer.
Pirolli, P. and Gumbrecht, M. (2006).
CHI 2006. [PDF]
|
Navigation in Degree of Interest Trees
We present an experiment that compares how people perform search tasks in a degree-of-interest browser and in a Windows-Explorer-like browser. Our results show that, whereas users do attend to more information in the DOI browser, they do not complete the task faster than in an Explorer-like browser. However, in both types of browser, users are faster to complete high information scent search tasks than low information scent tasks. We present an ACT-R computational model of the search task in the DOI browser. The model describes how a visual search strategy may combine with semantic aspects of processing, as captured by information scent. We also describe a way of automatically estimating information scent in an ontological hierarchy by querying a large corpus (in our case, Google's corpus).
Budiu, R., Pirolli, P. and Fleetwood, M. (2006).
AVU 2006. [PDF]
|
Modeling Informattion Scent: A Comparison of LSA, PMI-IR, and GLSA Similarity Measures on Common Test and Corpora
In this paper we describe a comparison among three systems that estimate semantic similarity between words: Latent Semantic Analysis [6], Pointwise Mutual Information [17], and Generalized Latent Semantic Analysis [8]. We compare all these techniques on a unique corpus (TASA) and, for PMI and GLSA, we also report performance on a different web-based corpus. The evaluation is carried out through two kinds of tests: (1) synonymy tests, and (2) comparison with human word similarity judgments.
Budiu, R., Royer, C. and Pirolli, P. (2006).
CHI 2006. [PDF]
|
SNIF-ACT: A Model of User Navigation on the World Wide Web
SNIF-ACT (Scent-based Navigation and Information Foraging in the ACT Cognitive architecture) is developed to explain navigation behavior on the World Wide Web when users are engaged in unfamiliar information-seeking tasks. SNIF-ACT makes navigation choices based on assessments of activation that spreads to the users' information goal from proximal cues (information scent) that occur in Web browsers. SNIF-ACT 1.0 was matched to the detailed protocol analyses from the N = 8 participants studied in Card, S., Pirolli, P., Van Der Wege, M., Morrison, J., Reeder, R., Schraedley, P., & Boshart, J. (2001). Information scent as a driver of Web Behavior Graphs: Results of a protocol analysis method for web usability. CHI 2001, ACM Conference on Human Factors in Computing Systems, CHI Letters, 3(1), 498-505. SNIF-ACT 1.0 was found to provide good fits to participants' choice of links to follow, and to their choice of when to leave a current patch of information. SNIF-ACT 2.0 incorporates several refinements based on a rational analysis of link selection on a web page. These refinements include a satisficing assessment of link selection from a Web page and an adaptive stopping rule that decides when to leave a Web page based on past and current experiences with a Web site. SNIF-ACT 2.0 was validated on a data set obtained from N = 244 participants studied by Chi, E. H., Rosien, A., Suppattanasiri, G., Williams, A., Royer, C., Chow, C., Robles, E., Dalal, B., Chen, J., & Cousins, S. (2003). The Bloodhound Project: Automating discovery of Web usability issues using the InfoScent simulator. CHI 2003, ACM Conference on Human Factors in Computing Systems, CHI Letters, 5(1), 505-512. Monte Carlo simulations of the model provided good fits to the Web navigation data. Theoretical and practical implications of the model are discussed.
Fu, W.-T. and Pirolli, P. (2005).
Journal Cognitive Science. [DOC]
|
Information Scent and Web Navigation: Theory, Models, and Automated Usability Evaluation
Within a more Information Foraging Theory, we have developed a rational analysis of Web use, which has shaped a cognitive model of Web navigation called SNIF-ACT. An automated and practical method for initializing the model with requisite knowledge of information scent was developed based on Pointwise Mutual Information (PMI) computations from a local document corpus with a Web back-off. An automated Web usability tool called Bloodhound was developed that implements an algorithm that approximates the operation of the cognitive model. We report on succesful empirical tests of the SNIF-ACT cognitive mode, the PMI method, and Bloodhound.
Pirolli, P., Chi, E. H. and Farahat, A. (2005).
HCI International. [DOC]
|
The Theory of Information Scent.
No Abstract Available
Pirolli, P. (2005).
Next Wave: The Research Associate Directorate technical trends publications.
|
Cognitive Models of Human-Information Interaction
No Abstract Available
Pirolli, P. (2005).
Handbook of Applied Cognition.
|
GLSA Server @ PARC
No Abstract Available
Royer, C., Farahat, A., Pirolli, P. and Budiu, R. (2005).
Twelfth Annual ACT-R Workshop. [PPT]
|
Navigation in DOI Trees
No Abstract Available
Budiu, R. and Pirolli, P. (2005).
Twelfth Annual ACT-R Workshop. [PDF]
|
The InfoCLASS model: Conceptual richness and inter-person conceptual coherence about information collections
Previous work on information foraging theory has addressed how people navigate through information systems to find information. This paper presents a new information foraging model called InfoCLASS that models the conceptual categories that people learn while interacting with information systems. InfoCLASS is based on previous work on the rational analysis of human category formation. InfoCLASS simulations can be used to make qualitative predictions about the richness of conceptual categories that will be learned from different experiential histories, and from different kinds of user interfaces. InfoCLASS simulations can also be used to assess the conceptual coherence among a group of users of information systems. It is argued that the degree of conceptual coherence among a group of users is an important determinant of the efficiency and effectiveness of a social organization engaged in making discoveries, such as scientific communities or intelligence agencies.
Pirolli, P. (2004).
Journal of the Japanese Cognitive Science Society. [DOC]
|
Information foraging on the Web: Rational analyses of human-information interaction
Presents rational analyses of information foraging on the Web
Pirolli, P. (2004).
Cognitive Science. [DOC]
|
The use of proximal information scent to forage for distal content on the World Wide Web
Summarizes the theory of information scent and related it to Egon Brunswik's probablistic functionalism
Pirolli, P. (2004).
Working with technology in mind: Brunswikian resources for cognitive science and engineering. [DOC]
|
The Web as a back off corpus
A number of real world applications require the evaluation and ranking of word pair association. The three most effective techniques for measuring word association are based on statistical techniques that use corpus co-occurrence counts, latent semantic analysis techniques, or a lexical ontology. A major limitation of latent semantic analysis and lexical ontology techniques is their inability to deal with out of corpus words. This limitation has motivated researchers to use the web as a corpus for computing co-occuranece counts. However, a significant number of real world applications require a large number of co-occuranece evaluations that cannot be readily computed from the Web due to search engine limitations on the number of queries. In this paper, we use a hybrid approach for computing world pair association. This approach uses statistical similarity measures computed from co-occurence counts in a local corpus in case of common terms or from the web in the case of infrequent terms. We compare the four models of semantic similarity. The models range in their access to syntactic and semantic informativeness as well as search engine access. We evaluate the models using human ratings of semantic similarity. The outcome shows that the best correlations with human similarity judgment comes from a statistical approach that uses a local corpus and backs off to the web. Experimental results on a number of evaluation data sets show a significant reduction in the number of search engine queries while maintaining the same performance levels. The storage and computational requirements of our hybrid approach can easily fit on a modern workstation.
Farahat, A. and Pirolli, P. (2004).
Association of Computational Lnguistics. [PDF]
|
The Use of Proximal Information Scent to Forage for Distal Content on the World Wide Web.
The particular focus of this chapter will be on a psychological theory of information scent (Pirolli, 1997, 2003; Pirolli & Card, 1999) that is embedded in a broader model (Pirolli & Fu, 2003) of information foraging on the Web. The notion of information scent also has been used in developing models of people seeking information in document-clustering browsers (Pirolli, 1997) and highly interactive information visualizations (Pirolli, Card, & Van Der Wege, 2003). Information scent refers to the cues used by information foragers to make judgments related to the selection of information sources to pursue and consume. These cues include items such as Web links or bibliographic citations that provide users with concise information about content that is not immediately available. The information scent cues play an important role in guiding users to the information they seek, and they also play a role in providing users with an overall sense of the contents of collections. The purpose of this paper is to present a theoretical account of information scent that supports the development of models of navigation choice.
Pirolli, P. (2004).
Working with Technology in Mind: Brunswikian Resources for Cognitive Science and Engineering. Oxford University Press. [PDF]
|
A theory of information scent
Information scent is a psychological theory of how people use perceptual cues, such as World
Wide Web (WWW) links in order to make information-seeking decisions and to gain an overall
sense of the contents of information collections. A spreading activation theory of information
scent is presented, as well as a theory of learning categories (e.g., genres) of available information.
The SNIF-ACT model incorporates information scent to model World Wide Web browsing. The
InfoCLASS model simulates the learning of information categories from browsing.
Pirolli, P. (2003).
10th International Conference on Human-Computer Interation.
|
Use of Critical Analysis Method to Conduct a Cognitive Task Analysis of Intelligence Analysts
Intelligence analysts (IAs) engage in information seeking, evaluation, prediction, and reporting behavior in an information-intensive work environment. A Cognitive Task Analysis (CTA) was conducted to capture data that will provide input to support development of a model of the analyst's processes, biases, and analytic strategies. A hybrid method was used to conduct the CTA, including a modified version of the critical decision method - the critical analysis method. The essential distinction was that participants were asked to describe an example of a strategic analysis problem. Procedures used to conduct the critical analysis method are described in this paper. Several factors contribute to making the IA's task challenging: (i) time pressure to produce reports in a shorter timeframe, (ii) a high cognitive workload, and (iii) difficult human judgments that are required regarding uncertain validity and reliability of the data. Human judgments are involved in considering the plausibility of information, deciding what information to trust, and determining how much weight to place on specific pieces of data. Intelligence analysis involves a complex process of assessing the reliability of information from a wide variety of sources and combining seemingly unrelated events. This problem is challenging because it involves aspects of data mining, data correlation and human judgment.
Card, S. K., Hutchins, S. and Pirolli, P. (2003).
6th International Conference on Naturalistic Decision Making.
|
AVID: Supporting the creation of scalable, responsive visualizations
In this paper we describe a visualization architecture (AVID) that employs a dynamic model of user interest to support the design and creation of highly responsive, scalable visualizations of hierarchical data. We present evidence of the architecture's efficacy, showcasing dynamic visualizations with near-immediate (<100ms) update times, even on structures of over 100,000 nodes. We discuss how the key concepts used generalize to arbitrary graph structures. Additionally, we present the results of a user study comparing a prototypical visualization built using AVID to a more traditional file-browser interface, showcasing up to 20% improvement in information access times.
Heer, J., Card, S. K., Heiser, J. and Pirolli, P. (2003).
Working Paper.
|
SNIF-ACT: A model of information foraging on the world wide web.
No Abstract Available
Pirolli, P. and Fu, W.-T. (2003).
Ninth International Conference on User Modeling, Johnstown, PA. [PDF]
|
Information Scent and the Adaptive Conceptual Lens on Information
Theoretical integration of cognitive models of memory, concept formation, Random Utility Model, and navigation
Pirolli, P. (2002).
Cognitive Science. [DOC]
|
Stochastic Models of Information Foraging by Information Scent
Theory for predicting user flow and inferring user goals on the WWW and similar systems
Pirolli, P. (2002).
ACM Transactions on Computer Human Interaction. [DOC]
|
A theory of information scent
Briefly summarize the new/novel technical contribution:
A thoery of how users process proximal cues to make navigation decision and develop conceptual representations of information sources.
Pirolli, P. (2002).
10th International Conference on Human Computer Interaction 2003. [DOC]
|
Adaptive Information Interaction
No Abstract Available
Pirolli, P. (2002).
ONR Workshop. [PPT]
|
A User-Tracing Architecture for Modeling Interaction with the World Wide Web
No Abstract Available
Pirolli, P., Fu, W.-T., Reeder, R. and Card, S. K. (2002).
Advanced Visual Interfaces, Trento, Italy. [PDF]
|
Using Information Scent to Model User Information Needs and Actions on the Web
On the Web, users typically forage for information by
navigating from page to page along Web links. Their surfing
patterns or actions are guided by their information needs.
Researchers need tools to explore the complex interactions
between user needs, user actions, and the structures and
contents of the Web. In this paper, we describe two
computational methods for understanding the relationship
between user needs and user actions. First, for a particular
pattern of surfing, we seek to infer the associated information
need. Second, given an information need, and some pages as
starting points, we attempt to predict the expected surfing
patterns. The algorithms use a concept called “information
scent”, which is the subjective sense of value and cost of
accessing a page based on perceptual cues. We present an
empirical evaluation of these two algorithms, and show their
effectiveness.
Chi, E. H., Pirolli, P., Chen, K. and Pitkow, J. (2001).
ACM CHI 2001 Conference on Human Factors in Computing Systems, Seattle, WA 490--497. [PDF]
|
WebEyeMapper and WebLogger: Tools for Analyzing Eye Tracking Data Collected in Web-use Studies
Eye trackers output a stream of points at which the eye was looking. To give these points meaning, researchers analyzing eye tracking data need to map the points onto the meaningful objects at which the eye was looking. Performing this mapping has proven to be a tedious, time-consuming task. We present a software system that automates this task for Web usability studies that incorporate eye tracking.
Reeder, R. W., Pirolli, P. and Card, S. K. (2001).
CHI 2001, Seattle. [PDF]
|
Using Thumbnails to Search the Web
We introduce a technique for creating novel, textually-enhanced thumbnails of Web pages. These thumbnails combine the advantages of image thumbnails and text summaries to provide consistent performance on a variety of tasks. We conducted a study in which participants used three different types of summaries (enhanced thumbnails, plain thumbnails, and text summaries) to search Web pages to find several different types of information. Participants took an average of 67, 86, and 95 seconds to find the answer with enhanced thumbnails, plain thumbnails, and text summaries, respectively. We found a strong effect of question category. For some questions, text outperformed plain thumbnails, while for other questions, plain thumbnails outperformed text. Enhanced thumbnails (which combine the features of text summaries and plain thumbnails) were more consistent than either text summaries or plain thumbnails, having for all categories the best performance or performance that was statistically indistinguishable from the best.
Woodruff, A., Faulring, A., Rosenholtz, R., Morrison, J. and Pirolli, P. (2001).
CHI 2001, Seattle. [PDF]
|
A Taxonomic Analysis of What World Wide Web Activities Significantly Impact People's Decisions and Actions
In this paper, we present three taxonomic classification schemes based on Web users' responses to what Web activities significantly impacted their decisions and actions. The taxonomic classifications focus on three variables: the Purpose of people's search on the Web, the Method people use to find information, and the Content of the information for which they are searching. These taxonomies are useful for understanding people's activity on the Web and for developing ecologically-valid tasks to be used when studying web behavior.
Morrison, J. B., Pirolli, P. and Card, S. K. (2001).
CHI 2001, Seattle. [PDF]
|
Visual Information Foraging in a Focus+Context Visualization
Eye tracking studies of the Hyperbolic Tree browser suggest that visual search in focus+context displays is highly affected by information scent (i.e., local cues, such as text summaries, used to assess and navigate towards distal information sources). When users detected a strong information scent, they were able to reach their goal faster with the Hyperbolic Tree browser than with a conventional browser. When users detected a weak scent or no scent, users exhibited less efficient search of areas with a high density of visual items. In order to interpret these results we present an integration of the CODE Theory of Visual Attention (CTVA) with information foraging theory. Development of the CTVA-foraging theory could lead to deeper analysis of interaction with visual displays of content, such as the World Wide Web or information visualizations.
Pirolli, P., Card, S. K. and Van Der Wege, M. (2001).
CHI 2001, Seattle. [PDF]
|
Information Scent as a Driver of Web Behavior Graphs: Results of a Protocol Analysis Method for Web Usability
The purpose of this paper is to introduce a replicable WWW protocol analysis methodology illustrated by application to data collected in the laboratory. The methodology uses instrumentation to obtain detailed recordings of user actions with a browser, caches Web pages encountered, and videotapes talk-aloud protocols. We apply the current form of the method to the analysis of eight Web protocols, visualizing the structure of the interaction and showing the strong effect of information scent in determining the path followed.
Card, S. K., Pirolli, P., Van Der Wege, M., Morrison, J., Reeder, R. W., Schraedley, P. and Boshart, J. (2001).
CHI 2001, Seattle. [PDF]
|
WebLogger: A Data Collection Tool for Web-use Studies
Considering the amount of interest in studying Web-browsing behavior, there is a relative lack of tools for data collection in this area. Those tools that do exist have significant limitations on the data they are able to collect or on their suitability for efficient analysis. We present WebLogger, a tool which instruments Microsoft's Internet Explorer Web browser. We have found that WebLogger alleviates some of the problems associated with other approaches to browser-based data collection methods.
Reeder, R. W., Pirolli, P. and Card, S. K. (2000).
Xerox PARC, Palo Alto, CA. [PDF]
|
The Effect of Information Scent on Searching Information Visualizations of Large Tree Structures
Focus + context information visualizations have sought to amplify human cognition by increasing the amount of information immediately available to the user. We study how the focus + context distortion of the Hyperbolic Tree browser affects information foraging behavior in a task similar to the CHI '97 Browse Off. In comparison to a more conventional browser, Hyperbolic users searched more nodes, searched at a faster rate, and showed more learning. However, the performance of the Hyperbolic was found to be highly affected by "information scent", proximal cues to the value of distal information. Strong information scent made hyperbolic search faster than with a conventional browser. Conversely, weak scent put the hyperbolic tree at a disadvantage. There appears to be two countervailing processes affecting visual attention in these displays: strong information scent expands the spotlight of attention whereas crowding of targets in the compressed region of the Hyperbolic narrows it. The results suggest design improvements.
Pirolli, P., Card, S. K. and Van Der Wege, M. (2000).
AVI 2000, Palermo, Italy. [PDF]
|
The Scent of a Site: A System for Analyzing and Predicting Information Scent, Usage, and Usability of a Web Site
Designers and researchers of users' interactions with the World Wide Web need tools that permit the rapid exploration of hypotheses about complex interactions of user goals, user behaviors, and Web site designs. We present an architecture and system for the analysis and prediction of user behavior and Web site usability. The system integrates research on human information foraging theory, a reference model of information visualization and Web data-mining techniques. The system also incorporates new methods of Web site visualization (Dome Tree, Usage Based Layouts), a new predictive modeling technique for Web site use (Web User Flow by Information Scent, WUFIS), and new Web usability metrics.
Chi, E. H., Pirolli, P. and Pitkow, J. (2000).
CHI 2000, The Hague, The Netherlands 161-168. [PDF]
|
Mining longest repeated subsequences to predict World Wide Web surfing
Modeling and predicting user surfing paths involves tradeoffs between model complexity and predictive accuracy. In this paper we explore predictive modeling techniques that appempt ot reduce model complexity while
retaining predictive accuracy. We show that compared to various Markov models, longest repeating subsequence
models are able to significantly reduce model size while retaining the ability to make accurate predictions. Sharp
increases in the overall predictive capabilities of these models are achievable by models increases to the size of the number of predictions made.
Pitkow, J. E. and Pirolli, P. (1999).
Second USENIX Symposium on Internet Technologies and Systems. [PDF]
|
Information Foraging
Information Foraging Theory is an approach to understanding how strategies and technologies for information seeking, gathering and consumption are adapted to the flux of information in the environment. The theory assumes that people, when possible, will modify their strategies or the structure of the environment to maximize their rate of gaining valuable information. Field studies inform the theory by illustrating that people do freely structure their environments and their strategies to yield high gains in information foraging. The theory is developed by (a) adaptation (rational) analysis of information foraging problems and (b) a detailed process model (ACT-IF). The adaptation analysis develops (a) information patch models, which deal with time allocation and information filtering and enrichment activities in environments in which information is encountered in clusters (e.g. bibliographic collections), (b) information scent models which address the identification of information value from proximal cues, and (c) information diet models which address decisions about the selection and pursuit of information items. ACT-IF is developed to instantiate these rational models and to fit the moment-by-moment behavior of people interacting with complex information technology. ACT-IF is a production system in which the information scent of bibliographic stimuli is calculated by spreading activation mechanisms. Time allocation and item selection heuristics make use of information sent to select production rules in ways that maximize information foraging activities.
Pirolli, P. and Card, S. K. (1999).
Psychological Review 106(4): 643-675. [PDF]
|
Cognitive Architectures and Cognitive Engineering Models in Human-Computer Interaction
(First paragraph) We engage our physical and social environments through highly evolved technologies, in interactions that often require sophisticated knowledge and virtuoso performance. Finding the order underlying the complexity of these interactions may be one of the most daunting challenges facing science. Over the past few decades, the study of human-computer interaction (HCI) has become an increasingly important arena for pursuing this challenge. HCI is a discipline concerned with the study and design of interactive computing systems used by people towards satisfying their goals. HCI has become an arena in which new computer applications can benefit from new cognitive engineering models that synthesize results from sound cognitive science. It has also become a useful testbed for cognitive architectures, which are integrated theories of psychological mechanisms that aim to predict complex learning, cognition, and performance. This chapter provides an overview of cognitive engineering models and cognitive architectures in the context of HCI.
Pirolli, P. (1999).
The handbook of applied cognition. John T. Wiley, Sussex, England. [PDF]
|
Distributions of Surfers' Paths Through the World Wide Web: Empirical Characterization
Surfing the World Wide Web (WWW) involves traversing hyperlink connections among documents. The ability to predict surfing patterns could solve many problems facing producers and consumers of WWW content. We analyzed WWW server logs for a WWW site, collected over ten days, to compare different path reconstruction methods and to investigate how past surfing behavior predicts future surfing choices. Since log files do not explicitly contain user paths, various methods have evolved to reconstruct user paths. Session times, number of clicks per visit, and Levenshtein Distance analyses were performed to show the impact of various reconstruction methods. Different methods for measuring surfing patterns were also compared. Markov model approximations were used to model the probability of users choosing links conditional on past surfing paths. Information-theoretic (entropy) measurements suggest that information is gained by using longer paths to estimate the conditional probability of link choice given surf path. The improvements diminish, however, as one increases the length of path beyond one. Information-theoretic (Total Divergence to the Average entropy) measurements suggest that the conditional probabilities of link choice given surf path are more stable over time for shorter paths than longer paths. Direct examination of the accuracy of the conditional probability models in predicting test data also suggests that shorter paths yield more stable models and can be estimated reliably with less data than longer paths.
Pirolli, P. and Pitkow, J. E. (1999).
World Wide Web 2(1-2): 29-45. [PDF]
|
A Theory of the Measurement of Knowledge Content, Access, and Learning
We develop an approach to the measurement of knowledge content, knowledge access and knowledge learning. This approach has two elements: First we describe a theoretical view of cognition, called the Newell-Dennett framework, which we see as being particularly favorable to the development of a measurement approach. Then, we describe a class of measurement models, based on Rasch modeling, which we see as being particularly favorable to the development of cognitive theories. Knowledge content and access are viewed as determining the observable actions selected by an agent in order to achieve desired goals in observable situations. To the degree that models within the theory fit the data at hand, one considers measures of observed behavior to be manifestations of intelligent agents having specific classes of knowledge content and varying degrees of access to that knowledge. Although agents, environment, and knowledge are constitutively defined (in terms of one another), successful application of our theory affords separation of parameters associated with the person from those associated with the environment. We present and discuss two examples of measurement models developed within our approach that address the evolution of cognitive skill, strategy choice and application, and developmental changes in mixtures of strategy use.
Pirolli, P. and Wilson, M. (1998).
Psychological Review 105: 58-82. [PDF]
|
Information Foraging Models of Browsers for Very Large Document Spaces
Information Foraging (IF) Theory addresses user strategies and technology for seeking, gathering, and using on-line information. We present IF-based models and evaluations of two interfaces: the Scatter/Gather browser for large document collections, and the Butterfly interface for surfing the citation link structure of scientific literatures. A computational cognitive model, ACT-IF, models observed users by assuming that they have heuristics that optimize their information foraging behavior in accordance with IF theory.
Pirolli, P. and Card, S. K. (1998).
AVI '98, L'Aquilla, Italy 83-93. [PDF]
|
Exploring Browser Design Trade-Offs Using a Dynamical Model of Optimal Information Foraging
Designers and researchers of human-computer interaction need tools that permit the rapid exploration and management of hypotheses about complex interactions of designs, task conditions, and user strategies. Dynamic programming is introduced as a such a tool for the analysis of information foraging technologies. The technique is illustrated in the context of the Scatter/Gather text clustering browser. Hypothetical improvements in browser speed and text clustering are examined in the context of variations in task deadlines and the quality of the document repository. A complex and non-intuitive set of tradeoffs emerge from even this simple space of factors, illustrating the general utility of the approach.
Pirolli, P. (1998).
Conference on Human Factors in Computing Systems, CHI '98, Los Angeles 33-40. [PDF]
|
Strong Regularities in World Wide Web Surfing
One of the most common modes of accessing information in the World Wide Web is surfing from one document to another along hyperlinks. Several large empirical studies have revealed common patterns of surfing behavior. A model that assumes that users make a sequence of decisions to proceed to another page, continuing as long as the value of the current page exceeds some threshold, yields the probability distribution for the number of pages that a user visits within a given Web site. This model was verified by comparing its predictions with detailed measurements of surfing patterns. The model also explains the observed Zipf-like distributions in page hits observed at Web sites.
Huberman, B. A., Pirolli, P., Pitkow, J. and Lukose, R. J. (1998).
Science 280: 95-97. [PDF]
|
Visualizing the Evolution of Web Ecologies
Several visualizations have emerged which attempt to visualize all or part of the World Wide Web. Those visualizations, however, fail to present the dynamically changing ecology of users and documents on the Web. We present new techniques for Web Ecology and Evolution Visualization (WEEV). Disk Trees represent a discrete time slice of the Web ecology. A collection of Disk Tress forms a Time Tube, representing the evolution of the Web over longer periods of time. These visualizations are intended to aid authors and webmasters with the production and organization of content, assist Web surfers making sense of information, and help researchers understand the Web.
Chi, E. H., Pitkow, J., Mackinlay, J., Pirolli, P., Gossweiler, R. and Card, S. K. (1998).
ACM Conference on Human Factors in Software (CHI '98), Los Angeles 400-407, 644-645. [PDF]
|
The Evolutionary Ecology of Information Foraging
We present Information Foraging Theory as an approach to understanding how strategies and technologies for information seeking, gathering, and consumption are adapted to the flux of information in the cultural environment. The theory is developed within an evolutionary-ecological framework that includes analysis of adaptation, knowledge, and cognition. The theory is applied to field studies, controlled experiments, and technology design. We present the Information Diet Model and Information Patch Residence Time Model as optimization models of information foraging under some strong constraints. These are used to develop a specific production system model called ACT-IF that predicts the fine-grained information seeking and gathering behavior of participants using a sophisticated document browsing system. We also present the Overlapped Patch Foraging with Queueing Model to address situations in which information search and information handling may occur in parallel, the Extreme Variance Rule which deals with information foraging under deadlines and uncertainty, a general class of Dynamic Information Foraging Models, and the Hogg-Huberman Model of the phase space of cost functions for heuristic information search.
Pirolli, P. and Card, S. (1997).
Xerox PARC, Palo Alto, CA. [PDF]
|
Silk from a Sow's Ear: Extracting Usable Structures from the Web
In its current implementation, the World Wide Web lacks much of the explicit structure and strong typing found in many closed hypertext systems. While this property probably relates to the explosive acceptance of the Web, it further complicates the already difficult problem of identifying usable structures and aggregates in large hypertext collections. These reduced structures, or localities, form the basis for simplifying visualizations of and navigation through complex hypertext systems. Much of the previous research into identifying aggregates utilize graph theoretic algorithms based upon structural typology, i.e. the linkage between items. Other research has focused on content analysis to form document collections. This paper presents our exploration into techniques that utilize both the topology and textual similarity between items as well as usage data collected by servers and page meta-information like title and size. Linear equations and spreading activation models are employed to arrange Web pages based upon functional categories, node types and relevancy.
Pirolli, P., Pitkow, J. and Rao, R. (1996).
Conference on Human Factors in Computing Systems, CHI '96, Vancouver, Canada. [PDF]
|
Scatter/Gather Browsing Communicates the Topic Structure of a Very Large Text Collection
Scatter/Gather is a cluster-based browsing technique for large text collections. Users are presented with automatically computed summaries of the contents of clusters of similar documents and provided with a method for navigating through these summaries at different levels of granularity. The aim of the technique is to communicate information about the topic structure of very large collections. We tested the effectiveness of Scatter/Gather as a simple pure document retrieval tool, and studied its effects on the incidental learning of topic structure. When compared to interactions involving simple keyword-based search, the results suggest that Scatter/Gather induces a more coherent conceptual image of a text collection, a richer vocabulary for constructing search queries, and communicates the distribution of relevant documents over clusters of documents in the collection.
Pirolli, P., Schank, P., Hearst, M. and Diehl, C. (1996).
Conference on Human Factors in Computing Systems (CHI '96), Vancouver, BC. [PDF]
|
Measuring Learning in LISP: An Application of the Random Coefficients Multinomial Logic Model
No Abstract Available
Draney, K. L., Wilson, M. and Pirolli, P. (1996).
Objective measurement: Theory in practice (Vol. 3): 195-218.
|
Table Lens as a Tool for Making Sense of Data
The Table Lens is a visualization for searching for patterns and outliers in multivariate datasets. It supports a lightweight form of exploratory data analysis (EDA) by integrating a familiar organization, the table, with graphical representations and a small set of direct manipulation operators. We examine the EDA process as a special case of a generic process, which we call sensemaking. Using a GOMS methodology, we characterize a few central EDA tasks and compare performance of the Table Lens and one of the best of the more traditional graphical tools for EDA i.e. Splus. This analysis reveals that Table Lens is more or less on par with the power of Splus, while requiring the use of fewer specialized graphical representations. It essentially combines the graphical power of Splus with the direct manipulation and generic properties of spreadsheets and relational database front ends. We also propose a number of design refinements that are suggested by our task characterizations and analyses.
Pirolli, P. and Rao, R. (1996).
Advanced Visual Interfaces Workshop, AVI '96, Gubbio, Italy. [PDF]
|
The Knowledge Crystallization Task: Information Foraging in the Wild
No Abstract Available
Pirolli, P., Card, S. K. and Gigch, J. P. v. (1996).
Xerox PARC, Palo Alto, CA.
|
Life, Death, and Lawfulness on the Electronic Frontier
To facilitate users' ability to make sense of large collections of hypertext we present two new techniques for inducing clusters of related documents on the World Wide Web. Users' ability to find relevant information might also be enhanced by finding lawful properties of document behavior and use. We present models and analyses of document use and change for the World Wide Web.
Pitkow, J. and Pirolli, P. (1997).
Conference on Human Factors in Computing Systems, CHI '97, Atlanta, GA 383-390. [PDF]
|
Computational Models of Information Scent-Following in a very Large Browsable Text Collection
An ecological-cognitive framework of analysis and a model-tracing architecture are presented and used in the analysis of data recorded from users browsing a large document collection. The users interacted with the Scatter/Gather browser, which clusters documents into groups of similar content and presents users with summaries of cluster content. Predictions made by a computational model of navigation and information foraging are matched against the observed activity.
Pirolli, P. (1997).
Conference on Human Factors in Computing Systems, CHI '97, Atlanta, GA 3-10. [PDF]
|
A Measurement Model for a Complex Cognitive Skill
No Abstract Available
Draney, K., Pirolli, P. and Wilson, M. (1995).
Cognitively diagnostic assessment. Lawrence Erlbaum Associates, Hillsdale, NJ: 103-125.
|
The Relationship between the Rasch Model and Conjoint Measurement Structures
No Abstract Available
Wilson, M. and Pirolli, P. (1995).
University of California, Berkeley, CA.
|
Information Foraging and Sensemaking in Strategic Management: An Analysis of an in-situ Study
No Abstract Available
Pirolli, P. and van Gigch, J. P. (1995).
Xerox PARC, Palo Alto, CA.
|
Information Foraging in Information Access Environments
Information foraging theory is an approach to the analysis of human activities involving information access technologies. The theory derives from optimal foraging theory in biology and anthropology, which analyzes the adaptive value of food-foraging strategies. Information foraging theory analyzes trade-offs in the value of information gained against the costs of performing activity in human-computer interaction tasks. The theory is illustrated by application to information-seeking tasks involving a Scatter/Gather interface, which present users with a navigable, automatically computer overview of the contents of a document collection arranged as a cluster hierarchy.
Pirolli, P. and Card, S. K. (1995).
ACM Conference on Human Factors in Software (CHI '95), Denver, Colorado 51–58. [PDF]
|
A Focus + Context Technique Based on Hyperbolic Geometry for Visualizing Large Hierarchies
We present a new focus+context (fisheye) technique for visualizing and manipulating large hierarchies. Our technique assigns more display space to a portion of the hierarchy while still embedding it in the context of the entire hierarchy. The essence of this scheme is to lay out the hierarchy in a uniform way on a hyperbolic plane and map this plane onto a circular display region. This supports a smooth blending between focus and context, as well as continuous redirection of the focus. We have developed effective procedures for manipulating the focus using pointer clicks as well as interactive dragging, and
for smoothly animating transitions across such manipulation. A laboratory experiment comparing the hyperbolic browser with a conventional hierarchy browser was conducted.
Lamping, J., Rao, R. and Pirolli, P. (1995).
ACM Conference on Human Factors in Computing Systems (CHI '95), Denver, Colorado. [PDF]
|
Training in Self-Explanation and Self-Regulation Strategies: Investigating the Effects of Knowledge Acquisition Activities on Problem Solving
No Abstract Available
Bielaczyc, K., Pirolli, P. and Brown, A. L. (1995).
Cognition and Instruction 13: 221-252.
|
Modelling Individual Differences in Students' Learning Strategies
No Abstract Available
Recker, M. and Pirolli, P. (1994).
Journal of the Learning Sciences 4: 1-38.
|
Learning Strategies and Transfer in the Domain of Programming
No Abstract Available
Pirolli, P. and Recker, M. (1994).
Cognition and Instruction 12: 235-275. [PDF]
|
The Cost-of-Knowledge Characteristic Function: Display Evaluation for Direct Walk Information Visualizations
In this paper we present a method, the Cost-of-Knowledge Characteristic Function, for characterizing information access from dynamic displays. The paper works out this method for a simple, but important, class of dynamic displays called direct-walk interactive information visualizations, in which information is accessed through a sequence of mouse selections and key selections. The method is used to characterize a simple calendar task for an application of the Information Visualizer, to compute the changes in characterization as the result of possible program variants, and to conduct empirical comparison between different systems with the same function.
Card, S. K., Pirolli, P. and Mackinlay, J. (1994).
CHI '94 Conference on Human Factors in Computing Systems, Boston, MA 238-244.
|
Collaborative Explanations and Metacognition: Identifying Successful Learning Activities in the Acquisition of Cognitive Skills
No Abstract Available
Bielaczyc, K., Pirolli, P. and Brown, A. (1994).
Sixteenth Annual Conference of the Cognitive Science Society, Atlanta, GA 39-44.
|
The Cost Structure of Sensemaking
Making sense of a body of data is a common activity in any kind of analysis. Sensemaking is the process of searching for a representation and encoding data in that representation to answer task-specific questions. Different operations during sensemaking require different cognitive and external resources. Representation are chosen and changed to reduce the cost of operations in an information processing task. The power of theses representational shifts is generally under-appreciated as is the relation between sensemaking and information retrieval. We analyze sensemaking tasks and develop a model of the cost structure of sensemaking. We discuss implications for the integrated design of user interfaces, representational tools, and information retrieval systems.
Russell, D. M., Stefik, M. J., Pirolli, P. and Card, S. K. (1993).
INTERCHI '93 Conference on Human Factors in Computing Systems, Amsterdam 269-276. [PDF]
|
Knowledge and the Simultaneous Conjoint Measurement of Activity, Agents, and Situations
No Abstract Available
Pirolli, P. and Wilson, M. (1993).
Annual Meeting of the Cognitive Science Society, Boulder, CO.
|
Student Strategies for Learning Programming from a Computational Environment
No Abstract Available
Recker, M. and Pirolli, P. (1992).
Intelligent Tutoring Systems. Springer-Verlag, Berlin: 382-394.
|
Explanation, Analogy and Transfer in an Intelligent Tutoring System Paradigm
No Abstract Available
Pirolli, P. (1992). . Taylor and Francis, New York: 65-84.
|
Measuring Learning Strategies and Understanding: A Research Framework
No Abstract Available
Pirolli, P. and Wilson, M. (1992).
Intelligent Tutoring Systems. Springer-Verlag, Berlin: 539-558.
|
Generative Knowledge and the Development of Situated Know-How in Intelligent Tutoring Stems: Implications for Student Modelling
No Abstract Available
Pirolli, P. (1992).
Cognitive modelling and interactive environments in language learning. Springer-Verlag, Berlin: 43-51.
|
Towards a Unified Model of Learning to Program
No Abstract Available
Pirolli, P. (1992).
NATO Advanced Research Workshop: Cognitive models and Intelligent Environments for Learning Programming, Santa Margherita, Italy 13-25.
|
The Structure of Design Problem Spaces
No Abstract Available
Goel, V. and Pirolli, P. (1992).
Cognitive Science 16: 395-429.
|
Computer-Aided Instructional Design
No Abstract Available
Pirolli, P. (1991).
Intelligent tutoring systems: Evolutions in design. Lawrence Erlbaum: 105-125.
|
Mind Bugs: The Origins of Procedural Misconceptions (book review)
No Abstract Available
Pirolli, P. (1991).
Artificial Intelligence 52: 329-340.
|
Effects of Examples and their Explanations in a Lesson on Recursion: A Production System Analysis
No Abstract Available
Pirolli, P. (1991).
Cognition and Instruction 8: 207-259.
|
The Instructional Design Environment: Technology to Support Design Problem Solving
No Abstract Available
Pirolli, P. and Russell, D. (1990).
Instructional Science Kluwer Academic Publishers, Dordrecht
Printed in the Netherlands 19: 121-144.
|
You Can't Get There from Here: Comments on R. W. Sperry's Resolution of Science and Ethics
No Abstract Available
Pirolli, P. and Goel, V. (1990).
American Psychologist 45: 71-73.
|
The Explanation of Programming Examples
No Abstract Available
Pirolli, P. and Recker, M. (1989).
Workshop on Models of Complex Human Learning, Cornell University, Ithaca, NY.
|
Empirical Analyses of Self-Explanation and Transfer in Learning to Program
No Abstract Available
Pirolli, P. and Bielaczyc, K. (1989).
Proceedings of the Annual Conference of the Cognitive Science Society, Hillsdale, NJ 450-457.
|
Strategies and Mechanisms for the Construction and Refinement of Programming Knowledge: A Unified Computational Model of Learning
No Abstract Available
Pirolli, P. (1989).
Army Research Institute.
|
On the Art of Building: Putting a New Instructional Design into Practice
No Abstract Available
Pirolli, P. (1989).
Proceedings of the 2nd Intelligent Tutoring Systems Research Forum 129-141.
|
Motivating the Notion of Generic Design Within Information-Processing Theory: The Design Problem Space
No Abstract Available
Goel, V. and Pirolli, P. (1989).
AI Magazine 10: 19-36.
|
The Problem Space of Instructional Design
No Abstract Available
Pirolli, P. L. and Greeno, J. G. (1988).
Intelligent Tutoring Systems: Lessons Learned. Lawrence Erlbaum, Hillsdale, NJ: 181-201.
|
Learning to Program Recursive Functions
No Abstract Available
Anderson, J. R., Pirolli, P. L. and Farrell, R. (1988).
The Nature of Expertise. Lawrence Erlbaum, Hillsdale, NJ: 153-183.
|
Towards Theory and Technology for the Design of Intelligent Tutoring Systems
No Abstract Available
Pirolli, P. and Russell, D. (1988).
Proceedings of the International Conference on Intelligent Tutoring Systems 350-356.
|
A Model of Purpose-Driven Analogy and Skill Acquisition in Programming
No Abstract Available
Pirolli, P. (1987).
Proceedings of the Tenth Annual Conference of the Cognitive Science Society 609-621.
|
Intelligent Tutoring Systems Technology: An Interface Between Instructional Theory and Practice
No Abstract Available
Pirolli, P. (1987).
Technology and Education 1: 2-4.
|
A Cognitive Model and Computer Tutor for Programming Recursion
No Abstract Available
Pirolli, P. (1986).
Human-Computer Interaction 2: 319-355.
|
The Role of Practice in Fact Retrieval
No Abstract Available
Pirolli, P. L. and Anderson, J. R. (1985).
Journal of Experimental Psychology: Learning, Memory, and Cognition 11: 136-153.
|
The Role of Learning from Examples in the Acquisition of Recursive Programming Skills
No Abstract Available
Pirolli, P. L. and Anderson, J. R. (1985).
Canadian Journal of Psychology 39: 240-272.
|
The Effect of Leading Questions on Prior Memory: Evidence for the Coexistence of Inconsistent Memory Traces
No Abstract Available
Pirolli, P. L. and Mitterer, J. O. (1984).
Canadian Journal of Psychology 38: 135-141.
|
Learning to Program Recursion
No Abstract Available
Pirolli, P. L., Anderson, J. R. and Farrell, R. (1984).
Proceedings of the Sixth Annual Conference of the Cognitive Science Society, Hillsdale, NJ 277-280.
|
Spread of Activation
No Abstract Available
Anderson, J. R. and Pirolli, P. L. (1984).
Journal of Experimental Psychology: Learning, Memory, and Cognition 10: 791-798.
|
SIMPLE: a Simple Language for Research in Programmer Psychology
No Abstract Available
Shrager, J. and Pirolli, P. L. (1983).
Carnegie-Mellon University, Department of Psychology, Pittsburgh, PA.
|
|
|
|