Xerox PARC, 3333 Coyote Hill Rd., Palo Alto CA 94304
E-mail: {stone, fishkin, bier}@parc.xerox.com
CR Categories and Subject Descriptors: I.3.6 [Computer Graphics]: Methodology and Techniques - interaction techniques; H.5.2 [Information Interfaces and Presentation]: User Interfaces - interaction styles; I.3.3 [Computer Graphics]: Picture/Image Generation - viewing algorithms; I.3.4 [Computer Graphics]: Graphics Utilities - graphics editors
Key Words: viewing filter, lens, transparent, visualization, editing, macro, graphics
Magic lens filters have a number of potential advantages over traditional methods of generating alternate views and filtering information. Binding the filter to a spatially bounded, movable region creates an easily understood user model based on experience with physical lenses. Limiting the view to a local region preserves context and can reduce clutter. Lenses can be parameterized, and can have arbitrary shape. The user can position different filters simultaneously over different parts of the displayed information to get multiple, simultaneous views. Lenses that overlap compose their effects, making it easy to create visual macros. These macros can be temporary, or can be "welded together" to create a compound lens that encapsulates a set of filters and parameters. Finally, the magic lens metaphor can be used uniformly across applications.
Magic lens filters perform a wide variety of operations that are currently performed by other user interface techniques, including: modifying the way a picture or solid model is rendered on the screen (e.g., wireframe versus shaded), querying application state (e.g., what font is this?), querying a database, activating alignment aids (e.g., turn on the grid), or identifying a region of interest (e.g., identifying the rectangular region of a VLSI drawing to work on next). By unifying these operations into a single technique, magic lens filters can reduce the time needed for users to learn to use a system and increase the consistency of the user interface. Users will more quickly learn to use new applications because familiar lenses can be applied to them. Conversely, programmers can add functionality to many applications at once by implementing a new filter.
The lens metaphor provides a rich set of useful interface tools. We have only begun to explore this area. Two exploratory implementations exist within the Cedar programming environment [20], which runs on SunOS; one focuses on graphical editing, using the Multi-Device Multi-User Multi-Editor (MMM) framework [6], and the other uses lenses for visualizing structure in text documents. A third implementation focused on visualizing maps is implemented in C++ in the X window system [16].
After describing the basic principles of operation of lenses, we describe in more detail each of the advantages their use provides, illustrated with examples. Further sections then describe the implementation, discuss related work, and present our conclusions and plans for future work.
Overlapping two lenses composes their functions in bottom to top order. There are several ways to implement composition [5]. For this paper, we assume that lenses compose by copying and modifying a sequence of models. The model output by the lower lens is the input to the upper lens.
In this context, model is defined quite broadly, including both data structures and application state. Furthermore, a model may incorporate a variety of data types from a variety of applications. For example, if a lens lies atop three other lenses and two applications, it receives input from five sources.
Many types of lenses are more effective if they can be parameterized by the user. For example, a magnification lens may have a slider along its edge, through which the user may change the magnification factor.
To illustrate the operation of magic lens filters, figure 1 shows two lenses for viewing a text document where each paragraph and character can have attached properties that affect the documents' appearance. Therefore, a specific appearance can be generated in a number of ways. For example, the word "Selection" may appear in a bold typeface because the paragraph style indicates it, or because there are "boldface" properties attached to the characters. In the figure, the upper lens boxes all characters that have properties attached, making it easy to see that the word "Selection" has none, unlike the words "MIDDLE" and "RIGHT" below. The lower lens marks whitespace with its underlying representation. Spaces are marked with small dots while tabs are marked with small arrows, similar to the global mode in Microsoft Word.(TM) Applying this operation locally reduces the visual clutter. The lenses overlap, showing composition of their effects.
Figure 1. The upper lens highlights text with properties and the lower lens shows the representation of whitespace.
Most of the illustrations in this section are snapshots of actual implemented examples, although the map examples have been touched-up slightly with a graphics editor to improve the presentation. Figures 1, 3, 4 and 15 are illustrations of straightforward extensions to the existing text and graphical editing implementations. The examples in figures 9 and 16 require access to external databases. We have not implemented this type of tool, although we believe they would be straightforward to produce.
Figure 2. A lens that provides a more detailed roadmap.
Figure 3. Labels to identify the font in a text document. The underlying text is from figure 1.
Figure 4 also shows the advantage of a locally bounded filter. This lens was designed to visualize the gravity algorithm used in snap-dragging [7]. Each dot indicates a potential cursor position. The line from the dot to the underlying shape indicates the point that would be selected using gravity. Applying this filter globally would be computationally expensive and would lose any context provided by the area colors of shapes.
Figure 4. Gravity algorithm debugging lens. Lines show where cursor positions will snap.
Other clutter-reducing lenses are magnification lenses or fisheye lenses that increase the amount of detail as well as increasing the size of the view. Such a lens could control the zooming feature of Pad [15] that makes it possible, for example, to read notes written "between the lines" of a document. Another variation would adjust the level of detail to control access to sensitive information.
Multiple lenses need not be of the same type. For example, a user who wishes to copy an object from one part of a document to another could have a magnification lens over the current object location to help in selection, and a grid lens over the destination location to help in placement.
Figure 5. Lenses showing detail in two different locations.
We have also experimented with shaped lenses for artistic effects. Figure 6 shows the words "Magic Lenses" overlaid by a magnification lens and a lens that shows a wireframe view. In this situation, the lenses are intended to remain as a permanent part of the scene. However, the fact that the illustration is constructed of text plus two lenses makes it easier to edit the text than if the text geometry had been modified to produce the lens effects.
Figure 6. The Magic Lenses logo.
Lenses can be included as a semi-permanent part of the presentation in domains other than graphics. One common use of data visualization systems is to make illustrations of the data (e.g. exploded or wire-frame views). Carefully shaped lenses can be used to create these illustrations. These illustrations can be displayed statically (e.g. printed), or presented interactively.
Magic lens filters can also be used as a debugging tool for programs that display data by displaying a textual description of the data. The text may be either the original representation or an intermediate form designed for debugging. Figure 7 shows a lens that displays the description of a selected segment of road.
Figure 7. The lens shows the text tags for the indicated segment of road.
In applications such as astronomy, medical imaging, comparative cartography and structural analysis, multiple images of the same object are compared to evaluate different characteristics of the object. Magic lens filters can be used to provide coordinated views of these images. Figure 8 shows a pseudo-color view indicating the sign and relative magnitude of curvature in the context of a shaded view of a 3D object.
Figure 8. Gaussian curvature pseudo-color lens. (Original images courtesy of Steve Mann)
Another example of alternate views that benefit by being shown in context is views that display temporal changes. For example, a lens could be used to show a region of a map as a function of time, making it easy to compare the current geography with that of ten years ago. Similarly, a lens could be used to show previous versions of text in an edited document. Lenses could be placed over a video frame in a video exploration application [12], each showing part of that frame at some temporal displacement.
An alternate view may provide a link to a separate but related object in a different model. For example, figure 9 shows a lens that displays the definition of the word selected through it. The lens starts out transparent and at any convenient size. The user clicks through the lens to select a word. The lens then resizes itself to display the definition underneath the line containing the selection, as shown in the figure. The original line of text is visible, but the text under the definition is obscured.
Figure 9. A lens displaying the definition of a word selected through it.
A similar lens positioned over a function call could display the function definition, or the function code itself. One positioned over a bibliographic reference could show the full reference. In a hypertext system [14], lenses could be used to highlight links and enable their operation. This provides a solution to the visual clutter caused by making the links always visible. Lenses could also be used to activate the links. In this case, the lens could display either the attributes or contents of the destination node.
Figure 10. Composing lenses to show waterways (dashed lines) and major roads (bold lines).
Figure 11. (a) An illustration made of layered shapes. (b) The lens shows the shapes in reverse order.
Similarly, the lens in Figure 12 scales objects based on their depth. The effect is to provide a 3D perspective view of their relative positions, enabling the user to see around the objects in front to the ones behind. Because the view changes with the position of the lens, this lens is especially effective in motion. Magic lens filters can also be used to explore true 3D scenes. For example, an exploding parts lens would show an exploded view of the objects viewed through it, or be used in molecular visualization as in the VIEW system [4].
Figure 12. 3D depth filter applied to three overlapping objects. (a) The large rectangle is the filter. (b) The filter moves right, changing the view.
Click-through tools with magic lens filters can be used to select objects that would otherwise be difficult to select. For example, figure 13 shows a lens that locally shrinks each object to separate coincident edges. A user wishing to select one edge of a particular shape can click through this lens to select the edge unambiguously.
Figure 13. The local scaling lens. (Tiling by Doug Wyatt)
Figure 14 shows a tool with three lenses that display grids. Clipping the grids to a lens reduces visual clutter. Combining the three grids into one tool makes it easy to switch between them; the user just moves the tool to apply a different grid.
Figure 14. Three grid lenses.
Figure 15 demonstrates a lens that shows recently deleted objects in a graphical editor: the leftmost shrub, deleted from the scene (a), appears under the lens (b). The user can click through the lens to restore the object to the scene (c). Other examples of lenses and click-through tools for editing have been previously published [5].
Figure 15. The previously deleted bush in front of the house (a) is visible in the lens (b) and can be restored by clicking on it (c).
Filters may also tune their effect based on the specific application in which they are applied. For example, figure 16 illustrates a "highlight schools" lens positioned over two different applications. In the map viewer on the left, school buildings are shown shaded, with bold outlines. In the text viewer on the right, words that refer to schools are shaded.
Figure 16. A "highlight schools" lens (dashed rectangle) over two different applications.
In addition to providing a consistent interface for performing operations in both of these applications, the lenses allow the user to strongly decouple the conceptual effect of an operation from the implementation of that operation. The "highlight schools" operation may be implemented quite differently within the two applications.
Such lenses are invisible to applications that move under them; the application(s) need not be modified. However, this architecture does require the modification of a fundamental part of the system, and lenses of this type cannot perform higher-level semantics. For example, the "highlight major roads" lens of figure 10 could not be implemented as the concept of a "road" is application-specific.
The lens semantics can be implemented in two very different ways. If a standard language exists for reading and writing application objects, then the lens filter can be implemented as a single entity. If it is over two applications, it will receive two data models as input, describing two semantically distinct sets of data, but both expressed in the same format. This is similar to what windowing systems presently require of applications which support clipboards, but lenses with more powerful semantics may require a richer object description than that presently found in clipboards.
If no such standard language exists, then the lens may be represented to the user as a single lens, but implemented as a set of application-specific lenses, one for each application region over which the lens finds itself. For example, in figure 16 (a lens whose left half is over a map viewer, and whose right half is over a text editor), the left half of the lens would be presented as a lens to the map viewer, and the right half of the lens would be presented as a second lens to the text editor. The parts of the lens which are not over an application must be handled specially.
Our current implementations incorporate all of these implementation classes. The Cedar environment has a common address space and data format for all applications and a ubiquitous, easily-modified graphics library, that makes it easy to explore lenses that operate across applications. The Unix/X environment, with its multiple address spaces and standard window manager, has been the testbed for exploring application-specific lenses in the context of a multi-address space window environment.
The Pad system [15] contains the idea of portals, magnifying glasses that provide access to different parts of the Pad information plane. Portals can also contain filters that modify the view of objects below them, making them similar to magic lens filters. Portals provide similar functionality to some of our lenses, but are presented primarily as a metaphor for navigation in Pad as opposed to a general purpose tool for exploring and modifying information.
Other systems provide special-purpose lenses that provide more detailed views in the context. For example, a fisheye lens [9] can enhance the presentation of complicated graphs [17]. The bifocal display provides similar functionality for viewing a large space of documents [19]. The MasPar Profiler uses a tool based on the magnifying lens metaphor to generate more detail (including numerical data) from a graphical display of a program [3]. The VIEW system moves a spherical probe around a molecule, revealing associated state [4].
Magic lens filters have many advantages over application-wide viewing modes. They can show local details in the context of larger-scale information, limit clutter to small regions, apply different filters in different places at the same time, and query features locally. Arbitrarily shaped filters can be used to refine queries, create dynamic illustrations, or produce artistic effects. They provide easy customization through visual macros and a way to explore models without modifying them. When combined with click-through tools, they provide enhanced editing operations including views of hidden objects, a reliable way to select objects, and a visual way to switch modes.
The direct-manipulation metaphor of physical lenses on overlapping layers seems to significantly enhance the usefulness of these local filters. The combination of user control and the animation inherent in the interactive motion helps the user create the context needed to interpret the potentially complex visualizations produced by these tools. While we have no formal evidence for this observation, the anecdotal evidence is interesting. People who have seen static pictures find a videotape of the tools in action much easier to understand. Furthermore, we have personally observed that examples which seem confusing when passively viewed on the videotape are easily understood when we are actively operating the system.
Magic lens filters are an important part of a new see-through user interface paradigm that allows the user to bring tools to the data instead of loading data into tools. This paper describes some of the benefits to the user of including filters in this kind of interface, and presents examples of useful tools in a number of domains including text editing, debugging, graphical editing, and map visualization. We anticipate that these filters will prove useful in many forms and in many applications beyond those described here.
Trademarks and Patents: Magic Lens and See-Through Interface are trademarks of the Xerox Corporation. Postscript is a trademark of Adobe Systems, Inc. UNIX is a trademark of AT&T. TIGER/Line is a trademark of the Bureau of the Census. Patents related to the concepts discussed in this paper have been applied for by the Xerox Corporation.
2. Gideon Avrahami, Kenneth P. Brooks, and Marc H. Brown. A two-view approach to constructing user interfaces. Proceedings of Siggraph '89 (Boston, MA, July), Computer Graphics Annual Conference Series, ACM, 1989, pp. 137-146.
3. Kent Beck, Jon Becher, and Liu Zaide. Integrating profiling into debugging. Proceedings of the 1991 International Conference on Parallel Processing, Vol. II, Software, August 1991, pp. II-284-II-285.
4. Lawrence D. Bergmen, Jane S. Richardson, David C. Richardson, and Frederick P. Brooks, Jr.. VIEW - An exploratory molecular visualization system with user-definable interaction sequences. Proceedings of Siggraph '93 (Anaheim, CA, August), Computer Graphics Annual Conference Series, ACM, 1993, pp. 57-64.
5. Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. Toolglass and Magic Lenses: The See-Through Interface. Proceedings of Siggraph '93 (Anaheim, CA, August), Computer Graphics Annual Conference Series, ACM, 1993, pp. 73-80.
6. Eric A. Bier and Steve Freeman. MMM: a user interface architecture for shared editors on a single screen. Proceedings of the ACM SIGGRAPH Symposium on User Interface Software and Technology (South Carolina, November), ACM, 1991, pp. 79-86.
7. Eric A. Bier and Maureen Stone. Snap-dragging. Proceedings of Siggraph '86 (Dallas, August), Computer Graphics, Vol. 20, No. 4, ACM, 1986, pp. 233-240.
8. Bureau of the Census. TIGER/Line(TM) Census Files, 1990. Washington, 1991.
9. George Furnas. Generalized Fisheye Views. Proceedings of CHI '86, (Boston, MA, April 1986), pp. 16-23.
10. Adele Goldberg and Dave Robson. A Metaphor for User Interface Design, Proceedings of the University of Hawaii Twelfth Annual Symposium on System Sciences, Honolulu, January 4-6, (1979), pp. 148-157.
11. Glenn Krasner and Stephen Hope. A Cookbook for Using the Model-View-Controller User Interface Paradigm in Smalltalk-80, Journal of Object-Oriented Programming, 1, 3, (1988), pp. 26-49.
12. Michael Mills, Jonathen Cohen and Yin Yin Wong. A magnifier tool for video data. Proceedings of CHI '92, (Monterey, CA, May 3-5, 1992) ACM, New York, (1992), pp. 93-98.
13. Greg Nelson. Juno, a constraint-based graphics system. Proceedings of Siggraph '85 Computer Graphics, Vol. 19, No. 3, ACM, 1985, pp. 235-243.
14. Jakob Nielsen. HYPERText & HYPERMedia. Academic Press. 1990.
15. Ken Perlin and David Fox. Pad: an alternative approach to the computer interface. Proceedings of Siggraph '93 (Anaheim, August), Computer Graphics Annual Conference Series, ACM, 1993, pp. 57-64.
16. Robert W. Scheifler, James Gettys, and Ron Newman. X Window System. Digital Press, Bedford MA, 1988.
17. Manojit Sarkar and Marc H. Brown. Graphical Fisheye Views of Graphs. Proceedings of CHI '92, (Monterey, CA, May 3-5, 1992) ACM, New York, (1992), pp. 83-91.
18. ImageVision, Silicon Graphics Inc., Mountain View, CA.
19. Robert Spence and Mark Apperley. Data Base Navigation: An Office Environment of the Professional. Behaviour and Invormation Technology, 1, 1, (1982), 43-54.
20. Daniel C. Swinehart, Polle T. Zellweger, Richard J. Beach, and Robert B. Hagmann. A structural view of the Cedar programming environment. ACM Transactions on Programming Languages and Systems, Vol. 8, No. 4, ACM, 1986, pp. 419-490.
21. Stephen A. Weyer and Alan H. Borning. A Prototype Electronic Encyclopedia, ACM Transactions on Office Systems, 3, 1, (1985), pp. 63-88.