reg:exp

HCI Comments VIII

Readings in Information Visualization: Using Vision to Think, Chapter 1, Stuart K. Card, Jock D. Mackinlay, Ben Shneiderman, Morgan Kaufmann Publishers, pp. 1-34

The grounding of visualization techniques in the biology of the human perceptual system is essential. The authors make this point but do not have enough space to elaborate sufficiently on the specific properties of the human eye and visual pathway. An accessible psychology text that goes into further detail describing vision and other senses is: E. Bruce Goldstein, Sensation and Perception. Wadsworth, Sixth Edition, 2002. ISBN 0534539645.

While concepts relating to static presentation of data are discussed at length ("spatial substrate", "marks", and their graphical properties) the dynamic, temporal properties are only glossed over briefly. I believe more interesting work can be done here. Animation itself can be used to show physical or abstract phenomena, just as static properties of data marks. In sonification for example, all conveyed information is temporal in nature. Interestingly, different frequency ranges are perceived as different phenomena - can similar characteristic ranges be found for visual frequencies?

What about aesthetics and production value? The article explicitly states that many of the demo systems mentioned were not concerned with creating beautiful graphics. But maybe they should be. In "Emotional Design" I believe Don Norman argues that "attractive things work better" (haven't read the book yet- it's sitting on my desk). Can established graphic design practices and guidelines inform information visualization? Can they be captured in a set of heuristics?

Other odds and ends:

- The "Cost-of-Knowledge Characteristic Function" maybe more significantly altered by non-visual tools such as Google. Structure isn't everything.

- Cell animators have been using selective distortion and exaggeration - such as squash and stretch - to create believable characters. How can exaggeration be employed in info vis to guide user attention towards salient/unusual data points or patterns?

- The "overview+detail" technique could be generalized to show n different viewpoints of the same frame of complex data sets to overcome human difficulty in understanding higher-dimensional spaces.

Comments on the demo videos linked from the course page:

FILMFINDER: User interface designers should be sensitive to the importance of production values (cf. in-class discussion of Nielsen's bad graph in the Heuristic Evaluation paper). This demo has an incredibly overdriven audio channel that makes it nearly impossible to listen to with headphones.

TREEMAP: Shneiderman's group places emphasis on the concept of "dynamic queries" - yet their examples in the Treemap demo are all based on Visualization of Excel files that I assume are then manipulated internally in Treemap. Integrating their work with a relational database system that can answer the changing queries directly would make their approach more powerful.
In the example, users can specify their own color gradients. Which gradients maximize perceptible differences between elements is not intuitive. Maybe the color picker could be based on perceptual distances to aid the user?

The Table Lens: Merging Graphical and Symbolic Representations in an Interactive Focus+Context Visualization for Tabular Information, Ramana Rao and Stuart K. Card, CHI 1994: ACM Conference on Human Factors in Computing Systems, pp. 318-22

The paper's abstract tone made it difficult for me to imagine the operation of Table Lens in practice - the demo was much more instructive as to how their technique works. The principal contribution seems to be that much larger data sets can be shown at once than in standard spreadsheet applications. Also, the direct visual comparison affords an intuitive understanding of the characteristics of the data sets. Simple comparisons can be read off and don't have to be calculated. Grouping and complex relationships between a larger set of different variables are still hard to comprehend though since display is restricted to the 2D constraints of a flat area display. The paper was short on evaluation.

HCI Comments VII

Thoughts on:
DENIM: An Informal Web Site Design Tool Inspired by Observations of Practice, Mark W. Newman, James Lin, Jason I. Hong, James A. Landay, Human-Computer Interaction, 2003. 18(3): pp. 259-324

Newman et al. show how fieldwork can play an important part in the UI design cycle. Their initial user study suggested the lack of tools for the early stages of website design. Consequently, the authors abandoned their original plan to design software for finished websites and refocused on developing an information organization tool that supports the early sketching phase of website design. Unwittingly, they further underline the importance of user-centered development in their implementation of "semantic zoom" levels within DENIM: three representation levels were suggested by the initial user study; two more were added ad-hoc by the programmers. The latter two levels then turned out to be of little use to practitioners during the reported evaluation study. Most insightful quote on this topic (pg. 317): "Much of [web design practice] literature is prescriptive rather than descriptive in nature and may not accurately reflect what designers are actually doing in the field. To learn what designers do, there is no substitute for direct connect."

Curiously, a discussion of data-driven, dynamic websites was absent from the otherwise very thorough and detailed article. In my own experience as a web designer and programmer, most sites have long moved from static HTML to database/CMS-backed, script-driven dynamic output. In this programming-heavy model, sub-page-level code modules are used as basic building blocks - logic widgets that often get reused in multiple different pages. DENIM focuses solely on surface realization of pages and does not address this underlying information infrastructure.

The authors write that "several designers were interested in having a tool that helped them keep track of project histories." This statement suggests an unfortunate disconnect between the realms of web design and general computer programming. The computer science community has long relied on freely available version control tools such as CVS. These systems readily work for HTML pages; given enough interest, it should also be possible to develop an extension for performing version comparisons on images - "visual diffs".

A major shortcoming of the DENIM system is the lack of downstream integration into the further production pipeline. The notion of "semantic zoom" should be carried through to its conclusion. Finished site sketches apparently have to be reprogrammed from scratch for final production - GIF image-map exports are not very useful to work off.

Finally, it seems to me that the custom set of pen gesture commands developed for DENIM would hinder rather than help its adoption. Web designers have to learn a new list of UI commands that are only useful within one application. This is possibly a general problem of pen-based interfaces where standards are still lacking or inherently difficult to define across applications (cf. Ken Hinckley's CS547 talk from last week).

HCI Comments VI

How to Conduct a Heuristic Evaluation, Jakob Nielson, http://www.useit.com/papers/heuristic/heuristic_evaluation.html
Multiple people are likely to find more usability problems than a single evaluator. Nielsen argues that from a cost-benefit point of view, three to five evaluators provide the best payoff. Heuristic evaluation is contrasted repeatedly with user testing; however, no clear succinct definition of what constitutes a user test is given. Benefits of heuristic evaluation are likely to be incremental/evolutionary rather than revolutionary: emphasis is placed on small individual problems rather than large, conceptual issues. Because of the reliance on a set of heuristics, evaluators are less likely to find structural problems not well covered by these heuristics. Real work done with an application may expose a different set of problems. Nielsen's cost-benefit analysis is based on commercial software products. Open-source software is developed and distributed through a distinctly different model that necessitates a different kind of analysis. Cheap automatic evaluation techniques combined with the short cycle period between OSS release versions may help achieve better coverage of more usability problems than a single small group heuristic evaluation. Example.: the Firefox browser interface went through many different incarnations; more so than iExplore or another browser. Even though the changes from version to version were rapid and took getting used to - something commercial vendors may not be able to subject their customers to - the UI has now become stable and is more functional than that of other browsers.

Methodology Matters: Doing Research in the behavioral and social sciences, Joseph E. McGrath, in Readings in Human-Computer Interaction: Toward the Year 2000 R. M. Baecker and J. Grudin and W. A. S. Buxton, ed. pp. 152-169.
McGrath presents a clear summary of social science research methodologies. In terms of knowledge communicated this article is the most dense so far and the most efficient in terms of "data ink" (concepts-per-page). The article touches upon all important issues to consider when planning social science research, but the limited length prohibits in-depth treatment - the buzzwords are there, but many are not explained exhaustively enough for a practitioner. A more detailed discussion can be found in: Investigating Communication - An Introduction to Research Methods. L.R. Frey, C.H. Botan, P. G. Friedman, G. L. Kreps. Prentice Hall, 1991 (my undergraduate textbook). The importance of triangulation between different methods is stressed to offset the inherent bias/weakness of any one particular methodology. For HCI researchers, adopting rigorous soc. science research methods means an two-step removal from usual computer science practices - first one needs to replace developers' intuition with external evaluation. One then needs to qualify any particular evaluation by considering the effect of the chosen measurement instrument. The importance of collaboration and cooperation between researches and institutions emerges since the scope of producing research validated by multiple methodologies may be too large of a project for any single individual.

Measuring API Usability, Steven Clarke, Dr. Dobb's Journal Special Windows/.NET Supplement, May 2004, pp. S6-S9

Clark recognizes that APIs are as much a user interface as GUIs - so similar techniques can be used to evaluate their usability (breakdowns). Scenario approaches can help determine desired API functionality. Comparisons between user expectations and API affordances according to the "cognitive dimensions framework" help pinpoint potential areas of mismatch where an API needs to be redesigned.
While I am usually a proponent of concision, I would have liked this article to be more substantial. The absence of any references exacerbates the problem. Hiding the information of how to read the radial graph in Figure 1 is in itself a usability breakdown of the article.

HCI Comments V

Kristen Blair, Bill Gaver, Tony Dunne, Elena Pacenti. Cultural Probes,ACM Interactions, February 1999, pp. 21-9

Researchers can improve the quality of their data gathering by consulting and applying principles from the artistic/design tradition. High production values generate better responses. Using hand-crafted artifacts reinforces the impression that the people being observed are important as individuals, not just as an amorphous "target group". Likelihood of cooperation is thus increased. Moreover, the unconstrained, evocative techniques encourage rich responses and enable "user-centered inspiration" that informs the developer about previously undiscovered issues. In some parts, the study appears to veer too much towards gratuitous design tinkering, and the step from viewing probes to proposing designs is not sufficiently described. Principal take-away message for me: sustaining regular communication on a personal level with the target audience is essential. Being creative about the means employed to achieve this conversation can help.

Prototyping for Tiny Fingers, Marc Rettig, Communications of the ACM, April 1994, pp. 21-7

Rettig presents powerful arguments for using quick-and-dirty low fidelity prototypes very early on in the development process. Changing designs and going through more iterations becomes easier since the investment (in terms of time, resources and ego) in each prototype is kept low. Results can be obtained quickly and the danger of getting mired in details too early on is minimized. I have experienced the point that testers tend to comment on "fit and finish" issues in a different area - paper writing. Draft reviewers often focus on surface details such as tone or even punctuation, without addressing more fundamental questions such as structure or soundness of arguments. Switching to other media for design prototypes is in itself valuable - physical, tangible objects have very different "paths of least resistance" and constructing a prototype under such altered circumstances forces the developer to question notions of what is natural/easy/desirable in the interface.

Looking Across the Atlantic: Using Ethnographic Methods to Make Sense of Europe, Genevieve Bell, Intel Technology Journal, 3rd Quarter 2001

The difficult task the ethnographer-as-outsider faces to truly understand her subject of inquiry became apparent to me through some of the reported details. Having grown up in Europe, I noticed a few choices made that likely prevent the study from producing a balanced view. The chosen time frame over the summer months is problematic. Especially in southern Europe, daily habits fluctuate greatly between seasons and the heat common in July/August strongly influences behavior patterns since air conditioning is not common in private residences. So people spend the middle of the day indoors to avoid the sun and go out at night much more frequently than during the rest of the year. Furthermore, the places chosen exhibit some peculiar idiosyncrasies not representative of their national environments. In Italy, Venice was picked as the "major urban metropolis." With roughly 300,000 inhabitants and an economy strongly skewed towards tourism, Venice is very much unlike other Italian urbanizations. In Spain, both the Basque and Catalan regions are known for fierce separatist movements that seek to uphold local culture against "Spanification." Both regions prize themselves on their independence and difference from mainstream Spanish culture.
Despite these mischaracterizations, the study still resulted in a better understanding of Intel's potential European customers. The "domains of significance" that emerged from the ethnography should help position products in these markets. However, the author's own concern, voiced in one of the concluding paragraphs, that "finding ways to make our ethnographic work relevant" remains the biggest challenge, points to the hurdles open-ended, qualtitative research faces in an industrial setting.

HCI Comments IV

Past, Present, and Future of User Interface Software Tools, Brad Myers, Scott E. Hudson, Randy Pausch, ACM Transactions on Computer-Human Interaction, March 2000, pp. 3-28

The authors offer a clear and well-structured exposition on which user interface tools have worked in the past and which ones have not, along with reasons for the successes and failures. (Relative-to-human) scale emerges again as a factor strongly influencing design. Because of the success of ubicomp devices, the authors argue that UIST will once again become an important research topic as the mature, relatively monolithic set of user interface principles dominating the desktop computing environment cannot be transferred to devices of different scale. In the section "Future Prospects and Visions", development trends for the next couple of years are sketched. It is noted that "user interfaces are becoming more cinematic". I strongly dislike the use of gratuitous animation in user interfaces - the cinematic components often force the user to abandon her own internal timing for interacting with a software and adhere to the developer's notion of speed. Many custom graphical interfaces seem sluggish to experienced users.
Another quote: "Furthermore, as people increasingly distribute their own computation across devices [...], there will be an increasing need for people to communicate with _themselves_ on different devices." I couldn't agree more. I work on at least three different machines every day and data synchronization has turned into a major headache. The trade off seems to be between working simple applications whose data can be synchronized easily (e.g., plain txt-files) and using complex applications with complex file formats that are troublesome to synchronize, especially if you modified versions of the file on more than one machine between syncs. A larger question is whether we want our data stored in a distributed or centralized fashion. Centralized servers assure consistency but may be more vulnerable to security risks. Also, what if you need access to your files but are temporarily off line? Maybe we can think of individual devices acting as local data caches for server-side storage. Then we could exploit existing caching algorithms from computer architecture to ensure data consistency.
Finally Myers et al mention designing for older adults. I agree that this topic needs to be addressed. You can already find pocket calculators, land line phones and wrist watches optimized for use by people with diminished perceptual skills. But trying to find a cell phone that is easily operatable by the elderly is near impossible.

Natural Programming Languages and Environments, Brad A. Myers, John F. Pane, Andy Ko, To appear in Communications of the ACM

As in previous papers, the idea of building "natural" interfaces is stressed. Here though, an attempt is at least made to define what naturalness means. However, two conflicting definitions are given. In the opening paragraph, natural is identified with "closer to the way people think about their tasks." Later on, natural becomes "faithfully representing nature of life". To me, these statement are not at all equivalent. The paper mentions the Alice graphical programming environment. In 2002, I was a teaching assistant for a class called "Virtual World Design" in which groups of students developed a series of small interactive 3D worlds in Alice. The main complaint from the students was that while basic tasks accessible through the user interface where quickly completed, any deeper programming was frustrated by the developer's efforts to hide the internals of the engine from the user. In the author's own terminology, the system had a low threshold, but also a low ceiling. Accommodating the different requirements posed by novices and expert programmers at the same time is likely to remain a challenge.

HCI Comments III

Thoughts on:
Anne Marie Piper & Nirav Mehta Tangible Bits: Towards Seamless Interfaces between People, Bits, and Atoms, Hiroshii Ishii and Brygg Ullmer, CHI 1997: ACM Conference on Human Factors in Computing Systems, pp. 234-41
Ishii and Ullmer outline their vision of tangible user interfaces - TUIs - that aim to leverage the haptic intelligence and processing capability which we have gained through interaction with physical objects for human computer interaction. Key concepts are 'interactive surfaces' (think Weiser's Tabs, Pads and Boards) that provide for manifold interfaces between physical and virtual worlds; the 'coupling of bits and atoms' which refers to the linking of computational processes to physical objects; and 'ambient media' which are concerned with peripheral perception as viable communication channels.

Beyond particular implementations, two fundamental design dichotomies (not exacly the right word) are at the core of this paper - figure/ground separation and generalized vs. specialized artifact design. The figure/ground distinction arises from selective human attention which qualitatively separates that which is attended to from its environment; it reoccurs as a basic constituting principle in the visual arts, music and probably any other field related to human perception. A recent example of amplifying figure/ground difference to increase UI legibility is Apple's use of dynamically resizing application windows that shrink and fade as they are "backgrounded". There seems to be a direct link between figure/ground distinction issues and Bayesian processes (conditional probabilities) at work in our perceptual system(s). Something to ponder in the future.

The generalized vs. specialized design question may help explain the uneasy position HCI has assumed within the computer science community. Predominantly, design and invention of physical artifacts has been concerned with building specialized tools optimized for helping humans perform one particular task. HCI folks follow in these footsteps by taking a user centric and thus application centric view. The mathematical tradition in contrast is based on axiomatic methods that are concerned with establishing generalizable results - while applications of these results to real world problems are accepted as welcome side effects, they do not represent the core problem. Arguably, the computer arose out of that tradition as a blank-slate general purpose device, capable of any number of feats, but optimized for none. The principal difference in approach inherently makes one group skeptical as to the merits of the other group's work.

A few more scatted notes:
Bimodal input seems to be indeed an important research direction. However, it is really independent of the question of whether phicons are useful or not. The two issues are muddled in the text. I also think there are two related yet distinct concepts to discuss in the ambientROOM: peripheral perception versus multimodal perception. Peripheral input can be restricted to one modality (presenting material in the edges of a large screen) while multimodal signals can all compete for the user's attention. The association is by no means mandatory. I can furthermore imagine that extended periods of multimodal interaction are tiring for users because of the greater cognitive load. The question of necessary rest periods or pauses comes up - how can one stop the constant information stream in an ambientROOM? On a desktop computer, you just have to lift your hands off the keyboard and look away from the screen. Finally, some nitpicking: the use of audio in the ambientROOM appears naive. The sampled sound of a raindrop - the chosen network activity indicator - is not a neutral signal. It has a complex and rich spectral structure in itself that carries connotations. An analogy in graphics would maybe be to use an intricate ornamental stencil mask or stamp to indicate every single data point in a graph plot.

Knowledge-Based Augmented Reality, Steven Feiner, Blair Macintyre, and Doree Seligman, Communications of the ACM, July 1993, pp. 53-62

Since the response to the previous article was overly elaborate, I'll keep it short here: The authors put their work in an interesting relation to Weiser's ubicomp vision - instead of embedding the computers in the environment itself, they simply project or overlay computer information onto the spaces we look at in the environment. The approach is much more economical and sidesteps the networking infrastructure required to integrate ubicomp devices. Reintroducing networking, I see promise for using augmented reality systems for cooperative tasks where groups of people need to work together. Each group member can have their own instructions shown, but all instructions can be synchronized. In this way, the complex sequential nature of team work (e.g., aircraft maintenance) could be accounted for. Late in the paper there is a reference to the utility of sound - the fact that we can hear things that are outside of our field of vision could be used to direct user attention to items in the real world that are relevant for his task but not currently visible. Generating 3D sound sources is an easy problem in psychoacoustics.

Reinventing the Familiar: Exploring an Augmented Reality Design Space for Air Traffic Control, Wendy E. Mackay, Anne-Laure Fayard, Laurent Frobert and Lionel Médini, CHI 1998: ACM Conference on Human Factors in Computing Systems, pp. 558-65

A very thorough exposition how close cooperation between users and designers in iterative cycles results in a superior understanding of the underlying task. To say it with the authors: "exploration of the design space is essential." For such critical systems as air traffic control, reliability becomes a major issue. What about fallback possibilities if the software fails? Keeping the paper strips in the work process enables a gracious degradation in case of software bugs. Related link: "Glitch Grounds U.K. Air Traffic"
http://www.cbsnews.com/stories/2004/06/20/world/main624974.shtml
Again, peripheral perception emerges as an important interaction variable. The Stanford iRoom with its big smartBoards could be used to explore aspecs of visual peripheral input.

HCI Comments II

Thoughts on:
The Computer for the 21st Century, Mark Weiser, Scientific American, September 1991, pp. 94-104

Positive/interesting points:
Weiser's reference to Polanyi's "tacit dimension" touched upon a topic I have been mulling about for a while. The concept of truly mastering a task/skill/activity by internalizing it through repetition beyond a barrier of conscious knowledge is found time and time again across disciplines and cultures (cf. Herrigel, "Zen in the Art of Archery") Conversely, note how hard it is to break out of established patterns, the "habitual mind" (is there a good citation for this term?). I know I am way off on a tangent here, but at least the paper was thought provoking. Larry Gross from the Annenberg School of Communication teaches (or used to teach) a great course entitled "Art as Communication" that spends a considerable amount of time on this topic.

My second item of interest is more directly related to the article's core message: interacting with many "boards" and "pads" on a daily basis may actually force the computer user to engage in regular physical activity. This is healthy. Ending the user's transfixion in front of the desktop PC could seriously lessen work-related health risks.

On the downside, ubiquitous computing seems to needlessly import some of the problems the physical world exhibits into the digital domain. Would you rather shuffle through a deck of "tabs" or use a search engine to find some files? Also, producing lots of limited-use electronics seems to be a waste of natural resources. Do we have a plan for how to disassemble and recycle these mini-computers when they start to fail or are deemed outdated? We cannot even take care of this task for our comparatively few full-grown PCs today (shipping the trash to China is not a sustainable solution). A meta-comment: the paper ends with a number of blanket statements such as "Computer access will penetrate all groups in society." that are not backed by any evidence. Irritating.

Charting Past, Present, and Future Research in Ubiquitous Computing, Gregory D. Abowd and Elizabeth D. Mynatt, ACM Transactions on Computer-Human Interaction, March 2000, pp. 29-58

What does it mean to build a natural interface? The authors mention writing as a natural action - but we all need to learn this skill through a long, possibly arduous process. Writing is common, yes, but natural? Similarly, playing a guitar does not resemble any other activity we routinely perform in our lives. Yet millions of people have learned how to use this "unnatural" instrument. Mastering these interfaces is difficult, but the inherent complexity also makes great virtuosity possible - it largely defines their value. (Upshot: we should not aim too low. Intuitiveness may be inversely related to usefulness)

I agree with the authors that scale is an extremely important attribute. Building devices that accommodate human scales is essential - issues of size, but also rhythm, nonlinear perception of time come to mind. Another good point: humans are effective - but far from perfect - recognizers. Computer recognition and context fusion techniques should therefore take an approach that incorporates the notion of uncertainty. Much of AI has already gone down this statistical route.

A little line on the importance of error handling caught my interest: how much research has been devoted not to the avoidance of errors, but to their constructive processing in interaction with the system user?

And yes, we need to introduce more associative models of information management. The image of a message morass perfectly described my own email account/folder mess. How can we visualize associative models appropriately? Graphs are good for showing connections, but don't mesh well with text.

HCI Comments I

I will post short responses to my reading assignments in HCI here from time to time. Most will not summarize the articles but rather take them as a starting point for various tangential thoughts.

As We May Think, Vannevar Bush, The Atlantic Monthly, July 1945
Bush, with striking insight, predicts that issues of knowledge management, meta-knowledges so to speak, will become the most important tasks to solve in the future. Key functions that science should enable are efficient extension, storage and consultation of the record of human knowledge. While enabling technologies are envisioned, the archival properties of these technologies are not addressed. Will his microfilm still be legible after spending 100 years on shelf? This problem is also underrated today - see the recent discovery that many CD-Rs will de-laminate and disintegrate after a few years.
Most of Bush's contraptions are directly bound to mechanical machines or chemical processes. The abstraction of the function of a particular device (think software) from the underlying architecture (think general purpose computer) has not taken place yet.

The Xerox Star: A Retrospective, Jeff Johnson, Teresa L. Roberts, William Verplank, David C. Smith, Charles Irby, Marian Beard, Kevin Mackey, IEEE Computer, September 1989, pp. 11-27

Through careful planning, a strong task-based focus, and meticulous attention to detail, the STAR team anticipated and introduced many lasting design features of office software systems. Part of the success was due to the developer's choice to not just rely on their own judgments but to leverage external expertise - graphic designers were hired and user studies were conducted. The article itself situates the STAR system very well in the general context of user interface research at the time, showing its lineage, but also concurrent competing technologies. As for criticism, it was uncanny how many of my immediate concerns about the system while reading the paper were acknowledged and addressed by the authors just a few pages later. There is the danger of pushing the desktop metaphor too far - the life of data is not like the life physical objects. Only allowing the user to act upon data in ways that have correspondences in the real world is limiting. There is also a problem of custom-tailoring a product too rigidly to an a priori model - what happens if the user's requirement profile changes - maybe as a function of becoming more computer proficient and reliant?

User Technology: From Pointing to Pondering, Stuart K. Card and Thomas P. Moran, ACM Conference on The history of personal workstations, 1986, pp. 183-98

Card and Moran outline a detailed "applied science of the user" - user behavior and processing capacity is rigorously studied. An interesting question at the level of their "conceptual interface": how can we as system designers/developers ensure that the user will have a reasonably accurate mental model of the system? The authors also point out the difficulty of aggregating scattered individual research studies from, e.g., psychology, into a unified computational model of human user behavior. An assumption is made that the user acts rationally to fulfill the given tasks. Is this always the case? When does irrationality come into play? Can we model it? Problematic on a technical level: repetitive use of terms that are overloaded with multiple, imprecise meanings in popular usage ("task") makes it hard to follow the flow of the argument at times.