October 12, 2004
HCI Comments III
Thoughts on:
Anne Marie Piper & Nirav Mehta Tangible Bits: Towards Seamless Interfaces between People, Bits, and Atoms, Hiroshii Ishii and Brygg Ullmer, CHI 1997: ACM Conference on Human Factors in Computing Systems, pp. 234-41
Ishii and Ullmer outline their vision of tangible user interfaces - TUIs - that aim to leverage the haptic intelligence and processing capability which we have gained through interaction with physical objects for human computer interaction. Key concepts are 'interactive surfaces' (think Weiser's Tabs, Pads and Boards) that provide for manifold interfaces between physical and virtual worlds; the 'coupling of bits and atoms' which refers to the linking of computational processes to physical objects; and 'ambient media' which are concerned with peripheral perception as viable communication channels.
Beyond particular implementations, two fundamental design dichotomies (not exacly the right word) are at the core of this paper - figure/ground separation and generalized vs. specialized artifact design. The figure/ground distinction arises from selective human attention which qualitatively separates that which is attended to from its environment; it reoccurs as a basic constituting principle in the visual arts, music and probably any other field related to human perception. A recent example of amplifying figure/ground difference to increase UI legibility is Apple's use of dynamically resizing application windows that shrink and fade as they are "backgrounded". There seems to be a direct link between figure/ground distinction issues and Bayesian processes (conditional probabilities) at work in our perceptual system(s). Something to ponder in the future.
The generalized vs. specialized design question may help explain the uneasy position HCI has assumed within the computer science community. Predominantly, design and invention of physical artifacts has been concerned with building specialized tools optimized for helping humans perform one particular task. HCI folks follow in these footsteps by taking a user centric and thus application centric view. The mathematical tradition in contrast is based on axiomatic methods that are concerned with establishing generalizable results - while applications of these results to real world problems are accepted as welcome side effects, they do not represent the core problem. Arguably, the computer arose out of that tradition as a blank-slate general purpose device, capable of any number of feats, but optimized for none. The principal difference in approach inherently makes one group skeptical as to the merits of the other group's work.
A few more scatted notes:
Bimodal input seems to be indeed an important research direction. However, it is really independent of the question of whether phicons are useful or not. The two issues are muddled in the text. I also think there are two related yet distinct concepts to discuss in the ambientROOM: peripheral perception versus multimodal perception. Peripheral input can be restricted to one modality (presenting material in the edges of a large screen) while multimodal signals can all compete for the user's attention. The association is by no means mandatory. I can furthermore imagine that extended periods of multimodal interaction are tiring for users because of the greater cognitive load. The question of necessary rest periods or pauses comes up - how can one stop the constant information stream in an ambientROOM? On a desktop computer, you just have to lift your hands off the keyboard and look away from the screen. Finally, some nitpicking: the use of audio in the ambientROOM appears naive. The sampled sound of a raindrop - the chosen network activity indicator - is not a neutral signal. It has a complex and rich spectral structure in itself that carries connotations. An analogy in graphics would maybe be to use an intricate ornamental stencil mask or stamp to indicate every single data point in a graph plot.
Knowledge-Based Augmented Reality, Steven Feiner, Blair Macintyre, and Doree Seligman, Communications of the ACM, July 1993, pp. 53-62
Since the response to the previous article was overly elaborate, I'll keep it short here: The authors put their work in an interesting relation to Weiser's ubicomp vision - instead of embedding the computers in the environment itself, they simply project or overlay computer information onto the spaces we look at in the environment. The approach is much more economical and sidesteps the networking infrastructure required to integrate ubicomp devices. Reintroducing networking, I see promise for using augmented reality systems for cooperative tasks where groups of people need to work together. Each group member can have their own instructions shown, but all instructions can be synchronized. In this way, the complex sequential nature of team work (e.g., aircraft maintenance) could be accounted for. Late in the paper there is a reference to the utility of sound - the fact that we can hear things that are outside of our field of vision could be used to direct user attention to items in the real world that are relevant for his task but not currently visible. Generating 3D sound sources is an easy problem in psychoacoustics.
Reinventing the Familiar: Exploring an Augmented Reality Design Space for Air Traffic Control, Wendy E. Mackay, Anne-Laure Fayard, Laurent Frobert and Lionel Médini, CHI 1998: ACM Conference on Human Factors in Computing Systems, pp. 558-65
A very thorough exposition how close cooperation between users and designers in iterative cycles results in a superior understanding of the underlying task. To say it with the authors: "exploration of the design space is essential." For such critical systems as air traffic control, reliability becomes a major issue. What about fallback possibilities if the software fails? Keeping the paper strips in the work process enables a gracious degradation in case of software bugs. Related link: "Glitch Grounds U.K. Air Traffic"
http://www.cbsnews.com/stories/2004/06/20/world/main624974.shtml
Again, peripheral perception emerges as an important interaction variable. The Stanford iRoom with its big smartBoards could be used to explore aspecs of visual peripheral input.