reg:exp

Project: The Slouch-Aware Chair

The Slouch-Aware Chair monitors its sitter's posture to detect unhealthy back positions and provides subtle visual and tactile feedback to encourage good postural habits. The chair senses orientation of the sitter's back through infrared distance rangers. A physical dial placed on the sitter's desk in the peripheral field of view indicates the current posture state - a green region in the 12 o'clock position indicates a healthy posture, while the two red regions off to the sides correspond to two common types of bad posture - backward slouching and forward slumping. If the sitter's bad posture continues over a prolonged period of time, the chair will notify the sitter via intermittent vibration. Vibration will cease when the sitter corrects her posture. The chair features instruction built into the armrest.

The Slouch-Aware chair was designed as a project on nearly invisible computing for CS247a, the Human Computer Interaction Design Studio class at Stanford together with Leor Vartikovski. Inspiration was taken from the Stanford posture class taught by Shawn McCracken.

List of Parts:
1 Ikea JULES desk chair (www.ikea.com, $40-50)
1 Phidgets InterfaceKit 8/8/8 (www.phidgets.com)
1 Phidgets Servo
2 AA Batteries w/ battery holder
2 Sharp GP2D120 IR distance rangers (www.acroname.com)
1 vibrating 3V DC motor (www.allelectronics.com)
1 4N4001 diode
1 D44H11 power transistor
1 generic switch
1 small solderless breadboard
2 custom sheet metal brackets (PRL)
tape, 1/4" foam core, wire

Technical details: coming soon...

Phidgets InterfaceKit, batteries for powering the motor, and protoboard to hold the simple circuit are all hidden in the armrest.

The 'Slouch-O-Meter' desktop display box.

Sensor and motor assembly on the back of the chair - sensors mounted with custom bent sheet metal brackets to guarantee minimum sensor distance of 1.5 inches to closest object.

Sensor and motor assembly on the back of the chair seen from above.

Flip-open instructions in the armrest.

In-class demonstration.

Notes on Ulrich, Eppinger - Product Design and Development

Karl T. Ulrich and Seven D. Eppinger
Product Design and Development - 2nd ed.
Irwin McGraw-Hill, 2000
ISBN 0-07-229647-X - Stanford call number HD 31 U47 2000 ENG

Chapter 12 of this textbook presents an overview of the functions of prototyping in the industrial product design process. A running example is used throughout the chapter - the redesign of the trackball for the Apple Duo notebook computer. A prototype is defined as "an approximatrion of the product along one or more dimensions of interest." Prototypes are classified along the following continua: physical--analytical, comprehensive--focused. Often, separate "looks-like" and "works-like" focused prototypes are created side-by-side. The reason given is that building comprehensive prototypes takes longer/is costlier. Comprehensive analyitcal (mathematically modeled) prototypes are not feasible in practice. Prototypes have four main uses: as learning tools; as communication artifacts for management, team members, customers; as integration tools to check if sub-assemblies fit together; and as milestones to demonstrate progress and existing functionality. (Note: user testing is not mentioned.)

Principles of prototyping: analytical prototypes are more flexible than physical prototypes since parametric models can be changed without rebuilding everything. Physical prototypes are necessary to check unanticipated behaviors due to the laws of physics in the real world. Prototyping can save money by moving iteration cycles away from the costly manufacturing stage. Prototypes may expedite other development stages such as mold design if prototype artifacts can be reused for production (e.g., CAD files?). Prototypes may restructure task dependencies?

Two particularly important prototyping technologies are 3D computer modeling and free-form fabrication (3D printing). Stereolithography itself is often referred to as rapid prototyping in the community - a source of confusion.

Prototyping brings with it the danger of sinking into Clausing's "hardware swamp" - spending time building and debugging prototypes that do not substantially further the larger product development project. Careful definition of the purpose and scope of the prototype is needed. Purpose: what are the learning, communication, and integration needs the prototype should fulfill? What is the level of approximation? Ignore all factors that are not part of the analysis. Come up with a testing plan for the prototype, make a schedule and stick to it. A real-world fact: it is usual to buld at least three milestone prototypes: alpha, beta, and pre-production. We are targettign alpha prototypes.

What are wire-wrapped boards?

Links of the day - DIY design

http://make.oreilly.com/
http://www.core77.com

Interview transcription and annotation tools

I was looking around for free audio labeling and segmentation tools to facilitate working with the hour-long interview files from contextual inquiry. Standard audio editors such as Goldwave allow you to drop markers but annotation is normally not possible, especially if it is desired to have hierarchical notes.

Praat and Transcriber are both made for linguists and can annotate on the phoneme-laval data, but both can also handle long audio files. I am currently test-driving both programs to see which one fits my work flow better.

LDC's Linguistic Annotation page may have more links.

Notes on "Contextual Design" (Beyer, Holtzblatt) - Part 1

Contextual Design
Hugh Beyer, Karen Holtzblatt
Morgan Kaufmann, 1998
ISBN: 1558604111
Info: google print | amazon | bibtex

Summary graphic of the approach from the authors' website:

Below is a summary of Beyer and Holtzblatt's main points in the first two parts of the book which are concerned with data collection in the field and initial processing and interpretation of that data. Chapters concerned with turning observation data into design ideas will be summarized as the need arises in our project.

Chapter 1:
Agony comes from not having a ground for making descisions or evaluating alternatives. Data breaks this deadlock. Data is the only reliable outside arbiter for people. But don't just ask what people want - you need a bidirectional conversation.

Chapter 2:
Marketing does not provide the right kind of data to make design decisions - it does not answer questions about the structure of a system. Marketing data gathering techniques may not be useful for the same reasons - they assume the right kind of questions are known a priori and the task of the data gathering is to find answers. Design exploration first has to find the right questions. A hard goal: make customer intuition explicit. You have to see work as it happens to reveal all aspects of it - contextual inquiry is the technique to do just this. CI gathers data in the field from a small number of participants. The goal is to then generalize form the observations. Principles of CI: context, partnership, interpretation, focus. Typical form is the contextual interview - 2-3hrs, introduction, observation with interruption, examination of artifacts. 10-20 interviews total are a good rule of thumb to cover an area of work.

Chapter 3:
Core premise of contextual inquiry: go where the customer works, observe, and talk to the customer about the work. The master/apprentice relationship model is especially useful for collecting data. Acting according to a relationship model is more effective than memorizing interviewing techniques and strategies. Explaining while doing is easier for the "master" than conceptualizing and abstracting their tasks - in the latter case, details get glossed over. Seeing the work reveals what matters, details, and structure. Remember: we don't just observe to understand and learn the tasks, but to then support them with new design.
Context: being at the place of work allows for gathering ongoing, concrete data. Summaries of experiences are not rich enough to generate data relevant for design. Abstraction: "If the customer is leaning back and looking at the ceiling, he is almost always talking in the abstract." Cure: "constantly pull the customer back into real experience". For retrospective accounts, ask questions that prompt interviewee to fill in the skipped details.
Partnership: in many interviews, control and power over conversation resides with interviewer - CI reverses this imbalance. What the interviewer offers: giving the interviewee a better understanding of their own work. Share your ideas that arise during interviews immediately so that your mind is clear and you get on-the-spot feedback.
Interpretation: Data gathering is not enough - you need to interpret the results to generate good design ideas. "Interpretation is the chain of reasoning that turns a fact into an action relevant to the designer's intent." (57) How can one insure the interpretation is right? Share it with the customer while they are engaged in the work - they will validate, refine, or reject it. Listen for "Huh"s, "Umm..could be" and "buts" in the responses.
Focus: necessary framework to make sense of work - set a deliberate project focus; each person also brings in their particular focus. Use group interpretation and interpersonal triggers (?). Listen to internal signals to find situations in which you should adjust your own interpretations: surprise and contradiction - always assume that a "seemingly pointless action hides a key secret of the trade"; Nods: look not for agreement, but for possible divergences; what you don't know (Technical jargon): admit yor ignorance! you are out to challenge your assumptions, not to validate them.
Structure of the contextual interview (p.64): most commonly one-on-one, 2-3hours; four parts:
1) conventional interview: introduction, summary data, 15 minutes.
2) transition: state rules of observation and introduction - 30 seconds.
3) contextual interview proper: keep it concrete, be nosy, take lots of notes
4) wrap-up: skim over notes, summarize with interviewee (15 mins).

Chapter 4: Contextual Inquiry in Practice
Set focus first - transform solution requirements into statement about the customer's work. What is the work we expect to support? How does this work fit into the customer's whole work life? What are the key work tasks?... (p.68) Who is involved in the work? Where does it happen physically? Study analogous work situations - look for metaphors {exactly what we are doing with the photo documentary} to expand focus. Inquiry starting points for commercial software products: designing a known product/addressing a new work domin/introducing a new technology. For IT project: upgrades/new systems/process redesign. Designing the interview situation - different kinds of tasks:
- normal: use standard CI
- intermittent: ask user to keep log
- uninterruptable: videotape, review with customer later
- extremely long: interview range of users at different stages of process; do retrospective walkthrough with a few; use project artifacts
- extremely focused: videotape and interpret with user
- internal- get at mental processes by interrupting a lot.
Who to interview: 2-3 people in every interesting role; 10-20 people total; 4-6 businesses for variety - go for diversity. Look for different business strategies (frelance/constultancies/large companies); cultural differences; physical/geographical situations/diffrerences of scale; *come up with short focus statement in simple language (77)*.

Part 2 - Chapter 5: A Language of Work
Formal languages are good; graphical formalisms are especially suitable for design - but not all are appropriate (cf. Nardi). The authors advocate their "work models" - five types of models are: flow, sequence, artifact, culture, physical - all are described in the following chapter.

Chapter 6: Work Models
Models describe work from the point of view of the interviewee - individual perspective.
The Flow Model: no work happens in isolation - the flow model makes communication and coordination explicit. Work flow model elements make the following distinctions: nodes (“bubbles”) are individuals or groups; their responsibilities are summarized within the node; flow is communication and passage of artifacts – shown as arrows. Artifacts themselves are boxes on top of flows. Communication topics and actions are shown on top of flows without boxes. Places are rectangular nodes with only incoming arrows(?). Breakdowns are shown as lightning bolts. Things to watch out for: coordination with other people – all contacts should be represented. Roles – what responsibilities are people taking on in practice (these are rarely identical with their formal job description). Informal structures: implicit and engrained actions that happen without reflection. Pay attention to real interactions – not the handbook definition of how things should be done.
Sequence model: understanding intent facilitates interpreting the steps taken to realize that intent. Sequence models are similar to flow diagrams or task analysis but make intents and triggers explicit. The show the linear ordering of events as perceived and acted upon by an individual – patterns and repetitions are handled later during consolidation. Graphical elements: intent on LHS of sheet; triggers as events in linear sequence; steps; order loops and branches by connecting steps with unlabeled arrows. Breakdowns are marked with lightning bolts as before. Finding the right level of action recording for your project. User hesitation and error are places where tools do not meet their understanding of the work process – these are design opportunities. Automatized systems have to provide triggers that are analogous to the physical world – users need reminders to take action. Infer intents after writing out sequence.
Artifact model: Artifacts make users' conceptual distinctions clear. The artifact model is a drawing or photocopy of an artifact. Highlight structure with lines and labels; annotate location, record which parts support which intent; lightning bolts show where tool encumbers process or interferes with it. Look how content/represented information is related to artifact structure. Watch out for informal annotations. What about the format and presentation of the artifact makes it an appropriate carrier of the recorded information? What is accentuated, what is downplayed?
Cultural Model (107): A new system has to fit with customer's culture, otherwise it will be rejected – so pay attention to the cultural context of work. Factors include self-perception, formal and informal business policy, etc. Components of the model: Influencers as bubbles; Extent of effect of influencer on work shown by size of overlap of bubbles. Direction of influence or pushback as arrows. Typical important influencers: standards and policy, power, values, identity, emotion, team styles and preferences. How to recognize cultural factors: what is the tone of the environment? Is it elegant, messy, minimal? What is the role of policy? How is it recorded? What are motivations for sticking to it?
Physical Model (115): Physical environment creates design constraints. Environments are organized by workers to reflect the work they do. Model distinctions: places – show size, private or open, cluttered or empty, etc. Physical structures – large objects. Usage and movement within space. Hardware, software, communication lines, and tools. Artifacts and their locations; layout; breakdowns. Pay special attention to: walls – imaginary or real; how does space group people and functions? What is close at hand, what further removed, what inaccessible? The physical model is a drawing of the aspects of the environment that matter. Don’t just draw a floor plan or an inventory list.

Chapter 7: Structure of the Interpretation Session
Shared understanding – let every team member experience all interviews. A method to to communicate to others what you have learned in person. Each interviewer walks through each of their interviews for the team. Team listens, questions, draws work models, records issues, interpretations, design ideas. Ideally the team should be made up of ~4 people - how are we condensing this process down for our two-man team?

Part 3 - Chapter 8: Consolidation (139)
Goal: design for a population of users, but have your solution be flexible enough to fill inividual's specific needs. Segmenting by demographics is a poor substitute for segmenting by work practice patterns. If work practices are common, work models from individual interviews can be fused into a consolidated model. If they cannot be fused, then there is no single market and you need to design multiple solutions.

Chapter 9: Creating one view of the customer (151)
The chapter introduces two new tools: affinity diagrams and consolidated work models. The underlying thread is generalization from examples by induction. Never depend on theoretical arguments - reality often doesn't follow principles of deductive logic. Variations exist within a structure; consolidation makes that structure explicit. Explicit representations of work are needed - nearly all design thinking requires props - only externalized points of view can be used effectively by other team members.
The affinity diagram: hierarchical organization of interpretation session notes; group common themes and issues. Color coded: white-blue-pink-green. Build this after interviewing 15-20 customers with 50-100 notes per customer. Can be done in one day with on person per 100 notes. Ban familiar words to avoid stereotypical grouping - let grouping arise from the data. Put up a note and find others that go with it - no need to articulate relationship yet. "Two notes have an affinity if they are saying similar things about the work as it relates to the design focus of the team" - they are expressing a similar intent, problem, or issue in the user's work."(156) Groups are given their own labels - in customer's voice. The labels are the new info generated by the affinity map. Introduce all the new data at once to think broadly in different paradigms - drop by drop introduction leads to assimilation into existing thought models and results in small fixes instead of holistic redesign.
Consolidating Flow Models: find communication patterns and common roles in different jobs. Roles are collections of responsibilities. THey arise from the needs of the work and are thus often consistent across organizations. Roles are constant but mapping from people to roles changes across orgs. Split all responsibilities of an individual into coherent roles, then merge. Keep track of how roles are linked to observed individuals with another level of color coding. Step-by-step instructions for consolidation are on pg 169.

Bibtex entry:

@book{286067,
 author = {Hugh Beyer and Karen Holtzblatt},
 title = {Contextual design: defining customer-centered systems},
 year = {1998},
 isbn = {1-55680-411-1},
 publisher = {Morgan Kaufmann Publishers Inc.},
 }

Cross-Media Prototyping Comparison

In my weekly meeting with Terry Winograd we hit upon an interesting question: What are the characteristic differences in prototyping practice in different media? What aspects are fundamentally the same or fundamentally different about protoyping mechanical devices and software? Some of the first realizations: In product design you build a prototype, then start from scratch for the next iteration. In code, existing files are changed and parts rewritten, but rarely do you simply discard the entire source tree. Are there analogies to writing software with and without versioning? Now how about sketches for paintings, musical composition, or architectural prototypes? A systematic investigation would be a nice side project.

Mike Kuniavsky, in his previously referenced post scored sketching/protoyping systems according to speed, provisionality, and history. What other schemes can we come up with?

To Read

My ever growing reading list... For current research:

• Eric Bregman, Information Appliances. Morgan-Kaufmann (hardcopy from Scott).
• B. Laurel, Design Research. MIT Press. (have hardcopy)
• Buchenau, Suri - Experience prototyping - DIS'00 (pdf) - look for prototyping references in here
• David J. Anderson, Brían O’Byrne - Lean Interaction Design and Implementation: Using Statecharts with Feature Driven Development (Proc. ForUse 2003)(pdf)
• Horrocks, Ian (1999), Constructing the User Interface with Statecharts, Addison-Wesley Longman (out of print - obtained from U Michigan via Interlibrary Loan)
• Jon Whittle, Johann Schumann - Generating statechart designs from scenarios (pdf via ACM DL)
• Jesse James Garnett - A visual vocabulary for describing information architecture and interaction design (website)
• U. Cugini, F. Folini, I. Vicini - A Procedural System for the Definition and Storage of Technical Drawings in Parametric Form - Eurographics 88 (source?)
• David Joshua Kurlander. Graphical Editing by Example. PhD thesis, School of Arts and Sciences, Columbia University, 1993. ( ps - unreadable)
• Card et al. - A morphological analysis of the design space of input devices (via ACM DL)
• Blair MacIntyre - Designer’s Augmented Reality Toolkit (DART) (old website | pdf)
• Lars Erik Holmgren - Smart-ITS (website)
• Joe Marks et al. - Design Galleries (TR from MERL)
• http://jerry.cs.uiuc.edu/~plop/plop98/final_submissions/P22.pdf
• Scott R. Klemmer, Michael Thomsen, Ethan Phelps-Goodman, Robert Lee, James A. Landay, Where Do Web Sites Come From? Capturing and Interacting with Design History. CHI 2002: ACM Conference on Human Factors in Computing Systems, CHI Letters, 4(1): pp. 1–8. (pdf from Scott's homepage)

General HCI Interest:

• Mike Kuniavsky - Observing the User Experience (google print info)
• BJ Fogg - Persuasive Technology (GREEN Call# BF637 .P4 F55 2003 - checked out)

Music:

• Sid Fels - GloveTalk (website)

Other:

• Kevin Lynch - The Image of the City (GREEN Call# NA9108 .L9)
• Creativity; Flow by Mihaly Csikszentmaihalyi

Recently finished:

• David Harel - Statecharts: A visual formalism for complex systems (ACM DL Citation | pdf from Harel's site | also available at MATH-CS lib)
• Ulrich, Eppinger - Product Design & Development (ENG call# HD31.U47 2000) - Ch.12 on prototyping
• Wiring:Prototyping Physical Interaction Design - Hernando Barragan's Thesis - pdf
• T. Winograd - "Interaction Spaces for 21st century computing" in Human-Computer Interaction in the New Millennium, Johm M. Caro, ed. ACM Press, p.259-276. (have hardcopy)

Adding 3rd party components to Phidgets

The Phidgets toolkit takes a plug'n'play approach to adding physical sensors and actuators to your applications: just connect one of the supplied Phidgets to a USB jack on your computer or to an input connector on the InterfaceKit, drag the corresponding software widget into your C#/VB/.. form, and everything should run. But what if you want to add a different device to your setup, one that did not come in the Phidgets box?

Adding external devices requires a larger toolbox of concepts, techniques, and &emdash; tools. You don't have to be an expert in EE or ME though - a combination of googling and hands-on trial and error will be sufficent for the kind of rough-and-ready prototypes we are building in class. Topics worth familiarizing yourself with include basic electronics (more), machine shop techniques (visit Stanford's product realization lab), and places to order electrical/mechanical parts from. Stanford's product design program has a large list of suppliers - Jameco (in Belmont) and Digikey are the main sources for electronics, but if you're in a hurry and want to stay local, you can get many parts at Fry's and Radio Shack in Palo Alto. McMaster-Carr has every conceivable part of hardware you will ever need, but making your own at PRL may be cheaper and faster. Useful general references are the Physical Computing book by O'Sullivan and Igoe and their web sites.

Let's first look at how to deal with new sensors such as foot switches, rotary encoders, or IR distance sensors. The Phidgets site has a documentation page that describes how to connect sensors with variable resistance/voltage output to the analog inputs of the Phidgets InterfaceKit. In many cases, you may be able to get away with simply connecting Phidgets' "analog input" cable to the appropriate pins of your sensor. The actual lead assignment for the Phidget cables can be found on this page. Unless you can find the right kind of connector at a local electronics shop I would suggest just cutting one of the supplied analog input cables in half, stripping some isolation off the cut end of the wire and soldering or otherwise connecting the wires to your sensor. An example: The Sharp GP2D12 IR distance ranger requires a 5V input and a connection to ground. Depending on the proximity of the closest objects in it's field, it outputs 0-5V on a third pin. The Phidget analog cables have leads for 5V supply voltage, ground, and "Vout" - the voltage coming back from the sensor. I simply soldered a cut Phidgets connector wire to the short cables supplied with the sensor, making sure to match ground to ground, etc. If you're not familiar/comfortable with soldering, take a look at this detailed beginner's guide (check out the photo gallery on soldering). I don't know of any publicly accessible labs with soldering irons though - maybe in the ME buildings? To prevent short circuits from dangling bare wires it is a good idea to cover your connections with electrical tape or heat shrink tubing.

Let us now try to connect a new output device to Phidgets - a motor. You can pick up cheap DC motors like the ones found in RC toy cars from surplus stores like allelectronics.com. Wendy supplied me with a 3V vibrating motor (this is a normal motors with an off-center mass attached to the rotating axis). Most motors draw more current than the InterfaceKit can provide/sink, so you will need a secondary power source (think 2xAA batteries) and some sort of switch that uses the signal from one of the InterfaceKit digital outs to switch the larger motor current. A power transistor (again supplied to me by Wendy) will do the job. Pick a "NPN" type: it will switch on the load (motor) when the chip output is high (boolean true in the Phidgets API). This corresponds to a "normally open" switch, i.e., the motor is off unless you flip the switch/put voltage on the transistor's base. A "PNP" transistor works the other way around - analogous to a "normally closed" switch. You will also need a "snubber diode" (which only lets electricity pass in one direction) to prevent blowback voltage from the motors damaging your other components. Don't worry about the details, simply following the recipies on the last link should get you there. Here is a diagram of how your circuit should look like:

To build simple circuits you probably want to get a solderless breadboard (Pololu | allelectronics) and a jumper wire kit.

Bill Verplank's breadboard sketch from the music250 website

The Phidgets website has a page on Using DC motors with the PhidgetInterfaceKit 0/16/16. Some of this information can be reused for the 8/8/8 kit we have but note that the 0/16/16 Kit has high voltage outputs, which our InterfaceKit does not. Don't connect the motor directly to your board!

Links of the day

Orange Cone:
I've come across this blog multiple times now during my research into prototyping tools and other HCI topics. The article on Sketching and Protoyping is especially relevant. Webmaster Mike Kuniavsky is also the authors of the book "Observing the User Experience."

A comment on the site pointed me towards Hernando Barragan from Ivrea (the Italian interaction design school Bill Verplank is involved with). His Wiring project is "a programming environment and electronics i/o board for exploring the electronic arts, tangible media, teaching and learning computer programming and prototyping with electronics" and is based on the Processing language.

Looking for a history of prototyping, I found the following pages:
http://archive.cpsr.net/program/workplace/PD-readings.html
Castle Island's Worldwide Guide to Rapid Prototyping: http://home.att.net/~castleisland/home.htm

Small Form Factor TFT + Touchscreen Survey

Issues: resolution; power requirements; how to address? via pc/microcontroller? max refresh rate - suitable for animation or not? color or bw? backlit? touchscreen?

Product links:
http://store.earthlcd.com/
- ezLCD-001 is a 2.7" 240x160 512 color LCD display - $200 with RS232/USB board
- Wintek WDG243216WEBA 320x240 mono display, 3.5" - $40 but needs controller board
- XLK-5002T (XLK5002T) 6.5" Analog Color TFT LCD Kit with Resistive Touch - $800
- mARMalade 7.8" 640x480 w/ integrated touchscreen - $400
- MTR-EVUE-4BW 4" Monochrome LCD Monitor w/ S-VIDEO in - $90
- Sharp 4LU4EB 4" B&W Video LCD; 383x234 Monochrome LCD - CCFL, NTSC, Analog RGB, 8 Volt DC Mono

Jameco in Belmont sells mostly B/W text-only LCDs.

http://www.circuitcellar.com/avr2004/grand.html ATMEL-based TFT controller - BYO-LCD. By Michal Sieluzycki - Winner of the AVR2004 Design Contest

http://www.hantronix.com/
Sells PDA-sized "Chip on glass" LCD graphics/character displays. this document describes how to address their displays from an 8bit microcontroller (such as the ATMEL)

General links:
http://www.usdc.org/ United Stated Display Consortium - This document has some projections on the future of small screens. Has lists of display manufacturers.

Society for Information Display

Sharp LCD Integration & Applications Guide
LH79520 System-on-Chip for color LCD applications

Touch screens

Notes on "Your Wish Is My Command" (Lieberman - ed.)

Henry Lieberman (ed.)
Your Wish is My Command
Morgan Kaufmann, 2001
ISBN 0262140535 (google print info | bibtex)

• Chapter 1 - Novice Programming Comes of Age
  Canfield Smith et al.
• Chapter 2 - Generalizing by Removing Detail
  Kahn
• Chapter 3 - Demonstrational Interfaces
  Myers/McDaniel
• Chapter 4 - Web Browsing by Example
  Sugiura
• Chapter 5 - Trainable Information Agents for the Web
  Bauer et al.
• Chapter 6 - End Users and GIS
  Traynor and Williams
• Chapter 7 - Bringing PBD to CAD Users
  Girard
• Chapter 8 - Demonstrating the Hidden Features That Make an Application Work
• Chapter 9 - A reporting tool using programming by example for format designation
• Chapter 10 - Composition by Example
• Chapter 11 - Learning Repetitive Text-editing Procedures with SMARTedit
• Chapter 12 - Training Agents to Recognize Text by Example
• Chapter 13 - SWYN: A Visual Representation for Regular Expressions
• Chapter 14 - Learning Users' Habits to Automate Repetitive Tasks
• Chapter 15 - Domain-independent programming by demonstration in existing applications
• Chapter 16 - Stimulus-Response PBD: Demonstrating When as Well as What
• Chapter 17 - Pavlov: Where PBD Meets Macromedia's Director
  Wolber
• Chapter 18 - Programming by Analogous Examples
  Repenning and Perone
• Chapter 19 - Visual Generalization in Programming by ExampleSt.Amant et al.

Chapter 1 - Canfield Smith et al.

This chapter describes the Stagecast Creator software. End users are defined as users who have never taken a programming class. The target audience are children. The authors want to do away with textual programming language altogether. Their approach: combine PBD and visual before-after rules. Their first 'insight': in programming languages, the language aspect itself is the problem. Furthermore, "we concluded that no conventional programming language would ever be widely accepted by end users." (9) A mentioned problem of PBD: how to represent a recorded program to users. The authors' second 'insight': do not represent each step; only the start and end states (10). Programming is kept in domain terms (11). AgentSheets by Repenning is cited as being similar in approach. The article refers to Don Norman's gulf of evluation/gulf of execution in explaining that the gap between a problem and its representation in a "conventional" programming language is too big for novices. Some theory: Sloman's approach to mental represenatations: analogical vs. Fregean is introduced - in the former, "the structure of the representation gives information about the structure of what is represented." Users fare better with the former, but conventional programming languages offer the latter. Bruner's approach: any domain of knowledge can be represented as: enactive, iconic, symbolic. The last type is Fregean, the others analogical.

Chapter 2 - Kahn

Description of the ToonTalk system. Target audience again are children. The authors use the term "animated programming" to describe their system - "programming abstractions are replaced by tangible analogs" or "concrete familiar objects". (22) Generalization step: remove details from the conditional part of a rule. Underlying idea: animation and computer game technology can make programming easier. No textual representation, no pictorial programs, instead a virtual animated world. Graphic design obscures rather than clarifies what is going on here. 41: "Pygmalion made the task of programming more concrete by letting programmer manipulate iconic representations of programs and sample data. Tinker made things more concrete by letting programmers manipulate sample data and descriptions of computations." Objects stand directly for computer terms, not analogical representations as in Creator from chapter 1.

Chapter 3 - Myers/McDaniel

Definition of demonstrational interfaces. Generalization can happen automatically through inferencing or explicitly by seeking user input. Inferencing can come up with incorrect behaviors. The authors describe, compare and contrast various levels of inferencing done in their own prior work, which includes the following systems: Peridot, Lapidary, Jade, Gilt, Tourmaline, C32, Pursuit, Gold, Marquise, Katie, Turquoise, Topaz, Gamut. Some of these work from single examples. Systems without inferencing: their Topaz, as well as Pygmalion and SmallStar. Simple rule-based inferencing: Peridot - test-feedback-action components. AI algorithms: Gamut infers from multiple examples. "It is still an open problem to balance the sophistication of the system with the expectations of users." Questions: what kind of feedback is provided about learned criteria? How can the user edit the learned program? Is there a visible representation of the generated code? Lookup InferenceBear (Frank and Foley 1994) for its event language and Pavlov (Wolber 1997) for its event-based model.

Chapter 4 - Sugiura

Presents Internet Scrapbook and SmallBrowse. Introduces the importance of the application domain to the discussion. Two main problems of PBE according to authors: 1) How can system correctly infer user intent? 2) How can PBE component interact with existing applications.

Chapter 6 - End Users and GIS - Traynor and Williams

Surrogate Users - a human interposition layer between users and a software system that helps in getting data in and information out. Could happen in our scenario if design studios have enough of a budget. GIS is hard because of 1) technical terms and concepts (different from domain knowledge of lay users), 2) required mental model, 3) a lacking record of how a display was created (think of Photoshop's action history palette). Heuristics for developing a better next-gen GIS: 1) present information in terms of user's task. 2) protect user from needing technical expertise from cartography etc. 3) protect the user from having to know about the software architecture 3) provide a program representation of the steps for creating an information display. (comment: applicability of these heuristics seems to depend on how regular the user will interact with the app - is it a daily tool or a sporadic instrument used intermittently?) reconceptualization is key for success of technical software in mainstream market. Discussion of C-SPRL follows - comic-strip metaphor; extension of Pursuit language. Doe it solve the problem of GUI complexity/

Chapter 7 - Bringing PBD to CAD Users - Girard

Describes the LIKE and EBP (Example-based Programming in Parametrics) systems. Targets knowledgable domain-expert users, not "novices" or "beginners." Support the design of "repetitive or conditional" parametric design. Lookup Glinert 1990 Visual programming, Roller 1990 Variant programming. Quote: "no PBD system, to our knowledge, has reached the same expressive power as conventional programming in its application area." (136) References Nardi's ASMOP. CAD is appropriate because relationships exist between graphical entities and designers know what the relationships are. Existing variational CAD systems combine sketching and constraint specification to compute an exact model - most efficiently through graph reduction (141). Parametric systems observe user

Chapter 17 - Pavlov - Wolber

Stimulus-response rules for a 2D Director-like animation system that does away with script-writing.

Chapter 18 - Repenning and Perrone

Describes AgentsSheets - pwered by Programming by Analogous Examples

Chapter 19 - St.Amant et al.

Visual generalization. The "data description problem": how and at what level of granularity should we describe actions and objects selected by user in a PBD system? Their argument: use visual appearance - screen pixels. Image processing is powerful enough and you won't have to rely on vendors supplying a nice API to access their apps' data model. "A PBE system might gain significant benefits if it could work in the same medium as a user, if it could process the visual environment with all its information." Reference to VisMap.

Notes on "A Small Matter of Programming" (Bonnie Nardi)

Bonnie Nardi
A Small Matter of Programming
MIT Press, 1993
ISBN 0262140535 (Amazon info | bibtex)

• Chapter 1 - Introduction
• Chapter 2 - Conversation and Computers
• Chapter 3 - Task-specific Programming Languages
• Chapter 4 - Interaction Techniques for End User Application Development
• Chapter 5 - Application Frameworks
• Chapter 6 - Collaborative Work Practices
• Chapter 7 - Scenarios of End User Programming

Chapter 1 - Introduction

The motivation for this book: end users should be able to program their own computers - we should give them control over the computational resources on their desks. That control should not be restricted to a small elite caste of professional programmers (page 2). "Our technical choices advance social goals." [Dunlop and Kling 1991] (3) - my interpretation: end user programming (EUP) is democratic. Who are end users? They are not casual, novice or naive - rather, they are serious domain specialists who rely on computers for daily work but who do not want to become professional programmers or learn general purpose programming languages (5). Nardi mentions two successful end-user computing systems she studied through a number of ethnographies: spreadsheets and CAD packages. These two examples recur throughout the rest of the text. Nardi's focus is on "complete end user application development" - not isolated development stages such as UI design or prototyping.

Chapter 2 - Conversation and Computers

Why do we need EUP systems at all? Casual human-human conversation is a particularly bad model for HCI - we rely heavily on context in our interpretation of speech. Context is hard to impossible to grasp for a computer system. So we need a different model: the critical observation is that conversation is fundamentally practical and goal-directed - we adapt it to the particular setting and function. The purpose of a language act is of vital importance. Court proceedings use formal language. Formal languages are good for making unambiguous precise statements. End users have problems with specific programming languages such as C or Lisp, but not with the concept of a formal language itself.

Chapter 3 - Task-specific Programming Languages

We should focus on accomplishing tasks and formal communications systems that best support that set of tasks (29). Examples of formal languages used by non-programmers are baseball notation and knitting notation. "Ordinary people unproblematically learn and use formal languages and notations in everyday life." (37) Formal languages are readily learned if their users have a particular interest in something and the formal language expresses elements and relations of that domain of interest. Gardner 1985: "The soundness, the speed, and the complexity of the reasoning that individuals exhibit seem primarily a function of the degree of familiarity and organization of the materials being processed, rather than a function of any special or general ability of the person doing the reasoning." (38) EUP languages should be task-specific. General purpose languages are too far removed from tasks faced by users (39). Nardi devotes a long section to the spreadsheet programming model. Important characteristics of the spreadsheet language are: "high-level task-specific programming primitives, accessibility, and simple but useful control constructs"(41) as well as the textual nature of the programming language (49). Problems of task-specific languages are that they are expensive to build; that users have to learn many different interfaces; and, most importantly to Nardi, that it is difficult to pinpoint right level of task-specificity (50). Languiages should address the problem of extensibility. And don't forget about ease of use (52). To develop right task-specific language, it is imperative to study the tasks - Nardi advocates Activity Theory by Leont'ev and Distributed Cognition (p54 ff).

Chapter 4 - Interaction Techniques for End User Application Development

There is no silver bullet for EUP, but various strategies and programming models may be effective for sub-systems. Models discussed include visual programming, form-based systems, programming by example-modification (my note: look at php.net site with user-contributed code snippets), programming by example (with a lengthy discussion of its shortcomings *SHOULD BE SUMMARIZED HERE*), and automatic programming by informal program specification

Chapter 5 - Application Frameworks

Successful EUP systems will be hybrid systems that employ some combination of graphical and textual elements. Examples of hybrid approaches in spreadsheets, Logo, and CAD software are discussed. Spreadsheets are very hard to debug. Visual Formalisms are advocated as application frameworks - their most important characteristics are: exploitation of human visual skills, manipulability, specializability, broad applicability, and familiarity. Existing visual formalisms are graphs, plots, panels, maps, outlines, and tables. Powerful programming capabilities are derived from coupling a visual formalism with a textual formal language.

Chapter 6 - Collaborative Work Practices

There exists a continuum of programming skill from end users via local developers (or tinkerers) to professional programmers. (Can we expect all designers to be at least tinkerers?) How much of the work is done collaboratively (synchronously or asynchronously)? Nardi reports that customization files are often shared. My own experience shows that user communities are a crucial component for making programming by example modification work. Look at the PHP documentation at php.net - more often that not I just have to go to a particular function's help page to find user contributed sample code that solves my problem. Supporting local developers (at another point called "gardeners" when their support responsibilities become more formalized) to develop new macros (or input types in our TUI settings) is desirable. It is a good idea to enable more complex expert customization by allowing macro languages or linking-in of add-on objects created in a general programming language (layering) (110). PBE assumes end users do not gain expertise over time and always work in isolation (115).

Chapter 7 - Scenarios of End User Programming

Summary: "[D]esigners should start with a clear sense of and respect for the tasks that end users will be doing, and then design a system that best supports thse tasks." (125) Further ingredients to success are task specific programming languages, visual application frameworks, and support for collaborative programming practices.

Bibtex entry:

@book{160165,
 author = {Bonnie A. Nardi},
 title = {A small matter of programming: perspectives on end user computing},
 year = {1993},
 isbn = {0-262-1405305},
 publisher = {MIT Press},
 }