| home| reg:exp| projects| music| papers| visuals| books| about|   

January 18, 2005

Programming by Demonstration

Overview of Programming by Demonstration (PBD):
PBD systems record user action within the application interface and try to infer rules by generalizing from the observed data. Thus program logic is generated from direct interface manipulation. The crucial component of inference and generalization makes these systems more powerful than mere macro recorders. While PBD was an active field in the early 1990, publishing activity has diminished markedly in recent years. Most researchers that were part of the first wave of PBD have exited the field. Few systems have made it past the toy stage into commercial production. This page attempts to provide a complete collection of links to PBD sites that are still active and maintained. It is broken up into various dimensions along which we may want to divide the PBD field.
Related terms and research areas: programming by example, instructible systems, end-user programming, natural programming.
Research foci:
PBD systems have mostly concentrated on augmenting a few classes of applications, described below:
  • End-user programming is concerned with providing non-experts with tools and techniques to control, automate, and customize their software applications or intelligent physical devices. Teaching basic programming paradigms is also an often-used objective. Techniques have to be simpler than procedural programming and often try to leverage real-world metaphors or visual programming techniques. Howie Goodell's EUP webpages have a plethora of information on the subject. See also Bonnie Nardi's A Small Matter of Programming
  • Repetitive task automation software functions like a macro recorder that learns patterns of user activity and can then execute those patterns autonomously. Tessa Lau's recent work falls into this category.
  • UI customization, personalization tries to detect idiosyncratic interface use over time and change interface appearance to optimize access to frequently used functionality.
  • PBD for Robots: probably most useful for our own project. Robotic manipulators (mostly "arms") are programmed by direct movement; human analogical performance picked up through vision or other sensors; or iconic demonstration in software.
Publications:
Two books have been published, each a collection of papers describing research systems. These books provide good coverage of the research area through 2001. Some papers published since 2001:
  • Dey, A.K., Hamid, R., Beckmann, C., Li, I., Hsu, D. a CAPpella: Programming by Demonstration of Context-Aware Applications. CHI 2004, ACM Conference on Human Factors in Computing Systems, CHI Letters 6(1): to appear. April 24-29, 2004. (pdf)
  • Montemayor et al., Physical programming: designing tools for children to create physical interactive environments, 2002. (abstract | long paper)
  • Markus Ehrenmann, Oliver Rogalla, Raoul Zöllner and Rüdiger Dillmann. Teaching Service Robots Complex Tasks: PBD for Workshop and Household Environments. In Proc. of the IEEE Int. Conf. on Field and Service Robotics 2001 (FRS), Finnland 2001. (pdf)
  • Chris Beckmann, and Anind Dey. SiteView: Tangibly Programming Active Environments with Predictive Visualization
    Intel Research Tech Report, 2003. (pdf)
  • Rob Hague, Alan F. Blackwell and Peter Robinson. End-User Programming in the Networked Home.
    Paper presented at The 1st Equator IRC Workshop on Ubiquitous Computing in Domestic Environments, The School of Computer Science and Information Technology, The University of Nottingham, 13-14th September 2001. http://www.equator.ac.uk/ (pdf)
People:
Researchers with active homepages:
Some Systems:
Events:
Other info collections:
Leftovers: Links that are not directly relevant or outdated:
Reachable, but stale: Wayne Citrin (tech reports@colorado), Martin Frank.
TUIs: Rob Jacob
Ullmer/Ishii - Emerging Frameworks for Tangible User Interfaces. In “Human-Computer Interaction in the New Millenium,” John M. Carroll, ed.; © Addison-Wesley, August 2001, pp. 579-601. (pdf) Posted by Bjoern Hartmann at January 18, 2005 6:42 AM