October 19, 2004
HCI Comments VI
How to Conduct a Heuristic Evaluation, Jakob Nielson, http://www.useit.com/papers/heuristic/heuristic_evaluation.html
Multiple people are likely to find more usability problems than a single evaluator. Nielsen argues that from a cost-benefit point of view, three to five evaluators provide the best payoff. Heuristic evaluation is contrasted repeatedly with user testing; however, no clear succinct definition of what constitutes a user test is given. Benefits of heuristic evaluation are likely to be incremental/evolutionary rather than revolutionary: emphasis is placed on small individual problems rather than large, conceptual issues. Because of the reliance on a set of heuristics, evaluators are less likely to find structural problems not well covered by these heuristics. Real work done with an application may expose a different set of problems. Nielsen's cost-benefit analysis is based on commercial software products. Open-source software is developed and distributed through a distinctly different model that necessitates a different kind of analysis. Cheap automatic evaluation techniques combined with the short cycle period between OSS release versions may help achieve better coverage of more usability problems than a single small group heuristic evaluation. Example.: the Firefox browser interface went through many different incarnations; more so than iExplore or another browser. Even though the changes from version to version were rapid and took getting used to - something commercial vendors may not be able to subject their customers to - the UI has now become stable and is more functional than that of other browsers.
Methodology Matters: Doing Research in the behavioral and social sciences, Joseph E. McGrath, in Readings in Human-Computer Interaction: Toward the Year 2000 R. M. Baecker and J. Grudin and W. A. S. Buxton, ed. pp. 152-169.
McGrath presents a clear summary of social science research methodologies. In terms of knowledge communicated this article is the most dense so far and the most efficient in terms of "data ink" (concepts-per-page). The article touches upon all important issues to consider when planning social science research, but the limited length prohibits in-depth treatment - the buzzwords are there, but many are not explained exhaustively enough for a practitioner. A more detailed discussion can be found in: Investigating Communication - An Introduction to Research Methods. L.R. Frey, C.H. Botan, P. G. Friedman, G. L. Kreps. Prentice Hall, 1991 (my undergraduate textbook). The importance of triangulation between different methods is stressed to offset the inherent bias/weakness of any one particular methodology. For HCI researchers, adopting rigorous soc. science research methods means an two-step removal from usual computer science practices - first one needs to replace developers' intuition with external evaluation. One then needs to qualify any particular evaluation by considering the effect of the chosen measurement instrument. The importance of collaboration and cooperation between researches and institutions emerges since the scope of producing research validated by multiple methodologies may be too large of a project for any single individual.
Measuring API Usability, Steven Clarke, Dr. Dobb's Journal Special Windows/.NET Supplement, May 2004, pp. S6-S9
Clark recognizes that APIs are as much a user interface as GUIs - so similar techniques can be used to evaluate their usability (breakdowns). Scenario approaches can help determine desired API functionality. Comparisons between user expectations and API affordances according to the "cognitive dimensions framework" help pinpoint potential areas of mismatch where an API needs to be redesigned.
While I am usually a proponent of concision, I would have liked this article to be more substantial. The absence of any references exacerbates the problem. Hiding the information of how to read the radial graph in Figure 1 is in itself a usability breakdown of the article.