"To some extent, usability is a narrow concern compared to the larger issue of system acceptability, which basically is the question of whether the system is good enough to satisfy all the needs and requirements of the users and other potential stakehold ers ..." ((Nielsen, 1993)).

This workshop addresses usability within Lindgaard's (1994) four categories of usability issues:

• navigation
• screen design and layout
• terminology
• consistency and match with the user's tasks)

• needs analysis
• information design
• stakeholder issues

Characteristics of usability testing

context-specific

Although the results of specific usability tests may be applicable to any number of similar design projects, the tests are designed to provide data to a specific project team about a specific audience and a specific set of goals. Thus, the methodolog y must be appropriate to and suffiecient for the context of that project - not all possible projects - and project teams should resist the temptation to generalize their findings across projects. In addition, tests are conducted as nearly as possible unde r the conditions expected to prevail when the product is put into use.

data-driven

The basis for design decisions in a usability-oriented design process must be data derived directly from observation and other methods, not from opinions or speculation. The design team uses every resource at its disposal (including opinion and spec ulation) to construct prototypes they consider likely to be successful, but they use data to find their errors and direct their design efforts.

descriptive, not prescriptive

While helpful and necessary prescriptive principles are emerging in the field of human-computer interface design ("Keep the user in control"), the usability process takes a descriptive approach ("These users are frustrated when the protoype program m akes them decide which disk format to use"). There is no presumption that a general principle will inevitably prove itself under the conditions of use peculiar to this product, and there is no expectation that the discovery of a design problem will automatically be accompanied by a clear remedy. It is this characteristic which renders inappropriate the "pass-fail" approach to usability. A product may reach a metric-based goal in usability testing (and in that sense "pass the test"), but it may neve r be declared to be unconditionally successful - only satisfactory.

flexible and pragmatic

Sequences of tests may be outlined for various development situations, but the details and timing of usability activities must be tailored to the emerging circumstances of a project if they are to achieve their best impact. Since the results of the t ests are not to be generalized, emphasis is placed on obtaining valid and useful data to guide design efforts - not on controlling variables to pinpoint specific causal relationships (although many usability tests are constructed quite rigorously, particu larly when a product must ultimately perform within very strict parameters.)

Why conduct usability tests during the development of a Web site?

• system development is expensive, and supporting a poorly designed system is even more expensive
• users are capable of rejecting or working around systems that do not meet their needs
• designers are demonstrably poor judges of what users want and need in systems
• even the best designers are not representative of the users of their systems
• usability testing is an inexpensive way to improve the first iteration of a new system before it is implemented
• people actively engaged in tasks typically do not expend more effort than necessary in learning the mechanics of the system they are using

Existing products/services

Often overlooked, the products and services that users are accustomed to using now are a rich source of data for every design team. In the zeal to replace old systems with new ones, we may forget several key points:

people have often developed comfortable relationships with existing systems; design teams should understand the current perceptions users have (not the ones we presume them to have) before a new system is developed

people may not perceive the extent to which they are adapting to inconveniences within existing systems; design teams need to discover all the problems to be remedied by a new system, whether users have reported them or not

people are often genuinely supported by existing systems, and are sometimes supported by the very features that might objectively be termed "undesirable"; design teams must not inadvertently eliminate helpful aspects of an existing system in the proc ess of replacing that system

people are frequently using existing systems in ways we did not suspect; design teams may capitalize on this knowledge to build "unexpectedly delightful" features into a new system

Needs, Goals and Tasks

Goals should be considered from at least two perspectives: the goals of the project stakeholders and those of the users (the goal of the design team is presumed to be constant - deliver an acceptable product - although, when primary stakeholders are also designers, the issue gets complicated).

Needs and goals may not be entirely synonymous ... for example, users may need critical information but decide to do without it because their overiding goals are "finishing quickly" or "not looking stupid." Administrators may need to provide complex instr uctions about enrolling in college, but be reluctant to undermine their goal of making the university feel like a "friendly place" to prospective students. Goals will usually win over needs in user and stakeholder behavior, so discrepencies here should be given special design attention.

Tasks are tricky because the new system under development is often presumed to be going to change the tasks people are currently doing. Task analysis should concentrate both on the way users currently satisfy their needs and on the underlying processes th at will have to be supported regardless of the system people are using.

Design

The various aspects of design tend to be covered in both paper prototype and electronic prototype testing, although information design may be most efficiently tested on paper and navigation design most effectively tested electronically.

• information design
• navigation
• screen design/layout
• symbols, buttons, and icons
• terminology
• consistency/match with user's task

Our workshop example for conducting usability testing will be the Indiana University Bloomington home page redesign project conducted over the summer and fall of 1995. Usability testing for this project was designed and coordinated by Dr. Ted Frick.

Report on Usability Testing

Current Indiana University Bloomington home page

New Indiana University Bloomington home page

Nielsen, J. (1993). Usability engineering. Boston: AP Professional.

Lindgaard, G. (1994). Usability testing and system evaluation. London: Chapman and Hall.