Keith Instone

This appeared in Web Review, October, 1997. This is part 1. Part 2 is Site Usability Heuristics for the Web.

One of the most important ways to make your Web site easier to use is to do testing with users. But there are several other techniques in the usability engineering arsenal that can be applied to the Web.

Today, we'll talk about heuristic evaluation. Heuristic evaluation involves the study of a user interface by a small set of evaluators (3-5 is recommended) who look for violations of common usability principles (rules of thumb, aka heuristics). The problems identified in a heuristic evaluation could be tackled right away, leading to a redesign; might be used to focus a usability test; or could be used elsewhere in your user-centered design process.

Heuristic evaluation is well-suited for the Web because it can be easy, fast and inexpensive. Often the evaluators are specially-trained usability specialists, but you can also use people with just an hour of training. If your site is already "live" or on a staging area, then your evaluators can be spread out across the globe.

Basically, heuristic evaluation involves identifying your heuristics, gathering opinions about the usability of your site, merging and rating the problems that were identified, and then trying to work toward solutions.

Heuristics

The rules of thumb you use are the most important part of a heuristic evaluation. They set the stage for identifying problems and provide the vehicle for finding solutions.

Jakob Nielsen originated heuristic evaluation in the early 1990s and has compiled the most comprehensive set of heuristics.

He did not just come up with this list off the top of his head, however. They come from analyzing more than 200 usability problems and statistically reducing the set to the 10 most important rules:

1. Visibility of system status
2. Match between system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation

His data came from software systems before the Web, but the heuristics are still very applicable. In an accompanying piece, I describe some of the specific ways that a Web site can follow these heuristics.

If you look at these heuristics and think "these are obvious," then good. Hopefully you have already been thinking about them in your designs. A heuristic evaluation can be just a way to formally consider each rule to make sure you are not forgetting anything.

Gather Opinions

After you have your heuristics, find a few evaluators, give them some basic information about your site (intended audience, purpose, etc.)and let them look at it. The site does not even have to be working, unlike in a usability test where users are trying to actually do something. You may only have prototype screens to be looked at. It might only exist on paper.

You might provide the evaluators with some scenarios of expected use or a site map to help them become acquainted with your site quicker. Then the evaluators make a few passes through your design and after a short while, they can start listing the usability problems they find.

The evaluators should list each error in the context of your well-known usability rules of thumb. "This is a system status problem and that is a flexibility problem," for example.

Merge and Rate

It is important to realize that an individual evalution is an opinion and should be treated as such. The benefit comes when you combine your 3-5 evaluations together into a larger list, removing duplicates. After merging, have your evaluators rate each problem, those they found themselves and those found by others.

Usually, you use a 5-point scale, with zero meaning "I do not think it is a problem," 4 indicating a "usability catastrophe," and less serious problems rated in between. Calculate the average rating for each problem and see what stands out as the most serious.

If all of your evaluators agree that something is a catastrophe, then it should be high on your list to correct. Other errors will be ranked lower and can be tackled as time and resources permit. Besides finding problems, a heuristic evaluation helps you prioritize what to focus on next.

Work toward Solutions

Still, this doesn't seem all that great, does it? So, you get a few people's opinions of what is wrong with your site. But how the heck do you correct the problems?

Sometimes the hard part is simply identifying the problem and the solution is then straightforward.

For example, under the "be consistent" heuristic, your evaluators might point out some very specific inconsistencies. Perhaps your list of products is alphabetical with bullets on one page and numerically-organized and bulletless on another. The fix may be as simple as deciding on one of the two formats and changing the other product list. Or, you may decide this bit of inconsistency is OK. But at least you thought about it.

Other times it is not so obvious how to deal with the problem. That's were the heuristics come in. Since each error was mapped to a general rule of thumb, you can work "top-down" from that heuristic to more specific implementation strategies for your particular site.

For example, if you have a lot of consistency problems, you could come up with your own conventions for ensuring a consistent site: same background, home button in the same location on the screen, a dictionary of terms, whatever. Write up a "conventions for ensuring consistency" document that everyone can refer to for solving future problems.

Also, you can use the heuristics as a basis for discussion with your colleagues. Their site may be different enough that they do not have the same usability problems, but they will have similar ones. Ask your friends how they tried to prevent errors or integrated their help. See if their solutions will work for you.