A heuristic analysis is a usability evaluation method that, contrary to usability tests, does not involve real users directly. Instead, it is based on a list of guidelines or good practices called heuristics.
During a heuristic analysis, the examiner, typically a user experience and usability expert, reviews the interface and determines whether or not it complies with established heuristic principles.
What heuristics to use?
A complete description of the Nielsen and Molich heuristics, as well as examples of them, is available in our article “The Nielsen heuristic principles explained with examples”
These are very generic principles and many uxers prefer to use custom listings with more guidelines that allow to assess more specific aspects of the interface.
Where do heuristics come from?
It is important to stress that heuristic principles should never arise from the personal opinion of the expert. All heuristics should be based on the observation of real users and describe the problems they face with the interfaces.
The Nielsen and Molich heuristics are born from having carried out an enormous number of tests with thousands of users over the years and not from the intuition of their two authors.
How to analyze data from a heuristic review?
In some cases, heuristics are used as a simple checklist applied to all the elements of an interface. The examiners simply indicates whether they comply or not, assigning a percentage that indicates to what extent the interface meets the heuristics.
However, this way of working with heuristics tells us little about interface problems and how to solve them if it is not accompanied by more qualitative work.
The ideal way to report the results of a heuristic review should include an analysis of each problem detected that focuses on understanding the reasons for the problem and, ideally, suggesting how to fix them.
At the same time, it is important that the examiner also assigns a priority or importance to the problems detected, in order to guide the teams when creating an improvement plan.
Typically, issues are prioritized based on three parameters:
Frequency: How many users could be affected by the problem? All or just a few?
Impact: What potential consequence would the problem have on the user experience? To cause a few doubts is not the same as preventing a purchase.
Persistence: Is it a problem that would appear every time the user will use the interface or just a few times?
Limitations and strengths of heuristic review.
Heuristic review is almost always a cheaper and faster technique than a user test since it requires very little preparation and does not involve additional expenses such as the recruitment of participants.
However, it is not as accurate as a test carried out with users. Although it allows us to find a large number of problems, there will always be a few more that we will be able to detect only if we involve real users.
The reason is simple: heuristics are approximations based on the behavior of very large groups of people. They cannot go down to the detail of the particular mental model of each individual.
In addition, they are general rules that, although they can be customized for interfaces of the same type (eg, e-commerce, travel websites, music apps), are almost never tailor-made for a specific interface.
Also, only by talking to real users will we be able to dig into the exact reasons for the problems they had.
Lastly, even if they are assigned to expert professionals, heuristic reviews always depend on the human factor and on the subjective assessment of the examiner. What is a problem for one, might not be for another.