Moderating a usability test? Don’t make these little big mistakes.

mm

Moderating a usability test is a task that requires good planning, concentration and attention to details.

Knowing the basics of good moderation is not always enough to ensure success.
There are many other things that nobody usually tells you and you discover only after hundreds of hours of testing – and several mistakes- .

This is an experience-based short list of things you should avoid to do when you are moderating a usability test.

Ignoring how the interface works.

The most obvious rule: if you are testing an interface, you should know how it works. Strangely enough, this does not always happen.

Bad facilitators revise and learn only the ideal path for accomplishing a task.
Participants, however, do not always succeed and follow different routes when trying to do something.
When a participant takes an unusual path, those researchers that haven’t inspected the entire interface may get lost exactly as their participants do.

Stimuli revision should not be limited to ideal paths.  It is always worth spending a couple of  hours to have an overall view of an interface in order to be ready to understand what your participants are trying to do.

 

Forgetting to provide a printed task description to your participants.

Even though the task is very simple, your participants may always need to recheck it during a usability test.

If you provide a printed task description rather than just telling them what they have to do, you will help them focus on the task and prevent interruptions during the sessions.

Always ask your users to read out loud the description before starting. In this way you will make sure they fully understand what they have to do.

Imagine a complex task with lot of details to remember such as:

“Book a return flight leaving from London on Monday and returning from Frankfurt on Sunday for two people. Choose the cheapest offer available.”

Without a printed description, your participants could make mistakes because they forget the task and not because of usability problems.
A printed task description will reduce those errors and help you discover real usability issues.

 

Letting user describe everything they see instead to think aloud.

Before starting a session, the facilitator should welcome the participant and explain him how the test works. Also she should tell him how to behave during the session.
Participants are often asked to think aloud when performing the task. That is to say, to express whatever comes to their mind during the session.

Thinking aloud is not something people do spontaneously. However, it provides information that the researcher would not be able to discover in any other way.

Sometimes, participants think that what they have to do is to describe and comment anything they see, even though they would not care about it in a real life situation.
They may also enter in what we call “expert mode”. They think they have to evaluate the interface and that the researcher is only looking for their opinions.

This behavior distracts participants from their task and make them behave in an very unnatural way.
Also, it is not providing any useful information to the researcher. On the contrary, it adds a lot of noise to the data collected during the session.
To avoid that, facilitators should explain clearly to their participants what they are expecting from them and eventually redirect them to their task if they start behaving like experts.

 

Providing fake data at the beginning of the task and not in context.

Certain tasks require participants to fill in forms and provide personal information. This is the case of online bookings or product purchases.

Sometimes, you can’t simply ask participants to use its real data because it may be too sensitive.
For example: credit card numbers, bank account information and national ID numbers. These are information you should avoid asking to provide in most of the cases.

When such information is required to accomplish the task, the facilitator usually provides some fake data the participant can use.

The how and when you give the information is very important.
If you provide it together with the task description, that may represent a bias.
The reason is that the availability of a certain piece of information implicitly suggests two things regarding the task:

  • participants will be asked for it during the process.
  • it is mandatory to complete the task.

In a real life situation, the users of a website can imagine that they will have to provide some personal data during a purchase. However, they might not know exactly which one.

Before starting, you can explain to your participant that, if the interface requests any data they don’t want to provide, they can ask you for it.  You can keep those data hidden until then.

Related Posts

Showing 2 comments
  • Vannessa Uhlein
    Reply

    Great post! I’d add “Using funny fake data” or “unreal fake data”, which can distract from the task and converts the context in unreal or not serious. Like strange names (= Luke Skywalker) or dashboards with thousands of leads. This is something the moderator needs to agree with the (playful) designers 😉

    • mm
      Stefano Serafinelli

      Thanks Vannessa,
      And yes, I definitely agree with you. I have tons of examples like this 🙂

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.