Getting ready to run your first user test

Share on facebook
Share on twitter
Share on linkedin

Running and facilitating a usability test is not a super-difficult task if you have a clear view of what you need to plan and execute it.

We’ve put together a brief checklist and some advices that will guide you trough the setup of your first usability test.

What to test

It can be a live website, a pre-production fully functioning version, a set of interactive prototypes or even some handmade sketches.

Usability tests can be performed at every stage of a design process, providing different insights according to what and when you are testing.

Prototype testing allows you to evaluate your design in an early-stage of the development process, where changes can be done easily and effortlessly. It also allows you to assess if the overall concept meets your users’ expectations and if it is going in the right direction.

On the other hand, only by testing a final design you are able to discover usability issues related to the visual aspect of your website.

Ideally, you should perform user tests all along the development process.

However, if you have limited resources, test as early as possible and make sure that the level of details of your stimuli (that is, what you are testing) is aligned with the goals of the study.

For instance, if you are testing a prototype and you just want to assess the reaction of your potential users to your new design, having some “lorem ipsum” text here and there is acceptable.

But, if you are ready to launch your website and you want to fine tune it, make sure that you are testing the final contents and design, otherwise you will miss important insights.

Goals & tasks

In order to conduct a user test you need a testing script. A testing script includes what you will ask your users to do during the study and the questions you will ask them.

As a general rule, sessions should be short enough to ensure participants won’t get too tired and they pay attention to you and the tasks they have to perform.
If you can, try to limit each session to 60-75 minutes, which means 4-5 tasks of 10-15 minutes each.

Choosing what to include in your testing script is easy if you are validating something very specific. For instance, whether your users are able to operate with a new checkout process or if your new product makes sense to them. When the object of your study is validating something , design your script focusing on that and resist the temptation of adding stuff that is not related to your new feature.

On the other hand, if your goal is to perform a general inspection of your website and to find as many usability issues as possible, you will need to prioritize. Not all your website features are equally important.

Choose the most relevant features, those that have a direct impact on your business and those that users are supposed to use more frequently or are especially important to them, and design your study accordingly.

For example, if you want to test an e-commerce website, you should spend at least one or two tasks on products search and purchase, trying to address different use cases.

On the other hand, unless you came out with a new innovative password recovery process and you have plenty of doubts about it, you probably won’t need to test the password recovery journey.

Having clear goals does not only determine which features to test but also when to test them.

If you are concerned about the visual design of your website, you should not test prototypes.
If the interaction is what you are unsure about, focus your test on it and plan your fieldwork for the day you can provide your participant with a simulated interaction.


Of course, you can’t perform a user test without some participants. But, how to find them?

The way you recruit them is not really important. If you have enough resources, you can contract a recruitment agency or, on the contrary, you can do it yourself and rely on friends and acquaintances.

What really matters is that all of them must be potential users of the product you are testing.

For instance, if you are testing a travel website, recruit people that use to purchase their travel tickets online. Is it a luxury hotels website? Then, probably it is not a good idea to recruit people on a low income, as they have different needs from your target user.

You should be very careful when defining who will take part to your study, since participants that are not your real users provide feedback that may lead you to take very bad decisions about your product.

You should also exclude what we call professional participants. Professional participants are used to take part to lots of usability studies, they know how they work and tend to behave in an unnatural way.

Finally, you should not invite anyone working in the same sector of the product you are testing nor people that are somehow related to it, since their comments will be biased by what they already know.

How many participants should you test? Jakob Nielsen had the answer: 5 of each kind are enough to discover most of the usability issues

A place where to run your test

You don’t need an ultra-modern usability lab to host your tests. Every room that is quiet and free of distractions can serve the scope.

Prior to the sessions you have to make sure that the testing material is working correctly on the device you will use by running the test yourself and checking all the possible user journeys.

You should also have the possibility to record the screen, the audio and, ideally, the face of the users during the sessions, so you can share them with other people involved in the project. Recordings are also very useful as you will be able to review them in case you missed some information or have doubts on how to interpret the results.

If your client wants to observe the sessions you don’t need to have an observation room with a one-way mirror. Recording tools like Morae allows you to stream up to two camera angles plus the audio to another computer on the same network.

If you are looking for something simpler and free, Skype and Google Hangouts can serve the scope, even if you can’t record the session with them.

A template where to take your notes

A well-structured document where you take note of the findings during the sessions can help you to save a lot of time and improve the quality of your analysis.

You can organize it by task, page (if you are testing an interface) or topics and should be built so that you can minimize the amount of data you have to insert in it (read my tips on how to build a good user test note template).

Before running your sessions you should have the chance to review the webpage you will be testing and to make some hypothesis about which aspects are more likely to present usability issues. If you had, make sure that your template includes a section where to take note about those aspects. 

We are experts in User Research. We test with real users to help you create usable products and great user experiences.

Tell us about your project and discover how we can help you.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.