UX research allows us to take decisions based on data backed by the opinion and observation of our users. Choosing the best technique for our analysis can be may be complicated due to the many techniques available.
As an example, the terms quantitative and qualitative are often confused in research. Contrary to what you might think, the difference between both is not referred exclusively to the number of users participating, but to the type of data collected during the research.
The choice between quantitative and qualitative techniques depends especially on our research purposes and with this article we want to explain and clear some doubts of the usage of each technique.
When to use quantitative or qualitative research
Before deciding which technique to use, it’s necessary to ask ourselves what we want to achieve with the study.
Do we want to understand the motivations of our clients? Or do we want to find out what version of our shopping cart sells more?
Qualitative research is focused on discovering emotions, experiences and motivations. With the qualitative techniques we can solve the reasons and questions that the users make when encountering our product or service.
The qualitative techniques are usually used in the early stages of a design project, with the goal of obtaining information that serves us as a base during the following phases of the process.
In this phases is common to use techniques that collect mostly qualitative data like interviews, focus groups, ethnographic studies or direct observation. This techniques are useful when we want to observe data as reasons for decision making, impressions, opinions or reasons that support the opinions of users.
The qualitative research also can give us much “collateral” information about the users that can serve us to communicate a service, like consumption habits, emotional impressions or even personal stories related with the brand or service and the language they use to describe them. This can be a very useful to create the brand storytelling.
It’s also common to develop qualitative tests throughout the design process before launching a product. With the usability tests we can find usage problems and get to know the perception that users have of our product.
To be able to perform the qualitative research, the number of participants don’t necessarily need to be big. With these methods, we are interested in detecting patterns and not quantifying our findings statistically.
Normally, user tests and interviews performed with 8 to 12 people, depending on the profile and objects of study, although using 5 users is also effective to use recurrent insights.
In the qualitative research is essential that users are selected and represented because generally the number of users is more reduced, that’s why it’s essential to choose the most adequate participant.
We could define as quantitative all the data that can be expressed numerically. Actually, we can be interested in finding metrics and understanding the metrics and the scale of our findings.
The data obtained with this type of investigation can be conformed to a statistical analysis and be used to optimize and improve products and services and to obtain a general and trustworthy vision of specific subjects.
The most common techniques that provide quantitative data would be: web analytics, A/B tests, surveys and quantitative evaluation techniques. They are useful to estimate the gravity of the problems, selecting the most successful design, comparing the ROI, etc.
They can be commonly used at the beginning or end of a design process (as surveys) ,to quantify the findings obtained with quantitative techniques, to find improvements and a better optimization in the current design.
When performing a quantitative research it’s necessary to question with what are we comparing and which key metrics are going to be considered during our study.
For example, during an user test, some of the most common metrics are:
- The time the user has to finish the task. If it’s compared with previous versions and the time is shorter, it’s easy to think that the design has been optimized.
- The difficulty of the task. It’s usual to compare the estimated difficulty before the task and compare it with the difficulty considered upon completion.
- Usability scale (SUS). It’s an usability questionnaire which use is very extended and gives data about general satisfaction and usability.
- Net promoter score. Measures user loyalty based on recommendation criteria.
The ideal user number for a quantitative study depends on the technique we want to use and the error margin we are able to assume.
Jakob Nielsen recommends testing with at least with 20 users when doing quantitative research, although for studies like eyetracking it’s recommended to test with at least 39 users to obtain trustworthy data.
Even though both types of research allow us to obtain valuable data, it’s rare to be able to get valuable insight if we use only use techniques that get one type of data.
Quantifying the recurrency of a problem tells us if it’s more or less important but it doesn’t give us clues about the reasons that cause it and how to solve it. Here is where qualitative data would help us out.
Furthermore, not all problems have the same relevance and urgency. This is the moment when quantitative information help us out to prioritize the findings and establish our strategic plan.
We are experts in User Research. We find answers to complicated questions.