Crafting surveys isn't just about stringing together a series of questions—it's an art form that requires finesse and strategy. As we explored in previous articles and literature on the topic, every word, every option, and every sequence can influence the responses you receive.
As UX researchers, understanding the nuances of open and closed-ended questions is crucial for designing surveys that yield accurate and meaningful insights. Let's delve into the complexities of survey design, exploring the implications of choosing between open and closed-ended questions, and how they can shape the outcomes of your research.
“The way you ask a question often determines the kind of answer you get—and one of the first decisions you have to make is: are you going to ask an open-ended or a closed-ended question?" - Fio Dossetto
The Power of Question Types: Open vs. Closed-Ended
In research, we employ two types of questions:
- Open-ended questions permit participants to provide unrestricted, free-form text responses.
- Closed questions, also known as closed-ended questions, limit participants to selecting from a predetermined set of possible answers.
Let’s look into each of them in detail.
Closed-Ended Questions: The Guided Path
Closed-ended questions restrict respondents to choose from a finite set of answer options. They can either select one of the options (single select - Radio Button) or multiple options (multiple-choice - Checkbox). These answer options can range from single-word responses (such as 'yes' or 'no') to rating scales (ranging from 'strongly agree' to 'strongly disagree'). Like the example below of a Likert Scale in a survey, another type of closed-ended question.
Closed-ended questions provide a structured path for respondents to follow, narrowing down the possibilities and simplifying the decision-making process.
One of the most widely used closed questions in research is the Net Promoter Score® (NPS) survey, where respondents are asked to rate their likelihood of recommending a product or service on a scale ranging from 0 to 10.
Example of an NPS question. https://www.customerthermometer.com/net-promoter-score-nps/nps-survey-questions/
Think of closed-ended questions as signposts along a well-marked trail, guiding respondents toward specific destinations.
Open-Ended Questions: The Uncharted Territory
These are questions that necessitate more than a simple 'yes' or 'no' response, prompting respondents to elaborate on their perspectives or provide additional details.
The primary advantage of open-ended questions lies in their capacity to unearth insights beyond initial expectations.
Let’s try to compare them in the following table:
How to choose between open and closed-ended questions?
As UX researchers, it's essential to navigate the terrain of survey design with care and precision. Here are some best practices to consider when choosing between open and closed-ended questions:
1. Do you know what you want to learn?
When you're after data you can plot on a graph or use to show percentages, throw out a closed-ended question. For example: "How often do you shop online? (a) Daily, (b) Weekly, (c) Monthly, (d) Rarely, (e) Never".
But if you're itching for deep insights into your customers and their wants, go for an open-ended question. It is also the ideal format if you are looking for spontaneous answers. For example: “Could you name all the cosmetic brands you have heard of or tried?”
2. Combine and conquer.
Let’s take this quite frequent structure as an example.
Start with a closed question: Please rate your experience, from 1 to 5, being 1= very unsatisfied and 5= very satisfied.
Then follow up with an open question: Can you briefly explain your rating?
This way, you can easily graph the satisfaction levels but also have a starting point on why the experience is positive and or negative.
3. Make a fun and short survey.
Also, switching and altering the type of questions keeps the motivation and interest in your respondents up. So make sure to have a mix of open and closed questions.
4. Open up questions when unsure or at risk of unnecessarily excluding.
If an open question is not possible, we should at least always offer an “other” option.
Conclusion
Happy surveying!
