TeaCup Lab Logo

Open vs. Closed-Ended Questions in Survey Design

Mónica Bohigas
June 13, 2024

Crafting surveys isn't just about stringing together a series of questions—it's an art form that requires finesse and strategy. As we explored in previous articles and literature on the topic, every word, every option, and every sequence can influence the responses you receive.

As UX researchers, understanding the nuances of open and closed-ended questions is crucial for designing surveys that yield accurate and meaningful insights. Let's delve into the complexities of survey design, exploring the implications of choosing between open and closed-ended questions, and how they can shape the outcomes of your research.

The way you ask a question often determines the kind of answer you get—and one of the first decisions you have to make is: are you going to ask an open-ended or a closed-ended question?" - Fio Dossetto

The Power of Question Types: Open vs. Closed-Ended

In research, we employ two types of questions:

  • Open-ended questions permit participants to provide unrestricted, free-form text responses.
  • Closed questions, also known as closed-ended questions, limit participants to selecting from a predetermined set of possible answers.

Let’s look into each of them in detail.

Closed-Ended Questions: The Guided Path

Closed-ended questions restrict respondents to choose from a finite set of answer options. They can either select one of the options (single select - Radio Button) or multiple options (multiple-choice - Checkbox). These answer options can range from single-word responses (such as 'yes' or 'no') to rating scales (ranging from 'strongly agree' to 'strongly disagree'). Like the example below of a Likert Scale in a survey, another type of closed-ended question.

Screenchot of a likert skale question in an online survey.

Closed-ended questions provide a structured path for respondents to follow, narrowing down the possibilities and simplifying the decision-making process.

One of the most widely used closed questions in research is the Net Promoter Score® (NPS) survey, where respondents are asked to rate their likelihood of recommending a product or service on a scale ranging from 0 to 10.

Screenshot of an NPS question and rating scale that is color coded.

Think of closed-ended questions as signposts along a well-marked trail, guiding respondents toward specific destinations.

Open-Ended Questions: The Uncharted Territory

In contrast, open-ended questions invite respondents to chart their own course, offering freedom and flexibility in their responses. Without predefined options, respondents have the opportunity to express themselves in their own words, providing rich, qualitative insights that delve deep into their thoughts, feelings, and experiences. They are most frequently used as probing questions to find out the reasoning behind a previous answer.

These are questions that necessitate more than a simple 'yes' or 'no' response, prompting respondents to elaborate on their perspectives or provide additional details.

The primary advantage of open-ended questions lies in their capacity to unearth insights beyond initial expectations.
Ilustration of Open ended questions vs closed ended questions

Let’s try to compare them in the following table:

Closed-ended
Open-ended
Respondent effort and time-cost

Closed-ended questions streamline the response process, making it quick and easy for respondents to answer.
Require more effort and time. Respondents may feel overwhelmed or fatigued if presented with too many open-ended questions, leading to rushed or incomplete responses.

Response flexibility
Once response options are determined, it can be challenging to capture unexpected or unanticipated responses.

Respondents have the freedom to articulate their thoughts and feelings in their own words, providing authentic and unfiltered feedback. Open-ended questions can lead to unexpected discoveries and insights
Output Quantifiability

Responses are easily quantifiable, allowing for straightforward analysis and comparison.
Responses to open-ended questions are not easily quantifiable, making comparisons across respondents challenging.
Output Depth
Closed-ended questions may restrict respondents' ability to fully express their thoughts or feelings, potentially overlooking valuable insights.
Open-ended questions allow for in-depth exploration of respondents' perspectives, uncovering nuances and complexities that may be missed with closed-ended formats.
Potential Bias
The options provided in closed-ended questions can influence respondents' choices, leading to biased results.
There is a higher risk of researcher bias (Researchers may unintentionally filter or interpret responses) together with respondent’s limitations in terms of communication skills.
The big question now is, how to choose. This leads us to the next point.

How to choose between open and closed-ended questions?

As UX researchers, it's essential to navigate the terrain of survey design with care and precision. Here are some best practices to consider when choosing between open and closed-ended questions:

1. Do you know what you want to learn?

Before you start writing each of your survey questions, ask yourself, “what do I need to learn from this question?” and only then decide which type of questions suits you best.
When you're after data you can plot on a graph or use to show percentages, throw out a closed-ended question. For example: "How often do you shop online? (a) Daily, (b) Weekly, (c) Monthly, (d) Rarely, (e) Never".

But if you're itching for deep insights into your customers and their wants, go for an open-ended question. It is also the ideal format if you are looking for spontaneous answers. For example: “Could you name all the cosmetic brands you have heard of or tried?”

2. Combine and conquer.

There are occasions where, for example, you want to know how good was the user’s experience but also need to know the why’s in order to inform the next research piece or product team; it’s the perfect time to combine open and closed.

Let’s take this quite frequent structure as an example.

Start with a closed question: Please rate your experience, from 1 to 5, being 1= very unsatisfied and 5= very satisfied.

Then follow up with an open question: Can you briefly explain your rating?

This way, you can easily graph the satisfaction levels but also have a starting point on why the experience is positive and or negative.
Example of an open question that follows up a closed rating question

3. Make a fun and short survey.

A survey's length can impact completion rates and respondent satisfaction. By limiting the number of open-ended questions, you can maintain a concise survey length, increasing the likelihood of participation and engagement.

Also, switching and altering the type of questions keeps the motivation and interest in your respondents up. So make sure to have a mix of open and closed questions.

4. Open up questions when unsure or at risk of unnecessarily excluding.

There are certain questions or topics that require flexibility to make your respondents feel at ease, like, for example, gender identity questions. Also, sometimes we are not aware of all the potential answers to a question, thus, we might bias our research results if we offer a closed list of options. (i.e. “select all the tasks you perform on a daily basis”). In theses cases, it is better to go for an open-ended question to enable flexibility and unbiased answers.

If an open question is not possible, we should at least always offer an “other” option.

Conclusion

By mastering the dynamics of open and closed-ended questions, UX researchers can unlock valuable insights that inform and elevate their design decisions. So, whether you're navigating the structured paths of closed-ended questions or venturing into the uncharted territory of open-ended exploration, remember: every question holds the potential to uncover a treasure trove of user insights.

Happy surveying!
Ready to elevate your user experience? Let's talk!
The answers you are looking for, one click away.
Get in touch ->
TeaCup Lab Logo
TeaCup Lab is a user experience consultancy agency founded in Madrid in 2016. We specialize in User Research & User testing for global brands.
Copyright 2019-2023 TeaCup Consulting SL

Contact us

TeaCup Lab 
Calle Jaén 2, 1ºG
28020, Madrid, Spain
+34 910 59 21 36
hola@teacuplab.com