TeaCup Lab Logo

5 reasons to test your product with users

Stefano Serafinelli
May 20, 2024

Fortunately, the times when testing a design with users was considered exotic and not very useful are behind us.

In the competitive world of product development, validation with real users is an established practice. From confirming concepts to identifying usability issues, testing with users brings us closer to success or, at the very least, reduces risks.

However, it is still common to find products that are not validated or are validated late due to poor planning ("We're going to start production next week. We'll make changes later" or "We don't have time or money for this") or excessive confidence in the design team's capabilities.

In this article, we want to review some reasons why validating our products with users on time is crucial, not only for our business but also for the users. And we will do so through examples of real cases.

1. A poor user experience has a direct impact on my business.

The history of design is filled with examples of products or entire businesses that have failed to provide a satisfactory experience to their users.

Victor Lombardi's book "Why We Fail" is probably one of those books that every designer (and entrepreneur) should have on their nightstand. It compiles several cases of designs that have failed for various reasons. Spoiler alert: many of these problems are related to a poor understanding of users, their needs, and the issues they face when using our products.

La historia de OpenID

One of the most significant examples found in the book is that of OpenID.
How? You've never heard of OpenID? Exactly.

OpenID emerged in 2007 as the first decentralized authentication protocol that allowed users to identify themselves using the same login credentials on all websites implementing the protocol. The problem that OpenID attempted to solve was real: with the rise of the internet and e-commerce, remembering all the usernames and passwords to access various websites was becoming increasingly complicated.

At the time, OpenID had modest success; it was integrated into Yahoo! and was on the verge of being integrated into other widely-used services on the web. However, the user experience of the service was plagued with issues, such as difficult-to-remember login credentials, lack of a standard process and shared terminology among different service providers (OpenID, ID, or username could be used interchangeably), and frequent technical problems.

The different OpenID providers presented the same service in a confusing and non-standard way.

 

The first user test of the service took place only in 2008, led by Yahoo. They were able to identify the issues and get to work to fix them, but it was too late. Shortly after, Facebook launched Facebook Connect, which basically did the same thing as OpenID but in a much simpler way.
The fact that they launched a service with a poor experience without testing it marked the downfall of OpenID, which was unable to recover the lost time

2. A good experience can also have a positive impact.

The success of Facebook Connect and similar services offered by Google, Apple, Twitter, and others is not solely explained by the failures of competitors. Facebook Connect succeeded because it provided a much smoother and simpler experience for its users.

The $300 million button

One of the most well-known cases demonstrating the benefits of validating a user experience is the legendary story of the $300 million button.

The protagonists of this story are two: Jared Spool, one of the most renowned UX experts, and a famous e-commerce website (Amazon) that is not explicitly mentioned.

Jared Spool recounts how a user test test de usuario conducted on Amazon's checkout form in 2009 revealed a problem that Amazon designers were not aware of until that moment.

At that time, Amazon required users to be logged in to make a purchase, something that is completely standard nowadays. The Amazon team was convinced that this would not be a problem for users. Those who already had an account would simply log in, while new users would understand the benefit of creating an account to make future purchases faster.

However, the test carried out revealed a very different reality: those who already had an account often struggled to remember their password, while new users did not want to register to make a purchase.

A famous quote from a participant in that study perfectly summed up common sense:
"I'm not here to start a relationship with Amazon, I just want to buy something."

"I'm not here to start a relationship with Amazon, I just want to buy something."

 

After the study, it was decided to remove the registration button and allow users to make purchases without being logged in, offering the option to create an account only after completing the purchase for those who wished to do so.

The estimated result of this change was $15 million more in sales during the first month and $300 million more during the first year. Not bad for removing a button.

3. A bad experience can also have a direct impact on people's security

Although the most common association with user experience is often in the realm of e-commerce websites and business, there are many other cases where design practice and research have a direct impact on security and even people's lives.

Consider, for example, the medical field, where not only patient experiences are frequently researched, but also user testing is conducted for medical devices.

In TeaCup Lab's laboratory, among other things, we frequently conduct tests on medical devices to ensure that they are easy to use and safe for both patients and healthcare professionals. In these cases, very rigorous study protocols are employed, as the consequences of launching an inadequate product to the market go beyond a simple failed purchase.

Image of one of the tests carried out in our laboratory

The story of Air Inter Flight 148

The book "Tragic Design" by Jonathan Shariat and Cynthia Savard Saucer is a very interesting resource for those wishing to understand the extent to which the consequences of poor design can reach. For example, it describes the story of Air Inter Flight 148, which crashed on approach to Strasbourg airport due to a series of causes, among which was a poorly conceived design that led to a tragic error by the pilots.

Airbus 320 aircraft, like the one that crashed, allow descent in autopilot mode by providing either the descent angle (in degrees) or the vertical descent rate (in feet per minute).
The control panel of an Airbus 320 has a single instrument for entering one of these two pieces of information: the user, in this case the pilot, must indicate which type of data is being entered by setting the instrument to Angle or Vertical Descent Rate mode before entering the data.


The cockpit of an Airbus A320-111, where we have highlighted in red the instrument where descent-related data is entered

 

Every day, we encounter very close examples of modes in interfaces. For example, on an iPhone, if we press and hold the home screen for a while, we enter editing mode and can delete or rearrange apps and folders. However, this is a feature that must be used very carefully and has historically led to errors.

In the case of the Airbus A320, to make matters worse, the two pieces of data are displayed very similarly on the plane's screen

In the image, we can see the two different states or modes of the same screen.

To the left, the instrument in Flight Path Angle mode, where -3.3 means a negative angle of 3.3 degrees.
To the right, the same screen in Vertical Speed mode, where -33 means -3300 feet per minute, a considerably faster descent rate than -3.3 degrees.

In the case of our flight, when setting the descent settings towards the airport, the pilots did not realize that the instrument was in Vertical Speed mode and not Flight Path Angle, which led them to descend much faster than planned and impact against the mountains before reaching Strasbourg, which they couldn't see due to bad weather.

This is a tragic example of poor design that inevitably makes us reflect on the importance of testing with users and in real-world usage environments any instrument or device that could harm people.

4. Researching with users provides us with a competitive advantage.

In any sector, it's likely that we'll face competitors offering products or services similar to ours. If we set aside other factors like product cost and consumer preferences, the success of our product largely depend on the experience we provide them.

Increasingly, companies are integrating user research into their processes. This allows them to optimize the consumer experience and, consequently, improve their business outcomes.

Not involving users in validating our experiences means falling behind the competition. On the contrary, establishing a consistent validation plan with our customers enables us to stay ahead of our competitors and meet market needs more effectively.

5. There are no bulletproof designs or infallible designers

We leave for last the perhaps most controversial reason: perfect designs do not exist. 

Regardless of who the designer is, whether it's a renowned design agency or a student preparing their master's thesis, every design and experience has its own issues, more or less severe, and areas for improvement. Although an experienced UX designer can create more satisfying experiences with fewer flaws, there will always be aspects that can be enhanced.

The empirical proof of this statement is before us every day. Over the years, we've been testing and validating products and services of all kinds with users and have never stopped encountering issues. These problems go beyond cosmetic ones and can have a real impact on the user experience, such as causing a user to abandon a purchase or have difficulty completing it.

The reason is simple: even though we're experts in user experience, we are not our own users. Despite the sensitivity we develop towards users over time, there are still aspects of their motivations, thoughts, and behaviors that escape us. Involving users in the analysis and validation of a product or service is the only way to discover the barriers preventing a completely satisfying experience.

Ready to elevate your user experience? Let's talk!
The answers you are looking for, one click away.
Get in touch ->
TeaCup Lab Logo
TeaCup Lab is a user experience consultancy agency founded in Madrid in 2016. We specialize in User Research & User testing for global brands.
Copyright 2019-2023 TeaCup Consulting SL

Contact us

TeaCup Lab 
Calle Jaén 2, 1ºG
28020, Madrid, Spain
+34 910 59 21 36
hola@teacuplab.com