Confirmation bias in UX research and design
Posted: Sat Jan 25, 2025 8:51 am
Confirmation bias can have serious consequences in UX research and design because it can distort your perspectives by excluding alternative options and delegitimizing disagreement. Recognizing and overcoming confirmation bias will lead to better decision-making, research, and eventually, better products and user experiences.
For example, let's imagine an e-commerce site that, despite having a lot of traffic and users putting products into carts, there are few sales. As a designer you hypothesize that the poor performance is due to a poorly designed checkout button (preconceived belief). You decide to collect feedback and ask the following survey question:
Was it difficult to locate the red payment button?
There are several problems with this question:
The focus on the payment button biases the study to effective changsha mobile numbers list gather evidence in favor of the prior hypothesis that the designers started with. Even if the responses indicate that yes, the button was difficult to find, that problem may not be the biggest issue users have with the site or the payment process.
Negative language (hard to spot) prompts participants to think about the problems with this button (rather than whether the button was effective or not).
The binary question leaves no room for users to provide context about the actual experience and issues they encountered.
A better, non-leading question is:
How was the checkout process? Explain everything you liked or didn't like about the process.
This question avoids priming participants and does not address designer confirmation bias. It acknowledges that the payment process might be the problem and allows respondents to elaborate and give contextual feedback.
In summary, there are many ways in which confirmation bias can affect UX professionals:
Asking biased questions and (more generally) setting up a test in a way that seeks to confirm researchers' assumptions rather than investigating other possible issues and causes of a problem.
Ignoring evidence that points in a different direction (for example, in a usability test)
Interpret ambiguous evidence in favor of researchers' prior hypotheses or assumptions.
Tips to prevent confirmation bias in UX research
Being aware of confirmation bias is the first step to avoiding it. Here are some ways you can avoid confirmation bias:
Investigate rather than validate: You should start with an open mindset and aim to test hypotheses and assumptions rather than validate them. The goal is to discover things you didn't know beforehand, not to confirm your expectation.
Get data: The less time, resources, and emotion you have invested in a particular design, the less biased you will be when interpreting user research observations.
Ask unbiased questions: When collecting user feedback, whether through usability testing, diary studies, or interviews, avoid asking leading questions.
For example, let's imagine an e-commerce site that, despite having a lot of traffic and users putting products into carts, there are few sales. As a designer you hypothesize that the poor performance is due to a poorly designed checkout button (preconceived belief). You decide to collect feedback and ask the following survey question:
Was it difficult to locate the red payment button?
There are several problems with this question:
The focus on the payment button biases the study to effective changsha mobile numbers list gather evidence in favor of the prior hypothesis that the designers started with. Even if the responses indicate that yes, the button was difficult to find, that problem may not be the biggest issue users have with the site or the payment process.
Negative language (hard to spot) prompts participants to think about the problems with this button (rather than whether the button was effective or not).
The binary question leaves no room for users to provide context about the actual experience and issues they encountered.
A better, non-leading question is:
How was the checkout process? Explain everything you liked or didn't like about the process.
This question avoids priming participants and does not address designer confirmation bias. It acknowledges that the payment process might be the problem and allows respondents to elaborate and give contextual feedback.
In summary, there are many ways in which confirmation bias can affect UX professionals:
Asking biased questions and (more generally) setting up a test in a way that seeks to confirm researchers' assumptions rather than investigating other possible issues and causes of a problem.
Ignoring evidence that points in a different direction (for example, in a usability test)
Interpret ambiguous evidence in favor of researchers' prior hypotheses or assumptions.
Tips to prevent confirmation bias in UX research
Being aware of confirmation bias is the first step to avoiding it. Here are some ways you can avoid confirmation bias:
Investigate rather than validate: You should start with an open mindset and aim to test hypotheses and assumptions rather than validate them. The goal is to discover things you didn't know beforehand, not to confirm your expectation.
Get data: The less time, resources, and emotion you have invested in a particular design, the less biased you will be when interpreting user research observations.
Ask unbiased questions: When collecting user feedback, whether through usability testing, diary studies, or interviews, avoid asking leading questions.