The market success of your software product depends on the user experience. Excellent user experience means higher user satisfaction, higher retention, increased customer loyalty, and steady income. To ensure a good user experience for your product, consider using UX benchmarking as one of your metrics. In this article, we will mostly focus on UX benchmarking and its key issues.
UX benchmarking - what is it?
UX benchmarking is a process where with the help of key metrics, you can assess your own product or compare it with competitor products. Using this technique helps you track your product’s performance and how it can be improved in the future. In other words, it is a comparative performance analysis against the market, past performance, or competition.
This competitive analysis improves overall user engagement and perceived usefulness. Design efforts can be time-consuming, but the proper UX benchmarking will facilitate the process even if you think it’s demanding.
How does UX benchmarking work?
Benchmarking is considered a process because once started, you can easily track your progress over time. That progress will lead to improvements and redesigns in your UX design system, the further development of your product and facilitate the UX team’s role.
So how does it work? To make the most of your UX benchmark study, you should follow a structured plan. Let’s look at how the process works:
Define what to measure
First, it is important to decide what you want to measure. You can pick a specific feature or a set of features that need improvement. For example, when you want to benchmark an ecommerce website, you can choose one of the top features. Given ecommerce’s focus on selling, the most important feature is usually the checkout process because it results in a purchase. When you have chosen a feature, you can define what you want to measure; for example, the average time of the checkout process, and how often the checkout process is abandoned (basket abandonment).
Also, there are two core elements that you need to implement in this step as well:
- Effective teamwork - Establish common goals of the study with your UX team. It’s important to set clear goals. When it comes to the example of an ecommerce website, you might want to reduce the number of users who abandon the basket by a certain percentage.
- Benchmarking aligned with the company’s KPIs - Benchmarking tied to business KPIs (e.g., customer satisfaction, revenue growth) will help you bring concrete value to your clients. Showing changes based on data can effectively influence clients’ decisions. For example, you can show that reducing the basket abandonment rate during the checkout process will give you more revenue.
Choose your test participants wisely
When it comes to testing with participants, remember your target audience (potential and existing users). Benchmark studies require quite a lot of data, but how many participants you need for testing depends on your goal.
According to the Nielsen Norman Group (NN/g), you might need only five participants to detect 85% of issues. To test a prototype: 5-10 participants; to derive insights from interviews: 20 participants or more are needed (but for example, in the ecommerce sector, the more quantitative data, the better.)
Define the type of research and stick to it
At this step, you should remember to keep all the information consistent for each test iteration because it might be difficult to compare data later if some of the information changes; for example, the number of participants or the questions you ask. Also, the device used for each iteration should remain the same. However, it doesn't exclude other tests depending on the goal.
Incorporate data-driven design
There is no point in conducting UX benchmarking if you do not intend to implement any changes based on the results. That is why it is crucial to make your insights useful and to implement improvements based on them. As you make improvements, conduct a second set of testing to assess how the UX is performing (using the baseline metrics you established previously).
Conduct benchmark studies regularly
UX benchmarking is extremely beneficial when done regularly; for example, monthly, quarterly, or yearly. It is clear when your design efforts are really enhancing the UX when you evaluate the user experience and compare it to past outcomes. In UX benchmarking, this type of activity is called longitudinal benchmarking, which simply means measuring over a specific period of time.
To assess how frequently you should carry out a benchmarking study, you need to consider your product design cycle and how often you make changes. These questions might help to decide when the next study is needed:
- How often do we make changes to our product?
- Do we have enough resources to make benchmarking studies on a regular basis?
UX metrics
Importantly, UX benchmarking involves collecting data-driven UX metrics (often called baseline usability metrics) that can be divided into these two categories:
- Qualitative data: usability, user satisfaction, user experience - in general, data describing a specific experience of a user.
- Quantitative data: analytics, surveys - all the measurable data.
These specific metrics are also named behavioral and attitudinal accordingly.
In the case of the UX benchmarking process, qualitative data is much more important because it helps improve specific issues. For example, many ecommerce platforms track how long the user takes to make a purchase. So, you might set specific expectations for how fast the user should complete the task (making a purchase) and conduct tests to measure the actual time it takes. Then, you can compare the data and the expectation and see where the issue is. For example, you discover that choosing the payment method definitely takes too long because the payment methods aren’t clearly visible and users can’t find what they need. With the use of UX benchmarking, you can improve this user journey and make the user experience better.
As mentioned before, this is a comparative process, so once you collect data, you have to compare it to something else; for example, a competitor, your previous performance, or industry-related standards. Remember also that even if it’s a comparative process when gathering insights, you might discover a specific user need that was not satisfied before. In that case, there won’t be anything to compare because it’s a new discovery.
Looking to build an engaging user experience? Check out the UX/UI design solutions we offer. With the help of our designers, your product will skyrocket in the blink of an eye.
Types of UX benchmarking
When it comes to UX benchmarking, it can be divided into two study categories:
- Retrospective - A retrospective study asks a question about the user’s past experience with the product or service. The most common technique to gather the information is a user interview or user satisfaction survey. This type of study might be biased by human memory, which is by no means fallible. It is also limited only to existing users.
- Task-based - During task-based benchmarking, test participants are asked to perform a specific task using the product. Before conducting this benchmarking study, you have to set a goal and success criteria to evaluate the user experience properly. In this approach, it is recommended to record the session and use the thinking-aloud protocol (basically, the test participants talk out loud during the task execution). It can give you a lot of valuable insights. For example, testing ticket machines at a tram stop outside during winter usually gives many insights into how they are underdeveloped. Moreover, this approach can give you more precise results, and you can examine the product with new and existing users.
As you can see on the graph below, these two types of study relate to different user experiences:
Good user experience design does not exist without proper research. In the next section, we will tell you why UX benchmarking is really important in product development.
Why is UX benchmarking valuable?
Why should you be interested in UX benchmarking? Here is a set of benefits that will help you understand why this process is so valuable for your organization.
- Uncovering strengths and weaknesses - Undoubtedly, UX benchmarking helps to establish the current state of the product. Knowing your product’s strengths and weaknesses will help you improve the parts that need it and take care of the well-designed ones.
- Making better design decisions - Benchmarking gives you a solid foundation for the decision-making process. It enables you to correct issues based on actual user research, not on a whim.
- Improved business objectives - Better understanding of the key user journeys of your product will help you change it according to the actual business KPIs. For example, by making changes based on user research, you can improve the conversion rate or avoid basket abandonment.
- Presenting positive results - Overall, even if UX benchmarking is more about improving and finding user insights to design better and more accessible products, your own product already has some strengths. UX benchmarking can help you highlight these strengths, which might have remained uncovered without a benchmark study.
Still, thinking about whether to add UX activities to your project? Read more about the business benefits and why UX design is important and worth investing in.
Conclusion
UX benchmarking plays a key role in measuring success or failure. Also, making a significant change in your product without proper user research is, to put it mildly, a bit risky. UX benchmarking is an effective method for measuring either the whole product or a specific feature. You might think you know how your product performs, but without usability testing, you will never truly know. Don’t miss out and incorporate UX benchmarking into your project!
