Usability testing guide
How to run usability test - a comprehensive guide
Elena Astakhova
UX designer
10 min read
10 min read
Usability testing, which involves a systematic evaluation of user-friendliness and ease of use, plays a central role in accomplishing a seamless user experience.
This article serves as an introduction to usability testing, exploring its purpose, benefits, and methodologies. Whether you’re a designer, product manager, or business owner, understanding usability testing is essential for creating intuitive interfaces, maximizing user engagement, and delivering exceptional customer experiences.
The usability test involves following steps
This article serves as an introduction to usability testing, exploring its purpose, benefits, and methodologies. Whether you’re a designer, product manager, or business owner, understanding usability testing is essential for creating intuitive interfaces, maximizing user engagement, and delivering exceptional customer experiences.
The usability test involves following steps
- Define Objectives
- Find participants
- Prepare documents
- Create scenario and tacks
- Pilot testing
- Conduct the test
- Analyse findings
Define Objectives
It is very important to have clear goals for the research - what exactly do we want to find out? Imagine we have designed a mockup or a prototype for a new feature/product, and the user’s input will depend on the questions we ask. If you don’t ask enough questions during a usability test, it can lead to incomplete or insufficient insights and potential usability issues.
In the planning stage, we need to set up clear objectives. Here are some general recommendations that might help you:
In the planning stage, we need to set up clear objectives. Here are some general recommendations that might help you:
- 1Test the key tacks performanceDigital products often feature core functionalities that serve as the primary source of value and utility for users. These features can vary widely based on the nature of the product. For example, an e-commerce website’s core features could include product search, shopping cart, and checkout process.
- 2Hot topicsSometimes we engage in discussions and debates over elements like layout, colour choices, element priority, and icon design. They are not critical, but everybody has an opinion about it, and it just makes meetings too long. When you come across a “hot topic”, like button size, just ask the users what they think and move on, once and for all!
Recruit participants
Test with employees
Having your employees as user-testing participants is cheaper, but may lead to incorrect results.
Team members often familiar with the company, product, its features, functionality, which can influence their judgments and feedback. This familiarity can make it more challenging to identify usability issues that might be evident to new or inexperienced users.
More than that, developers tend to be more demanding — they will be annoyed by many imperfections that most target users will never see.
see.
Having your employees as user-testing participants is cheaper, but may lead to incorrect results.
Team members often familiar with the company, product, its features, functionality, which can influence their judgments and feedback. This familiarity can make it more challenging to identify usability issues that might be evident to new or inexperienced users.
More than that, developers tend to be more demanding — they will be annoyed by many imperfections that most target users will never see.
see.
You are not the user
If you are running an internal test, make sure you have participants with different ages, backgrounds and computer literacy. Remember that engineers are much smarter than a general user and they may not represent the target audience.
External participants.
Start by clearly defining the characteristics, demographics, and behaviours of your target users. Consider factors such as age, gender, education level, technical proficiency, experience with similar products, and any other relevant attributes.
Based on the target user profile, develop specific screening criteria that participants must meet. These criteria will help filter potential participants who closely match your target user group. For example, if you are testing a mobile app for young professionals, you need participants aged 25–35, with a certain level of education and familiarity with smartphones.
While it’s important to find participants who resemble your target users, also aim for diversity within your selected group. Include participants with different backgrounds, experiences, and perspectives to gather a broader range of insights.
Start by clearly defining the characteristics, demographics, and behaviours of your target users. Consider factors such as age, gender, education level, technical proficiency, experience with similar products, and any other relevant attributes.
Based on the target user profile, develop specific screening criteria that participants must meet. These criteria will help filter potential participants who closely match your target user group. For example, if you are testing a mobile app for young professionals, you need participants aged 25–35, with a certain level of education and familiarity with smartphones.
While it’s important to find participants who resemble your target users, also aim for diversity within your selected group. Include participants with different backgrounds, experiences, and perspectives to gather a broader range of insights.
How to find test participants
- Recruitment agenciesThey can help identify and recruit participants based on specific demographics, behaviours, or target audience criteria. They often have pre-screened participants who are willing to participate in usability testing.
- Inernal DatabaceIf you have an existing user database or customer base, reach out to them directly, and don't forget about the reward system!
- Personal referralsPersonal referrals often result in more committed and engaged participants.
This also might be very useful if your target user is not interested in rewards (business owners, busy professionals). - Social mediaPost announcements in relevant groups, forums, or communities where your target audience is likely to be active. Don’t forget to motivate people to spend their own time for your research.
Prepare Documents
For a smooth and stress-free process, make sure you prepare documents in advance. Make Make sure you have:
I prefer having a Script, where I have everything together - introduction for the participant, tacks and closing questions. This document helps to set up a good structure, but we also should keep it conversational and be able to adapt to any unexpected situations.
- A test plan (objectives, scope, methodology, and logistics of the usability test. It provides an overview of the test's purpose, target audience, tasks, and success criteria.
- Product’s overview. Before we start testing a user, we want to make sure that participants understand the product’s functionality. Prepare a presentation about the product or show some basic functionality.
- Prepare a document that includes the scenarios and tasks that participants will undertake during the usability test.
- Non-Disclosure Agreement (NDA): If your usability test involves confidential or proprietary information, you may need participants to sign an NDA to protect sensitive data and ensure confidentiality.
- Depending on the approach you choose, you may need questionnaires or interview guides to collect qualitative and quantitative feedback from participants. These documents contain a series of questions related to participants' experiences, perceptions, and opinions about the product or interface.
- Prepare a presentation or reporting template that will help you summarise and communicate the findings and insights from the usability test effectively. This document can include key findings, recommendations, and visual representations of data collected during the test. It is much easier to run this process if you understand the final output.
I prefer having a Script, where I have everything together - introduction for the participant, tacks and closing questions. This document helps to set up a good structure, but we also should keep it conversational and be able to adapt to any unexpected situations.
Create Scenario and tacks.
Decide on the scope of the usability test: this will help in identifying the areas to create scenarios and tasks.
Scenarios are fictional or realistic situations that users will encounter during the test.
Develop scenarios that reflect real-world use cases or goals of your target users.
For example, if you are testing an e-commerce website, a scenario could be "You want to purchase a specific product and have it delivered to your home."
Scenarios are fictional or realistic situations that users will encounter during the test.
Develop scenarios that reflect real-world use cases or goals of your target users.
For example, if you are testing an e-commerce website, a scenario could be "You want to purchase a specific product and have it delivered to your home."
- Once you have the scenarios, break them down into specific tasks that users will need to perform. Tasks should be clear, concise, and measurable. Each task should have a specific goal or objective. For example, a task for the e-commerce website scenario could be "Find the product page for 'Product X' and add it to your cart."
- Arrange the tasks in a logical order, considering any dependencies or prerequisites. Ensure that tasks flow naturally and simulate a realistic user journey.
Pilot testing
Before conducting the actual usability test, conduct a pilot test with a small group of representative users to validate the scenarios and tasks. This can help identify any potential issues or areas for test improvement in the early stage.
You may find that some tasks are too easy for users, and there is no need to dive too deep into those tasks. Also, a pilot test helps us to uncover some difficult areas that we didn’t consider before.
You may find that some tasks are too easy for users, and there is no need to dive too deep into those tasks. Also, a pilot test helps us to uncover some difficult areas that we didn’t consider before.
Test for the test
A pilot test helps us to uncover some difficult areas that we didn’t consider for the usability test before.
Conduct the test
- Start the session with a warm and friendly welcome. Introduce yourself and express gratitude for the participant’s willingness to take part in the usability test. Make them feel valued and appreciated.
- Briefly explain the purpose of the usability test and its importance in improving the product or interface. Emphasize that you are testing the product, not the participant’s abilities.
- Introduce the test purpose. Get their informed consent to participate and record the session.
- “Think-aloud” technique. Encourage participants to ask questions at any point during the test: there are no right or wrong answers, and their curiosity and insights are valuable. It is also very useful for participants to verbalize their thoughts and explain their actions as they navigate through the tasks.
- Clearly explain each task to the participants, one at a time. Provide them with the necessary context, such as the scenario or user goal associated with the task.
- Observe and take notes: As the participants perform the tasks, observe their interactions closely. Take detailed notes on their actions, comments, and any difficulties they encounter. Pay attention to their thought processes, frustrations, and areas of confusion.
- Ask follow-up questions: After each task or at the end of the test, ask participants open-ended questions to gather more detailed feedback. Ask about their overall impressions, likes, dislikes, and any specific issues.
Analyse the findings
Synthesizing usability report findings involves summarizing and presenting the key insights and recommendations from the usability test clearly and concisely. Here are some steps to help you synthesize the findings effectively:
- Group similar items. Look for recurring patterns or themes among the usability test findings. Group similar issues or feedback together to identify overarching themes. This can help you simplify the synthesis process and provide a cohesive structure to the report.
- Prioritize the findings. Determine the relative importance and impact of each finding. Consider the severity of the usability issues, the frequency of occurrence, and their potential impact on user experience. Prioritize the findings based on their significance and relevance to the usability objectives.
- Summarize each finding. Write concise summaries for each usability issue or insight. Clearly describe the problem, its impact, and any relevant context or observations. Use bullet points or short paragraphs to keep the summaries focused and easy to read.
- Provide supporting evidence. Back up each finding with relevant data or examples from the usability test. This could include direct quotes from participants, task success rates, video clips, or any other evidence that supports the identified issues. Including evidence strengthens the credibility of your synthesis. If you are not using any specific app for user research, just record sessions in Teams.
- Connect the dots. Look for connections or relationships between different findings. Identify dependencies or interactions between usability issues and explain how they may compound or influence each other. This helps provide a holistic view of the usability challenges and opportunities for improvement.
- Generalize the findings. Consider the broader implications of the usability test findings. Look for insights that can be generalized beyond the specific test participants or tasks. For example, if multiple participants struggled with a particular navigation element, it may indicate a systemic design issue.
- Formulate recommendations. Based on the identified usability issues, develop clear and actionable recommendations for improvement. Each recommendation should address a specific finding and guide how to resolve or mitigate the issue. Include practical suggestions and consider the feasibility of implementing the recommendations.
- Structure the report. Organize the findings, summaries, evidence, insights, and recommendations into a logical structure. Use headings, subheadings, and sections to make the report easy to navigate. Consider including a summary or executive summary at the beginning to provide an overview of the main findings.