5 Common Pitfalls of Usability Testing

Tara Calihman - June 02, 2014

Usability testing is one of the best ways to quickly identify issues that prevent people from using your product the way it was intended. And while it’s easy to understand the benefits of talking to real users and watching them interact with your product, it’s not always clear where to start. The important thing is to start somewhere and refine your process over time.


It’s possible to get high-quality feedback without a big upfront investment of time or money, by recognizing some of the pitfalls that can derail your testing.

1. Waiting too long to test

Begin testing as soon as you have something to put in front of real or potential users. Even paper prototypes are a great way to help inform the design process when considering a new user interface. Don’t make the mistake of thinking it’s too early to get feedback.

2. Uncertainty about what to test

Start simple with goal-based scenarios. Describe a situation for your tester and ask how they would accomplish the goal you’ve outlined. Initially, what you test is less important than just picking a starting point. Even a seemingly routine interaction can generate valuable insights. And don’t feel that your early tests have to cover everything. The goal is not to fix every potential issue, but to build a discipline around testing.

3. Testing fatigue

Whether you’re facilitating or observing, usability testing can be frustrating when something that seems obvious isn’t apparent to your tester. Now imagine being the tester, confronted with an unfamiliar interface and unable to complete a seemingly simple task.

If either the facilitator or tester hits the point of fatigue, the remainder of the test may be a frustrating experience for everyone. The awkward silence when a user struggles to find that new feature is critical feedback about your product, but when they’ve clearly reached a dead-end, there’s no harm jumping in to explain and keep the session moving.

It can also be helpful to limit tests to an hour each and avoid scheduling sessions back-to-back.

4. Too much feedback

One of the challenges of usability testing is dealing with the sheer amount of feedback that can come out of a few short sessions. Remember that you’re not trying to resolve every issue. Instead, look for common themes by using the same script across a group of tests. Interactions that are truly problematic will make themselves known by surfacing repeatedly.

Choose a test schedule that makes sense for your team and your release cycle. Three to five tests per month can be a good starting point.

5. Feedback decays quickly

Before you begin testing, have a plan to share results with your team as soon as possible. Not only do the details fade quickly, but failing to identify next steps will result in wasted feedback and erodes confidence in the benefits of testing.

How you summarize the results depends largely on the audience. An executive summary requires much less detail than what you might share with your product and engineering teams. But no matter who the audience, you’ll get better engagement and follow-up from your team when you’ve taken the time to filter out what’s not relevant. Close the review by identifying key action items that can be completed before the next round of testing.

In the end, usability testing doesn’t need to be painful or time-consuming. By developing good habits and building testing into your existing process, you’ll find ongoing benefits that are easy to achieve.