Identifying the Ideal Remote UX Testing Size

Lots of attention has been paid over the years to trying to find the ideal size for Remote UX testing.

UX Remote Testing Methodologies

Whatusersdo.com recently wrote an insightful article tracing the historical origins, standard practices and common exceptions for Remote UX testing, which was published on Business2Community.com.

As Duddells points out, while many people focus on the group size as a concern, the main focus should be on collecting insights that drive design, not accumulating numbers to impress people in a PowerPoint presentation.

The industry standard for Remote UX testing is a group of five users. Duddells explains that this number first fell into popular consensus with the publication of a popular article written by Jakob Nielsen that cited a diminishing ROI for users six, seven and eight in larger group sizes.

Some of the factors Nielsen cites as causing the diminishing ROI for groups over five members include repeating issues, less insightful differentiations between users and increased staff time time for analysis, which is why Duddells writes, “it’s more important to focus on writing effective user tasks, rather than worrying too much about the number of participants.”

Doing so allows application development staff to keep the focus on garnering new insights, instead of on administrative or organizational tasks.

Why 5 UX Testers Are Almost Never Enough?

Professional User Experience Design professionals will tell you that there are several exceptions for when more than five test users are necessary for best results. Some of the exceptions Duddells lists include:

• Multiple user segments, “where you have distinct groups of customers a minimum of five users per group should be applied, particularly when your site services multiple territories.”

• Design comparison tests, and

• Tree and card sorting tests, which are blends of both quantitative and qualitative methods requiring a statistically significant number of users (Duddells suggets a minimum of 20)

When it comes to Remote User Experience testing, or any User Experience design principle, there is never going to be a one-size fits all solution. The very nature of User Experience is to provide customizable solutions unique to each product and user.

However, there are many patterns and trends that emerge through repetition, and from those trends we can extrapolate best practice standards. In the article cited above, Duddells does a great job of explaining a how many users to test with Remote User Experience testing.

At DOOR3, we specialize in testing various areas of user experience to help you figure out how to build a UX design strategy. Contact us today!