UX.training Blog

  • Home
  • Resources
  • What Clients Should Know About Usability Testing

What Clients Should Know About Usability Testing

Research Most Popular Posts

blog-what-clients-should-know-about-usability-testing

Occasionally clients contact us about conducting a usability test without knowing exactly what they’re asking for. Sometimes this happens because they’ve been told testing sounds like a good idea, sometimes it happens because they read something somewhere that intrigued them, or sometimes it’s because they know they need to test their product but don’t know if they should outsource or not. Regardless of the reason, there’s occasionally a learning process that clients go through when it comes to usability testing.

We’d like to share a few of the more common things that surprise clients when it comes to usability testing:

Usability testing is a defined process.
Clients often don’t realize just how much of a process usability testing is. A carefully planned and coordinated usability testing has several moving parts and a delay in any one slows down the entire process. A typical usability testing process looks something like this:
 
 
Step 1: Planning: The first thing that usually happens in a usability test is a series of meetings are held to understand both the requirements of the testing process (including who to test, where the testing will occur, and any deadline) and exactly what will be tested. Now that everyone is on the same page, the testing agency (either internal or external) will likely want to spend some time getting familiar enough with the system being tested that they can confidently design a test plan
 
Step 2: Recruitment: Once a test plan has been generated and approved by stakeholders, user recruitment can begin. Depending on the user base, this can be as simple as using a service like Ethn.io or as involved as cold calling (or emailing) dozens or even hundreds of users. Using a recruitment agency requires further expenditure, planning, and oversight.
 
Step 3: Testing: Everyone’s agreed on what to test with who, the test plan details the how, and all the participants are lined up. Now it’s time to gather all the testing materials and equipment and start testing users.
 
Step 4: Analysis: Usability testing has a tendency to generate enormous amounts of both qualitative and quantitative data. The next step in the process is to dive in and start wading through all that data looking for the answers to your research questions.
 
Step 5 Reporting: The individual testing sessions have all been completed and the data’s been analyzed. It’s time to gather the team and stakeholders and present the findings and any recommendations.
 

Moderated usability testing takes time.
As a result of being a process, usability testing also takes time to complete. Clients are often surprised to learn that a properly designed usability test can take four to six weeks to complete. Sometimes less, sometimes more.
What takes so long? 
There are a number of things in each step of the usability testing process that need to happen, and when they don't happen quickly it can cause delays:

 

Step 1: Planning: Waiting on client approvals, waiting for funding to come through, defining the who, when, and where of testing, plus getting familiar with the to-be-testing system, and generating a detailed test plan all take time and are sources of potential delays.
 
Step 2: Recruitment: Recruitment issues are by far the most common source of delay. Users can be hard to find, difficult to recruit, difficult to schedule, or often, all of the above. Regulatory or ethical issues associated with incentivizing some classes of users and inter-office politics or buy-in can also be hurdles.
 
Step 3: Testing: An hour for setup and takedown each day, 60-90 minute sessions with 30 minutes of reset buffer between each session plus an hour for lunch and you’ll burn through a typical work day pretty quickly. It’s not uncommon to spend an entire week on the testing phase of a moderately sized usability study.
 
Step 4: Analysis: As noted above, usability testing has a tendency to generate enormous amounts of both qualitative and quantitative data. The time it takes to analyze the data is directly related to the number and detail of the research questions being asked.
 
Step 5 Reporting: Generating reports is usually straightforward but, again, the number and detail of the research questions being asked is the biggest factor in how long it takes.

 

Taking the most common delays into account, we tend to block off six weeks of full-time work for up to three individuals for moderately sized usability studies. A typical schedule might look something like this:

Phase 1: Planning & Recruiting (2-3 weeks)
Phase 2: Testing (1 week)
Phase 3: Analysis & Reporting (2 weeks)

 

Usability testing doesn’t work without actual users.
There’s a saying the automotive world: “there’s no replacement for displacement.” In a nutshell, it means that, all things being equal, bigger engines will always be better than smaller engines; assuming your goal is to go faster, that is. When it comes to usability testing, the same is true for recruiting actual users. While there are numerous techniques and data sources you can use to estimate a user’s likely response, there is no replacement for insight gained by recruiting actual users. Users often provide critical insight and reasoned responses that simply aren’t captured by less direct methods.

 

Usability testing is a team effort.
While alluded to above, it’s worth explicitly stating that usability testing is a team effort. Occasionally clients hire us with the expectation that they can be 100% hands-off. Unfortunately, this is almost never the case. In order to be sure we’re going to get the answers a client needs, we’ll need to work closely WITH the client to make sure we understand the system being tested, that we understand the functional requirements of the testing process, and that we all the access to client systems and operations that we need.

 

What do you wish your clients knew about usability testing? Let us know in the comments below! 

Are you interesting in conducting usability testing on your website or application? Normal Modes has the experience and expertise to get you the answers you need. Learn how Normal Modes can use usability testing and other methodologies to answer your user research questions today:

 

Comments