How do I do usability research?

Usability research can be very costly.  There are companies that specialize in doing it the right way, with high end audio/video equipment and multi-stream video editing and compositing integrated into the program.  The deliverable is typically an incredibly insightful presentation with snippets of video that tell a compelling story all by themselves.

With startups, it is almost never the right decision to spend the big bucks and hire an agency to do this “right”.  First hand learning is key and some research trumps no research. There are many options for you to crowdsource research, such as usertesting.com which is a terrific service - but it is a great idea to start with a face to face, high touch usability benchmark study for a few subjects to get all the low hanging fruit usability problems out of the way quickly. 

It is ok to do this yourself, provided you put some structure around what you are doing to guard against common mistakes like confirmation bias and leading the witness. 

As with all research projects, you should start with a research protocol that clearly states the questions you want to answer, provides a guideline for recruitment, and lists the props required for the session.  Common props might include the means to take video footage, as well as a photo and video release form indicating that all footage is meant for internal use only.  This is very important, because video footage allows you to go back and look at where the subject is struggling.  It also helps you communicate your findings with the rest of the team.  If you manage it correctly, most subjects forget the camera is running within 5 minutes- but you must get their consent so they know and are ok with being on camera.  

Best practices usability benchmark studies suggest you keep the session inside one hour. Let’s say you are looking into the usability of a smart phone app or a web app. The session could look something like this:

• Introductions and orientation – explain purpose of research to subject and let them know what to expect (5 min)
• Execute any paperwork, such as an NDA, a photo and video release forms, and a profiling questionnaire (5 min)
• Provide a brief introduction to the app, then let them play with the app for a short time. Be around to answer questions. (5 min)
• Repeat with Device 2. (5 min)
• Ask the subject to execute a scripted task list for the app.  Tasks tend to be fairly specific – for instance, you could ask them to sign up for an account, then fill out their personal profile and opt into some product or service option which you are offering in the app.  Ask them to verbalize what they are doing as they try things out (but do not offer hints or commentary – you are there to watch and learn, not to talk.)  (15 min)
• Debrief – loosely guided interview to ask subjects to rate the usability of each task on a scale of 1 to 5, as well as answer some open ended questions about their general impressions and perceptions (20 minutes)
• Optional: Present the incentive check (typically $50-100, depending on the nature of the study).

This format is great at providing a sanity check for the out-of-box experience for any kind of product or service with a software and/or hardware component.  Can the end user figure out how to set up a new account and get going without too much groaning and gnashing of teeth?  Lots of times they can’t.  It is amazing how much you can learn about the onboarding process for new customers by watching subjects struggle through the product setup process.   It is very hard to keep quiet and not offer suggestions along the way… but the learnings are priceless.

As with all other kinds of research, it is healthy to invite the entire team to watch - particularly engineering team members. This is the best way to help them understand who they are designing for, and why certain feature enhancements are necessary to ensure an awesome user experience.

At some point you will be ready to go beyond the qualitative approach and get into quantitative research.  Have a look at this article about when and how to go from qualitative to quantitative research, and how to make it work for you. 

This post originally appeared on the ConceptSpring blog by Elaine Chen.