Remote Usability Testing – What, When, and How?

Situation: As a national leader in your field, you understand user experiences may vary based on their roles and functions of using the site or product. For better insight into the needs and pressure points of the site and gather ideas on how to address those concerns, you would like to do some usability testing in California , Texas, and New York but as usual, you have a strict budget and a short time frame to complete a study. What should you do? 
In the past, companies have had very few options that would answer these questions and allow them to coordinate a study of this size quickly or frugally, outside of the standard usability test or web analytics study. However in recent years, usability professionals have utilized web conferencing software and a host of communication devices to develop remote usability testing as an alternative to conducting standard “on location” usability tests.
When is remote testing appropriate?  

Remote usability testing is a methodology that allows moderators to gather feedback about a website, software product, or wire frame via an online meeting space or web conference forum. Remote usability testing is most beneficial to clients who:

• Are interested in multi-market studies
• Have a tight deadline
• Have user groups that are more difficult to schedule
• Limited budget

It has been our experience that remote usability testing is currently the most cost efficient alternative to standard usability testing as it allows clients more flexibility in project scheduling, access to geographically dispersed users groups, and is recourse to replace or schedule difficult user profiles. Using the scenario above, this article will compare remote usability to a standard domestic usability study that includes one or all of the factors above. Please see International Testing Volume 46 for remote testing considerations specific to international testing.

What are the advantages of Remote Testing? 

• Cost Savings: Typically, in addition to the standard cost of a usability test, multi-market tests require facility rental fees ranging from $1200-$1800 per day and travel expenses of the moderator(s), often including airfare, car rental, food, and hotel expenses. In a remote usability experience, moderators conduct the study in house and eliminate facility rental and travel expenses.
• Shortened project timeline: When conducting a multi city study, a large percentage of time is spent coordinating facility reservations and traveling between locations, increasing the standard 3 week life cycle of a project upwards of 4 or 5 weeks (partly dependent upon the number of locations tested). By conducting the sessions remotely, there is no need to coordinate with facility schedules or allocate additional time to travel between testing sites. Moderators will usually host and conduct the sessions from their own usability labs, and again, eliminate the need to rent a space or make travel arrangements.
• Geographically dispersed user groups: There are a number of companies who have developed websites and products for a smaller population of people who are usually not centrally located. As previously discussed, trying to conduct usability testing in all locations is expensive, time consuming and not often possible.  Through the use of web conferencing software, moderators are able to circumvent the need to get all the participants in one location. So as in the case of the previous example, instead of scheduling facilities in 3 states over the course of two weeks to assess the needs of 3 different markets, 12 participants can be tested in the span of a week, remotely to gather a fuller representation of the each user group.

Disadvantages of remote testing:

• When viewing a video of participant reaction is important:  While remote viewing software will capture the desktop activity, clients are not able to see a capture of the participants.  To alleviate this as an issue, consider requiring participants to own and have the ability to operate a webcam to use during the session.  However, keep in mind this may require more setup/prep time with each user and slightly longer sessions are in order.
• When secure content is of high concern:  Though remote sessions are conducted in private online meeting spaces, moderators are unable to guarantee users are viewing the session without observers in the room.

Decision: Remote usability testing, when executed correctly, provides companies a cost-effective and time-efficient alternative to get valid user feedback without the added burden of travelling to multiple locations, renting additional facilities or dealing with recruiting challenges. For these reasons, an increasing number of companies in recent months are opting to conduct remote usability testing in lieu of the standard usability test.
For more information about usability testing and other solutions, please visit http://www.usabilitysciences.com/services.