unraveled

Serco: an independent usability consultancy

Preface

This is the fifth installment of my graduate school work: Serco: an independent usability consultancy. For a background on my graduate work, please read the preface of the first installment, the interaction design process.

On 21 January 2004, our class visited Serco Usability Services, a consulting firm located in central London. Serco offers a number of services to supplement the design process including workshops, feasibility studies, field research, competitive analysis, prototyping and training sessions. In addition, they base much of their business on the usability testing of mobile devices, interactive television and desktop computer products in their usability facilities. This is the topic which our host, Simon Herd, focused on for the majority of our visit.

The typical usability evaluation will last approximately a month. The process begins with a stakeholder meeting. During this meeting, the participants develop a scenario that provides a basis for the evaluation, and discuss the required level of validity. That is, the participants establish the goal of the usability evaluation in order to create useful metrics towards measuring product success. The project plan and evaluation session plans are also formalized during this stage. The next step is to recruit evaluation participants. Before any participants can be recruited, a user profile must first be created. This profile is based on the product’s audience and any information the client has about this audience. After the profile is created, the formal usability evaluation can take place. Finally, after the evaluation is completed and all of the required data has been gathered, a report is created for the client summarizing the outcomes.

While this process remains generally consistent across different projects, the considerations will change according to different commercial contexts. In other words, the client, budget and project schedule can greatly affect the course of a project. A large media organization such as the BBC has a large number of stakeholders. Therefore, the product will have to meet different goals for each type of stakeholder. The BBC also has a very wide and varied audience, so it will be important to consider the real users of the product as opposed to the general BBC audience. These same considerations would be much different for a smaller organization with a more niche market. The stakeholders would likely be much less in number, which allows the product goals to be more focused. A smaller market would also allow the user profile to be more easily understood, which allows recruiters to have a better idea of the evaluation participants they need. However, since the user profile is narrower, actually finding the evaluation participants may prove difficult. These are just several examples of how considerations can change under difference circumstances. Depending on the project, there can be many other considerations that are prone to change.

In addition to constantly changing considerations, there are a number of other pitfalls in the practice of usability evaluation. There are both business and user goals so one of the most difficult tasks is finding a balance between these goals. In addition, since the evaluators can put a stamp of approval on products, there can sometimes be business pressures that have the potential to sway the evaluation results.

One example of usability testing we were able to directly observe was the evaluation of a web site. In this example, the goal was to have the participant book a cinema ticket via a web site. Much of this evaluation was based on a script that Serco has developed and routinely uses for desktop product evaluations. This script is used in order to keep the interaction between the facilitator and participant objective and consistent.

Separate from the script itself but just as important are general guidelines for interacting with the user and conducting the evaluation. In terms of the user, it is important:

In terms of the evaluation itself, it should be focused on:

These guidelines are set so that as much information as possible can be drawn from the user during the evaluation.

Usability evaluation has an important place in the product development cycle and can play a major role in improving the product user experience. The visit to Serco helped me to better understand the usability evaluation process, the different considerations that HCI practitioners must take when carrying out the evaluation process and several of the pitfalls that can occur during the process. The visit also made me realize that usability evaluation, though interesting, is not something that I want to focus on in my professional career. It can also be very educational as I realize that you can more effectively design for users when you better understand them. However, I think I would be more comfortable solving usability problems from the design end of the development cycle rather than the usability end of the cycle.

All installments in this series:

  1. The interaction design process
  2. A review of DataTiles
  3. Designing a cashpoint
  4. Culture and web design
  5. Serco: an independent usability consultancy
  6. An evaluation of exhibit design at the London Science Museum
  7. Thesis report: Accessible and Usable Web Design

Comments

Go back to the top of this entry ↑

Comments are now closed for this entry