October 2005 Archive

unraveled notes: what you’re missing

A while back I launched a second weblog, unraveled notes. Since then, I’ve been posting “short and sweet thoughts, links and ideas” there about 3-5 times a week. It has a relatively small readership (~56 readers according to FeedBurner) compared to unraveled (516 readers) so I thought a small sample of unraveled notes would help all the unraveled readers know what they’re missing and decide if they would like to subscribe. So here are the last ten unraveled notes. If you like what you’ve read, information about the unraveled notes XML feed follows.

Mobile Monday comes to London. The first event will by 7 November at Vodafone! Response to this has been massive so if you want to attend, sign up to the mailing list (in the aforementioned Yahoo! Group) and RSVP pronto! Looking forward to seeing everyone there. 27 Oct 05

Janine’s award winning short film, Torque, is now available to view online at Hurluberlu Films. If you have six minutes and a broadband connection, please have a look! 26 Oct 05

unraveled furl, my Furl feed, is no longer active. It’s been replaced by unraveled web, a feed of my My Web 2.0 archive. Please update your news readers. 26 Oct 05

Praise for simple tools that do one thing really well: BURL is like TinyURL, in that it shortens long URLs but it also leaves some context so the recipient of your slick new URL knows what the hell you’re pointing them at. Nice work Andrew Ferguson! 26 Oct 05

Any Pandora users out there? If yes, look up my shared stations using . I currently have three which I’m constantly refining: Work Lounge, Club Box and Folktronica. 24 Oct 05

Has anyone noticed that Google Desktop automatically changes the default search in Firefox from whatever it’s set on to Google? I expect more than that from Google. And since I recently changed my default search to Yahoo!, it’s reason enough for me to remove Google from the search list in Firefox. 19 Oct 05

I just tried Goowy Mail, a Flash based email client and was more than unimpressed. Slow and non-innovative, it features a lame copy of the OSX Dock and has very little new ideas to offer email management. 15 Oct 05

Metaphorical.net is a fine collection of “interactive studies and strange thoughts by William Ngan.” Enjoy. 12 Oct 05

I’ve been a happy user of NetNewsWire lite for a long time. I never went to NetNewsWire because I couldn’t justify benefits for the money. That was until I realized that it syncs with Bloglines, the slick browser based newsreader. Now I can read at home and at work and never worry about what I’ve already read. 10 Oct 05

Remember that sweet UI that was featured in Minority Report? Well someone has gone and done something similar with an OSX app. Check out this demo of TactaDraw used with the TactaPad, “The Tactile, Immersive, Two-Handed Input Device.” 10 Oct 05

If you enjoyed unraveled notes, please subscribe to its XML feed. (Read more about my XML feeds.)

Serco: an independent usability consultancy

Preface

This is the fifth installment of my graduate school work: Serco: an independent usability consultancy. For a background on my graduate work, please read the preface of the first installment, the interaction design process.

On 21 January 2004, our class visited Serco Usability Services, a consulting firm located in central London. Serco offers a number of services to supplement the design process including workshops, feasibility studies, field research, competitive analysis, prototyping and training sessions. In addition, they base much of their business on the usability testing of mobile devices, interactive television and desktop computer products in their usability facilities. This is the topic which our host, Simon Herd, focused on for the majority of our visit.

The typical usability evaluation will last approximately a month. The process begins with a stakeholder meeting. During this meeting, the participants develop a scenario that provides a basis for the evaluation, and discuss the required level of validity. That is, the participants establish the goal of the usability evaluation in order to create useful metrics towards measuring product success. The project plan and evaluation session plans are also formalized during this stage. The next step is to recruit evaluation participants. Before any participants can be recruited, a user profile must first be created. This profile is based on the product’s audience and any information the client has about this audience. After the profile is created, the formal usability evaluation can take place. Finally, after the evaluation is completed and all of the required data has been gathered, a report is created for the client summarizing the outcomes.

While this process remains generally consistent across different projects, the considerations will change according to different commercial contexts. In other words, the client, budget and project schedule can greatly affect the course of a project. A large media organization such as the BBC has a large number of stakeholders. Therefore, the product will have to meet different goals for each type of stakeholder. The BBC also has a very wide and varied audience, so it will be important to consider the real users of the product as opposed to the general BBC audience. These same considerations would be much different for a smaller organization with a more niche market. The stakeholders would likely be much less in number, which allows the product goals to be more focused. A smaller market would also allow the user profile to be more easily understood, which allows recruiters to have a better idea of the evaluation participants they need. However, since the user profile is narrower, actually finding the evaluation participants may prove difficult. These are just several examples of how considerations can change under difference circumstances. Depending on the project, there can be many other considerations that are prone to change.

In addition to constantly changing considerations, there are a number of other pitfalls in the practice of usability evaluation. There are both business and user goals so one of the most difficult tasks is finding a balance between these goals. In addition, since the evaluators can put a stamp of approval on products, there can sometimes be business pressures that have the potential to sway the evaluation results.

One example of usability testing we were able to directly observe was the evaluation of a web site. In this example, the goal was to have the participant book a cinema ticket via a web site. Much of this evaluation was based on a script that Serco has developed and routinely uses for desktop product evaluations. This script is used in order to keep the interaction between the facilitator and participant objective and consistent.

Separate from the script itself but just as important are general guidelines for interacting with the user and conducting the evaluation. In terms of the user, it is important:

to let them make mistakes, which reveals aspects of the interface that may need improvement.
to answer questions with questions, which forces the user to give more feedback.
not to tell the user what to do. Doing so counters one of the goals of the evaluation, which is to find out what users do without instructions.
not to explain the interface. Again, this is counter to the goal of evaluation and will not provide you with any information that you do not already know.

In terms of the evaluation itself, it should be focused on:

goals for all stakeholders, including the users, business and design team.
the 20/80 rule (20% of the functionality is used 80% of the time).
new or changed functions.
significant edge cases.
most difficult functions.

These guidelines are set so that as much information as possible can be drawn from the user during the evaluation.

Usability evaluation has an important place in the product development cycle and can play a major role in improving the product user experience. The visit to Serco helped me to better understand the usability evaluation process, the different considerations that HCI practitioners must take when carrying out the evaluation process and several of the pitfalls that can occur during the process. The visit also made me realize that usability evaluation, though interesting, is not something that I want to focus on in my professional career. It can also be very educational as I realize that you can more effectively design for users when you better understand them. However, I think I would be more comfortable solving usability problems from the design end of the development cycle rather than the usability end of the cycle.

All installments in this series:

Roundup: recent events I’ve attended

In the past three weeks, I’ve attending a few excellent events. Rather than try for posts that may not ever happen, here’s the standby roundup:

Interact! Games meets interaction design was another wonderful AIGA Experience Design event coordinated by Nico MacDonald. Ben Cerveny, previously from Frog Design and Ludicorp, discussed the historical and theoretical background of play, describing games as pattern languages along with a ton of other ideas that I couldn’t begin to describe but were extremely fascinating. Afterwards, Durrell Bishop of Luckybite provided a few notes on how interaction design can move towards play, citing the iPod click wheel as great example of how playfulness meets interaction. He encouraged the audience to avoid falling into interaction design preconceptions and use play more often to encourage interaction and increase the pleasurability of products.
Next was Creating value: Exploiting design innovation in the real world, another event chaired by the omnipresent Nico MacDonald. Chris Downs of live|work and Nina Warburton of Alloy gave some nice presentations on the importance of developing intellectual property in order to establish your design business, but the meat of the event was presented by Mat Hunter of IDEO. This is probably nothing new to those who know anything about making money from design, but for newbies like me, it was creating design IP in a nutshell. Some of his most useful points: making money from design is very expensive; ideas have a shelf life so go as fast as you can and focus on the business value of the design; you must prototype your ideas to make money from them; and most importantly, if you have an idea, you or a business partner needs to be able to speak to its business value, i.e. don’t focus on the design, focus on the value of design.
Finally, after hearing that Simon Willison was in London, I told him I’d like to meet up sometime. To my delight, a gathering of web geeks had already been planned for that very evening! Besides Simon, I was also able to meet Mark Norman Francis of Yahoo UK and Richard Hallows of Tonic. Great fun was had by all. Let’s do it again sometime, shall we?

← September 2005 | Archives | November 2005 →