June 2005 Archive

A review of DataTiles

Preface

This is the second installment of my graduate school work: a critical review of DataTiles, by Rekimoto et al. For a background on my graduate work, please read the preface of the first installment, the interaction design process.

In the paper DataTiles: A Modular Platform for Mixed Physical and Graphical Interactions (PDF), the authors, Jun Rekimoto, Brygg Ullmer and Haruo Oha, describe DataTiles as a system that integrates the benefits of two major interaction paradigms: graphical and physical user interfaces. They mention several previous efforts towards integrating the two interfaces as well as the advantages of this approach. However, the authors felt that many of these efforts fell short. Most prior work has focused on special purpose systems that do not scale to support different applications. Additionally, while many previous systems have used physical objects as a means of interaction, they do not retain the flexibility and power of graphical user interfaces.

To address these issues, the authors introduce the use of “graphically augmented physical objects.” Their prototype system, DataTiles, uses transparent tiles as a modular unit of interaction. These tiles are combined with a sensor-enhanced flat panel display allowing users to control digital data “within” physical DataTiles while retaining the flexibility and power of graphical user interface.

As an example, to view a specific remote camera, users can place a map tile next to a portal tile on the display. The map tile then shows camera locations, allowing the user to select which camera they would like to view in the portal tile. This is a simple example that only uses two tiles, but much more complex combinations can be composed by using the five different types of tiles.

I found the motivations behind DataTiles intriguing, especially the idea of mixed physical and graphical interactions. As the authors mentioned, interactions with physical objects can often be much richer than interactions with graphical interfaces. People can use sophisticated motor skills to manipulate objects more complexly than pointing and clicking. In addition, interactions can involve two hands or allow several people to interact cooperatively. These are all powerful interaction ideas that deserve to be investigated in the context of computing. Unfortunately, beyond basic physical interactions derived mostly from graphical interactions such as clicking and dragging, DataTiles use as a physically interactive system is limited. Once a tile is placed on the display, little more can be physically accomplished. Most interactions after that point are, in fact, graphical.

Regardless of this shortcoming, the authors presented a worth while case for DataTiles and future research in mixed physical and graphical interactions. Three key interaction ideas are summarized in the introduction and then mentioned throughout the paper:

Tagged transparent objects as graspable interaction modules. These serve as physical windows for digital information and trigger specific actions when placed on the display surface.
Mixed physical and visual [graphical] interactions.
Physical language for combining multiple tiles.

As mentioned, there has been a good deal of previous work on mixed physical and graphical interactions. The authors support their research mostly by comparing this previous work to DataTiles. For example, several times in the related work section, the authors describe previous work on mixed physical and graphical interactions and then compare their own work to it. They describe DataTiles as an extended and more integrated system, but they sometimes fail to provide solid reasoning for these developments. One of the claimed advancements of DataTiles over previous systems is a tightly coupled input and output space. In terms of graphical interactions, this claim makes sense: it is better to have the output display closer to the input device in most contexts. However, they do not state the advantages of this approach in terms of physical interactions. When building new interactive systems, it is important to consider the interaction within the entire system and not simply within the more familiar aspects of the system.

A qualitative evaluation of the DataTiles system took place during a laboratory open house event where the authors recorded user’s reactions and comments. Two key points were highlighted: physical features and tile composition. Many users appreciated the tile size and tactile feelings, but some felt that of the guides on the tiles interfered with their interaction. In addition, some users had parallax and registration issues with the tiles and were unable to determine “which side was up.” Regarding tile composition, many users enjoyed the idea of composing tiles, but there was sometimes confusion when two or more combinations of tile existed. While interesting, their evaluation was somewhat limiting. It would have been more valuable if they compared the DataTiles system to current graphical interfaces through a series of quantitative tests.

There are some notable applications of DataTiles: media editing and educational platforms are two examples the authors provide. But for the most part, I had difficulty appreciating the usefulness of physical interactions in an increasingly graphical world. While reading, I could not help but envision a graphical interface accomplishing the same user goals with similar outcomes. DataTiles seem to take the graphical windowing environment and simply replace it with physical tiles. An application the authors mentioned was a time-wheel tile next to a container tile that held a movie. By dragging a pen around the time-wheel, the movie could be moved forward or rewound. The computer I am using right now has a movie player with forward and rewind controls that do the same thing as the time-wheel tile but within a different interface. Each interface has its own benefits, but in the case of DataTiles, I do not believe the advantages of interacting with a physical object outweigh the disadvantages. Some concerns that come to mind are storage, transportation and durability of glass tiles.

Mixed physical and graphical interactions have clear benefits in the context of computing, and I believe that we should continue to explore these interactions. However, these mixed interfaces should not duplicate what graphical interfaces already accomplish reasonably well. They should instead focus on interaction that graphical interfaces are unable to accomplish.

DataTiles system

All installments in this series:

The interaction design process

Preface

During grad school I often wished that I was writing for unraveled and the UX community rather than for one of my professors. So much so that much of the work I submitted for my portfolio was pulled directly from this weblog. But I wasn’t able to write all of it for unraveled, and most of my papers have remained locked up on my iBook since I submitted them. I knew that at some point I would publish these papers here for everyone else to read and enjoy. I’ve decided that time is now. Over the next few weeks, I’m going to be publishing all of the work I did for my HCI graduate course that’s fit to print. I welcome your feedback on these papers, but please keep in mind that they were written primary as an academic exercise, not as a designer writing for the community.

The first installment is also the first paper that I submitted: the interaction design process.

Introduction

Whether the operating system is Windows, Macintosh or Linux, a common problem is that users often aren’t able to locate saved documents on their computers efficiently. More to the point, file retrieval on most computers is hard. An experienced computer user might be surprised to read this. “File retrieval is hard? What’s so hard about using the open command, navigating to the file and clicking open?”

The problem with the open command is that it places all responsibility for finding a file on the user’s memory and folder structure. For a user to open a file efficiently, he needs to quickly recall the file’s name and position in the folder structure from his memory. Alternatively, the user could use a find or search command that can locate files based on their name and contents, but this command is usually not integrated with the open command and often doesn’t return useful results.

In this paper, I’ll outline the various steps that an HCI expert might take towards finding a solution to the file retrieval problem. Generally speaking, the interaction design process doesn’t change, whether it’s being used to create an automobile dashboard display or a single command within an operating system. Thus, the following interaction design process could in fact be used to create a dashboard display. The point is that although the problem might change, the general approach can remain the same.

Research

The first stage of the approach is the research stage, which will allow the designer (HCI expert) to gain a thorough understanding of the file retrieval problem and the users who are affected by it. The first step is this stage is user interviews, which allow the designer to ask users direct questions about how they use the open command on their computer. Some questions may include when they use the open command, their frustration with the open command and their knowledge of the file retrieval process. While interviews provide a great deal of information about how users think they behave, user observation will allow the designer to see how users behave firsthand. It’s often surprising to hear users explain their actions, and then to see them act completely different. One user may say that they never use the open command in a particular situation, but user observation may find that they not only use the open command in that situation, they use it often.

Literature review is what most people think of when they hear the word, “research.” Indeed, it’s an integral part of any research process. Two papers that may be useful on the subject of file retrieval are Finding and Reminding: File Organization from the Desktop by Deborah Barreau and Bonnie A. Nardi and Why Can’t I Find My Files? New Methods for Automating Attribute Assignment (PDF), by Craig A. N. Soules and Gregory R. Ganger. The first paper summarizes and synthesizes two independent studies of the ways users organize files on their computers. The second discusses approaches to automatically assign attributes — metadata that describes a file — to files.

A final step of the research process is brainstorming. By the end of the research process, the designer will likely have a lot of ideas brewing in her head. Before moving any further into the design process, it’s a good idea to record these ideas. Some ideas may seem strange or even impossible at first, but some of them may prove useful during the remainder of the process. Never say no to an idea until you thoughtfully rule it out.

Requirements

The first part of the requirements stage is to gather user expectations. This includes functions that users expect or desire from the new open command. To gain a better understanding of user expectations, it’s also helpful to know what influences the user’s expectations and desires such as environmental, social or cognitive factors.

The second part of the requirements stage is to gather technical requirements, including the data and functional needs of the system. In this case, the system is the computer operating system. For example, if the operating system’s file system can only be structured one way, it should be noted that the new open command would need to work within that specific file structure.

Design

The design stage defines the new open command, including its new interaction and flow. The first step is to define the interaction using all the knowledge gained through the research and requirements stages. The goal is to create a framework that the interaction takes place within. This can be as simple as making sketches on paper or detailed as numerous flow charts.

Next, the designer must define the interaction elements required to carry out the interaction designed in the previous step. Interaction elements include forms, inputs, buttons, scrollbars and menus that may be needed to create the new command.

Finally, using the interaction design definition as a blueprint, a prototype of the new open command can be built. It’s important that the prototype have all the general characteristics of a finished product, but it need not be perfect as the command will be refined after it’s completed and actual users are given the opportunity to use the new command.

User Testing

After the prototype is completed, the next step is a solid round of user testing, in which users of the final open command will be able to use the prototype. Many of the same techniques employed in the research stage can also be used in the user testing stage in order to gain further insight on the usability and overall experience of the new open command.

Refine

The final step in the design process is to refine the prototype. This involves finalizing the design of the command, after which it can be integrated into the operating system and delivered to users. Since the design process is iterative, further user testing and refinement may be necessary in order to achieve a usable version of the command.

All installments in this series:

UXnet London June Social Event

You’re invited to the monthly social meeting in London for the User Experience Design community.

Date: Thursday, 23 June 2005
Time: 6:30 PM
Place: Ruby Lounge
34 Lower Marsh, near Waterloo station
Find it with Google Maps
Please email any queries to

Notes from UXnet London Local Ambassador, Joshua Kaufman:

Start the weekend by joining your UX colleagues at Ruby Lounge, a friendly bar just around the corner from Waterloo station. Word has it that it fills up quickly so arrive early if you can!
Many of you are probably wondering “Exactly what is UXnet doing besides hosting social meetings?” Over the past few months, UXnet local ambassadors around the world have been busy designing locale sites for their regions. I’m happy to report that the design is now in its final stages, and locale sites will be popping up in major cities - including London - over the next few months. More details to come next month.
Just when you thought there was a day for everything… World Usability Day, sponsored by the Usability Professionals Association, is on 3 November, 2005. It aims to promote the fields of usability engineering and user-centered design by encouraging, organizing, and sponsoring activities at the local level around the globe. Learn more about World Usability Day: http://worldusabilityday.net/about.
As always, I encourage you to visit the UK UXnet calendar to learn about the UX events happening in the UK over the next few months: http://www.louiseferguson.com/events.htm. If there’s a UK UX event that you feel should be included on this calendar, please email the event details to louise at gmail.com.

This is a UXnet (http://uxnet.org) event and was organized through the coordinated efforts and resources of IAI, ASIS&T, AIGA, IxDG, STC UK, UK UPA and BCS-HCI, LondonIA, London Usability, UCLIC and BIMA. If you represent an organization that you feel should also be part of future UXnet event coordination in London, or an organization that would like to host future UXnet events, please contact .

Things my train driver actually said on my commute home

I commute to work in London via the Northern Line every day. Usually, the train driver says very little. If they say anything, it’s usually an explanation for why the train hasn’t moved in the last minute. So I was happily surprised to hear my happy-go-lucky train driver say the following things on my commute home last Friday:

For those getting off at Embankment, I hope you have a good evening and a great weekend. And you can start your weekend off right by minding the gap at the next station.

The next stop on this magical mystery tour is Leicester Square.

If you see someone who is elderly or disabled, please do give up your seat to those less able to stand.

I see there’s a High Barnet train from this station in about 9 minutes. However, I recommend staying on this train and changing at Camden Town as you’ll probably get to your destination sooner.

If you’re going out clubbing while in Camden Town, do be careful. There are some undesirable places in Camden Town and definitely some undesirable characters.

He didn’t say much more after Camden Town, but what he said up until that point was enough to make my Friday commute so much more enjoyable. It takes so little energy from train drivers to make so many people smile; it’s a shame this doesn’t happen more often.

Three OS X Apps: Tiger Update

Back in January 2003 I talked about my three favorite OS X apps for their simplicity and functionality. A lot has changed since then. My primary computer is an iBook instead of an iMac, Panther and Tiger have replaced Jaguar and Safari appeared. So I thought I should update the list.

Quicksilver by Blacktree does everything that Launchbar did, but is faster, more extensible and free. In a nutshell, if you want to get things done with you Mac, and you haven’t tried Quicksilver, you’re missing out. The latest reason to use it is the excellent Backpack plugin, beautifully integrating your Mac with Backpack.
Safari by Apple is small, fast and simple. I know it doesn’t have the simple extensibility that Firefox has, but with PithHelmet, Saft and SafariStand, there isn’t much more I need. I actually tried switching to Firefox a while back but speed was an issue on OS X Panther. And since I spend about 75% of my time in the browser, I need it to be fast. This is where Safari excels the most. (Can anyone comment on Firefox speed in Tiger vs. Panther?)
LiteSwitch X by Proteron and SideTrack by Raging Menace tied for third. Even with the newer bezel interface, app switching in OS X still leaves me with many desires, and LiteSwitch fills each of them. I can hide, quit, force quit and relaunch, all within the same interface. It may seem overpriced for its functionality but when you consider how often you switch applications in OS X, $14.95 doesn’t seem so bad. Sidetrack, which is also $15, allows me to vertically scroll by sliding my finger down the right edge of my iBook trackpad. Is there anything more I need to say? Yes, I know the new PowerBooks have a trackpad scrolling feature, but using two fingers to scroll seems unnecessary.

← May 2005 | Archives | July 2005 →