unraveled

A review of DataTiles

Preface

This is the second installment of my graduate school work: a critical review of DataTiles, by Rekimoto et al. For a background on my graduate work, please read the preface of the first installment, the interaction design process.

In the paper DataTiles: A Modular Platform for Mixed Physical and Graphical Interactions (PDF), the authors, Jun Rekimoto, Brygg Ullmer and Haruo Oha, describe DataTiles as a system that integrates the benefits of two major interaction paradigms: graphical and physical user interfaces. They mention several previous efforts towards integrating the two interfaces as well as the advantages of this approach. However, the authors felt that many of these efforts fell short. Most prior work has focused on special purpose systems that do not scale to support different applications. Additionally, while many previous systems have used physical objects as a means of interaction, they do not retain the flexibility and power of graphical user interfaces.

To address these issues, the authors introduce the use of “graphically augmented physical objects.” Their prototype system, DataTiles, uses transparent tiles as a modular unit of interaction. These tiles are combined with a sensor-enhanced flat panel display allowing users to control digital data “within” physical DataTiles while retaining the flexibility and power of graphical user interface.

As an example, to view a specific remote camera, users can place a map tile next to a portal tile on the display. The map tile then shows camera locations, allowing the user to select which camera they would like to view in the portal tile. This is a simple example that only uses two tiles, but much more complex combinations can be composed by using the five different types of tiles.

I found the motivations behind DataTiles intriguing, especially the idea of mixed physical and graphical interactions. As the authors mentioned, interactions with physical objects can often be much richer than interactions with graphical interfaces. People can use sophisticated motor skills to manipulate objects more complexly than pointing and clicking. In addition, interactions can involve two hands or allow several people to interact cooperatively. These are all powerful interaction ideas that deserve to be investigated in the context of computing. Unfortunately, beyond basic physical interactions derived mostly from graphical interactions such as clicking and dragging, DataTiles use as a physically interactive system is limited. Once a tile is placed on the display, little more can be physically accomplished. Most interactions after that point are, in fact, graphical.

Regardless of this shortcoming, the authors presented a worth while case for DataTiles and future research in mixed physical and graphical interactions. Three key interaction ideas are summarized in the introduction and then mentioned throughout the paper:

  1. Tagged transparent objects as graspable interaction modules. These serve as physical windows for digital information and trigger specific actions when placed on the display surface.
  2. Mixed physical and visual [graphical] interactions.
  3. Physical language for combining multiple tiles.

As mentioned, there has been a good deal of previous work on mixed physical and graphical interactions. The authors support their research mostly by comparing this previous work to DataTiles. For example, several times in the related work section, the authors describe previous work on mixed physical and graphical interactions and then compare their own work to it. They describe DataTiles as an extended and more integrated system, but they sometimes fail to provide solid reasoning for these developments. One of the claimed advancements of DataTiles over previous systems is a tightly coupled input and output space. In terms of graphical interactions, this claim makes sense: it is better to have the output display closer to the input device in most contexts. However, they do not state the advantages of this approach in terms of physical interactions. When building new interactive systems, it is important to consider the interaction within the entire system and not simply within the more familiar aspects of the system.

A qualitative evaluation of the DataTiles system took place during a laboratory open house event where the authors recorded user’s reactions and comments. Two key points were highlighted: physical features and tile composition. Many users appreciated the tile size and tactile feelings, but some felt that of the guides on the tiles interfered with their interaction. In addition, some users had parallax and registration issues with the tiles and were unable to determine “which side was up.” Regarding tile composition, many users enjoyed the idea of composing tiles, but there was sometimes confusion when two or more combinations of tile existed. While interesting, their evaluation was somewhat limiting. It would have been more valuable if they compared the DataTiles system to current graphical interfaces through a series of quantitative tests.

There are some notable applications of DataTiles: media editing and educational platforms are two examples the authors provide. But for the most part, I had difficulty appreciating the usefulness of physical interactions in an increasingly graphical world. While reading, I could not help but envision a graphical interface accomplishing the same user goals with similar outcomes. DataTiles seem to take the graphical windowing environment and simply replace it with physical tiles. An application the authors mentioned was a time-wheel tile next to a container tile that held a movie. By dragging a pen around the time-wheel, the movie could be moved forward or rewound. The computer I am using right now has a movie player with forward and rewind controls that do the same thing as the time-wheel tile but within a different interface. Each interface has its own benefits, but in the case of DataTiles, I do not believe the advantages of interacting with a physical object outweigh the disadvantages. Some concerns that come to mind are storage, transportation and durability of glass tiles.

Mixed physical and graphical interactions have clear benefits in the context of computing, and I believe that we should continue to explore these interactions. However, these mixed interfaces should not duplicate what graphical interfaces already accomplish reasonably well. They should instead focus on interaction that graphical interfaces are unable to accomplish.

DataTiles system

All installments in this series:

  1. The interaction design process
  2. A review of DataTiles
  3. Designing a cashpoint
  4. Culture and web design
  5. Serco: an independent usability consultancy
  6. An evaluation of exhibit design at the London Science Museum
  7. Thesis report: Accessible and Usable Web Design

  1. Briefly: I agree, the instantiation of tangible computing here seems gratuitous. Better to take the approach Tom Igoe at NYU ITP does (“In physical computing, we take the human body as a given, and attempt to design within the limits of its expression.”)

    Seems to me a surer way to arrive at design solutions where the physical interface is necessary, rather than a cool hack tossed in “because we can.”

    AG on 1 Jul 05

Comments are now closed for this entry