by Carsten Schwesig
Figure 1. Mockup credit-card bendable display (courtesy SONY CSL ).
Movies can make us forget that we are sitting in a cinema among strangers, looking at images projected onto a wall. Instead, we feel as though we are observing real people in real situations and we become emotionally involved in the narrative. User interfaces can trigger a similar suspension of disbelief: we forget that we are operating a machine to manipulate virtual, digital data. Instead, we experience media and applications as part of our physical environment. Such interfaces feel “natural”, or rather “organic”.
Used in this sense, the term “organic” refers to subjective experience. It is therefore difficult to define or quantify, but we can observe some qualities that make interfaces feel organic. In this article I want to concentrate on techniques that achieve this effect by introducing physical properties to the user interface.
Elements of physics can be added to user interfaces in a number of ways: one approach is to imbue real physical objects with digital properties, as in the work of the Tangible Media Group at the MIT Media Lab. Another approach is to simulate physical environments on screen: some car racing games, for example, derive much of their appeal from convincing physics modeling.
But physics – real or simulated – can also be very limiting: computers are so useful as a media platform precisely because digital media are not bound by the laws of physics: Hypertext links break when printed on paper. YouTube is incompatible with the real or simulated physics of film projectors.
Organic interface design represents a less literal approach which, rather than focusing on physical objects or metaphors, emphasizes the analog, continuous and transitional nature of physical reality and human experience. By combining sensitive analog input devices with responsive graphics, we can create user experiences that acknowledge the subtleties of physical interaction.
A ubiquitous example of analog input coupled with responsive graphics is the computer mouse: continuous mapping of physical hand movement to virtual pointer position results in the experience of direct manipulation that is central to the WIMP interface. The introduction of the mouse, in combination with a responsive screen interface, transported digital information from the abstract, cerebral world of the command line right into our tangible, physical environment.
We explored the use of analog sensors in the development of the Gummi interface concept. Gummi was inspired by a new generation of organic, flexible electronics. Our reasoning was that, at some point in the future, it would be possible to build credit-card sized, flexible computers composed of layers of organic electronic components: flexible batteries, circuits, sensors and a FOLED display. What kind of new interfaces would be possible with such a hypothetical device?
The resulting interface concept and prototype allows users to browse digital media by a combination of physical deformation and 2D position control. Held in both hands, the device can be bent along one axis while a touchpad mounted on the back of the device is used to control 2D position (see Fig. 1). With a simple, consistent vocabulary of physical interactions, it is possible to navigate file structures, maps, hyperlinks, photo albums and movies.
Figure 2. Potential interactions with bendable credit-card displays.
We also implemented a street map that can be scrolled with the touchpad and zoomed by physically bending the device. The flat prototype can be bent either away from or towards the user to control zoom direction (Fig 2). The amount of bending is mapped to zoom speed – slight deformation results in a slow, continuous zoom; stronger bending increases the zoom speed.
This kind of analog interaction has two benefits. It gives the user finer control over the feature in question and it feels organic because it is as sensitive to the nuances and transitions of physical interaction as a real-world object.
Although Gummi was inspired by new hardware technology, the use of analog sensors in the resulting prototype became a focus for the project. We realized that capturing analog, continuous physical interaction led to an interface that offered interesting new functional possibilities, while at the same time feeling very organic.
Two recently released products feature analog interaction techniques. The Nintendo Wii gaming platform includes an input device that adds hand and arm gestures to the interaction vocabulary of game interfaces. Apple’s iPhone represents the first mass-marketed product that uses multi-touch interaction. Its responsive graphical user interface is peppered with subtle, animated visual behaviors that add up to create a very tactile, organic user experience.
Analog interaction techniques arguably played an important role in the success of both the Wii and the iPhone and recent technological developments suggest that there is a lot of room for exploration in this area: small, cheap sensors can capture a wide range of analog physical inputs such as gestures, pressure, deformation, multi-point touch, orientation and location. New materials, such as organic LED displays, point towards flexible computers that can sense their own shape. Mobile devices with fast processors and high-resolution displays support graphical user interfaces that can match the subtle, analog nature of physical interaction.
The above mentioned examples share a number of characteristics that contribute to the organic feel of these interfaces: By using analog sensors in input devices, continuous and subtle changes in physical interaction can be measured. The analog complexity of input is reflected in a highly responsive graphical user interface, featuring smooth animation and consistent visual behavior. As a result, input device and graphical user interface are experienced as a whole, not as independent elements of the interface.
The cumulative effect of these characteristics is a user interface that inspires a suspension of disbelief: intangible information feels as though it is part of our tangible physical environment.
1. Schwesig, Carsten, Poupyrev, Ivan and Mori, Eijiro (2004): Gummi: a bendable computer. In: Dykstra-Erickson, Elizabeth and Tscheligi, Manfred (eds.) Proceedings of ACM CHI 2004 Conference on Human Factors in Computing Systems April 24-29, 2004, Vienna, Austria. pp. 263-270.
Carsten Schwesig is an interaction designer and researcher living in Tokyo. After graduating from the Royal College of Art in 2001, he became a member of the Interaction Laboratory at Sony CSL. In 2007, he joined Sony’s Creative Center, where he continues to explore the possibilities of analog interaction.