'Get what you need with the click of a button' is the mantra for many technology designers. But how do you create that kind of a user experience when you have a screen to touch, not a button to push‾
Touch-based systems with rich graphics have evolved from their earliest incarnations such as information kiosks to encompass a wide array of beneficial tools for users, including handheld remote controls, tablet PCs and GPS navigation devices. Now, researchers are developing the next frontier for touch: truly integrating this technology into mobile phones and other mobile devices.
Called 'visually rich, direct manipulation' (VDRM) products, they are aimed at offering users significant productivity gains, compelling new capabilities and improved visual engagement.
Creating the ideal VDRM experience, however, is challenging. The current crop of VDRM-based applications strives to make life easier for the mobile user by lowering the cognitive requirement for abstract thinking, simplifying the requirement for hand-eye coordination, affording free-form input and enabling the use of more durable form factors.
These are wonderful attributes that benefit multi-tasking mobile users who need to devote their split attention between operating their device, launching multiple applications and navigating the real world. Yet, the mobile VDRM experience can also pose drawbacks as diverse as finger stress from overuse, the demand of constant visual attention, the lack of tactile feedback, and even the accumulation of undesired smudges, oils and dirt.
Consider the airline-ticket-dispensing kiosk. When users are asked to select a seat on the airplane, they are presented with a graphical depiction of the airplane's seating arrangement, typically showing occupied and available seats by using simple color codes or symbols. Consumers can indicate the seat they wish to occupy by touching it on this seat map. This requires no separate data entry and it clearly places the seat in context so that customers can select the desired seat.
Choosing a seat using text-only descriptions is far less desirable. Imagine selecting a seat from a table like 'Seat 14F is a window seat, on the left side of the airplane, located in front of the leading wing edge' - or, worse, by direct keyboard entry with no contextual information. The touch-screen experience is clearly superior for the customer.
Helpful or frustrating‾
However, for the airline reservation agent, the VRDM experience may become a burden. Airline reservation agents likely already know the context information of the seating arrangement. The visual representation offers no extra information and can slow down the interaction. Plus, from an ergonomics perspective, the manipulation could be a burden. Accordingly, it is much easier for a reservation agent to enter data in a prompted data field with a few easy keystrokes.
Using this example, mobile device application designers need to be especially mindful of their users' familiarity, contextual knowledge and frequency of use in designing VRDM experiences. For those customers who use a service sparingly, visually rich information is helpful; for others, it may cause frustration.
Another benefit for mobile devices is that VRDM can allow for new capability in free-form data entry unmatched by technology currently in the marketplace. To get full value from direct manipulation, the technology should include support for stylus input. For example, if a touch-based application consists of pushing finger-pad-size buttons, selecting icons or dragging objects from one location to another, using a finger may be adequate for the task. However, if the goal is to input a multi-stroke graphical character such as those found in Asian languages, a stylus is a very important and useful tool for a detailed drawing. A stylus is helpful in making precise markings and capturing natural handwriting if creating and transmitting a hand-drawn free-form image, such as an annotated map or picture.
Laying the content
VRDM products can enable more informative, intuitive and exciting visually-engaging experiences. Techniques such as information-layering using transparency and compositing can present a rich image, overlaid with text to create a visually pleasing effect. For example, a movie poster could be changed dynamically, based on the consumer's direct manipulation. This is the beginning of a new, more powerful kind of interface.
More intuitive experiences can be enabled when they create a high-fidelity simulation of the familiar physical world. Gesture-initiated scrolling, page flipping and zooming are all more satisfying when coupled with hand or finger movements that use gross motor skills. For example, the Nintendo Wii makes playing simulated tennis more fun because your handheld controller becomes a symbolic tennis racket and requires customers to wave their arm appropriately. Technology designers can and should evaluate when the occasional use of this kind of system may be fun or when it may become a burden for repetitive tasks.
An even more exciting experience can be enabled by VRDM products when they use techniques such as perspective, reflections, lighting and shadowing to improve object identification, discrimination and understanding. Designers should avoid repetitive images and color; this will help ensure that the emotional excitement generated remains visually stimulating.
Tom MacTavish is VP and director of Human Interaction Research at Motorola Labs