Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces

Overview

This paper goes over freehand gestural interaction with direct-touch computation surfaces. The paper then goes over the design principles: gesture registration, gesture relaxation, and gesture & tool reuse. They tested the design principles using the annotate gesture, wipe gesture, cut/copy & paste gesture, and pile-n-browse gesture and they conducted these in a user evaluation.

Design Principles

One drawback, mentioned in the gesture relaxation section, was that the height of a table - have it be a coffee table or desk height - may impact the system’s ability to read the gesture. If a user attempts to put down three fingers, the coffee table height while standing may only read the finger tips and the desk table height when sitting down may read the finger tips and much of the rest of the fingers. If these were two different gestures, the system would not be able to differentiate them.

For the “gesture and tool reuse section”, they are hinting towards using the same gesture to mean different things in different contexts. This is similar to having modes, which is typically not a good thing according to most HCI studies. This could confuse a user and have them doing a certain gesture expecting a certain result, but having a different result. However, there must be cases where it is clearly evident that using the same gesture while in a different case would yield an expected result, but designers would have to be careful.

Another note, a user may be in the middle of one gesture, post gesture registration, and have the urge to perform another gesture, without finishing the first gesture. This could cause some confusion amongst users and yield unpredicted results. Possibly, some system feedback could keep user aware that the prior gesture has not finished yet.

It is good that reusing gesture primitives, allowing users to learn a smaller vocabulary, is possible. Designers must be careful to not use these primitives for completely different tasks.

User Evaluation

In the study, the users appeared to have learned the gestures fairly easy, but some struggled with the “piling” gestures. It would be interesting to see if users remembered these gestures after not using them for some period of time.

Discussion

• A common vocabulary of gesture primitives
• Using them in different contexts should be for similar tasks and yield similar results. For example, dragging one finger should be used for moving something in all contexts, not resizing in one context and moving the screen in another context. To me, that would be confusing. There should be a common vocabulary where, say, four fingers would drag the screen and not ever move an item on the screen.
• Memory of gestures
• It would be interesting to see a study of how users memorize these common gestures and gesture primitives over time. They may become second nature or users would keep having to refer back to some guide.
• Primitives as letters, and gestures as words
• It is interesting to compare the gestures & primitives to a full human language
• Each primitive would be like a letter or word and gestures would be like a sentence. Users would be able to “speak” to the computers with this language.