I recently read an interesting CSCW 2008 paper by Wendy Ju: Range: Exploring Implicit Interaction through Electronic Whiteboard Design. She describes a framework for implicit interaction and applies it to the design of an interactive whiteboard application called Range.

The paper is situated in the field of ubiquitous computing. The goal of Mark Weiser‘s vision of ubiquitous computing was calm computing, where calm reflects the desired state of mind of the user. Invisibility in ubicomp is more about enabling seamless accomplishment of task than staying beneath notice. Just as a good, well-balanced hammer “disappears” in the hands of a carpenter and allows him or her to concentrate on the big picture, computers should participate in a similar magic disappearing act. Calm computing moves between the center and the periphery of attention. The periphery informs without overwhelming the user, but the user can still move to the center to get control. The implicit interaction framework presented in this paper contributes to how calm computing can be effectively realized. It allows to reason about the way users can mitigate actions the system takes, and is a good complement to Eric Horvitz’s work on mixed-initiative interaction.

Implicit interactions enable communication and action without explicit input or output.One way that an action can be implicit is if the exchange occurs outside the attentional foreground of the user (e.g. auto-saving files, filtering spam, and ubicomp interaction). The other way is if the exchange is initiated by the computer system rather than by the user (e.g. an email alert, screen saver, etc.). Although it may seem strange that something that grabs attention is implicit, the key factor is that the interaction is based on an implied demand for information or action, not an explicit one.

The implicit interaction framework divides the space of possible interactions along the axes of attentional demand and initiative:

Attentional demand is the degree of cognitive and perceptual load imposed on the user by the interactive system. Foreground interactions require a greater degree of focus, concentration and consciousness and are exclusive of other focal targets. Background interactions are peripheral, have less demand and can occur in parallel with other interactions.

Initiative is an indicator of how much presumption the interactive system uses in the interaction. Interactions that are initiated and driven by the user explicitly are called reactive interactions, while interactions initiated by the system based on inferred desire or demand are proactive interactions.

The implicit interaction framework builds on Bill Buxton’s background/foreground model. Buxton’s model assumes attention and initiative are inherently linked. On the contrary, this framework decouples attention and initiative in two separate axes. Buxton’s foreground corresponds to the reactive/foregound quadrant, while his background corresponds to the proactive/background quadrant.

An example: a word processing program that …

  • auto-saves because you command it to is situated in the reactive/foreground quadrant
  • auto-saves because you have set it to do so every 10 minutes is situated in the reactive/background quadrant
  • auto-saves because it feels that a lot of changes have been made is situated in the proactive/background quadrant

Being proactive means the word processing program is acting with greater presumption with respect to the needs and desires of the user.

Designers can manipulate the proactivity and reactivity by (1) dictating the order of actions (does the system act first or wait for the user to act?); or by (2) choosing the degree of initiative (does the sytem act, offer to act, ask if it should act, or merely indicate that it can act?); or by (3) gathering more data to ensure the certainty of the need for an action or when they design features to mitigate the potential cost of error for the action. Even in the reactive realm, the degree of initiative can vary based on the amount that the user needs to maintain ongoing control and oversight of an action in progress.

Ju discusses 3 implicit interaction techniques:

  • user reflection
  • system demonstration
  • override

User reflection is how the system indicates what it feels the users are doing or would like to have done.

A good example are modern spell-checking programs. Early versions had to be invoked explicitly, and engaged the user in an explicit dialog about potentially misspelled words to repair. Current spell-checking programs run continuously in the background allowing users to more easily notice potential errors. The implicit alert of this interaction is far more seamless than that of earlier spell-check programs. A similar example is the continous compilation used by modern IDEs such as Eclipse and Visual Studio. Earlier programs only showed compile errors when the user explicitly activated the compile command.

System demonstration is how the system shows the user what it is doing or what it is going to do.

In Range, the whiteboard animates its transition from ambient display mode (where it displays a set of images related to the workspace) to drawing mode (where users can make sketches and diagrams) as a demonstration-of-action that calls more attention to the mode change than a sudden switch would, and provides a handle for override (see later).

Override techniques allow users to repair misinterpretation of the user’s state or to interrupt or stop the system from engaging in proactive action.

This usually occurs after one of the previous techniques (user reflection or system demonstration) alert the user to some inference or action that is undesirable. Override is distinct from “undo” because it is targeted at countering the action of the system rather than reverting a command by the user.

An example of override in Range is that in the transition between modes users are able to “grab” digital content to use it as part of the whiteboard contents, or to stop the motion of objects that are being moved to make space for drawings.

The main contribution of this framework compared to prior models for implicit interaction lies in the key variable of initiative. Without this variable, it would not be possible to distinguish user reflection techniques from system demonstration techniques or to map the role of override.

In conclusion, a very interesting paper that offers a framework to reason about proactive user interfaces and make sure that users are always in control.