Goal-oriented gaze strategies afforded by object interaction

A. Belardinelli, O.Herbort, M.V. Butz (published in Vision Research, 2015)

Task influence has long been known to play a major role in the way our eyes scan a scene. Yet, most studies focus either on visual search, or on a sequence of active tasks in complex real world scenarios. Only few studies have contrasted the distribution of eye fixations during viewing and grasping objects.

Here we address how attention is deployed when different actions have to be planned on objects, in contrast to when the same objects have to be categorized.

In this respect, we are particularly interested in the role every fixation plays in the unfolding dynamics of action control. To investigate these issues, we conducted an eye-tracking experiment in which participants were shown images of real-world objects. Subjects were to either assign the displayed objects to one of two classes (categorization task), to mimic lifting (lifting task), or to mimic opening the object (opening task). Results suggest that even on simplified, two dimensional displays the eyes reveal the intentions of the participant in an anticipatory fashion.

In the case of active tasks, already the second saccade after the stimulus onset was directed towards the central region between the two locations where the thumb and the rest of the fingers would be placed. An analysis of saliency at fixation locations showed that fixations in active tasks have a higher correspondence with salient features than fixations in the passive tasks. We suggest that attention flexibly coordinates visual selection for information retrieval and motor planning and hence works as a gateway between three components, linking the task (action), the object (target), and the effector (hand) in an effective way.

Here we show for every used stimulus the original image, the corresponding saliency map, the fixation map and the touch map (for the lift and open task) as computed from collected data.