IEEE TVCG (ISMAR Journal-track), 2024.
Zhimin Wang, and Feng Lu
With eye tracking finding widespread utility in augmented reality and virtual reality headsets, eye gaze has the potential to recognize users’ visual tasks and adaptively adjust virtual content displays, thereby enhancing the intelligence of these headsets. However, current studies on visual task recognition often focus on scene-specific tasks, like copying tasks for office environments, which lack applicability to new scenarios, e.g., museums. In this paper, we propose four scene-agnostic task types for facilitating task type recognition across a broader range of scenarios. We present a new dataset that includes eye and head movement data recorded from 20 participants while they engaged in four task types across 15 360-degree VR videos. Using this dataset, we propose an egocentric gaze-aware task type recognition method, TRCLP, which achieves promising results. Additionally, we illustrate the practical applications of task type recognition with three examples. Our work offers valuable insights for content developers in designing task-aware intelligent applications. Our dataset and source code will be released upon acceptance.
Our related work:
Gaze-Vergence-Controlled See-Through Vision in Augmented Reality
Interaction With Gaze, Gesture, and Speech in a Flexibly Configurable Augmented Reality System
Edge-Guided Near-Eye Image Analysis for Head Mounted Displays