Computer Laboratory

Projects

Tracking eye position and gaze direction in near-eye volumetric displays

Marek Wernikowski(1, 2) Joseph G. March(1) Radosław Mantiuk(2), Özgür Yöntem(1), and Rafał K. Mantiuk(1).

(1)University of Cambridge, (2)West Pomeranian University of Technology, Szczecin

Presented at ISMAR 2024

Our eye-tracking solution predicts the position of the nodal point of each eye through the use of advanced computer vision techniques. We demonstrate this working on a near-eye volumetric display.

Abstract

Near-eye volumetric displays, showing multiple focal planes, require knowledge of the accurate position of the nodal point of the eye to correctly render a 3D scene. This is because pixels seen through multiple planes must be accurately aligned with the eye’s visual axis to ensure consistency across focal planes. While most eye-tracking methods focus on determining a gaze position within a designated target space, this work aims to track both the eye position and the corresponding gaze direction expressed in coordinates relative to the physical location of the volumetric display planes. To achieve this, we rely on a near-infra-red (NIR) camera image of the pupil and corneal reflections (glints). The existing eye model is used to establish the relationship between the pupil and glint positions in a NIR image and the eye position and rotation in a 3D space. We address the key challenge of robust tracking of the glints in a system that introduces multiple reflections. We also demonstrate that the system reduces the need for recalibration on subsequent uses. Our experiments on a multiple-focal plane display demonstrate that the method can maintain an accurate projection point for volumetric displays.

Materials

  • Paper:
    Tracking eye position and gaze direction in near-eye volumetric displays.
    Marek Wernikowski, Joseph G. March, Radosław Mantiuk, Özgür Yöntem and Rafał K. Mantiuk.
    In ISMARR 2024
    [paper PDF]
  • Supplementary document [PDF]
  • Code [Github]

Related projects