A High-Level Description and Performance Evaluation of Pupil Invisible

Head-mounted eye trackers promise convenient access to reliable gaze data in unconstrained environments. Due to several limitations, however, often they can only partially deliver on this promise. Among those are the following: (i) the necessity of performing a device setup and calibration prior to every use of the eye tracker, (ii) a lack of robustness of gaze-estimation results against perturbations, such as outdoor lighting conditions and unavoidable slippage of the eye tracker on the head of the subject, and (iii) behavioral distortion resulting from social awkwardness, due to the unnatural appearance of current head-mounted eye trackers. Recently, Pupil Labs released Pupil Invisible glasses, a head-mounted eye tracker engineered to tackle these limitations. Here, we present an extensive evaluation of its gaze-estimation capabilities. To this end, we designed a data-collection protocol and evaluation scheme geared towards providing a faithful portrayal of the real-world usage of Pupil Invisible glasses. In particular, we develop a geometric framework for gauging gaze-estimation accuracy that goes beyond reporting mean angular accuracy. We demonstrate that Pupil Invisible glasses, without the need of a calibration, provide gaze estimates which are robust to perturbations, including outdoor lighting conditions and slippage of the headset.

Labeled pupils in the wild: A dataset for studying pupil detection in unconstrained environments

We present labelled pupils in the wild (LPW), a novel dataset of 66 high-quality, high-speed eye region videos for the development and evaluation of pupil detection algorithms. The videos in our dataset were recorded from 22 participants in everyday locations at about 95 FPS using a state-of-the-art dark-pupil head-mounted eye tracker. They cover people with different ethnicities, a diverse set of everyday indoor and outdoor illumination environments, as well as natural gaze direction distributions. The dataset also includes participants wearing glasses, contact lenses, as well as make-up. We benchmark five state-of-the-art pupil detection algorithms on our dataset with respect to robustness and accuracy. We further study the influence of image resolution, vision aids, as well as recording location (indoor, outdoor) on pupil detection performance. Our evaluations provide valuable insights into the general pupil detection problem and allow us to identify key challenges for robust pupil detection on head-mounted eye trackers.