Labeled pupils in the wild: A dataset for studying pupil detection in unconstrained environments

We present labelled pupils in the wild (LPW), a novel dataset of 66 high-quality, high-speed eye region videos for the development and evaluation of pupil detection algorithms. The videos in our dataset were recorded from 22 participants in everyday locations at about 95 FPS using a state-of-the-art dark-pupil head-mounted eye tracker. They cover people with different ethnicities, a diverse set of everyday indoor and outdoor illumination environments, as well as natural gaze direction distributions. The dataset also includes participants wearing glasses, contact lenses, as well as make-up. We benchmark five state-of-the-art pupil detection algorithms on our dataset with respect to robustness and accuracy. We further study the influence of image resolution, vision aids, as well as recording location (indoor, outdoor) on pupil detection performance. Our evaluations provide valuable insights into the general pupil detection problem and allow us to identify key challenges for robust pupil detection on head-mounted eye trackers.

3D Gaze Estimation from 2D Pupil Positions on Monocular Head-Mounted Eye Trackers

We collected eye tracking data from 14 participants aged between 22 and 29 years. 10 recordings were collected from each participant, 2 for each depth (calibration and test) at 5 different depths from a public display (1m, 1.25m, 1.5m, 1.75m and 2m). Display dimensions were 121.5cm × 68.7cm. We use a 5×5 grid pattern to disply 25 calibration points and an inner 4×4 grid for displaying 16 test points. This is done by randomly moving a target marker on these grid positions and capturing images from eye/scene camera at 30 Hz. We further perform marker detection using ArUco library on target points to compute their 3D coordinates w.r.t. scene camera. In addition, we are given the 2D position of pupil center in each frame of the eye-camera from a state-of-the-art dark-pupil head-mounted eye tracker (PUPIL). the eye tracker consists of a 1280×720 resolution scene camera and a 640×360 resolution eye camera. the PUPIL software used was v0.5.4.

Data is collected in an indoor setting and adds up to over 7 hours of eye tracking. Current dataset includes marker tracking results using ArUco per frame for every recording along with pupil tracking results from PUPIL eye tracker also for every frame of the eye video. We have also included camera intrinsic parameters for both eye camera and scene camera along with some post processed results such as frames corresponding to gaze intervals for every grid point. For more information on data format and how to use it please refer to the README file inside the dataset. In case you want to access the raw videos from both scene and eye camera please contact the authors.

Discovery of Everyday Human Activities From Long-term Visual Behaviour Using Topic Models

We recruited 10 participants (three female) aged between 17 and 25 years through university mailing lists and adverts in university buildings. Most participants were bachelor’s and master’s students in computer science and chemistry. None of them had previous experience with eye tracking. After arriving in the lab, participants were first introduced to the purpose and goals of the study and could familiarise themselves with the recording system. In particular, we showed them how to start and stop the recording software, how to run the calibration procedure, and how to restart the recording. We then asked them to take the system home and wear it continuously for a full day from morning to evening. We asked participants to plug in and recharge the laptop during prolonged stationary activities, such as at their work desk. We did not impose any other restrictions on these recordings, such as which day of the week to record or which activities to perform, etc.

The recording system consisted of a Lenovo Thinkpad X220 laptop, an additional 1TB hard drive and battery pack, as well as an external USB hub. Gaze data was collected using a PUPIL head-mounted eye tracker connected to the laptop via USB. The eye tracker features two cameras: one eye camera with a resolution of 640×360 pixels recording a video of the right eye from close proximity, as well as an egocentric (scene) camera with a resolution of 1280×720 pixels. Both cameras record at 30 Hz. The battery lifetime of the system was four hours. We implemented custom recording software with a particular focus on ease of use as well as the ability to easily restart a recording if needed.

We recorded a dataset of more than 80 hours of eye tracking data. The dataset comprises 7.8 hours of outdoor activities, 14.3 hours of social interaction, 31.3 hours of focused work, 8.3 hours of travel, 39.5 hours of reading, 28.7 hours of computer work, 18.3 hours of watching media, 7 hours of eating, and 11.4 hours of other (special) activities. Note that annotations are not mutually exclusive, i.e. these durations should be seen independently and sum up to more than the actual dataset size.

The dataset consists of 20 files. Ten files contain the long-term eye movement data of the ten recorded participants of this study. The other ten files describe the corresponding ground truth annotations.

More information can be found here.

Appearance-Based Gaze Estimation in the Wild

We present the MPIIGaze dataset that contains 213,659 images that we collected from 15 participants during natural everyday laptop use over more than three months. The number of images collected by each participant varied from 34,745 to 1,498. Our dataset is significantly more variable than existing ones with respect to appearance and illumination.

The dataset contains three parts: “Data”, “Evaluation Subset” and “Annotation subset”.

• The “Data” includes “Original”, “Normalized” and “Calibration” for all the 15 participants.
• The “Evaluation Subset” contains the image list that indicates the selected samples for the evaluation subset in our paper.
• The “Annotation Subset” contains the image list that indicates 10,848 samples that we manually annotated, following the annotations with (x, y) position of 6 facial landmarks (four eye corners, two mouth corners) and (x,y) position of two pupil centers for each of above images.

More information can be found here.

Prediction of Search Targets From Fixations in Open-World Settings

We recorded fixation data of 18 participants (nine male) with different nationalities and aged between 18 and 30 years. The eyesight of nine participants was impaired but corrected with contact lenses or glasses.

To record gaze data we used a stationary Tobii TX300 eye tracker that provides binocular gaze data at a sampling frequency of 300Hz. Parameters for fixation detection were left at their defaults: fixation duration was set to 60ms while the maximum time between fixations was set to 75ms.The stimuli were shown on a 30 inch screen with a resolution of 2560×1600 pixels.Participants were randomly assigned to search for targets for one of the three stimulus types.

The dataset contains three categories: “Amazon”, “O’Reilly” and “Mugshots”. For each category, there is a folder that contains 4 subfolder: search targets, Collages, Gaze data and binary mask that we used to get the position of each individual image in the collages.

• In the subfolder search targets you can find the 5 single images. The participants were looking for this image in the collages.
• In the folder Collages there are 5 subfolder. Subfolder with the same name as the search target indicate that users saw those collages for the search target. There are 20 collages per search target.
• In the folder gaze data you can find Media name, Fixation order, Fixation position on the screen, pupil size for left and right eye.

More information can be found here.

Recognition of Visual Memory Recall Processes Using Eye Movement Analysis

This dataset was recorded to investigate the feasibility of recognising visual memory recall from eye movements. Eye movement data was recorded of participants looking at familiar and unfamiliar pictures from four picture categories: abstract, landscapes, faces, and buildings. The study was designed with two objectives in mind: (1) to elicit distinct eye movements by using a large screen and well-defined visual stimuli, and (2) to record natural visual behaviour without any active visual search or memory task by not asking participants for real-time feedback.

The dataset has the following characteristics:

• ~7 hours of eye movement data recorded using a wearable Electrooculography (EOG) system
• 7 participants (3 female, 4 male), aged between 25 and 29 years
• one experimental run for each participant, involving them to look at four continuous, random sequences of pictures (exposure time for each picture 10s). Within each sequence, 12 pictures were presented only once; five others were presented four times at regular intervals. In between each exposure, a picture with Gaussian noise was shown for five seconds as a baseline measurement.
• separate horizontal and vertical EOG channels, joint sampling frequency of 128Hz
• fully ground truth annotated for picture type (repeated, non-repeated) and picture category

Eye Movement Analysis for Activity Recognition Using Electrooculography

This dataset was recorded to investigate the problem of recognising common office activities from eye movements. The experimental scenario involved five office-based activities – copying a text, reading a printed paper, taking handwritten notes, watching a video, and browsing the Web – and periods during which participants took a rest (the NULL class).

The dataset has the following characteristics:

• ~8 hours of eye movement data recorded using a wearable Electrooculography (EOG) system
• 8 participants (2 female, 6 male), aged between 23 and 31 years
• 2 experimental runs for each participant, each run involving them in a sequence of five different, randomly ordered office activities and a period of rest
• separate horizontal and vertical EOG channels, joint sampling frequency of 128Hz
• fully ground truth annotated (5 activity classes plus NULL)

Robust Recognition of Reading Activity in Transit Using Wearable Electrooculography

This dataset was recorded to investigate the problem of recognising reading activity from eye movements. The experimental setup was designed with two main objectives in mind: (1) to record eye movements in an unobtrusive manner in a mobile real-world setting, and (2) to evaluate how well reading can be recognised for persons in transit. We defined a scenario of travelling to and from work containing a semi-naturalistic set of reading activities. It involved subjects reading freely chosen text without pictures while engaged in a sequence of activities such as sitting at a desk, walking along a corridor, walking along a street, waiting at a tram stop and riding a tram.

The dataset has the following characteristics:

• ~6 hours of eye movement data recorded using a wearable Electrooculography (EOG) system
• 8 participants (4 female, 4 male), aged between 23 and 35 years
• 4 experimental runs for each participant: calibration (walking around a circular corridor for approximately 2 minutes while reading continuously), baseline (walk and tram ride to and from work without any reading), two runs of reading in the same scenario
• separate horizontal and vertical EOG channels, joint sampling frequency of 128Hz
• fully ground truth annotated (reading vs. not reading) using a wireless Wii Remote controller