pervasive gaze estimation
Gaze estimation is an active topic of research in several fields, most notably mobile and ubiquitous computing, computer vision, and robotics. Advances in head-mounted eyes tracking and egocentric vision promise continuous visual behaviour sensing in mobile everyday settings over days or even weeks. We have been working on advancing the state of the art in both remote and head-mounted gaze estimation for several years. For example, we have developed computer vision methods for appearance-based gaze estimation in the wild using large-scale and learning-by-synthesis methods. We have further presented computational methods for head-mounted eye tracker self-calibration, seamless gaze estimation across multiple hand-held and ambient displays, as well as for robust pupil detection and tracking under challenging real-world occlusion conditions.
selected publications
visual behaviour modelling and analysis
User modelling is among the most fundamental problems in human-computer interaction and ubiquitous computing. We have shown that everyday activities, such as reading or common office activities, can be predicted in both stationary and mobile settings from eye movements alone. Eye movements are closely linked to human visual information processing and cognition, such as perceptual learning, experience, or visual search. We have therefore further explored eye movement analysis as a promising approach towards the vision of cognition-aware computing: Computing systems that sense and adapt to covert aspects of user state. The vast majority of previous works focused on short-term visual behaviour lasting only a few minutes. We have contributed methods for recognition of high-level contextual cues, such as social interactions, as well as for discovery of everyday activities from visual behaviour.
selected publications
everyday gaze-based human-computer interfaces
Despite considerable advances in eye tracking, previous work on eye-based human-computer interfaces mainly developed use of the eyes in settings that involved single user, single device, and WIMP-style interactions. This is despite the fact that the eyes are involved in nearly everything that we do and thus potentially hold a lot of valuable information for interactive systems. In this spirit, we have introduced smooth pursuit eye movements — the movements we perform when latching onto a moving object — as a novel gaze interaction technique for dynamic interfaces. We have demonstrated the use of pursuits for eye tracker calibration as well as interaction with smart watches. Inspired by how visual attention mediates interactions between humans, we have further proposed social gaze as a new paradigm for designing user interfaces that react to visual attention. Another important research direction is to use gaze for interaction in unconstrained everyday settings, in particular with the increasing number of personal devices and ambient displays.
selected publications
