Two papers accepted at LREC-COLING

We are pleased to announce that the following two papers were accepted for publication in the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Congratulations to the authors!

1. 

   OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog

   Adnen Abdessaied, Manuel Hochmeister, Andreas Bulling

   Proc. 31st Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), pp. 1–11, 2024.

   Abstract Links BibTeX Project

   We present the Object Language Video Transformer (OLViT) – a novel model for video dialog operating over a multi-modal attention-based dialog state tracker. Existing video dialog models struggle with questions requiring both spatial and temporal localization within videos, long-term temporal reasoning, and accurate object tracking across multiple dialog turns. OLViT addresses these challenges by maintaining a global dialog state based on the output of an Object State Tracker (OST) and a Language State Tracker (LST): while the OST attends to the most important objects within the video, the LST keeps track of the most important linguistic co-references to previous dialog turns. In stark contrast to previous works, our approach is generic by nature and is therefore capable of learning continuous multi-modal dialog state representations of the most relevant objects and rounds. As a result, they can be seamlessly integrated into Large Language Models (LLMs) and offer high flexibility in dealing with different datasets and tasks. Evaluations on the challenging DVD (response classification) and SIMMC 2.1 (response generation) datasets show that OLViT achieves new state-of-the-art performance across both datasets.

   Paper: abdessaied24_coling.pdf

   Code: https://git.hcics.simtech.uni-stuttgart.de/public-projects/OLViT

   @inproceedings{abdessaied24_coling, author = {Abdessaied, Adnen and von Hochmeister, Manuel and Bulling, Andreas}, title = {OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog}, booktitle = {Proc. 31st Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING)}, year = {2024}, pages = {1--11} }
2. 

   InteRead: An Eye Tracking Dataset of Interrupted Reading

   Francesca Zermiani, Prajit Dhar, Ekta Sood, Fabian Kögel, Andreas Bulling, Maria Wirzberger

   Proc. 31st Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), pp. 1–16, 2024.

   Abstract Links BibTeX Project

   Eye movements during reading offer a window into cognitive processes and language comprehension, but the scarcity of reading data with interruptions – which learners frequently encounter in their everyday learning environments – hampers advances in the development of intelligent learning technologies. We introduce InteRead – a novel 50-participant dataset of gaze data recorded during self-paced reading of real-world text. InteRead further offers fine-grained annotations of interruptions interspersed throughout the text as well as resumption lags incurred by these interruptions. Interruptions were triggered automatically once readers reached predefined target words. We validate our dataset by reporting interdisciplinary analyses on different measures of gaze behavior. In line with prior research, our analyses show that the interruptions as well as word length and word frequency effects significantly impact eye movements during reading. We also explore individual differences within our dataset, shedding light on the potential for tailored educational solutions. InteRead is accessible from our datasets web-page: https://www.ife.uni-stuttgart.de/en/llis/research/datasets/.

   doi:

   Paper: zermiani24_coling.pdf

   @inproceedings{zermiani24_coling, title = {InteRead: An Eye Tracking Dataset of Interrupted Reading}, author = {Zermiani, Francesca and Dhar, Prajit and Sood, Ekta and Kögel, Fabian and Bulling, Andreas and Wirzberger, Maria}, year = {2024}, booktitle = {Proc. 31st Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING)}, pages = {1--16}, doi = {} }

Here are some related news you might like to read next:

• Paper accepted at CogSci 2024
• Four papers accepted at ETRA 2024
• Two papers accepted at CHI
• Paper accepted at AAAI 2024
• Paper accepted at PACM HCI