Evaluating Webcam-based Gaze Data as an Alternative for Human Rationale Annotations (2024.lrec-main)
Copied to clipboard
| Challenge: | We compare webcam-based eye-tracking recordings with human-annotated rationales to evaluate importance scores. |
| Approach: | They compare webcam-based eye-tracking recordings with attention-based importance scores for 4 different multilingual Transformer-based language models. |
| Outcome: | The proposed method is comparable to human rationales in linguistic analysis. |
Similar Papers
ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement (2025.emnlp-main)
Copied to clipboard
Ali Salamatian, Amirhossein Abaskohi, Wan-Cyuan Fan, Mir Rayat Imtiaz Hossain, Leonid Sigal, Giuseppe Carenini
| Challenge: | Chart question answering (CQA) is a key research challenge for large vision-language models . recent efforts focus on leveraging LVLMs directly on chart images . |
| Approach: | They propose a gaze-guided attention refinement that aligns image-text attention with human fixations to improve chart reasoning quality and interpretability. |
| Outcome: | The proposed approach improves answer accuracy and attention alignment yielding gains of up to 2.56 percentage points across multiple models. |
From Human Reading to NLM Understanding: Evaluating the Role of Eye-Tracking Data in Encoder-Based Models (2025.acl-long)
Copied to clipboard
| Challenge: | integrating eye-tracking features into Neural Language Models does not degrade downstream task performance, enhances alignment between model attention and human attention patterns, and compresses the embedding space. |
| Approach: | They used eye-gaze data from the Ghent Eye-Tracking Corpus to investigate how integrating knowledge of human reading behavior impacts Neural Language Models. |
| Outcome: | The proposed approach does not degrade downstream task performance, enhances alignment between model attention and human attention patterns, and compresses the embedding space. |
Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patterns (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work to predict gaze patterns during naturalistic reading has not been conducted on general text characteristics. |
| Approach: | They propose to use two eye-tracking corpora of naturalistic reading and two language models to test their performance. |
| Outcome: | The proposed models predict eye-tracking measures during naturalistic reading and language processing. |
Analyzing Interpretability of Summarization Model with Eye-gaze Information (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have provided saliency scores for neural summarization models . eye-gaze information is often used as a proxy for human attention in reading tasks . |
| Approach: | They propose to compare model saliency to human eye-gaze data to determine whether it conforms to human gaze during summarization. |
| Outcome: | The proposed framework compares the model behavior to human summarization performance. |
Classifying Referential and Non-referential It Using Gaze (D18-1)
Copied to clipboard
| Challenge: | a particular problem for anaphora resolution systems is the pronoun it, which can be used both referentially and non-referentially. |
| Approach: | They use eye-tracking data to learn how humans perform disambiguation and use it to improve automatic classification. |
| Outcome: | The proposed system outperforms a baseline and outperformed linguistic-based approaches. |
Does Listener Gaze in Face-to-Face Interaction Follow the Entropy Rate Constancy Principle: An Empirical Study (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that nonverbal behaviours are rich in communicative functions, such as gaze, head movements, and speech-accompanying manual gestures. |
| Approach: | They train a transformer-based neural sequence model to process gaze data extracted from video-recorded conversations and compute its information density. |
| Outcome: | The proposed model computes listeners’ gaze behaviour and the information density of speech using a pre-trained language model. |
At a Glance: The Impact of Gaze Aggregation Views on Syntactic Tagging (D19-64)
Copied to clipboard
| Challenge: | Recent work uses gaze data at the type level or at the token level and mostly from a single eye-tracking corpus. |
| Approach: | They propose to use gaze data to capture central tendency or variability of gaze data and to integrate binary phrase chunking and part-of-speech tagging. |
| Outcome: | The proposed approaches capture the central tendency or variability of gaze data better than proposed local views which retain individual participant information. |
Eye Movement Features Can Predict Human Preferences on Machine-Generated Texts (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing studies on eye movement in text quality assessment are limited . eye-movement features are important predictors of human judgments of text quality, but are costly and inconsistent. |
| Approach: | They propose to capture eye-movement features during screen reading of LLM-generated text using a dataset that includes eye-motion recordings, reading-time measurements, and post-reading evaluations. |
| Outcome: | The proposed dataset shows that eye-movement features can significantly improve models over other probabilistic metrics, including negative log-likelihood (NLL). |
Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-text Rationales (2023.acl-long)
Copied to clipboard
Brihi Joshi, Ziyi Liu, Sahana Ramnath, Aaron Chan, Zhewei Tong, Shaoliang Nie, Qifan Wang, Yejin Choi, Xiang Ren
| Challenge: | Existing metrics like task performance of the LM generating the rationales or similarity between generated and gold rationale are not good indicators of their human utility. |
| Approach: | They propose to use a large language model to generate rationales with better human utility by estimating its conciseness and novelty. |
| Outcome: | The proposed model can measure human utility to a better extent by estimating its usefulness in answering similar unseen instances. |
AERA Chat: An Interactive Platform for Automated Explainable Student Answer Assessment (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing systems that use pretrained language models to score student answers are noisy and unreliable. |
| Approach: | They propose a visualization platform for automated student answer assessment that leverages multiple LLMs to generate rationales. |
| Outcome: | The proposed platform enables educators to mark tasks and researchers to evaluate rationale quality from different models. |