Papers with eye-tracking
Cue-bot: A Conversational Agent for Assistive Technology (2022.acl-demo)
Copied to clipboard
Shachi H Kumar, Hsuan Su, Ramesh Manuvinakurike, Maximilian C. Pinaroc, Sai Prasad, Saurav Sahay, Lama Nachman
| Challenge: | Large-scale pre-training has achieved significant performance gains across many tasks within NLP, including intent prediction and dialogue state tracking. |
| Approach: | They propose to use eye-tracking, mouse controls and an intelligent agent Cue-bot to represent the user in a conversation. |
| Outcome: | The proposed system can be used by people with different levels of disabilities to interact with the world, supported by eye-tracking, mouse controls and an intelligent agent Cue-bot. |
Design of BCCWJ-EEG: Balanced Corpus with Human Electroencephalography (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent research has focused on the fusion of NLP and neuroscience of language. |
| Approach: | They propose to use a balanced corpus of written Japanese (BCCWJ) annotated with human electroencephalography to improve annotations and annotations. |
| Outcome: | The proposed language resource is annotated with human electroencephalography (EEG) and can improve on annotations, genres, languages, etc. |
Controlling Reading Ease with Gaze-Guided Text Generation (2026.eacl-long)
Copied to clipboard
| Challenge: | Using a gaze-based model, we generate texts with controllable reading ease. |
| Approach: | They propose a method that predicts gaze patterns to steer language model outputs towards eliciting certain reading behaviors by predicting eye-tracking measures. |
| Outcome: | The proposed method generates texts with controllable reading ease using eye-tracking with native and non-native speakers of English. |
Unsupervised Induction of Linguistic Categories with Records of Reading, Speaking, and Writing (N18-1)
Copied to clipboard
| Challenge: | a few researchers have shown that data traces from human processing can be used to improve NLP models. |
| Approach: | They propose to use data readily available for most languages to improve unsupervised induction . they find that english unsupervised POS induction achieves an error reduction of 1.5% . |
| Outcome: | The proposed model improves on Ontonotes domains with a word embeddings. |
CogBERT: Cognition-Guided Pre-trained Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods fine-tune pre-trained models on cognitive data, ignoring the semantic gap between texts and cognitive signals. |
| Approach: | They propose a framework that can induce fine-grained cognitive features from cognitive data and incorporate them into pre-trained language models by adaptively adjusting the weight of cognitive features for different NLP tasks. |
| Outcome: | The proposed framework can induce fine-grained cognitive features from cognitive data and incorporate them into BERT by adaptively adjusting weight of cognitive features for different NLP tasks. |
Surprisal Estimators for Human Reading Times Need Character Models (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that character models can be applied to a structural parser-based processing model to calculate word generation probabilities. |
| Approach: | They propose to use a character model to calculate word generation probabilities from a structural parser-based processing model. |
| Outcome: | The proposed model performs better on self-paced reading, eye-tracking, and fMRI data than large-scale language models trained on much more data. |
CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signals (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies integrate word embeddings with cognitive features into neural models of natural language processing (NLP) but there are some issues in the use of cognitive features in NLP. |
| Approach: | They propose a cog-align approach that aligns textual and cognitive inputs to capture differences and commonalities. |
| Outcome: | The proposed model improves on three NLP tasks with multiple cognitive features over state-of-the-art models. |
Using Eye-tracking Data to Predict the Readability of Brazilian Portuguese Sentences in Single-task, Multi-task and Sequential Transfer Learning Approaches (2020.coling-main)
Copied to clipboard
Sidney Evaldo Leal, João Marcos Munguba Vieira, Erica dos Santos Rodrigues, Elisângela Nogueira Teixeira, Sandra Aluísio
| Challenge: | Sentence complexity assessment is a relatively new task in Natural Language Processing. |
| Approach: | They propose to use Brazilian Portuguese to evaluate sentences with linguistic features to improve readability. |
| Outcome: | The proposed model reaches the state-of-the-art for Brazilian Portuguese with 97.8% accuracy with linguistic features. |
Eye-Tracking Features Masking Transformer Attention in Question-Answering Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | Eye movement features are considered to be direct signals reflecting human attention distribution with a low cost to obtain, inspiring researchers to augment language models with eye-tracking (ET) data. |
| Approach: | They select first fixation duration (FFD) and total reading time (TRT) as the cognitive signals to guide Transformer attention in question-answering tasks. |
| Outcome: | The proposed models improve but compromise stability when augmenting with ET data. |