Papers by David Reich
Pre-Trained Language Models Augmented with Synthetic Scanpaths for Natural Language Understanding (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for augmenting language models with human scanpaths have been developed, but the potential of synthetic gaze data across NLP tasks remains unexplored. |
| Approach: | They propose to combine synthetic scanpath generation with a scanpath-augmented language model, eliminating the need for human gaze data. |
| Outcome: | The proposed model outperforms the underlying language model and achieves comparable performance to a language model augmented with real human gaze data. |
Reverse-Engineering the Reader (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have sought to determine to what extent language models can serve as useful models of human cognition by aligning them to human psychometric data. |
| Approach: | They propose a method to fine-tune a language model to implicitly optimize parameters of a linear regressor that directly predicts humans’ reading times of in-context linguistic units. |
| Outcome: | The proposed technique improves language models’ psychometric predictive power but also its perplexity on held-out test data. |
ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Texts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Eye movements in reading are a key part of psycholinguistic research, but the lack of eye movement data and its unavailability at application time pose a major challenge for this line of research. |
| Approach: | They propose a novel sequence-to-sequence diffusion model that generates synthetic scanpaths on texts by leveraging pre-trained word representations and jointly embedding both the stimulus text and the fixation sequence. |
| Outcome: | The proposed model outperforms state-of-the-art models in psycholinguistic analysis and is able to exhibit human-like reading behavior. |
Fine-Tuning Pre-Trained Language Models with Gaze Supervision (2024.acl-short)
Copied to clipboard
| Challenge: | Existing pre-trained language models lack a gaze module to exploit cognitive signals. |
| Approach: | They propose to integrate a gaze module into pre-trained language models at the fine-tuning stage to exploit cognitive signals. |
| Outcome: | The proposed model improves performance on the GLUE benchmark and standard fine-tuning and text augmentation baselines. |
Ensemble Distillation for Structured Prediction: Calibrated, Accurate, Fast—Choose Three (2020.emnlp-main)
Copied to clipboard
| Challenge: | Modern neural networks do not always produce wellcalibrated predictions . post-hoc calibration methods require a held-out calibration dataset, which may not be available in all circumstances. |
| Approach: | They validate ensemble distillation framework for producing well-calibrated structured prediction models without the prohibitive inference-time cost of ensembles. |
| Outcome: | The proposed framework produces well-calibrated predictions without the prohibitive inference-time cost of ensembles. |
Eye Tracking and NLP (2025.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial combines eye tracking during reading with NLP . outlines how eye movements in reading can be leveraged for NLP methods . |
| Approach: | The tutorial combines eye tracking during reading with NLP . it covers eye movements in reading, integrating eye movement data in NLP models . |
| Outcome: | The tutorial outlines how eye movements in reading can be leveraged for NLP . it provides the essential background for conducting research on joint modeling of eye movements and text. |