Papers by Nora Hollenstein
Multilingual Language Models Predict Human Reading Behavior (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that cognitively motivated "attention" mechanism in neural models is not a good indicator for relative importance. |
| Approach: | They compare the performance of language-specific and multilingual pretrained transformer models to predict reading time measures reflecting natural human sentence processing. |
| Outcome: | The proposed models predict reading time measures on Dutch, English, German, and Russian texts. |
Interpreting Character Embeddings With Perceptual Representations: The Case of Shape, Sound, and Color (2022.acl-long)
Copied to clipboard
| Challenge: | Character-level information is included in many NLP models, but evaluating the information encoded in character embeddings is an open issue. |
| Approach: | They propose to use shape, sound, and color embeddings to evaluate the information encoded in character representations in five languages to perform cross-lingual analysis. |
| Outcome: | The proposed classifiers evaluate phonological information encoded in character embeddings and LSTM models. |
ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation (2020.lrec-1)
Copied to clipboard
| Challenge: | a new dataset of eye-tracking and electroencephalography captures language understanding . eye movement data provides millisecond-accurate records of where humans look when reading . |
| Approach: | They recorded and preprocessed eye-tracking and electroencephalography data during natural reading and during annotation. |
| Outcome: | The study combines eye-tracking and electroencephalography to capture the reading process . the data can be used to evaluate state-of-the-art machine learning systems . |
Dynamic Human Evaluation for Relative Model Comparisons (2022.lrec-1)
Copied to clipboard
| Challenge: | Automated metrics have reported flaws when applied to measure quality aspects of generated text and have been shown to correlate poorly with human judgements. |
| Approach: | They propose an agent-based framework to measure the required number of human annotations when evaluating generated outputs in relative comparison settings. |
| Outcome: | The proposed model can be compared with a crowdsourced case study and a simulation with simulated human judgements. |
The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts (2022.lrec-1)
Copied to clipboard
| Challenge: | Corpora of eye movements during reading of contextualized running text is a way of making such records available for natural language processing. |
| Approach: | They present CopCo, the first eye tracking corpus of its kind for the Danish language. |
| Outcome: | The Copenhagen corpus of eye tracking recordings from natural reading of Danish texts is the first of its kind for the Danish language. |
Entity Recognition at First Sight: Improving NER with Eye Movement Information (N19-1)
Copied to clipboard
| Challenge: | Previous studies have shown eye-tracking data can be used to improve natural language processing models. |
| Approach: | They leverage eye movement features from three corpora with recorded gaze information to augment a neural model for named entity recognition with gaze embeddings. |
| Outcome: | The proposed model outperforms baseline models on both individual datasets and in cross-domain settings. |
Every word counts: A multilingual analysis of individual human alignment with model attention (2022.aacl-short)
Copied to clipboard
| Challenge: | Using eye-tracking data, fixation durations are often not considered in generalisation studies because of individual differences. |
| Approach: | They analyse eye-tracking data from speakers of 13 different languages reading . they find significant differences between languages but also individual reading behaviour . |
| Outcome: | The proposed model can be used to improve the generalization of ML models and allow for more personalized and fair applications. |
Reading Does Not Equal Reading: Comparing, Simulating and Exploiting Reading Behavior across Populations (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing corpora of eye-tracking-while-reading corporata lack diversity, limiting their ability to include primarily native speakers. |
| Approach: | They expand the eye-tracking-while-reading dataset CopCo by incorporating a new dataset of L2 readers with diverse L1 backgrounds. |
| Outcome: | The extended CopCo corpus comprises neurotypical L1 and L1 readers with dyslexia as well as L2 readers reading the same materials. |
CogniVal in Action: An Interface for Customizable Cognitive Word Embedding Evaluation (2020.coling-demos)
Copied to clipboard
| Challenge: | Existing tools for evaluation of word embeddings are extrinsic and intrinsic methods, but they do not accurately reflect the meaning of words. |
| Approach: | They present a command-line interface for CogniVal with multiple improvements over the original framework and the possibility to evaluate custom embeddings against custom cognitive data sources. |
| Outcome: | The proposed system improves and extends the CogniVal framework and provides scalable and customized experiments. |
Relative Importance in Sentence Processing (2021.acl-short)
Copied to clipboard
| Challenge: | In natural language processing, the relative importance of words is usually interpreted with respect to a specific task. |
| Approach: | They compare the relative importance of words in English language processing by humans and neural language models by using saliency methods. |
| Outcome: | The proposed method could be used to interpret neural language models. |
Long-Range Language Modeling with Selective Cache (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that use transformers to model language cost quadratically increase with sequence length. |
| Approach: | They propose a selective cache which stores key-value pairs from previous contexts. |
| Outcome: | The proposed selective cache outperforms XL cache and compressive cache by considerable margins. |
Synthesizing Human Gaze Feedback for Improved NLP Performance (2023.eacl-main)
Copied to clipboard
| Challenge: | Prior work on eye tracking and NLP reveals that human scanpaths can aid in understanding and performance of NLP models. |
| Approach: | They propose a model for generating human scanpaths over text that approximates meaningful cognitive signals in human gaze patterns. |
| Outcome: | The proposed model can approximate meaningful cognitive signals in human gaze patterns. |
Eye-Tracking Features Masking Transformer Attention in Question-Answering Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | Eye movement features are considered to be direct signals reflecting human attention distribution with a low cost to obtain, inspiring researchers to augment language models with eye-tracking (ET) data. |
| Approach: | They select first fixation duration (FFD) and total reading time (TRT) as the cognitive signals to guide Transformer attention in question-answering tasks. |
| Outcome: | The proposed models improve but compromise stability when augmenting with ET data. |
Evaluating Webcam-based Gaze Data as an Alternative for Human Rationale Annotations (2024.lrec-main)
Copied to clipboard
| Challenge: | We compare webcam-based eye-tracking recordings with human-annotated rationales to evaluate importance scores. |
| Approach: | They compare webcam-based eye-tracking recordings with attention-based importance scores for 4 different multilingual Transformer-based language models. |
| Outcome: | The proposed method is comparable to human rationales in linguistic analysis. |
Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a new architecture for controlling, generating and augmenting text is being developed for supervised NLP tasks. |
| Approach: | They propose a conditional VAE architecture to control, generate, and augment text. |
| Outcome: | The proposed model shows high quality, diversity and attribute control in an ablation task. |
Cross-Lingual Transfer of Cognitive Processing Complexity (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies indicate that multilingual language models utilize structural similarities between languages to facilitate cross-lingual transfer. |
| Approach: | They propose a multilingual model that uses structural similarities between languages to facilitate cross-lingual transfer by a meaningful bias towards sentence length and cross-linguistic differences. |
| Outcome: | The proposed model can predict varied patterns for 13 languages, despite being fine-tuned only on English data. |