Papers by Anne Beyer
Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a common approach to coherence evaluation is shuffling the sentence order of a text, creating incoherent text samples that need to be discriminated from the original. |
| Approach: | They propose an extendable set of test suites addressing different aspects of discourse and dialogue coherence. |
| Outcome: | The proposed evaluation paradigm is suited to evaluate linguistic qualities that contribute to the notion of coherence. |
New or Old? Exploring How Pre-Trained Language Models Represent Discourse Entities (2022.coling-1)
Copied to clipboard
| Challenge: | Recent research shows pre-trained language models learn to encode syntactic knowledge to a certain degree. |
| Approach: | They propose to investigate the information-status of entities as discourse-new or discourse-old . they use binary classification and sequence labeling to investigate their ability to encode syntactic knowledge . |
| Outcome: | The proposed models encode information on whether an entity has been introduced before or not in the discourse. |
Embedding Space Correlation as a Measure of Domain Similarity (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing work on domain similarity using text-based features of corpus is limited by pre-trained word embeddings. |
| Approach: | They propose a measure of domain similarity based on dimension-wise correlations between embedding spaces . they find a threshold at which the measure indicates that two corpora come from the same domain . |
| Outcome: | The proposed measure can be used to determine which corpora are more similar to each other in a cross-domain sentiment detection task. |
Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models (2025.coling-main)
Copied to clipboard
Sherzod Hakimov, Yerkezhan Abdullayeva, Kushal Koshti, Antonia Schmidt, Yan Weiser, Anne Beyer, David Schlangen
| Challenge: | Existing evaluation paradigms for text-only models are largely limited to a limited number of tasks and require little or no data and training cost. |
| Approach: | They propose to use a game-based evaluation paradigm to evaluate multimodal models by a goal-oriented game (self) play. |
| Outcome: | The proposed evaluation paradigm is more efficient than current methods for text-only models and is more cost-effective than existing methods. |