Papers by Amber Shore
Explain-then-Process: Using Grammar Prompting to Enhance Grammatical Acceptability Judgments (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can explain grammatical rules, but fail to apply those rules when judging sentence acceptability. |
| Approach: | They propose a grammar prompting paradigm that feeds an LLM's metalinguistic explanation back to the target model before deciding which sentence of a minimal pair is grammatical. |
| Outcome: | The proposed model improves on the English BLiMP, Chinese SLING, and Russian RuBLimp benchmarks. |
MEEP: Is this Engaging? Prompting Large Language Models for Dialogue Evaluation in Multilingual Settings (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing metrics for engagingness evaluate the response without the conversation history, are designed for one dataset, or have limited correlation with human annotations. |
| Approach: | They propose to use large language models to evaluate engagingness in dialogue . they propose to include prompts and translated prompts in the model . |
| Outcome: | The proposed model outperforms existing methods on evaluation of engagingness in dialogue across languages. |
Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs? (2025.naacl-long)
Copied to clipboard
| Challenge: | Among various types of ambiguity, this study focuses on syntactic ambiguities, specifically relative 1 Dataset available at https://github.com/PortNLP/ MultiWHO. |
| Approach: | They propose to use a dataset to fine-grained evaluate relative clause attachment preferences in ambiguous and unambiguous contexts. |
| Outcome: | The proposed dataset shows that large language models perform well in unambiguous cases, but lack flexibility in human language processing. |
Correct-Detect: Balancing Performance and Ambiguity Through the Lens of Coreference Resolution in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are intended to reflect human linguistic competencies . but when context is absent or insufficient, ambiguity resolution becomes more tenuous . |
| Approach: | They propose a CORRECT-DETECT trade-off between large language models and ambiguity detection . they show that large language model models can achieve good performance with minimal prompting . |
| Outcome: | The proposed models can achieve good performance with minimal prompting in coreference disambiguation and detection of ambiguity in corefertility tasks, but they cannot do both at the same time. |