Papers by Emmy Liu
When Does Translation Require Context? A Data-driven, Multilingual Exploration (2023.acl-long)
Copied to clipboard
| Challenge: | Recent studies in context-aware MT attempt to target a small set of discourse phenomena during evaluation, however not in a fully systematic way. |
| Approach: | They develop a multilingual discourse-aware benchmark to evaluate model performance on discourse phenomena in a given dataset. |
| Outcome: | The proposed model improves on previously studied phenomena while uncovering others which were not addressed. |
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions (2025.emnlp-main)
Copied to clipboard
Emmy Liu, Amanda Bertsch, Lintang Sutawika, Lindia Tjuatja, Patrick Fernandes, Lara Marinov, Michael Chen, Shreya Singhal, Carolin Lawrence, Aditi Raghunathan, Kiril Gashteovski, Graham Neubig
| Challenge: | Language model performance is largely dependent on pretraining decisions, but scaling laws based on only these two aspects do not always explain downstream task performance. |
| Approach: | They meta-analyze 92 open-source pretrained models to quantify their impact on performance. |
| Outcome: | The framework lays a foundation for more systematic investigation of how model development choices shape final capabilities. |
Syntax and Semantics Meet in the “Middle”: Probing the Syntax-Semantics Interface of LMs Through Agentivity (2023.starsem-1)
Copied to clipboard
| Challenge: | a recent study examined how large language models handle interactions in meaning across words and larger syntactic forms. |
| Approach: | They propose to use a dataset to examine the linguistic properties of optionally transitive English verbs to examine their agentivity. |
| Outcome: | The proposed model outperforms all other models in the evaluation dataset . the results are better correlated with human judgements than syntactic and semantic corpus statistics . |
Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting (2023.emnlp-main)
Copied to clipboard
| Challenge: | idioms are common in everyday language, but often pose a challenge to translators because their meanings do not follow from the meanings of their parts. |
| Approach: | They propose to use retrieval-augmented models to increase the accuracy of a strong pretrained machine translation model on idiomatic sentences by up to 13%. |
| Outcome: | The proposed techniques improve the accuracy of a strong pretrained model on idiomatic sentences by up to 13% in absolute accuracy, and holds potential benefits for non-idiomatic phrases. |
Multi-lingual and Multi-cultural Figurative Language Understanding (2023.findings-acl)
Copied to clipboard
Anubha Kabra, Emmy Liu, Simran Khanuja, Alham Fikri Aji, Genta Winata, Samuel Cahyawijaya, Anuoluwapo Aremu, Perez Ogayo, Graham Neubig
| Challenge: | Figures permeate human communication, but are understudied in NLP. |
| Approach: | They create a figurative language inference dataset for seven languages associated with a variety of cultures, using cultural and regional concepts for figurativ expressions. |
| Outcome: | The results show that the most common figurative expressions are found in Hindi, Indonesian, Javanese, Kannada, Sundanese, Swahili and Yoruba. |
Testing the Ability of Language Models to Interpret Figurative Language (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on figurative language has not been done on literal language models. |
| Approach: | They propose a Winograd-style task to evaluate figurative phrases with divergent meanings by interpreting paired figurativ phrases with a human input. |
| Outcome: | The proposed task outperforms state-of-the-art models on a nonliteral language understanding task in zero-shot settings. |
Program-Aided Reasoners (Better) Know What They Know (2024.naacl-long)
Copied to clipboard
| Challenge: | Prior work shows that program-aided reasoning improves accuracy but also requires reasoners to "know what they know". |
| Approach: | They compare the calibration of program-aided language models (PAL) and text-based Chain-of-thought (COT) prompting techniques over 5 datasets and 2 model types . |
| Outcome: | The proposed methods improve accuracy and calibrate the models over 5 datasets and 2 model types. |
Language Modeling with Editable External Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Continual learning methods tackle the problem of a changing world by incrementally training on new information. |
| Approach: | They propose a retrieval-augmented generation approach that incrementally deletes or rewrites other entries in the knowledge base each time a document is added. |
| Outcome: | The proposed model improves accuracy relative to conventional retrieval-augmented generation by 7-13% and 6-10% absolute. |
Are representations built from the ground up? An empirical examination of local composition in language models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Compositionality is a hallmark of human language, but many phrases are non-compositional . a study by a team of researchers shows that LMs may not be able to distinguish between compositional and non-composable phrases. |
| Approach: | They propose to predict LM-internal representations of longer phrases given their constituents . they find that the representation of a parent phrase can be predicted with some accuracy . |
| Outcome: | The proposed model can predict a parent phrase with some accuracy given its children's transformations, but this is not the case. |