Papers by Yunmo Chen
FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) lack the capacity to handle multimodal inputs effectively. |
| Approach: | They introduce a reference-free and fine-grained evaluation metric that measures the faithfulness of the generated free-form answers from large vision-language models. |
| Outcome: | The proposed metric measures the faithfulness of free-form answers from large vision-language models. |
Condensing Multilingual Knowledge with Lightweight Language-Specific Modules (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to boost performance in multilingual models but scalability is difficult to manage. |
| Approach: | They propose a method that incorporates language-specific (LS) modules to boost model performance. |
| Outcome: | The proposed method outperforms state-of-the-art methods while outperforming existing methods. |
A Unified View of Evaluation Metrics for Structured Prediction (2023.emnlp-main)
Copied to clipboard
| Challenge: | a framework that unifies evaluation metrics for structured prediction tasks is presented . metric design decisions are motivated by specific characteristics of tasks, and we suggest modifications to existing metrics to meet those motivations. |
| Approach: | They propose a framework that unifies a variety of evaluation metrics for different structured prediction tasks. |
| Outcome: | The proposed framework can be used to create new metrics based on the output structure of a number of tasks. |
When Do Decompositions Help for Machine Reading? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on decompositions of complex questions has focused on multi-step reasoning . but, in machine reading, it is unclear when decomposing is helpful . |
| Approach: | They conduct experiments on decompositions in machine reading to unify recent work . they find that decomposing complex questions can be helpful in zero or limited-data settings . |
| Outcome: | The proposed model can learn decompositions implicitly even with limited data, the study shows . the results are consistent with previous work on decomposing complex questions . |
PExA: Parallel Exploration Agent for Complex Text-to-SQL (2026.acl-short)
Copied to clipboard
| Challenge: | Recent work in text-to-SQL has explored toolaugmented LLMs, deep planning, and agentic workflows to address complex challenges. |
| Approach: | They validated a framework for text-to-SQL, Spider 2.0, with 70.2% execution accuracy. |
| Outcome: | The proposed framework achieves 70.2% execution accuracy on a state-of-the-art benchmark for text-to-SQL, Spider 2.0. |
MultiMUC: Multilingual Template Filling on MUC-4 (2024.eacl-long)
Copied to clipboard
William Gantt, Shabnam Behzad, Hannah An, Yunmo Chen, Aaron White, Benjamin Van Durme, Mahsa Yarmohammadi
| Challenge: | We present multilingual parallel template filling datasets for MUCs . systems were required to extract one template per incident, containing details about perpetrators, victims, weapons used . |
| Approach: | They introduce MultiMUC, the first multilingual parallel corpus for template filling . they obtain automatic translations from a strong multilingual machine translation system . |
| Outcome: | The proposed dataset includes translations of the classic MUC-4 template filling benchmark into Arabic, Chinese, Farsi, Korean, and Russian. |
LOME: Large Ontology Multilingual Extraction (2021.eacl-demos)
Copied to clipboard
Patrick Xia, Guanghui Qin, Siddharth Vashishtha, Yunmo Chen, Tongfei Chen, Chandler May, Craig Harman, Kyle Rawlins, Aaron Steven White, Benjamin Van Durme
| Challenge: | LOME is a system for performing multilingual information extraction with large ontologies. |
| Approach: | They propose a system for multilingual information extraction with a framenet parser . LOME is available as a Docker container on Docker Hub and a lightweight version is available on the web . |
| Outcome: | The proposed system outperforms or is competitive with the (monolingual) state-of-the-art . it can be used to build knowledge graphs with large ontologies and across multiple languages . |
Learning to Retrieve Iteratively for In-Context Learning (2024.emnlp-main)
Copied to clipboard
Yunmo Chen, Tongfei Chen, Harsh Jhamtani, Patrick Xia, Richard Shin, Jason Eisner, Benjamin Van Durme
| Challenge: | In-context learning is a powerful tool for learning large language models. |
| Approach: | They propose an iterative retrieval framework that empowers retrievers to make iterable decisions through policy optimization. |
| Outcome: | The proposed framework outperforms existing methods on semantic parsing datasets with 4M additional parameters for state encoding. |
On Event Individuation for Document-Level Information Extraction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a bomb exploded in a restaurant in Lima, and a second device was deactivated by the police . |
| Approach: | They argue that the task demands definitive answers to thorny questions of *event individuation* they argue that even human experts disagree on the task . |
| Outcome: | The proposed task demands definitive answers to thorny questions of *event individuation* . the proposed task also raises concerns about the usefulness of template filling metrics . |
Hierarchical Entity Typing via Multi-level Learning to Rank (2020.acl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a canonical information extraction task that assigns spans to one of a handful of types. |
| Approach: | They propose a hierarchical entity classification method that embraces ontological structure at training and during prediction. |
| Outcome: | The proposed method outperforms previous work on strict accuracy and significantly outperformed previous work. |
Narrowing the Gap between Zero- and Few-shot Machine Translation by Matching Styles (2024.findings-naacl)
Copied to clipboard
Weiting Tan, Haoran Xu, Lingfeng Shen, Shuyue Stella Li, Kenton Murray, Philipp Koehn, Benjamin Van Durme, Yunmo Chen
| Challenge: | Recent work shows that large language models can generalize to machine translation using zero-shot examples with in-context learning. |
| Approach: | They investigate the factors contributing to this gap by matching the writing styles of the target corpus. |
| Outcome: | The proposed methods can be enhanced without the need for parallel demonstration examples. |
Iterative Document-level Information Extraction via Imitation Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Documents may feature zero or more instances of a template of any given type, and the task of template extraction entails identifying the templates in a document and extracting each template’s slot values. |
| Approach: | They propose to use iterative extraction to extract complex relations, i.e., N-tuples representing a mapping from named slots to spans of text within a document. |
| Outcome: | The proposed model leads to state-of-the-art results on two established benchmarks and a strong baseline on the new BETTER Granular task. |
Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction (2021.emnlp-main)
Copied to clipboard
Mahsa Yarmohammadi, Shijie Wu, Marc Marone, Haoran Xu, Seth Ebner, Guanghui Qin, Yunmo Chen, Jialiang Guo, Craig Harman, Kenton Murray, Aaron Steven White, Mark Dredze, Benjamin Van Durme
| Challenge: | Zero-shot cross-lingual information extraction (IE) is a technique for training data in a source language but not in . |
| Approach: | They explore techniques including data projection and self-training to improve zero-shot cross-lingual information extraction (IE) IE is a construction of an IE model for some target language given existing annotations exclusively in English. |
| Outcome: | The proposed techniques show that they perform better than any single strategy. |
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts (2024.findings-acl)
Copied to clipboard
Lingfeng Shen, Weiting Tan, Sihao Chen, Yunmo Chen, Jingyu Zhang, Haoran Xu, Boyuan Zheng, Philipp Koehn, Daniel Khashabi
| Challenge: | Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs. |
| Approach: | They compare how state-of-the-art LLMs respond to malicious prompts in different languages . they find that LLM's generate unsafe responses more often when a prompt is written in a lower-resource language . |
| Outcome: | The proposed model can generate unsafe responses more often when a malicious prompt is written in a lower-resource language, and less irrelevant responses when written in lower-source languages. |
An Empirical Study on Finding Spans (2022.emnlp-main)
Copied to clipboard
| Challenge: | Various information extraction tasks require a span finding component, which either directly yields the output or serves as an essential component of downstream linking. |
| Approach: | They propose methods for span finding, the selection of consecutive tokens in text for some downstream tasks. |
| Outcome: | The proposed methods perform better on masked language models and pre-trained encoders than on encoder-decoder models. |