Papers by Yuling Gu
What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations (2023.findings-emnlp)
Copied to clipboard
Kavel Rao, Liwei Jiang, Valentina Pyatkin, Yuling Gu, Niket Tandon, Nouha Dziri, Faeze Brahman, Yejin Choi
| Challenge: | Moral or ethical judgments rely heavily on the contexts in which they occur . a student model that produces defeasible contexts with improved validity, diversity, and defasibility is superior to intermediate student models . |
| Approach: | a new study uses a student model to provide contextualizations that make an action morally acceptable . the model is based on a dataset of 115K defeasible moral actions rated highly by human annotators . |
| Outcome: | The proposed model outperforms all intermediate models in a high-quality dataset . the model is based on 1.2M entries of contextualizations and rationales for 115K moral actions . |
Digital Socrates: Evaluating LLMs through Explanation Critiques (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can provide reasoned explanations, but the nature and quality of those explanations are still poorly understood. |
| Approach: | They propose to define a task of explanation critiquing and train an open-source automatic critique model using this data. |
| Outcome: | The proposed model can provide high-quality, nuanced evaluations without expensive API calls or human annotations. |
WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | a global dataset for multi-cultural value prediction task is lacking in the computer science community . a multi-culture awareness of LMs is critical to generating safe and personalized responses . |
| Approach: | They present a global multi-cultural value prediction task using a world value survey dataset . they construct more than 20 million examples of the type "(demographic attributes, value question) answer" they show that the task is challenging for strong open and closed-source models . |
| Outcome: | The proposed model can generate a rating response to a value question based on demographic contexts on 11.1%, 25.0%, 72.2%, and 75.0% of the questions. |
OLMES: A Standard for Language Model Evaluations (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing models claim to perform better on tasks measuring model capabilities, but there is no standard setup for reproducible evaluations. |
| Approach: | They propose a document that is documented and practical for reproducible LLM evaluations and includes recommendations from existing literature and new experiments. |
| Outcome: | The proposed standard identifies and reviews the varying factors in evaluation practices adopted by the community, such as prompt formatting, choice of in-context examples, probability normalizations, and task formulation. |
EvoHyper: Evolving Hypergraph Topologies for Unified Collaboration in Multi-Agent Communication (2026.findings-acl)
Copied to clipboard
Heng Zhang, Yihao Zhong, Lubin Gan, Zhihe Chen, Jiajun Wu, Yuling Shi, Xiaodong Gu, Hao Zhang, Haochen You, Jin Huang
| Challenge: | Existing methods for multi-agent collaboration use a fixed communication graph and manage collaboration structure and shared memory in separate modules. |
| Approach: | They propose a framework that uses an evolving hypergraph topology for multi-agent collaboration. |
| Outcome: | The proposed framework achieves 3.2% to 7.8% accuracy gains over state-of-the-art methods and efficient, reducing token consumption by up to 23.5%. |
SWE-QA: Can Language Models Answer Repository-level Code Questions? (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for understanding and reasoning about entire soft-ware repositories focus on small, self-contained code snippets. |
| Approach: | They propose a repository-level code question answering benchmark to facilitate research on automated QA systems in real-world repositories. |
| Outcome: | The proposed benchmarks are designed to facilitate research on automated QA systems in real-world repositories. |
GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing routing strategies rely on local token probabilities or post-hoc verification, introducing significant inference overhead. |
| Approach: | They propose a step-wise collaboration framework that generates only the first token of each reasoning step and routes it to a larger model only when initial token entropy exceeds a threshold. |
| Outcome: | The proposed approach reduces inference latency while preserving accuracy. |
Do language models have coherent mental models of everyday things? (2023.acl-long)
Copied to clipboard
| Challenge: | Psychologists and cognitive scientists hypothesize that humans develop mental models of the world, namely internal, conceptual representations of the environment which we base our decisions and actions on. |
| Approach: | They propose to add a constraint satisfaction layer to the LM's raw predictions to apply commonsense constraints to reduce incoherence. |
| Outcome: | The proposed extension removes inconsistencies and improves accuracy by 16-20%. |
ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction (2026.findings-acl)
Copied to clipboard
Zichun Guo, Yuling Shi, Wenhao Zeng, Chao Hu, Haotian Lin, Terry Yue Zhuo, Jiawei Chen, Xiaodong Gu, Wenping Ma
| Challenge: | Empirical evaluations on state-of-the-art MLLMs reveal a significant performance gap . ML models lack the fine-grained cross-modal reasoning required to bridge visual discontinuities. |
| Approach: | They propose a benchmark that renders fragmented documents directly from Markdown to facilitate evaluation of VRDU tasks. |
| Outcome: | The proposed benchmark renders fragmented documents directly from Markdown. |
DREAM: Improving Situational QA by First Elaborating the Situation (2022.naacl-main)
Copied to clipboard
| Challenge: | Cognitive science has long promoted the formation of mental models as central to understanding and question-answering. |
| Approach: | They train a new model, DREAM, to answer questions that elaborate the scenes that situated questions are about and then provide those elaborations as additional context to a question-answering (QA) model. |
| Outcome: | The proposed model is able to create better scene elaborations than a representative state-of-the-art, zero-shot model. |
LastingBench: Defend Benchmarks Against Knowledge Leakage (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect and safeguard LLMs against knowledge leakage fail to address the long-term challenge of mitigating it. |
| Approach: | They propose a method to reinforce and safeguard existing benchmarks against knowledge leakage by perturbation-based detection and counterfactual rewriting to disrupt memorization while preserving original intent. |
| Outcome: | The proposed method reduces memorization effects in long-context QA benchmarks, providing a more accurate assessment of model reasoning and generalization abilities. |