Papers by Zeming Chen
RETAIL: Towards Real-world Travel Planning for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing travel planning systems assume users provide explicit queries, limiting their practical utility. |
| Approach: | They propose a dataset RETAIL which supports decision-making for implicit queries while covering explicit queries. |
| Outcome: | The proposed model achieves a 1.0% pass rate, suggesting real-world travel planning remains challenging. |
Discovering Knowledge-Critical Subnetworks in Pretrained Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models encode implicit representations of knowledge in their parameters, but localizing these representations and disentangling them from each other remains an open problem. |
| Approach: | They propose a masking scheme that can be applied to weights and neurons to discover such subnetworks. |
| Outcome: | The proposed method can remove specific knowledge from models while minimizing adverse effects on the original model. |
Complex Reasoning over Logical Queries on Commonsense Knowledge Graphs (2024.acl-long)
Copied to clipboard
| Challenge: | Currently, language models struggle to generate commonsense inferences for complex tasks due to data scarcity and the difficulty of reasoning over multiple pieces of information. |
| Approach: | They propose a dataset to generate commonsense inferences from commonsensible data . they use a commonsence knowledge graph to extract and form questions from existing commonseense knowledge graphs. |
| Outcome: | The proposed dataset improves the ability of language models to reason about complex events without expensive human annotations. |
Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evaluation methods do not provide insight into how well a language model captures distinct linguistic skills essential for language understanding and reasoning. |
| Approach: | They propose a new format of NLI benchmark for evaluation of broad-coverage linguistic phenomena using a set of datasets and an evaluation procedure for diagnosing how well a language model captures reasoning skills. |
| Outcome: | The proposed model can diagnose model behavior and verify model learning quality. |
DISCO: Distilling Counterfactuals with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | high-quality counterfactual data is scarce for most tasks and not easily generated at scale. |
| Approach: | They propose a method for automatically generating high-quality counterfactual data at scale . they use a large general language model to generate phrasal perturbations and filter them . |
| Outcome: | The proposed method is task-agnostic and can be applied to the task of natural language inference. |
Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for analyzing the performance of Large Language Models (LLMs) focus on single knowledge updates and fact recall, but do not consider how these updates affect downstream reasoning. |
| Approach: | They propose a benchmark to study how LLMs propagate new knowledge when it conflicts with the model's parametric knowledge. |
| Outcome: | The proposed benchmark compared models with no updated facts to show that the new methods worsen performance and improve reasoning performance. |
Automatic Evaluate Dialogue Appropriateness by Using Dialogue Act (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations of dialogue quality rely on human judgments, which are time-consuming, labor-intensive, prone to biases, and lacking objectivity. |
| Approach: | They propose a method that utilizes the underlying patterns of dialogue act transitions to evaluate the appropriateness of chatbot responses. |
| Outcome: | The proposed method proves that human judgments are time-consuming, labor-intensive, and lacking objectivity. |
Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to GMNER use MLLMs as auxiliary tools, causing cumulative error propagation and a lack of rigorous cross-modal verification. |
| Approach: | They propose a model that enforces structured cross-modal reasoning through Multi-style Reasoning Schema Injection and Constraint-guided Verifiable Optimization. |
| Outcome: | The proposed model enforces structured cross-modal reasoning through multi-style Reasoning Schema Injection and Constraint-guided Verifiable Optimization. |
Live-Aid: A Large-Scale Dialogue Dataset and Benchmark for Interleaved Multi-party Interactions in Live Streaming (2026.findings-acl)
Copied to clipboard
Yiming Lei, Yize Fan, Zeming Liu, Jiaji Dong, Hui Qiu, Haitao Leng, Qingjie Liu, Kehai Chen, Tingting Gao, Yunhong Wang
| Challenge: | Existing Multimodal Large Language Models struggle with dynamic interactions due to the scarcity of high-quality interleaved data. |
| Approach: | They propose a large-scale interleaved live interaction Chinese dataset with human-annotated video responses. |
| Outcome: | The proposed model can be used to evaluate live interactions in Chinese over 1,100 hours and 80,037 dialogue turns. |
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)
Copied to clipboard
Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu
| Challenge: | Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent. |
| Approach: | They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs. |
| Outcome: | The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models. |
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning (2021.starsem-1)
Copied to clipboard
| Challenge: | Currently, symbolic and deep learning approaches to NLI are receiving less attention. |
| Approach: | They propose a symbolic-based inference framework that integrates symbolic reasoning and semantic formalism to solve NLI tasks. |
| Outcome: | The proposed framework improves accuracy on the NLI task and on the SICK and MED datasets. |
Mitigating Label Biases for In-context Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to categorize label biases in in-context learning (ICL) have not addressed all three types of label bias. |
| Approach: | They propose a method that estimates a language model’s label bias using random in-domain words from the task corpus to categorize and detect label biases in ICL. |
| Outcome: | The proposed method significantly improves the performance of GPT-J and GPT-3 on a wide range of tasks. |
False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalize (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work has leveraged probing-based approaches to study the separability of malicious and benign inputs in Large Language Models’ internal representations. |
| Approach: | They propose to use probing-based methods to study separability of malicious and benign inputs in LLMs' internal representations to detect harmful and benign content. |
| Outcome: | The proposed methods show that they learn superficial patterns rather than semantic harmfulness. |