Papers by Yidong Liu
RubricBench: Aligning Model-Generated Rubrics with Human Standards (2026.acl-long)
Copied to clipboard
Junyi Zhou, Qiyuan Zhang, Yufei Wang, Fuyuan Lyu, Yidong Ming, Can Xu, Qingfeng Sun, Kai Zheng, Peng Kang, Xue Liu, Chen Ma
| Challenge: | Existing benchmarks lack discriminative complexity and ground-truth rubric annotations required for rigorous evaluation. |
| Approach: | They propose a curated benchmark with 1,147 pairwise comparisons to assess the reliability of rubric-based evaluation. |
| Outcome: | The proposed benchmarks show that they support diverse domains, exhibit discriminative ability, provide high-quality annotations, and include human-authored rubrics. |
Out-of-Distribution Generalization in Natural Language Processing: Past, Present, and Future (2023.emnlp-main)
Copied to clipboard
Linyi Yang, Yaoxian Song, Xuan Ren, Chenyang Lyu, Yidong Wang, Jingming Zhuo, Lingqiao Liu, Jindong Wang, Jennifer Foster, Yue Zhang
| Challenge: | Existing literature on the generalization of machine learning models to out-of-distribution data is lacking. |
| Approach: | They propose to present the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
| Outcome: | The proposed survey provides the first comprehensive review of recent progress, methods, and evaluations on the generalization challenge from an OOD perspective in natural language understanding. |
Always the Best Fit: Adaptive Domain Gap Filling from Causal Perspective for Few-Shot Relation Extraction (2023.findings-emnlp)
Copied to clipboard
Ge Bai, Chenji Lu, Jiaxiang Geng, Shilong Li, Yidong Shi, Xiyan Liu, Ying Liu, Zhang Zhang, Ruifang Liu
| Challenge: | Existing approaches to cross-domain relation extraction have been limited by domains . data bias between domains can be difficult to fill, especially in few-shot scenarios . |
| Approach: | They propose a framework to bridge the semantic gap caused by data bias between domains . they use syntactic structure, label distribution, and entities to calculate causal effects . |
| Outcome: | The proposed framework fills the domain gap and yields better results on the few-shot task. |
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective (2023.findings-acl)
Copied to clipboard
Linyi Yang, Shuibai Zhang, Libo Qin, Yafu Li, Yidong Wang, Hanmeng Liu, Jindong Wang, Xing Xie, Yue Zhang
| Challenge: | Pre-trained language models (PLMs) have improved generalization performance but the out-of-distribution (OOD) generalization problem remains a challenge in many NLP tasks. |
| Approach: | They propose to create a benchmark for evaluating out-of-distribution (OOD) generalization in NLP models. |
| Outcome: | The proposed benchmarks highlight the importance of OOD robustness and provide insights on how to measure it and improve it. |
Intuitive or Dependent? Investigating LLMs’ Behavior Style to Conflicting Prompts (2024.acl-long)
Copied to clipboard
| Challenge: | Extensive experiments with seven Large Language Models reveal their varying behaviors. |
| Approach: | They investigate the behaviors of Large Language Models when faced with conflicting prompts versus their internal memory. |
| Outcome: | Extensive experiments with seven LLMs reveal their varying behaviors. |
PersLEARN: Research Training through the Lens of Perspective Cultivation (2023.acl-demo)
Copied to clipboard
Yu-Zhe Shi, Shiqian Li, Xinyi Niu, Qiao Xu, Jiawen Liu, Yifan Xu, Shiyu Gu, Bingru He, Xinyang Li, Xinyu Zhao, Zijian Zhao, Yidong Lyu, Zhen Li, Sijia Liu, Lin Qiu, Jinhao Ji, Lecheng Ruan, Yuxi Ma, Wenjuan Han, Yixin Zhu
| Challenge: | PersLEARN is a tool designed to facilitate the cultivation of scientific perspectives . junior researchers struggle to identify the perspectives reflected in the literature and struggle to develop their own viewpoints. |
| Approach: | They propose a tool to facilitate the cultivation of scientific perspectives by interacting with a prompt-based model and allowing students to develop their own perspectives explicitly. |
| Outcome: | The proposed tool outperforms baseline approaches across multiple domains of literature from different perspectives. |
Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environments (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for legal intelligence are limited to static evaluation paradigms or simplified scenarios. |
| Approach: | They introduce J1-ENVS, the first interactive and dynamic legal environment tailored for LLM-based agents. |
| Outcome: | The proposed framework assesses task performance and procedural compliance across legal proficiency levels. |
Exploiting Unlabeled Data for Target-Oriented Opinion Words Extraction (2022.coling-1)
Copied to clipboard
Yidong Wang, Hao Wu, Ao Liu, Wenxin Hou, Zhen Wu, Jindong Wang, Takahiro Shinozaki, Manabu Okumura, Yue Zhang
| Challenge: | Existing methods to extract opinion words from sentences are limited due to the expensive annotation process. |
| Approach: | They propose to exploit massive unlabeled data to reduce distribution shift risk . they propose to use two filters specifically for TOWE to filter noisy data . results indicate superiority of MGCR over current state-of-the-art methods . |
| Outcome: | The proposed method reduces the risk of distribution shifts by increasing the exposure of the model to varying distribution shift. |
Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that sequence-to-sequence (seq2sequ) models struggle with compositional generalization (CG) a crucial property of human language learning is its compositional globalization (GC), the algebraic ability to understand and produce a potentially infinite number of novel combinations from known components. |
| Approach: | They propose a sequence-to-sequence (seq2sequ) extension which learns to compose representations of different encoder layers dynamically for different tasks. |
| Outcome: | The proposed model achieves competitive results on two comprehensive and realistic benchmarks, which empirically demonstrates the effectiveness of the proposed model. |