Papers by Yilun Yang
CodeMixBench: Evaluating Code-Mixing Capabilities of LLMs Across 18 Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models (LLMs) are limited by their narrow language pairs and tasks, failing to adequately assess their code-mixing abilities. |
| Approach: | They propose a benchmark to assess large language models' (LLMs) code-mixing abilities that covers eight tasks and 18 languages from seven language families. |
| Outcome: | The proposed method combines word substitution with GPT-4 prompting to generate large-scale synthetic code-mixed texts. |
TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexity (2026.findings-acl)
Copied to clipboard
Zheyuan Yang, Liqiang Shang, Junjie Chen, Xun Yang, Chenglong Xu, Bo Yuan, Chenyuan Jiao, Yaoru Sun, Yilun Zhao
| Challenge: | TableVista evaluates multimodal table reasoning under visual and structural complexity . current models struggle to maintain reasoning consistency when structural complexity combined with visually integrated presentations. |
| Approach: | They propose a benchmark for evaluating multimodal table reasoning under visual and structural complexity. |
| Outcome: | The proposed model performs poorly on visual and structural complexity. |
Latent Inter-User Difference Modeling for LLM Personalization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly integrated into users’ daily lives, leading to a growing demand for personalized outputs. |
| Approach: | They propose a framework that models inter-user differences in the latent space instead of relying on language-based prompts. |
| Outcome: | The proposed framework outperforms baseline methods on personalized review generation. |
Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Tools (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle to generate correct plans for multi-constraint planning problems . a recent study showed that large language models have significant potential in solving planning problems. |
| Approach: | They propose an LLM-based planning framework that formalizes and solves multi-constraint planning problems as constrained satisfiability problems. |
| Outcome: | The proposed framework achieves a success rate of 93.9% and is effective with diverse paraphrased prompts. |
Table-R1: Inference-Time Scaling for Table Reasoning Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | In this study, we explore inference-time scaling on table reasoning tasks. |
| Approach: | They propose a large-scale dataset of reasoning traces and a reinforcement learning with verifiable rewards approach to enable inference-time scaling on table reasoning tasks. |
| Outcome: | The proposed model matches or exceeds GPT-4.1 and DeepSeek-R1 models on diverse table reasoning tasks. |
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation (2024.emnlp-main)
Copied to clipboard
Yuan Ge, Yilun Liu, Chi Hu, Weibin Meng, Shimin Tao, Xiaofeng Zhao, Mahong Xia, Zhang Li, Boxing Chen, Hao Yang, Bei Li, Tong Xiao, JingBo Zhu
| Challenge: | Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. |
| Approach: | They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). |
| Outcome: | The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations. |
Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization (2025.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on real-world datasets demonstrate that DPL significantly enhances LLM personalization. |
| Approach: | They propose a novel approach that emphasizes extracting inter-user differences to enhance LLM personalization. |
| Outcome: | The proposed approach extracts inter-user differences to enhance LLM personalization. |
Part Represents Whole: Improving the Evaluation of Machine Translation System Using Entropy Enhanced Metrics (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing machine translation metrics have poor correlations with human assessments . entropy-based evaluations are often limited to a limited number of samples . |
| Approach: | They propose a fast and unsupervised approach to enhance machine translation metrics using entropy by introducing sentence-level difficulty. |
| Outcome: | The proposed method outperforms existing metrics on five sub-tracks in the WMT19 Metrics shared tasks. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving (2025.findings-acl)
Copied to clipboard
| Challenge: | a benchmark for university-level physics problem solving contains 1,297 expert-annotated problems . a proprietary model, o3-mini, achieves only 59.9% accuracy, highlighting fundamental weaknesses in scientific reasoning, conceptual understanding, and mathematical precision. |
| Approach: | They introduce Physics, a benchmark for university-level physics problem solving. |
| Outcome: | The proposed model achieves only 59.9% accuracy on the most advanced model, o3-mini . the proposed model is a powerful tool for evaluating models on advanced problems . |
ARC: Active and Reflection-driven Context Management for Long-Horizon Information Seeking Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to managing context are based on raw accumulation or passive summarization, treating it as static artifact and allowing early errors or misplaced emphasis to persist. |
| Approach: | They propose a framework that treats context as a dynamic internal reasoning state during execution. |
| Outcome: | Experiments on long-horizon information-seeking benchmarks show that ARC outperforms passive context compression methods. |
PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. |
| Approach: | They propose a method to optimize prompts for LLM-driven multi-step tasks using a human-designed feedback rule. |
| Outcome: | The proposed method outperforms human-engineered prompts and several other prompt optimization methods on 11 representative multi-step tasks. |
FanChuan: A Multilingual and Graph-Structured Benchmark For Parody Detection and Analysis (2025.findings-acl)
Copied to clipboard
Yilun Zheng, Sha Li, Fangkun Wu, Yang Ziyi, Lin Hongchao, Zhichao Hu, Cai Xinjun, Ziming Wang, Jinxuan Chen, Sitao Luan, Jiahao Xu, Lihui Chen
| Challenge: | Parody is an emerging phenomenon on social media, where individuals imitate a role or position opposite to their own . limited available data and deficient diversity in current datasets hinder study of parody . |
| Approach: | They build a dataset of parody users and annotated comments from both English and Chinese corpora to test parody detection and comment sentiment analysis. |
| Outcome: | The proposed datasets provide richer contextual information, which is lacking in existing datasets. |
Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance (2025.emnlp-main)
Copied to clipboard
Yilun Liu, Minggui He, Feiyu Yao, Yuhe Ji, Shimin Tao, Jingzhou Du, Justin Li, Jian Gao, Zhang Li, Hao Yang, Boxing Chen, Osamu Yoshie
| Challenge: | Existing solutions for text-to-image synthesis are sensitive on textual prompts, posing a challenge for novice users. |
| Approach: | They propose a dialogue-based TIS prompt generation model that emphasizes user experience for novice users. |
| Outcome: | The proposed model emphasizes user experience for novice users . it improves user-centricity score while maintaining a competitive quality of synthesized images. |
A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning (2026.acl-long)
Copied to clipboard
Tianyu Yang, Sihong Wu, Yilun Zhao, Zhenwen Liang, Lisen Dai, Chen Zhao, Minhao Cheng, Arman Cohan, Xiangliang Zhang
| Challenge: | Multimodal mathematical Reasoning (MMR) has attracted increasing attention for its ability to solve mathematical problems involving both textual and visual modalities. |
| Approach: | They review the theoretical frameworks of multimodal reasoning and examine the challenges they face in visual math tasks. |
| Outcome: | The proposed models can solve problems involving both textual and visual modalities. |
MMSciCode: Real-world Evaluation of Multilingual Multi-Discipline Scientific Research Coding (2026.acl-long)
Copied to clipboard
| Challenge: | MMSciCode is a benchmark for evaluating foundation models in scientific code generation. |
| Approach: | They propose a multilingual, multi-discipline benchmark for evaluating foundation models in scientific code generation that integrates domain-specific knowledge with algorithmic reasoning. |
| Outcome: | The new benchmark is annotated by domain experts and features rigorous quality controls to ensure dataset integrity and authenticity. |
A Corpus of Adpositional Supersenses for Mandarin Chinese (2020.lrec-1)
Copied to clipboard
| Challenge: | Adpositions are frequent markers of semantic relations, but they are highly ambiguous and vary significantly from language to language. |
| Approach: | They propose to annotate Chinese adpositions in a corpus with all aforementioned supersenses . they adapt a framework that defined a set of supersens according to ostensibly language-independent criteria . |
| Outcome: | The proposed corpus is the first to be broadly annotated with adposition semantics in Chinese . it shows that the supersense categories are well-suited to Chinese adepositions despite syntactic differences from English . |
Can LLMs Generate High-Quality Test Cases for Algorithm Problems? TestCase-Eval: A Systematic Evaluation of Fault Coverage and Exposure (2025.acl-short)
Copied to clipboard
| Challenge: | TestCase-Eval focuses on Fault Coverage and Fault Exposure tasks . authors provide insights into their strengths and limitations in generating effective test cases . correctness and robustness of algorithmic solutions hinge on quality of test suites . |
| Approach: | They introduce TestCase-Eval, a benchmark for systematic evaluation of LLMs in test-case generation. |
| Outcome: | The new benchmark measures the performance of LLMs in test-case generation. |
AMALGUM – A Free, Balanced, Multilayer English Web Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | a corpus of 4M tokens is available online with a large number of high-quality annotation layers. |
| Approach: | They propose to use a genre-balanced English web corpus with multiple annotation layers . they harness knowledge from multiple annotation layer to achieve a "better than NLP" benchmark . |
| Outcome: | The proposed corpus is genre-balanced and features high-quality automatic annotation layers. |