Papers by Yanjun Chen
PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks (2025.emnlp-main)
Copied to clipboard
Yunuo Liu, Dawei Zhu, Zena Al-Khalili, Dai Cheng, Yanjun Chen, Dietrich Klakow, Wei Zhang, Xiaoyu Shen
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, such as code generation, mathematical problem-solving, and general-purpose human instruction following. |
| Approach: | They propose to use large language models to process questions expressed in natural language to automate tourism-booking prices when multiple, overlapping farerules apply. |
| Outcome: | The proposed model can automate tourism-booking prices when multiple, overlapping farerules apply. |
PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit (2022.naacl-demo)
Copied to clipboard
Hui Zhang, Tian Yuan, Junkun Chen, Xintong Li, Renjie Zheng, Yuxin Huang, Xiaojie Chen, Enlei Gong, Zeyu Chen, Xiaoguang Hu, Dianhai Yu, Yanjun Ma, Liang Huang
| Challenge: | PaddleSpeech is an open-source speech toolkit that supports speech-to-text and text-to speech tasks. |
| Approach: | They describe the design philosophy and core architecture of PaddleSpeech to support several essential speech-to-text and text-to speech tasks. |
| Outcome: | The proposed framework achieves competitive or state-of-the-art performance on various speech datasets and implements the most popular methods. |
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning (2025.findings-acl)
Copied to clipboard
Xinghao Chen, Zhijing Sun, Guo Wenjin, Miaoran Zhang, Yanjun Chen, Yirong Sun, Hui Su, Yijie Pan, Dietrich Klakow, Wenjie Li, Xiaoyu Shen
| Challenge: | Large Language Models (LLMs) excel in reasoning tasks through Chain-of-Thought prompting. |
| Approach: | They examine the factors influencing CoT distillation including granularity, format and teacher model. |
| Outcome: | The proposed model is based on four teacher models and seven student models across seven mathematical and commonsense reasoning datasets. |
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents (2025.findings-acl)
Copied to clipboard
Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on calibration and uncertainty quantification focuses on individual models, overlooking the potential of model diversity. |
| Approach: | They propose a method that uses Jensen-Shannon Divergence to identify and aggregate well-calibrated subsets of large language models (LLMs) to improve calibration. |
| Outcome: | The proposed method improves accuracy on binary prediction tasks compared to single-model and naive ensemble baselines. |
A Gentle Introduction to Deep Nets and Opportunities for the Future (2022.acl-tutorials)
Copied to clipboard
| Challenge: | a tutorial on deep nets will introduce a new language for fine tuning deep net programs . the tutorial will be divided into two parts: Part A will make deep net programming accessible to a broader audience . |
| Approach: | This tutorial introduces a new language for fine tuning deep nets with short (1-line) programs that are as easy to code as regression in statistics packages such as R. |
| Outcome: | This tutorial will introduce gft (general fine tuning), a new language for deep nets . glm is a "little language" similar to gslm in statistics package R . |
PAPERMIND: Benchmarking Agentic Reasoning and Critique over Scientific Papers in Multimodal LLMs (2026.findings-acl)
Copied to clipboard
Yanjun Zhao, Tianxin Wei, Jiaru Zou, Xuying Ning, Yuanchen Bei, Lingjie Chen, Simmi Rana, Wendy H. Yang, Hanghang Tong, Jingrui He
| Challenge: | Existing benchmarks assess integrated and agent-oriented scientific reasoning in isolation . Existing systems assess integrated reasoning in isolated tasks . |
| Approach: | They propose a benchmark to evaluate integrated and agent-oriented scientific reasoning over research papers. |
| Outcome: | The proposed benchmark evaluates integrated and agent-oriented scientific reasoning over scientific papers. |
When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications? (2024.findings-emnlp)
Copied to clipboard
Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy Miller, Danielle Bitterman, Matthew Churpek, Majid Afshar
| Challenge: | Numerical data is pivotal for medical questions and answers, but tabular data is not fully integrated into LLMs. |
| Approach: | They examine the effectiveness of vector representations from last hidden states of LLMs for medical diagnostics and prognostics using electronic health record data. |
| Outcome: | The proposed representations outperform those using raw numerical EHR data in medical diagnostics and prognostics. |
Learning to Maximize Mutual Information for Chain-of-Thought Distillation (2024.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation is a technique of transferring knowledge from large, complex models to smaller ones. |
| Approach: | They propose a method utilizing chain-of-thought distillation to transfer knowledge from large, complex models to smaller ones by maximizing mutual information of the representation features of the two tasks. |
| Outcome: | The proposed method outperforms the state-of-the-art knowledge distillation method on four datasets. |
QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm (2025.findings-acl)
Copied to clipboard
Qirui Zhou, Shaohui Peng, Weiqiang Xiong, Haixin Chen, Yuanbo Wen, Haochen Li, Ling Li, Qi Guo, Yongwei Zhao, Ke Gao, Ruizhi Chen, Yanjun Wu, Zhao Chen, Yunji Chen
| Challenge: | Existing LLMs cannot comprehend the complex data flow and computation process of the attention operator and utilize low-level primitive to exploit GPU performance. |
| Approach: | They propose an LLM-friendly Thinking Language (LLM-TL) that can decouple the generation of high-level optimization logic and low-level implementation on GPU and enhance LLMs’ understanding of attention operator. |
| Outcome: | The proposed method outshines existing LLMs on A100, RTX8000, and T4 GPUs, achieving a speed-up of up to 35.16. |
CFBench: A Comprehensive Constraints-Following Benchmark for LLMs (2025.acl-long)
Copied to clipboard
Tao Zhang, ChengLIn Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, Zenan Zhou
| Challenge: | Existing evaluations of Large Language Models (LLMs) focus on fragmented constraints or narrow scenarios, but they overlook the comprehensiveness and authenticity of constraints from the user’s perspective. |
| Approach: | They propose a Chinese Comprehensive Constraints Following Benchmark for LLMs that compiles constraints from real-world instructions and constructs a systematic framework for constraint types. |
| Outcome: | The proposed framework integrates multi-dimensional assessment criteria with requirement prioritization, covering various perspectives of constraints, instructions, and requirement fulfillment. |
The Accuracy Paradox in RLHF: When Better Reward Models Don’t Yield Better Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) significantly enhances Natural Language Processing by aligning language models with human expectations. |
| Approach: | They propose to integrate feedback from humans into RLHF to improve language models by capturing human-like preferences. |
| Outcome: | The proposed model outperforms models trained with moderately accurate reward models on relevance, factuality, and completeness tasks. |
Fine-Grained and Multi-Dimensional Metrics for Document-Level Machine Translation (2025.naacl-srw)
Copied to clipboard
| Challenge: | Large language models excel in machine translation, but most studies focus on sentence-level translation. |
| Approach: | They propose to use LLMs as a judge paradigm to evaluate document-level translations by directly prompting them to translate entire documents in a single pass. |
| Outcome: | The proposed method improves translation quality even without document-level fine-tuning compared to translating sentences separately . |