Papers by Siyu Chen
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)
Copied to clipboard
| Challenge: | a number of tools are used to perform complex tasks, but the tool utilization process can cause errors. |
| Approach: | They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks. |
| Outcome: | The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios. |
SELFGOAL: Your Language Agents Already Know How to Achieve High-level Goals (2025.naacl-long)
Copied to clipboard
Ruihan Yang, Jiangjie Chen, Yikai Zhang, Siyu Yuan, Aili Chen, Kyle Richardson, Yanghua Xiao, Deqing Yang
| Challenge: | Existing approaches to improve the performance of language agents without training are not available. |
| Approach: | They propose an automatic approach to break down high-level goals into tree structure of more practical subgoals during interaction with environments while identifying the most useful subgoal. |
| Outcome: | The proposed approach significantly improves the performance of language agents across various tasks, including competitive, cooperative, and deferred feedback environments. |
WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing RAG frameworks rely on Automatic Speech Recognition to process speech input, which discards crucial audio information and increases computational overhead. |
| Approach: | They propose a retrieval augmented generation framework with native, end-to-end audio support that integrates audio and text into a unified knowledge representation. |
| Outcome: | The proposed framework can perform 10x faster than current pipelines while delivering 10x acceleration. |
Modeling Multi-Dimensional Cognitive States in Large Language Models under Cognitive Crowding (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) mainly address isolated tasks such as emotion analysis or stance detection. |
| Approach: | They propose a large-scale model that combines large-level annotations with hyperbolic space to model human cognitive states. |
| Outcome: | The proposed model outperforms baseline models on cognitive dimensions on single dimension tasks while retaining strong hierarchical structure. |
DEEPER Insight into Your User: Directed Persona Refinement for Dynamic Persona Modeling (2025.acl-long)
Copied to clipboard
Aili Chen, Chengyu Du, Jiangjie Chen, Jinghan Xu, Yikai Zhang, Siyu Yuan, Zulong Chen, Liangyue Li, Yanghua Xiao
| Challenge: | Existing methods for generating personas from static historical data fail to capture dynamic behaviors and evolving preferences in real-world interactive scenarios. |
| Approach: | They propose a novel approach that iteratively updates personas using streaming user behavior data to continually enhance their quality. |
| Outcome: | The proposed approach delivers 32.2% reduction in user behavior prediction error over four update rounds, outperforming the best baseline by 22.92%. |
Learning from Adjective-Noun Pairs: A Knowledge-enhanced Framework for Target-Oriented Multimodal Sentiment Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to determine sentiment polarity of opinion target are inconsistent and lack visual attention. |
| Approach: | They propose a framework which can exploit adjective-noun pairs extracted from images to improve visual attention and sentiment prediction capability of the TMSC task. |
| Outcome: | The proposed framework outperforms state-of-the-art on two public datasets. |
Revealing the Barriers of Language Agents in Planning (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies show language agents lack human-level planning abilities . limitations and mechanisms to address them remain insufficiently understood . |
| Approach: | They apply a feature attribution study to identify key factors hindering agent planning . they identify the limited role of constraints and diminishing influence of questions . |
| Outcome: | The proposed model achieves 15.6% on a real-world planning benchmark. |
Past Meets Present: Creating Historical Analogy with Large Language Models (2025.acl-long)
Copied to clipboard
Nianqi Li, Siyu Yuan, Jiangjie Chen, Jiaqing Liang, Feng Wei, Zujie Liang, Deqing Yang, Yanghua Xiao
| Challenge: | Historical analogies are important abilities that help people make decisions and understand the world. |
| Approach: | They propose a historical analogy acquisition task that uses large language models to acquire historical analogies. |
| Outcome: | The proposed method mitigates hallucinations and stereotypes when LLMs generate historical analogies. |
EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing work on extending specialized agents to multi-agent systems is dependent on human-designed frameworks, limiting the functional scope and scalability of agent systems. |
| Approach: | They propose a generic method to automatically extend specialized agents to multi-agent systems via evolutionary algorithm . they consider existing agent frameworks as the initial individual and apply evolutionary operators to generate multiple agents with diverse settings. |
| Outcome: | The proposed method can extend specialized agents to multi-agent systems . it can generate multiple agents with diverse settings, and improves performance across tasks . |
Towards Trustworthy Smart Contract Synthesis: A Multi-Agent Framework with Lean-Based Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Smart Contracts are the foundation of Decentralized Finance (DeFi), executing financial logic without trusted intermediaries. |
| Approach: | They propose a framework that integrates LLM-based generation with Lean-based auto-formalization and verification. |
| Outcome: | LeVer is the first trustworthy smart contract synthesis framework that integrates LLM-based generation with Lean-based auto-formalization and verification. |
Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality. |
| Approach: | They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model. |
| Outcome: | The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs. |
Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to generate adversarial examples for NMT use the meaning-preserving restriction. |
| Approach: | They propose a new definition for adversarial examples based on the Doubly Round-Trip Translation (DRTT) they introduce masked language models to construct bilingual adversarials based upon DRTT . |
| Outcome: | The proposed approach significantly improves the robustness of the NMT model on clean and noisy test sets. |
Pre3: Enabling Deterministic Pushdown Automata for Faster Structured LLM Generation (2025.acl-long)
Copied to clipboard
Junyi Chen, Shihao Bai, Zaijun Wang, Siyu Wu, Chuheng Du, Hailong Yang, Ruihao Gong, Shengzhong Liu, Fan Wu, Guihai Chen
| Challenge: | Existing methods for structured generation of outputs are inefficient under large inference batches. |
| Approach: | They propose a new LLM-based method that parses LR(1) grammars into a pushdown automaton and exploits deterministic pushdown automation to optimize the constrained LLM decoding efficiency. |
| Outcome: | The proposed method improves time per output token (TPOT) by 40% and throughput by 36% . |
AgentRM: Enhancing Agent Generalization with Reward Modeling (2025.acl-long)
Copied to clipboard
Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing LLM-based agents have strong performance on held-in tasks, but their generalizability to unseen tasks remains poor. |
| Approach: | They propose a reward-based generalizable reward model to guide the policy model for effective test-time search. |
| Outcome: | The proposed agentRM outperforms existing agents on held-in tasks by 8.8 points on average. |
Curse of Knowledge: Your Guidance and Provided Knowledge are biasing LLM Judges in Complex Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has focused on simple settings, but their reliability in complex tasks remains understudied. |
| Approach: | They propose to use large language models as judges to evaluate reliability in complex tasks . they use a challenge benchmark to expose and quantify Auxiliary Information Induced Biases . |
| Outcome: | The proposed benchmark exposes and quantifies Auxiliary Information Induced Biases across 12 basic and 3 advanced scenarios. |
CODIS: Benchmarking Context-dependent Visual Comprehension for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Fuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang, Qidong Yan, Yingjie Li, Xiaolong Wang, Siyu Wang, Ziyue Wang, Xiaoyue Mi, Peng Li, Ning Ma, Maosong Sun, Yang Liu
| Challenge: | Multimodal large language models have demonstrated promising results in a variety of tasks that combine vision and language. |
| Approach: | They propose a benchmark to assess the ability of models to use contextual information in free-form text to enhance visual comprehension. |
| Outcome: | The proposed model fails to extract and utilize contextual information to improve understanding of images. |
InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews (2024.acl-long)
Copied to clipboard
Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, Yanghua Xiao
| Challenge: | Existing methods focus on knowledge and linguistic patterns of characters. |
| Approach: | They propose to evaluate character fidelity of role-playing agents with psychological scales . they propose to use psychological scale to measure personality traits of RPAs based on personality traits. |
| Outcome: | The proposed model reproduces character fidelity with psychological scales and shows that it is effective in measuring personality traits. |
DocEE: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing datasets focus on sentence-level event extraction, but document-level EE is limited due to the lack of large-scale and practical training and evaluation datasets. |
| Approach: | They propose a document-level event extraction dataset with 27,000+ events and 180,000+ arguments. |
| Outcome: | The proposed dataset includes 27,000+ events, 180,000+ arguments and large-scale manual annotations, fine-grained argument types and application-oriented settings. |
RolePlot: A Systematic Framework for Evaluating and Enhancing the Plot-Progression Capabilities of Role-Playing Agents (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research has focused on role-playing agents’ ability to portray specified characters, but their ability to advance the plot requires substantial improvements to deliver more engaging interaction. |
| Approach: | They propose a role-playing framework to evaluate and enhance the plot-progression capabilities of role-players. |
| Outcome: | The proposed framework improves RPAs’ ability to time plot developments and yields a significant increase in conversation turns and sustained higher arousal levels. |
ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base (2024.acl-long)
Copied to clipboard
| Challenge: | ANALOGYKB is a million-scale analogy knowledge base based on existing knowledge graphs (KGs) based upon relational knowledge triples, we can discover new analogies using the corresponding relations between concepts. |
| Approach: | They propose a million-scale analogy knowledge base derived from existing knowledge graphs (KGs) ANALOGYKB identifies analogies of the same relations and analogies from analogous relations . |
| Outcome: | The proposed model enables both smaller LMs and LLMs to gain better analogical reasoning capabilities. |
TimeArena: Shaping Efficient Multitasking Language Agents in a Time-Aware Simulation (2024.acl-long)
Copied to clipboard
| Challenge: | e.g., GPT-4 still lag behind humans in effective multitasking, a study finds . current textual simulations do not adequately address the notion of time . |
| Approach: | They propose a textual simulated environment that incorporates complex temporal dynamics and constraints that better reflect real-life planning scenarios. |
| Outcome: | The proposed model incorporates complex temporal dynamics and constraints that better reflect real-life planning scenarios. |
Meta-LMTC: Meta-Learning for Large-Scale Multi-Label Text Classification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale multi-label text classification tasks often face long-tailed label distributions, where many labels have few or even no training instances. |
| Approach: | They propose a meta-learning approach that incorporates the objective of adapting to new low-resource tasks into the meta-Learning phase. |
| Outcome: | The proposed approach achieves state-of-the-art against strong baselines and can still enhance powerful BERTlike models. |
T⋆: Progressive Block Scaling for Masked Diffusion Language Models Through Trajectory Aware Reinforcement Learning (2026.acl-short)
Copied to clipboard
| Challenge: | Autoregressive (AR) modeling via next-token prediction dominates scaling practice and deployed systems. |
| Approach: | They propose a TraceRL-based curriculum for progressive block-size scaling in masked diffusion language models. |
| Outcome: | The proposed curriculum outperforms direct large-block TraceRL on two SDAR scales and three benchmarks and retains block-size-specific non-monotone updates while improving accuracy. |
Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have catalyzed numerous AI applications, among which role-playing agents (RPAs) are particularly popular. |
| Approach: | They propose to evaluate LLMs' character understanding capability via the character profiling task, i.e., summarizing character profiles from corresponding materials, a widely adopted yet understudied practice for RPA development. |
| Outcome: | The proposed model outperforms existing models and literature summarization methods and proves its ability to understand fictional characters in downstream tasks. |
Towards Preference Following in Tool Calling Language Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Currently, large language model (LLM)-based agents can't follow user preferences when calling tools. |
| Approach: | They propose a benchmark to evaluate agents' ability to identify personalized user preferences from interaction histories and to adhere to these preferences when calling tools. |
| Outcome: | The proposed model achieves 51.16% accuracy on the APOLLO benchmark, while GPT-4o achieves only 51.13% accuracy. |
EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction (2025.naacl-long)
Copied to clipboard
| Challenge: | EASYTOOL combines tools from diverse tool documentation into a single tool instruction. |
| Approach: | They propose a framework that transforms tool documentation into a unified tool instruction. |
| Outcome: | EASYTOOL combines extensive tool documentation into a concise tool instruction . it reduces token consumption and improves performance of LLM-based agents . |
A Survey of Inductive Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Kedi Chen, Dezhao Ruan, Yuhao Dan, Yaoting Wang, Siyu Yan, Xuecheng Wu, Yinqi Zhang, Qin Chen, Jie Zhou, Liang He, Biqing Qi, Linyang Li, Qipeng Guo, Xiaoming Shi, Wei Zhang
| Challenge: | Inductive reasoning is an important task for large language models (LLMs). |
| Approach: | They propose a survey of inductive reasoning for large language models . they categorize methods into three main areas: post-training enhancement, test-time exploration, and data augmentation. |
| Outcome: | The proposed method improves inductive reasoning in large language models. |
Graph Enhanced Cross-Domain Text-to-SQL Generation (D19-53)
Copied to clipboard
| Challenge: | Existing deep learning approaches for semantic parsing do not generalize to unseen data sets . existing benchmarks have shown text-to-SQL parsers do not generally perform well to unsen SQL queries. |
| Approach: | They propose a new cross-domain learning scheme to perform text-to-SQL translation . they demonstrate its use on a large-scale cross- domain text- to-Sql data set Spider . |
| Outcome: | The proposed learning scheme improves on a large-scale text-to-SQL data set. |
Human-Agent Collaborative Paper-to-Page Crafting (2026.findings-acl)
Copied to clipboard
Qianli Ma, Siyu Wang, Chen Yilin, Yinhao Tang, Yixiang Yang, Chang Guo, Bingjie Gao, Zhening Xing, Yanan Sun, Zhipeng Zhang
| Challenge: | Existing approaches to create project pages from academic papers have focused on static slides and posters, but the dynamic nature of webpages remains an unaddressed challenge. |
| Approach: | They propose a novel multi-agent system that deconstructs paper-to-page creation into a coarse-to fine pipeline from narrative planning to multimodal content generation and interactive rendering. |
| Outcome: | The proposed system generates high-quality, visually appealing pages in under 15 minutes for less than $0.1 . |
“A good pun is its own reword”: Can Large Language Models Understand Puns? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the understanding of puns in large language models (LLMs) have not explored the use of pun in creative writing and humor creation. |
| Approach: | They propose to use pun recognition, explanation and generation tasks to evaluate the capabilities of large language models (LLMs) they adopt automated evaluation metrics from prior research and introduce new evaluation methods and metrics that align more closely with human cognition. |
| Outcome: | The proposed methods align more closely with human cognition than previous evaluation metrics. |
ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use (2025.acl-long)
Copied to clipboard
Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing Huang, Jiecao Chen
| Challenge: | Effective evaluation of multi-hop tool use is critical for analyzing the understanding, reasoning, and function-calling capabilities of large language models. |
| Approach: | They propose a dataset that provides rigorous evaluation of multi-hop tool use. |
| Outcome: | The proposed model achieves 49.04% accuracy across five model families. |
Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have focused on word analogies, but they neglect structures that underpin analogical reasoning. |
| Approach: | They propose a task to abduct structures that form an analogy between two systems to evaluate their analogical reasoning abilities. |
| Outcome: | The proposed task is based on 400 scientific analogies from 13 different fields and is compared with a standard SCAR benchmark. |
Saliency-based Multi-View Mixed Language Training for Zero-shot Cross-lingual Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent multilingual pre-trained models have been demonstrated effective in many cross-lingual tasks. |
| Approach: | They propose a framework that leverages code-switched data with multi-view learning to fine-tune XLM-R. |
| Outcome: | The proposed model achieves state-of-the-art on zero-shot cross-lingual sentiment classification and dialogue state tracking tasks. |
Distilling Script Knowledge from Large Language Models for Constrained Language Planning (2023.acl-long)
Copied to clipboard
Siyu Yuan, Jiangjie Chen, Ziquan Fu, Xuyang Ge, Soham Shah, Charles Jankowski, Yanghua Xiao, Deqing Yang
| Challenge: | Existing work exploits language models to plan for abstract goals of stereotypical activities, but leaves more specific goals with multi-facet constraints understudied. |
| Approach: | They propose an over-generate-then-filter approach to improve large language models on constrained language planning task by distilling a constrained script dataset. |
| Outcome: | The proposed approach improves the constrained language planning ability of large language models on constraint faithfulness and also in smaller LMs. |
Character is Destiny: Can Persona-assigned Language Models Make Personal Choices? (2025.findings-emnlp)
Copied to clipboard
Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, null Xiaoqingdong, Yanghua Xiao
| Challenge: | Recent research has demonstrated the potential of using LLMs to develop role-playing language agents (RPLAs) however, imitative decision-making necessitates a more nuanced understanding of personas. |
| Approach: | They propose a method that uses persona-based memory retrieval to improve RPLAs. |
| Outcome: | The proposed method significantly advances RPLAs on this task. |