Papers by Ziyang Huang
TurnaboutLLM: A Deductive Reasoning Benchmark from Detective Games (2025.emnlp-main)
Copied to clipboard
| Challenge: | evaluating large language models' reasoning abilities via detective stories is often infeasible due to the large answer space and diverse reasoning types presented by its questions. |
| Approach: | They propose a framework and dataset for evaluating the deductive reasoning abilities of Large Language Models (LLMs) by leveraging the interactive gameplay of detective games Ace Attorney and Danganronpa. |
| Outcome: | The proposed framework and dataset are based on the detective games Ace Attorney and Danganronpa and show that they are more efficient than current strategies for enhancing deductive reasoning. |
Unifying Inference-Time Planning Language Generation (2026.findings-acl)
Copied to clipboard
Prabhu Prakash Kagitha, Bo Sun, Ishan Desai, Andrew Zhu, Cassie Huang, Manling Li, Ziyang Li, Li Zhang
| Challenge: | Large language models (LLMs) are used to generate a formal representation of a plan in a planning language. |
| Approach: | They propose a unifying organizational framework based on intermediate representations to unify the inference-time LLM-as-formalizer methodology for classical planning. |
| Outcome: | The proposed framework subsumes most existing work and proposes new ones that involve syntactically similar but high-resource intermediate languages. |
R³A: Reinforced Reasoning for Relevance Assessment for RAG in User-Generated Content Platforms (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing approaches to query–document relevance assessment are limited . ambiguous user intent and asymmetric relevance are challenges for RAG platforms . |
| Approach: | They propose a decomposed reasoning model for relevance assessment that decomposes query intent into intent inference and evidence grounding. |
| Outcome: | The proposed model outperforms strong baselines on offline benchmarks and achieves significant gains in large-scale online A/B testing. |
Improve Rule Retrieval and Reasoning with Self-Induction and Relevance ReEstimate (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing rule retrieval methods suffer from low accuracy due to semantic gap between instantiated facts and abstract representations of rules. |
| Approach: | They propose a method that induces inferential rules that might offer benefits for reasoning by abstracting the underlying knowledge and logical structure in queries. |
| Outcome: | The proposed method improves retrieval effectiveness and accuracy across settings. |
AdamMeme: Adaptively Probe the Reasoning Capacity of Multimodal Large Language Models on Harmfulness (2025.acl-long)
Copied to clipboard
| Challenge: | Existing models that assess mLLMs on harmful meme understanding are inaccurate and lack accuracy. |
| Approach: | They propose a framework that adaptively probes the reasoning capabilities of mLLMs . their framework systematically reveals the varying performance of different target mllms a . |
| Outcome: | The proposed framework systematically reveals the performance of different target mLLMs. |
Efficient Prior-Guided Reasoning for Robust Retrieval-Augmented Generation under Conflicts (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) has become a standard paradigm for grounding Large Language Models (LLMs) however, performance degrades substantially when faced with noisy, outdated, or conflicting retrieved information. |
| Approach: | They propose a framework that explicitly elicits the model’s parametric knowledge as prior information to guide reasoning on retrieved documents. |
| Outcome: | The proposed framework achieves robust performance across varying degrees of external inconsistency and noise. |
Towards Adaptive Mechanism Activation in Language Agent (2025.coling-main)
Copied to clipboard
| Challenge: | Existing Language Agents rely on a fixed mechanism or a set of mechanisms activated in a predefined order, limiting their adaptation to varied potential task solution structures. |
| Approach: | They propose to use language agents to learn to activate different mechanisms without relying on expert models to optimize their adaptation to different task solutions. |
| Outcome: | The proposed approach improves agent performance by enabling it to activate the appropriate mechanisms according to the potential characteristics of the task. |
DiffusionSL: Sequence Labeling via Tag Diffusion Process (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Sequence Labeling (SL) is a long-standing field of natural language processing. |
| Approach: | They propose a framework that utilizes a conditional discrete diffusion model for generating discrete tag data. |
| Outcome: | The proposed framework outperforms gpt-3.5-turbo on multiple benchmark datasets and tasks. |
Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection (2026.findings-acl)
Copied to clipboard
Zhiwei Liu, Yupeng Cao, Yuechen Jiang, Mohsinul Kabir, Polydoros Giannouris, Chen Xu, Ziyang Xu, Tianlei Zhu, Md. Tariquzzaman, Triantafillos Papadopoulos, Yan Wang, Lingfei Qian, Xueqing Peng, Zhuohan Xie, Ye Yuan, Saeed Almheiri, Abdulrazzaq Alnajjar, Ming-Bin Chen, Harry Stuart, Paul Thompson, Prayag Tiwari, Alejandro Lopez-Lira, Xue Liu, Jimin Huang, Sophia Ananiadou
| Challenge: | Existing research on LLM biases has focused on direct questioning or general-purpose settings . pronounced behavioral biase despite their growing deployment in financial analysis, forecasting, and decision support. |
| Approach: | They propose a benchmark to evaluate behavioral biases of large language models in MFMD . they use a multilingual financial misinformation dataset to integrate these with misinformation claims . |
| Outcome: | The proposed benchmark evaluates behavioral biases of large language models across economic scenarios. |
Social Welfare Function Leaderboard: On the Emergence of LLM Agents as the Welfare Dictator (2026.findings-acl)
Copied to clipboard
Zhengliang Shi, Ruotian Ma, Jen-tse Huang, Xinbei Ma, Xingyu Chen, Mengru Wang, Qu Yang, Yue Wang, Fanghua Ye, Ziyang Chen, Shanyi Wang, Cixing LI, Wenxuan Wang, Zhaopeng Tu, Xiaolong Li, Zhaochun Ren, Liefeng Bo
| Challenge: | Large language models (LLMs) are increasingly entrusted with high-stakes decisions that affect human welfare. |
| Approach: | They evaluate 20 state-of-the-art Large language models (LLMs) and 20 LLM dictators to create a social welfare function benchmark. |
| Outcome: | The proposed model creates dilemma between maximizing collective efficiency and ensuring distributive fairness. |
Experience Retrieval-Augmentation with Electronic Health Records Enables Accurate Discharge QA (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve the reliability of Large Language Models (LLMs) in clinical applications require factual knowledge from open-ended datasets and clinical case-based knowledge to provide context grounded in real-world patient experiences. |
| Approach: | They propose a retrieval-augmented generation framework based on the electronic health record to offer contextual information from other patients’ discharge reports. |
| Outcome: | The proposed framework outperforms a text-based ranker in a clinical QA dataset with 1,280 discharge-related questions . |
Exploiting Contextual Knowledge in LLMs through 𝒱-usable Information based Layer Enhancement (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to enhance the context-faithfulness of Large Language Models (LLMs) ignore the fundamental mechanism of how contextual information is processed within LLMs’ internal states. |
| Approach: | They propose a method that enhances the utilization of contextual knowledge within LLMs’ internal representations by employing V-usable information analysis. |
| Outcome: | The proposed method improves context-faithfulness generation in Question-Answering tasks, particularly in scenarios involving unknown or conflicting contextual knowledge. |
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)
Copied to clipboard
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection (2026.acl-long)
Copied to clipboard
Yuechen Jiang, Zhiwei Liu, Yupeng Cao, Yueru He, Ziyang Xu, Chen Xu, Zhiyang Deng, Prayag Tiwari, Xi Chen, Alejandro Lopez-Lira, Jimin Huang, Junichi Tsujii, Sophia Ananiadou
| Challenge: | RFC-Bench evaluates large language models on financial misinformation under realistic news . current models struggle to maintain coherent belief states without external grounding, study finds . |
| Approach: | They propose a benchmark for evaluating large language models on financial misinformation under realistic news. |
| Outcome: | The proposed model performs better when context is available, while reference-free settings expose significant weaknesses. |
Towards Low-Resource Harmful Meme Detection with LMM Agents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for harmful meme detection are limited due to the dynamic nature of memes . eliciting knowledge-revising behavior within the LMM agent is a key factor in achieving this goal . |
| Approach: | They propose an agency-driven framework for low-resource harmful meme detection . they use annotated memes to leverage label information as auxiliary signals for model . |
| Outcome: | The proposed framework achieves superior performance than state-of-the-art methods on the low-resource harmful meme detection task. |
MASSW: A New Dataset and Benchmark Tasks for AI-Assisted Scientific Workflows (2025.findings-naacl)
Copied to clipboard
Xingjian Zhang, Yutong Xie, Jin Huang, Jinge Ma, Zhaoying Pan, Qijia Liu, Ziyang Xiong, Tolga Ergen, Dongsub Shim, Honglak Lee, Qiaozhu Mei
| Challenge: | Scientific innovation is driven by detailed workflows, which include critical steps such as contextualizing literature, generating ideas, validating ideas, and planning new research. |
| Approach: | They propose to use large language models to extract five key aspects from scientific publications to optimize scientific workflows. |
| Outcome: | The proposed dataset includes more than 152,000 peer-reviewed publications from 17 leading computer science conferences spanning the past 50 years. |
FPT: Feature Prompt Tuning for Few-shot Readability Assessment (2024.naacl-long)
Copied to clipboard
| Challenge: | Prompt-based methods lack crucial linguistic knowledge for readability assessment tasks such as word length, sentence length, and usage of different difficulty-level words. |
| Approach: | They propose a new prompt-based tuning framework that incorporates linguistic knowledge and a loss function to calibrate the similarity ranking order between categories. |
| Outcome: | The proposed framework outperforms the large language model gpt-3.5-turbo-16k in most cases. |
KMatrix-2: A Comprehensive Heterogeneous Knowledge Collaborative Enhancement Toolkit for Large Language Model (2025.emnlp-demos)
Copied to clipboard
Shun Wu, Di Wu, Wangtao Sun, Ziyang Huang, Xiaowei Yuan, Kun Luo, XueYou Zhang, Shizhu He, Jun Zhao, Kang Liu
| Challenge: | Existing studies on K-LLMs systems focus on declarative knowledge and procedural knowledge (rules) . |
| Approach: | They propose to build a toolkit that supports comprehensive heterogeneous knowledge collaborative enhancement for Large Language Models (LLMs). |
| Outcome: | The proposed toolkit provides unified knowledge integration and joint knowledge retrieval methods to achieve more comprehensive heterogeneous knowledge collaborative enhancement. |
Evaluating the Expressive Appropriateness of Speech in Rich Contexts (2026.acl-long)
Copied to clipboard
Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang, Jianwu Dang
| Challenge: | Existing methods for evaluating expressive speech focus on word accuracy, naturalness, signal quality, or emotional intensity at the utterance level. |
| Approach: | They propose a framework for Evaluating Expressive Appropriateness in speech that assesses whether a speech sample aligns with the underlying communicative intent implied by its discourse-level narrative context. |
| Outcome: | The proposed framework outperforms existing speech evaluation and analysis systems on a human-annotated test set. |
SemPA: Improving Sentence Embeddings of Large Language Models through Semantic Preference Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing sentence embedding methods rely on fixed prompt templates or involve modifications to the model architecture, compromising its generative capabilities. |
| Approach: | They propose a sentence-level direct preference optimization approach that boosts the sentence representations while preserving the generative ability of LLMs. |
| Outcome: | The proposed method improves representations of semantically meaningful vectors without sacrificing generation capability. |
Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained large language models struggle to perform logical reasoning reliably despite advances in scale and compositionality. |
| Approach: | They propose a Differentiable Symbolic Reasoning framework that uses symbolic programming to improve LMs' logical reasoning abilities. |
| Outcome: | The proposed framework outperforms competitive baselines when faced with systematic changes in sequence length. |
MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Programming often involves translating detailed and complex specifications into code . current state-of-the-art models struggle to solve these problems, a new study shows . |
| Approach: | They propose a multi-modal coding dataset to evaluate algorithmic problem-solving skills in visually rich contexts. |
| Outcome: | The proposed model lacks powerful vision-code models due to the extreme demand for reasoning abilities. |
Aria-UI: Visual Grounding for GUI Instructions (2025.findings-acl)
Copied to clipboard
| Challenge: | Using a multimodal model, GUI agents can ground from language instructions to target elements . relying on HTML or AXTree inputs is a challenge for GUI agents . |
| Approach: | They propose a large multimodal model specifically designed for GUI grounding that adopts a pure vision approach instead of auxiliary inputs. |
| Outcome: | The proposed model outperforms vision-only and AXTree-reliant models on offline and online agents. |