Papers by Yang Fang

VoxpopuliTTS: a large-scale multilingual TTS corpus for zero-shot speech generation (2025.coling-main)

Challenge:	Existing methods focus on replicating dialogues in textual form, neglecting the role’s voice traits as a crucial effect in interaction, which tends to be more immersive experiences in realistic scenarios.
Approach:	They propose a first seamless speech-language personality interaction model to achieve immersive RPAs with low latency.
Outcome:	The proposed model exhibits role-specific personality traits and vocal traits throughout the interaction, enabling a mixture of speech and language responses.

Copied to clipboard

Wenrui Liu, Jionghao Bai, Xize Cheng, Jialong Zuo, Ziyue Jiang, Shengpeng Ji, Minghui Fang, Xiaoda Yang, Qian Yang, Zhou Zhao

GQLBench: A Large-Scale Cross-Domain, Cross-Dialect Benchmark for NL2GQL (2026.acl-long)

Challenge:	Existing multilingual TTS datasets are limited in speech generation fields due to lack of quality data.
Approach:	They propose to use 30,000 hours of high-quality speech data across 3 languages . they filter out low-quality text-text pairs and concatenate short transcripts .
Outcome:	The proposed dataset comprises 30,000 hours of high-quality speech data, across 3 languages with multiple speakers and styles, suitable for various speech tasks such as TTS and ASR.

Copied to clipboard

Yanning Su, Yuhang Zhou, Yang Fang, Sen Liu, Guangnan Ye, Hongfeng Chai

CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling (2025.acl-long)

Challenge:	despite growing interest in NL2GQL, benchmarking progress has been constrained by the lack of resources that are simultaneously large-scale, cross-domain, and cross-dialect.
Approach:	They propose a framework that integrates NL2SQL-to-NL2GQL conversion with graph-native data generation.
Outcome:	The proposed framework supports execution-based evaluation on Cypher and ISO-GQL, covering hundreds of graph databases and over 20k natural language questions for each dialect.

Copied to clipboard

Minghui Fang, Shengpeng Ji, Jialong Zuo, Hai Huang, Yan Xia, Jieming Zhu, Xize Cheng, Xiaoda Yang, Wenrui Liu, Gang Wang, Zhenhua Dong, Zhou Zhao

Model Composition for Multimodal Large Language Models (2024.acl-long)

Challenge:	Cross-modal retrieval tasks are used to retrieve data from one modality or another based on a query from another modality.
Approach:	They propose a generative cross-modal retrieval framework based on coarse-to-fine semantic modeling . they propose combining K-Means and RQ-VAE to discretize multimodal data into token sequences that support autoregressive generation.
Outcome:	The proposed framework achieves excellent performance and efficiency in multimodal retrieval tasks.

Copied to clipboard

Chi Chen, Yiyang Du, Zheng Fang, Ziyue Wang, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu

Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation (2023.emnlp-main)

Challenge:	Existing methods for creating versatile MLLMs rely on joint training with paired instruction data, which is resource-intensive and challenging to extend to new modalities.
Approach:	They propose a new paradigm for multimodal large language models by reusing modality encoders and merging LLM parameters.
Outcome:	The proposed model retains the modal understanding capabilities of each original model.

Copied to clipboard

Wenyu Guo, Qingkai Fang, Dong Yu, Yang Feng

Multilinguality

Challenge:	Existing models require associated image with input sentence, which is difficult to satisfy at inference.
Approach:	They propose to use synthetic and authentic images to generate translations using text-to-image generation models.
Outcome:	The proposed model achieves state-of-the-art performance on En-De and En-Fr datasets while remaining independent of authentic images during inference.

Neural Machine Translation with Phrase-Level Universal Visual Representations (2022.acl-long)

Copied to clipboard

Qingkai Fang, Yang Feng

Multilinguality

Challenge:	Existing multimodal machine translation methods require paired input of source sentence and image, which makes them suffer from shortage of sentence-image pairs.
Approach:	They propose a phrase-level retrieval-based method to get visual information from existing sentence-image data sets.
Outcome:	The proposed method significantly outperforms strong baselines on multiple MMT datasets, especially when the textual context is limited.

Gaussian Process based Deep Dyna-Q approach for Dialogue Policy Learning (2021.findings-acl)

Copied to clipboard

Guanlin Wu, Wenqi Fang, Ji Wang, Jiang Cao, Weidong Bao, Yang Ping, Xiaomin Zhu, Zheng Wang

Dialogue Systems & Conversational Agents

Challenge:	Reinforcement learning (RL) is the main dialogue policy learning method in recent years.
Approach:	They propose a Gaussian Process based Deep Dyna-Q approach to dialogue policy learning . they propose evaluating the quality of experiences generated by the world model using a discriminator .
Outcome:	The proposed approach improves the effectiveness and efficiency of dialogue policy learning by 20% with fewer human-machine interactions.

HTMuon: Improving Muon via Heavy-Tailed Spectral Correction (2026.findings-acl)

Copied to clipboard

Tianyu Pang, Yujie Fang, Zihang Liu, Shenyang Deng, Lei Hsiung, Shuhua Yu, Yaoqing Yang

LLaMA-Omni 2: LLM-based Real-time Spoken Chatbot with Autoregressive Streaming Speech Synthesis (2025.acl-long)

Challenge:	Muon’s orthogonalized update rule suppresses the emergence of heavy-tailed weight spectra and over-emphasizes the training along noise-dominated directions.
Approach:	They propose to preserve Muon's ability to capture parameter interdependencies while producing heavier-tailed updates and inducing heavier-tail weight spectra.
Outcome:	The proposed algorithm suppresses the emergence of heavy-tailed weight spectra and over-emphasizes training along noise-dominated directions.

Copied to clipboard

Qingkai Fang, Yan Zhou, Shoutao Guo, Shaolei Zhang, Yang Feng

Natural Language Interfaces

Challenge:	LLaMA-Omni 2 is a series of speech language models (SpeechLMs) based on large language models.
Approach:	They introduce a series of speech language models capable of real-time speech interaction . LLaMA-Omni 2 trains on 200K multi-turn speech dialogue samples .
Outcome:	The proposed speech language models surpass state-of-the-art models on spoken question answering and speech instruction.

CLHA: A Simple Yet Effective Contrastive Learning Framework for Human Alignment (2024.lrec-main)

Copied to clipboard

Feiteng Fang, Liang Zhu, Xi Feng, Jinchang Hou, Qixuan Zhao, Chengming Li, Xiping Hu, Ruifeng Xu, Min Yang

Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning (2020.emnlp-main)

Challenge:	Large language models (LLMs) have attracted considerable attention from academic and industrial communities due to their outstanding performance in various natural language processing tasks.
Approach:	They propose a Contrastive Learning Framework for Human Alignment to evaluate the noise within the data and dynamically adjust the training process.
Outcome:	The proposed framework surpasses other algorithms in terms of reward model scores, automatic evaluations, and human assessments on the widely used dataset "Helpful and Harmless"

Copied to clipboard

Zhiyuan Fang, Tejas Gokhale, Pratyay Banerjee, Chitta Baral, Yezhou Yang

Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search (2026.findings-acl)

Challenge:	Observable changes in the scene are reflected in captions, but actions are also linked to social aspects such as intentions, effects, and attributes that describe the agent.
Approach:	They propose to generate captions from videos that describe latent aspects of the human agent's actions.
Outcome:	The proposed model can be used to describe latent aspects of human actions in video clips and answer questions about videos.

Copied to clipboard

Yifei Zhang, Xu Yang, Xiao Yang, Bowen Xian, Qizheng Li, Shikai Fang, Jingyuan Li, Jian Wang, Minrui Xu, Yuge Zhang, Weiqing Liu, Jiang Bian

Context-DPO: Aligning Language Models for Context-Faithfulness (2025.findings-acl)

Challenge:	LLM-based agents for machine learning engineering rely on tree search to rank candidates.
Approach:	They propose an LLM-based agent that operationalizes gradient-based optimization.
Outcome:	The proposed agent achieves a state-of-the-art 35.1% any-medal rate on MLE-Bench with a limited budget on a single GPU.

Copied to clipboard

Baolong Bi, Shaohan Huang, Yiwei Wang, Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei, Junfeng Fang, Zehao Li, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, Shenghua Liu

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories (2024.findings-acl)

Challenge:	Context-DPO is the first alignment method specifically designed to enhance contextfaithfulness for large language models.
Approach:	They propose a benchmark that simulates Retrieval-Augmented Generation scenarios with knowledge conflicts to evaluate context-faithfulness.
Outcome:	The proposed method improves LLMs' context-faithfulness by 35% to 280% over open-source models.

Copied to clipboard

Jia Li, Ge Li, Yunfei Zhao, Yongmin Li, Huanyu Liu, Hao Zhu, Lecheng Wang, Kaibo Liu, Zheng Fang, Lanshen Wang, Jiazheng Ding, Xuanming Zhang, Yuqi Zhu, Yihong Dong, Zhi Jin, Binhua Li, Fei Huang, Yongbin Li, Bin Gu, Mengfei Yang

Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training (2024.acl-long)

Challenge:	Existing benchmarks are poorly aligned with real-world code repositories and are insufficient to evaluate the coding abilities of Large Language Models (LLMs).
Approach:	They propose a repository-level benchmark named DevEval to evaluate LLMs' coding abilities in real-world code repositories.
Outcome:	The proposed benchmarks show that the LLMs perform better in real-world code repositories than existing benchmarks.

Copied to clipboard

Feiteng Fang, Yuelin Bai, Shiwen Ni, Min Yang, Xiaojun Chen, Ruifeng Xu

LearnAlign: Data Selection for LLM Reinforcement Learning with Improved Gradient Alignment (2026.findings-acl)

Challenge:	Large language models (LLMs) exhibit substantial capabilities yet face challenges such as hallucination, outdated knowledge, and untraceable reasoning processes.
Approach:	They propose a retrieval-augmented generation approach that leverages adaptive adversarial training to dynamically adjust the model’s training process in response to retrieval noises.
Outcome:	The proposed approach improves the performance of the LLaMA-2 7B model under diverse noise conditions.

Copied to clipboard

Shipeng Li, Zhiqin Yang, Shikun Li, Xiaobo Xia, Hengyu Liu, Xinghua Zhang, Gaode Chen, Dong Fang, Ying Tai, Zhe Peng

Arbitrary Time Information Modeling via Polynomial Approximation for Temporal Knowledge Graph Embedding (2024.lrec-main)

Challenge:	Reinforcement learning with verifiable rewards (RLVR) is a key technique for enhancing LLMs’ reasoning abilities, yet its data inefficiency remains a major bottleneck.
Approach:	They propose a gradient-alignment-based method which intelligently selects the learnable and representative training reasoning data for RLVR post-training.
Outcome:	Experiments on five reasoning benchmarks show that the proposed method significantly reduces training data requirements while improving performance.

Copied to clipboard

Zhiyu Fang, Jingyan Qin, Xiaobin Zhu, Chun Yang, Xu-Cheng Yin

Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs (2026.findings-acl)

Challenge:	Existing knowledge graphs lack rich inference patterns and the limited ability to model arbitrary timestamps continuously.
Approach:	They propose a temporal knowledge graph-based temporal representation method that decomposes time information by polynomials and then enhances the model's capability to represent arbitrary timestamps flexibly.
Outcome:	The proposed method can encode arbitrary time information or even unseen timestamps while capturing rich inference patterns and higher-arity relations of the knowledge base.

Copied to clipboard

Guangtao Lyu, Qi Liu, Chenghao Xu, Jiexi Yan, Muli Yang, Xueting Li, Fen Fang, Cheng Deng

Modeling Adversarial Attack on Pre-trained Language Models as Sequential Decision Making (2023.findings-acl)

Challenge:	Existing training-free methods are vulnerable to the attention sink phenomenon . Existing methods include contrastive decoding and auxiliary expert models .
Approach:	They propose a training-free attention intervention that constructs a PAD map to identify semantically core visual regions and applies per-head Median Absolute Deviation Scaling to adaptively control the intervention strength.
Outcome:	The proposed intervention improves visual grounding and reduces hallucinations on multiple LVLMs and benchmarks.

Copied to clipboard

Xuanjie Fang, Sijie Cheng, Yang Liu, Wei Wang

NER-guided Comprehensive Hierarchy-aware Prompt Tuning for Hierarchical Text Classification (2024.lrec-main)

Challenge:	Pre-trained language models (PLMs) have shown strong potential in various downstream tasks.
Approach:	They propose to model adversarial attack task as a sequential decision-making problem where the whole attack process is sequential with two decision- making problems, i.e., word finder and word substitution.
Outcome:	The proposed approach achieves the highest attack success rate with a comparable modification rate and semantic similarity to attack fine-tuned BERT.

Copied to clipboard

Fuhan Cai, Duo Liu, Zhongqiang Zhang, Ge Liu, Xiaozhe Yang, Xiangzhong Fang

Information Extraction & Text Mining

Challenge:	Hierarchical text classification (HTC) is a challenging task in natural language processing due to its complex taxonomic label hierarchy.
Approach:	They propose to use prompts to model hierarchical text classification (HTC) they propose to introduce conditional random fields and Global Pointer to establish hierarchic dependencies .
Outcome:	The proposed approach achieves state-of-the-art (SoTA) performance on three public datasets.

SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)

Copied to clipboard

Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke

Challenge:	Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis.
Approach:	They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis.
Outcome:	SciAssess evaluates 11 LLMs on multiple tasks across scientific fields.

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)

Copied to clipboard

Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang

AudioVSR: Enhancing Video Speech Recognition with Audio Data (2024.emnlp-main)

Challenge:	i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data.
Approach:	They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data.
Outcome:	i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks.

Copied to clipboard

Xiaoda Yang, Xize Cheng, Jiaqi Duan, Hongshun Qiu, Minjie Hong, Minghui Fang, Shengpeng Ji, Jialong Zuo, Zhiqing Hong, Zhimeng Zhang, Tao Jin

CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy (2025.naacl-long)

Challenge:	Recent work has shown poor performance with non-Indo-European languages . previous work primarily utilizes video information to build VSR models .
Approach:	They propose a generative model for data inflation that integrates synthetic data with authentic visual data to enhance the VSR model.
Outcome:	The proposed model improves on the audio-visual alignment problem in audio-video tasks.

Copied to clipboard

Mian Zhang, Xianjun Yang, Xinlu Zhang, Travis Labrum, Jamie C. Chiu, Shaun M. Eack, Fei Fang, William Yang Wang, Zhiyu Chen

CTC-based Non-autoregressive Textless Speech-to-Speech Translation (2024.findings-acl)

Challenge:	Existing research has explored mental health condition classifications, empathetic conversations, and chatbots designed for simple discourse structures.
Approach:	They propose a benchmark for systematic evaluation of cognitive behavioral therapy assistance using Large Language Models (LLMs).
Outcome:	The proposed benchmark includes three levels of tasks covering key aspects of cognitive behavioral therapy that could be enhanced through AI assistance.

Copied to clipboard

Qingkai Fang, Zhengrui Ma, Yan Zhou, Min Zhang, Yang Feng

Expanding before Inferring: Enhancing Factuality in Large Language Models through Premature Layers Interpolation (2025.emnlp-main)

Challenge:	Existing direct speech-to-speech translation models require text supervision during training, which is not feasible for numerous unwritten languages.
Approach:	They propose a non-autoregressive (NAR) model that generates discrete units from the source speech and employs a unit-based vocoder to synthesize the target.
Outcome:	The proposed model achieves translation quality comparable to the autoregressive model while preserving up to 26.81 decoding speedup.

Copied to clipboard

Dingwei Chen, Ziqiang Liu, Feiteng Fang, Chak Tou Leong, Shiwen Ni, Ahmadreza Argha, Hamid Alinejad-Rokny, Min Yang, Chengming Li

Efficient Training for Cross-lingual Speech Language Models (2026.findings-acl)

Challenge:	Existing approaches to generating factually inconsistent outputs are resource-intensive.
Approach:	They propose a plug-and-play intervention designed to enhance factuality by inserting premature layers formed through mathematical interpolation with adjacent layers.
Outcome:	The proposed intervention reduces hallucinations while outperforming baselines on four datasets.

Copied to clipboard

Yan Zhou, Qingkai Fang, Yun Hong, Yang Feng

Roleplay-doh: Enabling Domain-Experts to Create LLM-simulated Patients via Eliciting and Adhering to Principles (2024.emnlp-main)

Challenge:	Currently, large language models (LLMs) focus on the text modality, making speech modeling difficult.
Approach:	They propose a cross-lingual speech language model that trains on discrete speech tokens to achieve cross-modal and cross-linguistic alignment through continual pre-training.
Outcome:	The proposed method achieves cross-modal and cross-lingual alignment through continual pre-training.

Copied to clipboard

Ryan Louie, Ananjan Nandi, William Fang, Cheng Chang, Emma Brunskill, Diyi Yang

MaskTab: Scalable Masked Tabular Pretraining with Scaling Laws and Distillation for Industrial Classification (2026.findings-acl)

Challenge:	Existing methods for improving LLMs in simulations are limited due to privacy concerns and limited domain knowledge.
Approach:	They propose a pipeline that elicits qualitative feedback from a domain-expert and transforms it into a set of principles that govern an LLM-prompted roleplay.
Outcome:	The proposed pipeline shows a 30% improvement in response quality and principle following for the downstream task.

Copied to clipboard

Bo Zheng, Yudong Chen, Zihua Xiong, Shuai Fang, Peidong He, Yang Yang, Sheng Guo

MolTC: Towards Molecular Relational Modeling In Language Models (2024.findings-acl)

Challenge:	Tabular data is high-dimensional, riddled with missing entries, and rarely labeled at scale.
Approach:	They propose a unified pre-training framework for industrial-scale tabular data . MaskTab encodes missing values via dedicated learnable tokens .
Outcome:	The proposed framework outperforms XGBoost and MaskTab-L on industrial-scale . it achieves +5.04% AUC and +8.28% KS over prior art under rigorous scaling .

Copied to clipboard

Junfeng Fang, Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li, Kun Wang, Wenjie Du, Xiang Wang

GASE: Graph-Aware Semantic Embedding Learning with Frozen LLMs for Text-Attributed Graphs (2026.acl-long)

Challenge:	Molecular Relational Learning (MRL) is a promising way to understand interactions between molecular pairs.
Approach:	They propose a novel LLM-based multi-modal framework for molecular interaction modeling following Chain-of-Thought (CoT) theory which integrates graphical information of two molecules in pair.
Outcome:	The proposed framework integrates graphical information of two molecules in pair.

Copied to clipboard

Mingqian Ding, Jianjun Li, Wenqi Yang, Zhibo Zhang, Yushen Fang

Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity (2026.acl-long)

Challenge:	Large Language Models (LLMs) have shown strong potential for text-attributed graph (TAG) learning, yet effectively integrating LLM semantics with graph structural information remains challenging.
Approach:	They propose a framework for learning Graph-Aware Semantic Embeddings using frozen LLMs.
Outcome:	The proposed framework outperforms state-of-the-art methods on node classification and achieves a 5 speedup over fine-tuning-based methods.

Copied to clipboard

Feiteng Fang, Dingwei Chen, Xiang Huang, Ting-En Lin, Yuchuan Wu, Xiong Liu, Jing Ye, Ziqiang Liu, Haonan Zhang, Liang Zhu, Hamid Alinejad-Rokny, Min Yang, Yongbin Li

Challenge:	Existing approaches to reward modeling in reinforcement learning tasks are limited when dealing with ambiguous preferences.
Approach:	They propose to use AAM to dynamically calibrate preference margins using the Bradley-Terry model's internal parameter knowledge to improve reward modeling in subjective tasks.
Outcome:	The proposed approach improves reward modeling by dynamically calibrating preference margins using the model’s internal parameter knowledge.

SrDetection: A Self-Referential Framework for Data Leakage Detection in Code Large Language Models (2026.findings-acl)

Copied to clipboard

Shuaimin Li, Liyang Fan, Zeyang li, Zhuoyue Wan, Yufang Lin, Shiwen Ni, Feiteng Fang, Hamid Alinejad-Rokny, Yuanfeng Song, Kun Jing, Chen Jason Zhang, Min Yang

Where to Attack: A Dynamic Locator Model for Backdoor Attack in Text Classifications (2022.coling-1)

Challenge:	Existing methods for evaluating code large language models assume access to proprietary training corpora or use external reference sets with manually tuned, non-generalizable thresholds.
Approach:	They propose a framework for self-referential leakage detection for gray-box and black-box settings.
Outcome:	The proposed framework improves average F1 by 21.52 points in the gray-box setting and 14.46 points in black-box settings over strong baselines.

Copied to clipboard

Heng-yang Lu, Chenyou Fan, Jun Yang, Cong Hu, Wei Fang, Xiao-jun Wu

Human-in-the-loop Robotic Grasping Using BERT Scene Representation (2022.coling-1)

Challenge:	BackDoor Attack (BDA) study aims to train a poisoned model with clean data and some trigger-embedded instances to perform normally on normal inputs.
Approach:	They propose to train a poisoned model with clean and poisonest inputs . they propose to use triggers to predict those poisonets as target labels .
Outcome:	The proposed model can predict P2P dynamically without human intervention.

Copied to clipboard

Yaoxian Song, Penglei Sun, Pengfei Fang, Linyi Yang, Yanghua Xiao, Yue Zhang

PresentAgent: Multimodal Agent for Presentation Video Generation (2025.emnlp-demos)

Challenge:	Existing approaches for robotic grasping in cluttered scenes are expensive and lack structure information.
Approach:	They propose a human-in-the-loop framework for robotic grasping in cluttered scenes . they substitute scene-graph representation with a text representation of the scene using BERT .
Outcome:	The proposed framework outperforms object-agnostic and scene-graph based methods on robots and physical robots.

Copied to clipboard

Jingwei Shi, Zeyu Zhang, Biao Wu, Yanjie Liang, Meng Fang, Ling Chen, Yang Zhao

Dual-Reasoner: Bridging Interleaved Atomicity and Streaming Latency via Thinking-while-Talking (2026.findings-acl)

Challenge:	Existing methods for generating static slides or text summaries are limited to producing narrated presentations.
Approach:	They propose a multimodal agent that transforms long-form documents into narrated presentations.
Outcome:	The present agent produces fully synchronized visual and spoken content that closely mimics human-style presentations.

Copied to clipboard

Yangzhuo Li, Shengpeng Ji, Yifu Chen, Tianle Liang, Haoyu Yang, null Junboli, Jun Fang, Lin Li, Qingyang Hong

End-to-end Knowledge Retrieval with Multi-modal Queries (2023.acl-long)

Challenge:	Existing methods to integrate Chain-of-Thought into spoken dialogue models incur prohibitive latency.
Approach:	They propose a Streaming Masking Mechanism to ensure uninterrupted audio streaming . they use a quadruple-constraint system to reconstruct logical atomicity .
Outcome:	Experimental results show that Dual-Reasoner improves speech generation performance with low latency.

Copied to clipboard

Man Luo, Zhiyuan Fang, Tejas Gokhale, Yezhou Yang, Chitta Baral

Beyond Quantity: Trajectory Diversity Scaling for Code Agents (2026.findings-acl)

Challenge:	a new task is proposed to learn knowledge retrieval with multimodal queries . a vision-language model can retrieve knowledge using images and text inputs .
Approach:	They propose a task for vision-language models to retrieve knowledge with multi-modal queries . they propose reViz, a model that integrates content from both text and image queries based on a multimodal query task .
Outcome:	The proposed task performs better under zero-shot settings than previous work on cross-modal retrieval.

Copied to clipboard

Guhong Chen, Chenghao Sun, Cheng Fu, Qiyao Wang, Zhihong Huang, ChaoPeng Wei, Guangxu Chen, Feiteng Fang, Ahmadreza Argha, Bing Zhao, Xander Xu, Qi Han, Hamid Alinejad-Rokny, Qiang Qu, Binhua Li, Shiwen Ni, Min Yang, HU Wei, Yongbin Li

Green & Sustainable NLP

Challenge:	Code large language models (LLMs) are becoming tool-interactive agents . quantity-centric scaling exhibits an early bottleneck that underutilizes trajectory data . et al.: a new approach to scale trajectory diversity improves tool-use generalization .
Approach:	They propose a Trajectory Diversity Scaling-based data synthesis framework for code agents that scales performance through diversity rather than raw volume.
Outcome:	Experiments on general tool-use benchmarks and code agent tasks show that TDScaling improves tool-user generalization and inherent coding proficiency.

Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning (2025.naacl-long)

Copied to clipboard

Rujing Yao, Yang Wu, Chenghao Wang, Jingwei Xiong, Fang Wang, Xiaozhong Liu

Information Retrieval

Challenge:	Existing approaches to large language models focus on semantic similarity, neglecting the intricate logical structures and reasoning essential for addressing complex legal issues.
Approach:	They propose a Logical-Semantic Integration Model (LSIM) that bridges semantic and logical coherence and a supervised framework that integrates semantic features with in-context learning.
Outcome:	The proposed framework significantly improves accuracy and reliability on a real-world legal QA dataset.

StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning (2024.acl-long)

Copied to clipboard

Shaolei Zhang, Qingkai Fang, Shoutao Guo, Zhengrui Ma, Min Zhang, Yang Feng

Text Generation

Challenge:	Existing simultaneous translation methods focus on text-to-text and speech-totext translation.
Approach:	They propose a Simul-S2ST model that jointly learns translation and simultaneous policy in a unified framework of multi-task learning.
Outcome:	The proposed model can perform offline and simultaneous speech recognition, speech translation and speech synthesis via an "All-in-One" seamless model.

UniGeM: Unifying Data Selection and Mixing via Geometric Exploration and Mining (2026.findings-acl)

Copied to clipboard

Changhao Wang, null Yunfeiyu, Xinhao Yao, Jiaolong Yang, Lu Yu, Junpeng Fang, Chaobo Li, Riccardo Cantoro, Qing Cui, Jun Zhou

Challenge:	Large Language Models (LLMs) scaling is limited by data quality and domain mixing and instance selection are two separate problems.
Approach:	They propose a framework that unifies mixing and selection without training proxy models or relying on external reference datasets.
Outcome:	The proposed framework achieves 2.0 data efficiency over a random baseline and further improves overall performance compared to SOTA methods in reasoning-heavy evaluations and multilingual generalization.

XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners (2024.naacl-long)

Copied to clipboard

Yun Luo, Zhen Yang, Fandong Meng, Yingjie Li, Fang Guo, Qinglin Qi, Jie Zhou, Yue Zhang

CascadeFix: Multi-Location Program Repair via Cascading Planning and Generation (2026.findings-acl)

Challenge:	Existing methods for active learning rely on model uncertainty or disagreement to pick unlabeled data, leading to over-confidence in superficial patterns and lack of exploration.
Approach:	They propose to use a bi-directional encoder and a uni-directional decoder to generate and score an explanation for low-resource text classification.
Outcome:	The proposed model improves on 9 strong baselines on six datasets and can generate explanations for its predictions.

Copied to clipboard

Huan Zhang, Li Kuang, Yang Yang, Yilei Fang, Yingjie Xia

Text Clustering

Challenge:	Existing methods for automating program repair face insufficient bug dependency modeling and inadequate global repair planning when addressing semantically complex multi-location bugs.
Approach:	They propose a multi-location automatic repair method via cascading planning and generation . they propose to model dependencies among bugs and cluster them to ensure rationality .
Outcome:	The proposed method resolves 84 multi-location bugs, achieving a 31% improvement over current methods.

PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play (2025.findings-acl)

Copied to clipboard

Wei Fang, Yang Zhang, Kaizhi Qian, James R. Glass, Yada Zhu

AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization via Multi-LLMs (2025.findings-naacl)

Challenge:	Existing solutions for large language models rely on manual rewriting or labeled data for validation . Existing approaches rely only on comprehensive tool documentation and in-context demonstrations .
Approach:	They propose a framework that "plays" with each tool to explore its input-output behaviors.
Outcome:	Experiments show that PLAY2PROMPT improves zero-shot tool performance across open and closed models.

Copied to clipboard

Jiawei Chen, Xiao Yang, Zhengwei Fang, Yu Tian, Yinpeng Dong, Zhaoxia Yin, Hang Su

Low-resource Neural Machine Translation with Cross-modal Alignment (2022.emnlp-main)

Challenge:	Existing jailbreak research exhibits limitations in universality, validity, and efficiency . Existing methods for jailbreaking LLMs have limited validity and effectiveness .
Approach:	They propose a black-box approach that uses wordplay-guided mapping rule sampling to create universal adversarial prompts.
Outcome:	The proposed method efficiently identifies security vulnerabilities across various LLMs, achieving an average success rate of over 80% with fewer than 10 queries.

Copied to clipboard

Zhe Yang, Qingkai Fang, Yang Feng

Multilinguality

Challenge:	Existing neural machine translation techniques rely on large monolingual corpus, which is costly for some low-resource languages.
Approach:	They propose a cross-modal contrastive learning method to learn a shared space for all languages by additional visual modality.
Outcome:	The proposed method can learn cross-modal and cross-lingual alignment with small amount of image-text pairs and achieves significant improvements over the text-only baseline.

MTPChat: A Multimodal Time-Aware Persona Dataset for Conversational Agents (2025.findings-naacl)

Copied to clipboard

Wanqi Yang, Yanda Li, Meng Fang, Ling Chen

Dialogue Systems & Conversational Agents

Challenge:	Existing time-aware datasets that focus on persona-grounded conversations focus on temporal dynamics, which narrows their scope and diminishes their complexity.
Approach:	They propose a multimodal, time-aware persona dialogue dataset that integrates linguistic, visual, and temporal elements within dialogue and persona memory.
Outcome:	The proposed framework integrates linguistic, visual, and temporal elements within dialogue and persona memory to assess a model’s ability to understand implicit temporal cues and dynamic interactions.

VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery (2026.findings-eacl)

Copied to clipboard

Jinchao Ge, Tengfei Cheng, Biao Wu, Zeyu Zhang, Shiya Huang, Judith Bishop, Gillian Shepherd, Meng Fang, Ling Chen, Yang Zhao

MedDialog: Large-scale Medical Dialogue Datasets (2020.emnlp-main)

Challenge:	MLLMs that use domain-specific data are limited in understanding cultural heritage artifacts such as ancient Greek pottery . supervised fine-tuning improves adaptation to domain knowledge, but it struggles with deeper reasoning tasks.
Approach:	They propose a visual question-answer tool that augments SFT with reinforcement learning using verifiable rewards.
Outcome:	The proposed model outperforms baseline models on reasoning-intensive questions on ancient Greek pottery.

Copied to clipboard

Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie

Dialogue Systems & Conversational Agents

Challenge:	telemedicine is a medical practice that provides patient care remotely using video conferencing tools.
Approach:	They build large-scale medical dialogue datasets to facilitate research . they pretrain several models on the Chinese MedDialog dataset and compare their performance .
Outcome:	The proposed datasets show that models trained on MedDialog can generate doctor-like medical dialogues.

MACSum: Controllable Summarization with Mixed Attributes (2023.tacl-1)

Copied to clipboard

Yusen Zhang, Yang Liu, Ziyi Yang, Yuwei Fang, Yulong Chen, Dragomir Radev, Chenguang Zhu, Michael Zeng, Rui Zhang

Information Extraction & Text Mining

Challenge:	Existing work on controllable summarization with mixed attributes lacks designated annotations.
Approach:	They propose a human-annotated summarization benchmark for controllable summarizing with mixed attributes based on news and dialogue sources .
Outcome:	The proposed dataset contains human-annotated summarization datasets with mixed attributes . hard prompt models yield the best performance on most metrics and human evaluations . mixed-attribute control is still challenging for summarizing tasks .

Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching (2022.findings-naacl)

Copied to clipboard

Kunbo Ding, Weijie Liu, Yuejian Fang, Zhe Zhao, Qi Ju, Xuefeng Yang, Rong Tian, Zhu Tao, Haoyan Liu, Han Guo, Xingyu Bai, Weiquan Mao, Yudong Li, Weigang Guo, Taiqiang Wu, Ningyuan Sun

Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching (2025.acl-long)

Challenge:	Existing studies have shown that cross-lingual knowledge distillation can improve the performance of pre-trained models for cross-linguistic similarity matching tasks.
Approach:	They propose a multi-stage distillation framework for constructing a small-size but high-performance cross-lingual model using contrastive learning, bottleneck, and parameter recurrent strategies.
Outcome:	The proposed model can compress the size of XLM-R and MiniLM by more than 50% while the performance is only reduced by about 1%.

Copied to clipboard

Jialong Zuo, Shengpeng Ji, Minghui Fang, Mingze Li, Ziyue Jiang, Xize Cheng, Xiaoda Yang, Chen Feiyang, Xinyu Duan, Zhou Zhao

Can MLLMs Understand the Deep Implication Behind Chinese Images? (2025.acl-long)

Challenge:	Existing methods focus on disentangling speakers and content, while others focus on preserving the source's prosody.
Approach:	They propose a rhythm-controllable and efficient zero-shot voice conversion model that transforms the source speaker’s timbre into an unseen one while retaining speech content.
Outcome:	The proposed model adapts the linguistic content duration to the desired speaking style, facilitating the transfer of the target speaker’s rhythm.

Copied to clipboard

Chenhao Zhang, Xi Feng, Yuelin Bai, Xeron Du, Jinchang Hou, Kaixin Deng, Guangzeng Han, Qinrui Li, Bingli Wang, Jiaheng Liu, Xingwei Qu, Yifei Zhang, Qixuan Zhao, Yiming Liang, Ziqiang Liu, Feiteng Fang, Min Yang, Wenhao Huang, Chenghua Lin, Ge Zhang, Shiwen Ni

DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenes (2025.findings-emnlp)

Challenge:	MLLMs perform poorly on traditional culture images, indicating limitations in understanding high-level semantics and lacking a deep knowledge base of Chinese traditional culture.
Approach:	They propose to use Chinese images to assess MLLMs' higher-order perception and understanding of Chinese visual content.
Outcome:	The proposed model incorporates images that represent Chinese traditional culture, such as famous Chinese traditional paintings, to ensure the authenticity of the Chinese context.

Copied to clipboard

Zhaowei Wang, Hongming Zhang, Tianqing Fang, Ye Tian, Yue Yang, Kaixin Ma, Xiaoman Pan, Yangqiu Song, Dong Yu

DEFT: Distribution-guided Efficient Fine-Tuning for Human Alignment (2024.findings-emnlp)

Challenge:	Large Vision-Language Models (LVLMs) have achieved significant progress in tasks like visual question answering and document understanding.
Approach:	They introduce DivScene, a large-scale dataset with 4,614 houses across 81 scene types and 5,707 kinds of target objects.
Outcome:	The proposed dataset provides a much greater diversity of target objects and scene types than existing datasets, enabling a comprehensive task evaluation.

Copied to clipboard

Liang Zhu, Feiteng Fang, Yuelin Bai, Longze Chen, Zhexiang Zhang, Minghuan Tan, Min Yang

EmoRes: Toward Adaptive Psychological Support via User-Agnostic Benchmark and Topic-Mining Agent (2026.findings-acl)

Challenge:	Experimental results show that the methods enhanced by DEFT outperform the original methods in both alignment capability and generalization ability, with significantly reduced training time.
Approach:	They propose a distribution-based alignment framework that integrates data filtering and distributional guidance to improve alignment efficiency and generalization ability.
Outcome:	The proposed framework outperforms existing methods in alignment capability and generalization ability with significantly reduced training time.

Copied to clipboard

Zhengwei Zou, Xuanming Jiang, Baoyi An, Dingyu Nie, Zhengxing Fang, Qingyu Liu, Xueming Qian, Guoshuai Zhao, Zhongyu Yang

Think before Go: Hierarchical Reasoning for Image-goal Navigation (2026.acl-long)

Challenge:	Large language models generate fragmented and emotionally inconsistent dialogues lacking the therapeutic structure necessary for reliable assessment.
Approach:	They propose a framework that boosts psychological reasoning via a Topic-Mining Emotional Agent and a multi-perspective Self-Reflection Agent.
Outcome:	The proposed framework improves topic continuity, emotional coherence, and clinical interpretability over baselines and validated by ablation studies and human evaluations.

Copied to clipboard

Pengna Li, Kangyi Wu, Shaoqing Xu, Fang Li, Lin Zhao, Long Chen, Zhi-Xin Yang, Nanning Zheng

STAF: Pushing the Boundaries of Test-Time Adaptation towards Practical Noise Scenarios (2024.lrec-main)

Challenge:	Existing methods for image-goal navigation fail to extract informative visual cues, leading agents to wander around.
Approach:	They propose a framework that decomposes image-goal navigation into high-level planning and low-level execution.
Outcome:	The proposed method is superior to existing methods in both simulation and real-world environments.

Copied to clipboard

Haoyu Xiong, Xinchun Zhang, Leixin Yang, Yu Xiang, Gang Fang

A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Any Translation (2024.acl-long)

Challenge:	Pre-trained language models have demonstrated superior performance on NLP tasks . however, when the training domain and testing domain are taken from different distributions, the deployed model often violates this assumption.
Approach:	They propose a Stable Test-time Adaptation Framework to stabilize the adaptation process.
Outcome:	The proposed framework boosts model robustness to noise distribution shifts while minimizing error accumulation and catastrophic forgetting.

Copied to clipboard

Zhengrui Ma, Qingkai Fang, Shaolei Zhang, Shoutao Guo, Yang Feng, Min Zhang

Text Generation

Challenge:	Existing translation pipelines require additional cascade components to achieve speech-to-speech translation.
Approach:	They propose a non-autoregressive generation framework for simultaneous speech translation . it integrates both text-to-text and speech-tospeech tasks into a unified framework .
Outcome:	The proposed framework outperforms state-of-the-art models in speech-to-text and speech- to-speech tasks.

AscendKernelGen: LLM-Driven Kernel Generation for NPUs (2026.findings-acl)

Copied to clipboard

Xinzi Cao, Jianyang Zhai, Pengfei Li, Zhiheng Hu, Cen Yan, null Mubingxu, Guanghuan Fang, Bin She, Jiayu Li, Yihan Su, Dongyang Tao, Feidiao Yang, Chang-Dong Wang, Yutong Lu, Weicheng Xue, Bin Zhou, Yonghong Tian

Challenge:	Neural Processing Units (NPUs) are critical for AI infrastructure, but their development remains a bottleneck due to vendor-specific Domain-Specific Languages (DSLs).
Approach:	They propose a framework for NPU kernel development that bridges the gap in hardware-specific coding . compiler success on complex Level-2 kernels improves from 0% to 95.5%, they say .
Outcome:	The proposed framework bridges the gap in hardware-specific coding, showing a near-zero success rate on complex kernels.

Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering (2026.acl-long)

Copied to clipboard

Shuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu

AMPO: Automatic Multi-Branched Prompt Optimization (2024.emnlp-main)

Challenge:	Existing approaches to overcome object hallucination are limited . Existing mitigations include costly retraining and a training-free inference framework .
Approach:	They propose a training-free inference framework that simulates a metacognitive self-correction process.
Outcome:	The proposed framework reduces object hallucination rates by 12.67% on MMHal-Bench and improves accuracy by 5.8% on POPE.

Copied to clipboard

Sheng Yang, Yurong Wu, Yan Gao, Zineng Zhou, Bin Zhu, Xiaodi Sun, Jian-Guang Lou, Zhiming Ding, Anbang Hu, Yuan Fang, Yunsong Li, Junyan Chen, Linjun Yang

Who Can Withstand Chat-Audio Attacks? An Evaluation Benchmark for Large Audio-Language Models (2025.findings-acl)

Challenge:	Existing prompt engineering techniques are limited to producing single flow instructions, struggling with handling diverse patterns.
Approach:	They propose an automatic prompt optimization method that iteratively develops a multi-branched prompt using failure cases as feedback.
Outcome:	The proposed method achieves the best results across five tasks and demonstrates significant optimization efficiency due to adoption of a minimal search strategy.

Copied to clipboard

Wanqi Yang, Yanda Li, Meng Fang, Yunchao Wei, Ling Chen

From Language to Driving: A Dual-Loop SLM-Enhanced Framework for Multi-Planner Scheduling via a Domain-Specific Language (2026.acl-long)

Challenge:	Existing research focused on model-specific adversarial methods, but real-world applications demand a more generalizable approach to audio adversarials.
Approach:	They propose a Chat-Audio Attacks benchmark to evaluate LALMs' robustness . they propose standard evaluation, GPT-4o-based evaluation and human evaluation .
Outcome:	The proposed benchmark aims to explore the robustness of six state-of-the-art LALMs with voice interaction capabilities.

Copied to clipboard

Jiawei Liu, Xun Gong, Muli Yang, Xingrui Yu, Fen Fang, Xulei Yang, Ivor Tsang, Yunfeng hu, Hong Chen, Qing Guo

Challenge:	Recent large language model-based AD research offers new avenues to address this challenge.
Approach:	They propose a small language model (SLM) for high-level semantic reasoning and schedule generation, while an inner loop performs low-level, high-frequency schedule execution and vehicle control.
Outcome:	The proposed framework improves instruction completion rates while maintaining high safety and compliance relative to multiple baselines.

Extract-Select: A Span Selection Framework for Nested Named Entity Recognition with Generative Adversarial Training (2022.findings-acl)

Copied to clipboard

Peixin Huang, Xiang Zhao, Minghao Hu, Yang Fang, Xinyi Li, Weidong Xiao

Named Entity Recognition

Challenge:	Existing studies treat named entity recognition as a sequential labeling problem.
Approach:	They propose a span selection framework for nested named entity recognition . they propose nesting entities with different input categories would be separately extracted .
Outcome:	The proposed framework outperforms competing models on four benchmark datasets.

Back Translation for Speech-to-text Translation Without Transcripts (2023.acl-long)

Copied to clipboard

Qingkai Fang, Yang Feng

Text Generation

Challenge:	End-to-end speech-totext translation (ST) is often achieved by utilizing source transcripts, but transcripts are only sometimes available since numerous unwritten languages exist worldwide.
Approach:	They propose an algorithm to synthesize pseudo ST data from monolingual target data to enhance ST without generating source transcripts.
Outcome:	The proposed method achieves an average boost of 2.3 BLEU on MuST-C En-De, En-Fr, and En-Es datasets.

Forge: Quality-Aware Reinforcement Learning for NP-Hard Optimization in LLMs (2026.findings-acl)

Copied to clipboard

Xiaozhe Li, Xinyu Fang, Shengyuan Ding, Yang Li, Linyang Li, Haodong Duan, Qingwen Liu, Kai Chen

Reasoning

Challenge:	Existing benchmarks focus on correctness, overlooking optimality . large language models excel at math, coding, logic and puzzles .
Approach:	They propose a framework for training and evaluating Large Language Models on NP-hard optimization problems through quality-aware RLVR.
Outcome:	The proposed framework outperforms existing benchmarks on math, coding, logic and puzzles.

Mitigating Hallucinations in Large Vision-Language Models without Performance Degradation (2026.acl-long)

Copied to clipboard

Xingyu Zhu, Junfeng Fang, Shuo Wang, Beier Zhu, Zhicai Wang, Yonghui Yang, Xiangnan He

Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data? (2024.acl-long)

Challenge:	Recent advances in large vision-language models produce hallucinations that compromise output reliability.
Approach:	They propose a dual-stage framework for mitigating hallucinations without performance degradation . they propose semantic-aware component disentanglement and interpretable parameter updates .
Outcome:	The proposed model reduces hallucinations by 23.4% while maintaining 97.4% of general generative capability.

Copied to clipboard

Qingkai Fang, Shaolei Zhang, Zhengrui Ma, Min Zhang, Yang Feng

Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)

Challenge:	Existing two-pass direct speech-to-speech translation models require parallel speech data to train, which is challenging to collect.
Approach:	They propose a two-pass direct speech-to-speech translation (S2ST) model that decomposes the task into speech- to-text translation (s2TT) and text-tospech (TTS) they propose 'composer' S2ST model that integrates pretrained S2TT and TTS models into a direct S2 ST model.
Outcome:	The proposed model integrates pretrained S2TT and TTS models into a direct S2ST model without parallel speech data.

Copied to clipboard

Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu

FARSS: Fisher-Optimized Adaptive Low-Rank and Singular-Vector Selection for Knowledge-Preserving Fine-Tuning (2026.findings-acl)

Challenge:	Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent.
Approach:	They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs.
Outcome:	The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models.

Copied to clipboard

Renxing Chen, Ziwei Xiang, Peisong Wang, Hongjian Fang, Meng Li, Fanhu Zeng, Yanan Zhu, Peipei Yang, Xu-Yao Zhang, Jian Cheng

When Slower Isn’t Truer: Inverse Scaling Law of Truthfulness in Multimodal Reasoning (2026.findings-acl)

Challenge:	Low-rank adaptation methods for large language models have limitations in preserving world knowledge and limiting updates to preserve world knowledge.
Approach:	They propose a Fisher-optimized adaptive low Rank and Singular-VectorSelection framework for knowledge-preserving fine-tuning that allows efficient and task-sensitive updates.
Outcome:	The proposed framework outperforms existing methods for knowledge-preserving fine-tuning.

Copied to clipboard

Sitong Fang, Wenjing Cao, Jiahao Li, Xuyao Wang, Chi-Min Chan, Sirui Han, Juntao Dai, Yike Guo, Yaodong Yang, Jiaming Ji

EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction (2024.findings-emnlp)

Challenge:	a study of slow reasoning models for multimodal reasoning finds that they are more prone to fabricating plausible yet false details when confronted with incomplete or misleading visual inputs.
Approach:	They conduct the first systematic study of the inverse scaling law in slow-thinking paradigms for multimodal reasoning.
Outcome:	The findings suggest that slower reasoning models are more prone to fabricating false details . the study analyzed 5,000-sample hierarchical prompt dataset by 50 participants .

Copied to clipboard

Li Yang, Qifan Wang, Jianfeng Chi, Jiahao Liu, Jingang Wang, Fuli Feng, Zenglin Xu, Yi Fang, Lifu Huang, Dongfang Liu

UCS-SQL: Uniting Content and Structure for Enhanced Semantic Bridging In Text-to-SQL (2025.findings-acl)

Challenge:	Existing methods to extract product attribute value require multiple extractions to obtain all corresponding values.
Approach:	They propose an Efficient product Attribute Value Extraction approach using lightweight sparse-layer interaction.
Outcome:	The proposed method achieves significant efficiency gains with neutral or marginal loss in performance when the context is long and number of attributes is large.

Copied to clipboard

Zhenhe Wu, Zhongqiu Li, JieZhangChinaTele JieZhangChinaTele, Zhongjiang He, Jian Yang, Yu Zhao, Ruiyu Fang, Bing Wang, Hongyan Xie, Shuangyong Song, Zhoujun Li

QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learning (2026.acl-long)

Challenge:	Existing methods overlook the challenge of effectively transforming structure information from NL to SQL.
Approach:	They propose a text-to-SQL framework that unites content and structure pipes to bridge the gap between NL and SQL.
Outcome:	The proposed framework bridges the gap between natural language questions and SQL by combining content and structure pipes.

Copied to clipboard

Songxin Qu, Tai-Ping Sun, Yun-Jie Wang, Huan-Yu Liu, Cheng Xue, Xiao-Fan Xu, Han Fang, Yang Yang, Yu-Chun Wu, Guo-Ping Guo, Zhao-Yun Chen

Reasoning

Challenge:	Large language models lack reliability in scientific domains that require strict adherence to physical constraints.
Approach:	They propose a large-scale dataset constructed via a task-adaptive strategy and a hybrid verification protocol that combines deterministic solvers with semantic auditing to guarantee scientific rigor.
Outcome:	The proposed model outperforms baselines and general-purpose preference models and is competitive with proprietary models.

DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Search (2026.acl-long)

Copied to clipboard

Zhibang Yang, Xinke Jiang, Rihong Qiu, Ruiqing Li, Yihang Zhang, Yue Fang, Yongxin Xu, Hongxin Ding, Xu Chu, Junfeng Zhao, Yasha Wang

Information Retrieval

Challenge:	Existing methods for ambiguous queries struggle to retrieve high-quality documents . DFAMS outperforms advanced FR methods by 14.37% in knowledge classification accuracy .
Approach:	They propose a framework that leverages dynamic information flow to identify latent query intents and construct semantically aligned knowledge partitions for accurate retrieval across heterogeneous sources.
Outcome:	The proposed framework outperforms existing methods in classification accuracy and retrieval recall tests.

CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation (2023.acl-long)

Copied to clipboard

Yan Zhou, Qingkai Fang, Yang Feng

Enhancing Temporal Sensitivity and Reasoning for Time-Sensitive Question Answering (2024.findings-emnlp)

Challenge:	Existing methods to translate speech signals into text are limited by the modality gap between speech and text.
Approach:	They propose Cross-modal Mixup via Optimal Transport to overcome the modality gap between speech and text by finding alignment between modalities.
Outcome:	Experiments on the MuST-C ST benchmark show that CMOT achieves an average BLEU of 30.0 in 8 translation directions, outperforming previous methods.

Copied to clipboard

Wanqi Yang, Yanda Li, Meng Fang, Ling Chen

Natural Language Interfaces

Challenge:	Existing language models have limited sensitivity to temporal information and inadequate temporal reasoning capabilities.
Approach:	They propose a framework that enhances temporal awareness and reasoning . they propose to use Temporal Information-Aware Embedding and Granular Contrastive Reinforcement Learning .
Outcome:	The proposed framework outperforms existing LLMs on time-sensitive question answering tasks.

Towards Scalable Lightweight GUI Agents via Multi-role Orchestration (2026.findings-acl)

Copied to clipboard

Ziwei Wang, Junjie Zheng, Leyang Yang, Sheng Zhou, Xiaoxuan Tang, Fang Zhouhua, Zhiwei Liu, Dajun Chen, Yong Li, Jiajun Bu

Challenge:	Advanced GUI agents suffer from prohibitive deployment costs on resource-constrained devices.
Approach:	They propose a lightweight GUI agent with GUI-specific knowledge and task scalability . LAMO-3B supports monolithic execution and MAS-style orchestration .
Outcome:	The proposed GUI agent LAMO-3B supports monolithic execution and MAS-style orchestration.

MARD: Module-Aware Reasoning Distillation for Language Models with Adaptive Supervision (2026.acl-long)

Copied to clipboard

Wenqi Yang, Jianjun Li, Zhibo Zhang, Mingqian Ding, Yushen Fang

MQuinE: a Cure for “Z-paradox” in Knowledge Graph Embedding (2024.emnlp-main)

Challenge:	Multi-step reasoning remains challenging for language models with limited capacity . et al., 2025) demonstrate remarkable reasoning capabilities across diverse tasks .
Approach:	They propose a module-aware reasoning distillation framework that explicitly targets key Transformer components for effective reasoning transfer.
Outcome:	The proposed framework targets key components for effective reasoning transfer . it adopts an offline distillation setting, where a strong teacher model provides reasoning trajectories in advance .

Copied to clipboard

Yang Liu, Huang Fang, Yunfeng Cai, Mingming Sun

STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation (2022.acl-long)

Challenge:	Existing knowledge graph embedding models suffer from Z-paradox, a deficiency in expressiveness . Embedding-based models map each entity and relation into a vector or matrix .
Approach:	They propose a new knowledge graph embedding model that does not suffer from Z-paradox while preserves strong expressiveness to model various relation patterns with theoretical justification.
Outcome:	The proposed model outperforms existing models on link prediction tasks while maintaining strong expressiveness.

Copied to clipboard

Qingkai Fang, Rong Ye, Lei Li, Yang Feng, Mingxuan Wang

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning (2025.findings-naacl)

Challenge:	Existing methods to learn speech representations for end-to-end speech-totext translation (ST) neglect the representation discrepancy across modalities.
Approach:	They propose a method to calibrate the representation discrepancy between modalities by mixing up the representation sequences of different modality inputs.
Outcome:	The proposed method alleviates the cross-modal representation discrepancy and improves on a strong baseline on eight translation directions.

Copied to clipboard

Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Moore Wang, Ruibin Yuan, Haihong Wu, Hongquan Lin, Wenhao Huang, Jiajun Zhang, Chenghua Lin, Jie Fu, Min Yang, Shiwen Ni, Ge Zhang

i-Code Studio: A Configurable and Composable Framework for Integrative AI (2024.emnlp-demo)

Challenge:	Existing datasets for Chinese instruction tuning are not well-aligned with Chinese users’ interaction patterns.
Approach:	They propose to use Chinese instruction tuning datasets to improve instruction fine-tuning for Chinese users.
Outcome:	The proposed dataset shows that Chinese models achieve competitive performance in diverse benchmarks.

Copied to clipboard

Yuwei Fang, Mahmoud Khademi, Chenguang Zhu, Ziyi Yang, Reid Pryzant, Yichong Xu, Yao Qian, Takuya Yoshioka, Lu Yuan, Michael Zeng, Xuedong Huang

KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering (2022.acl-long)

Challenge:	Existing frameworks for Integrative AI lack flexibility and composability to handle multimodal tasks.
Approach:	They propose a configurable framework for Integrative AI that orchestrates multiple pre-trained models to conduct complex multimodal tasks.
Outcome:	The proposed framework achieves impressive results on zero-shot multimodal tasks . it can communicate and personalize for users, and it can be used in a multimodal agent .

Copied to clipboard

Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng

Task Compass: Scaling Multi-task Pre-training with Task Prefix (2022.findings-emnlp)

Challenge:	Open-Domain Question Answering (ODQA) models typically include a retrieving module and a reading module.
Approach:	They propose a new open-domain question-answering framework that uses a knowledge-enhanced version of FiD to improve the approach.
Outcome:	The proposed model improves on ODQA benchmark datasets with less than 40% computation cost.

Copied to clipboard

Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng

LIME: Less Is More for MLLM Evaluation (2025.findings-acl)

Challenge:	Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks.
Approach:	They propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks.
Outcome:	The proposed model can be used as a foundation backbone for a wide range of tasks and as augmentation tool for data augmentation with complementary tasks.

Copied to clipboard

King Zhu, Qianbo Zang, Shian Jia, Siwei Wu, Feiteng Fang, Yizhi Li, Shuyue Guo, Tianyu Zheng, Jiawei Guo, Bo Li, Haoning Wu, Xingwei Qu, Jian Yang, Ruibo Liu, Xiang Yue, Jiaheng Liu, Chenghua Lin, Hamid Alinejad-Rokny, Min Yang, Shiwen Ni, Wenhao Huang, Ge Zhang

AiraXiv: An AI-Driven Open-Access Platform for Human and AI Scientists (2026.acl-demo)

Challenge:	Existing MLLM benchmarks and unified evaluation frameworks cannot accurately and efficiently reflect the ability of MLMLs.
Approach:	They propose a semi-automated benchmark curated using a pipeline that filters out uninformative samples and eliminates answer leakage by focusing on tasks that require image-based understanding.
Outcome:	The proposed benchmark reduces the number of samples by 76% and evaluation time by 77% while it can more effectively distinguish different models’ abilities.

Copied to clipboard

Junshu Pan, Panzhong Lu, Yixuan Weng, QiYao Sun, Fang Guo, Zijie Yang, Qiji Zhou, Yue Zhang

Challenge:	Recent advances in artificial intelligence (AI) have accelerated the growth of both human-authored and AI-generated research outputs.
Approach:	They propose an AI-driven open-access platform built on open preprints, AI-augmented analysis and review, and reader feedback.
Outcome:	The proposed platform supports human scientists through an interactive UI and AI scientists through Model Context Protocol (MCP)-based interactions.

Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data (2022.acl-long)

Copied to clipboard

Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng

Information Retrieval

Challenge:	Experimental results show that REtrieving from the traINing datA only can lead to significant gains on multiple NLG and NLU tasks.
Approach:	They propose to retrieve training instances from traINing datA and concatenate them with input to generate output.
Outcome:	The proposed method achieves state-of-the-art results on XSum, BigPatent, and CommonsenseQA.

Understanding and Bridging the Modality Gap for Speech Translation (2023.acl-long)

Copied to clipboard

Qingkai Fang, Yang Feng

OmniAlign-V: Towards Enhanced Alignment of MLLMs with Human Preference (2025.acl-long)

Challenge:	Existing methods to improve end-to-end speech translation (ST) use multitask learning, but there is always a modality gap between ST and MT due to the differences between speech and text.
Approach:	They propose a method to bridge the modality gap between ST and MT by leveraging (text) machine translation data.
Outcome:	The proposed method bridges the modality gap and achieves significant improvements over baseline in all eight directions.

Copied to clipboard

Xiangyu Zhao, Shengyuan Ding, Zicheng Zhang, Haian Huang, Maosongcao Maosongcao, Jiaqi Wang, Weiyun Wang, Xinyu Fang, Wenhai Wang, Guangtao Zhai, Hua Yang, Haodong Duan, Kai Chen

CTAP for Chinese:A Linguistic Complexity Feature Automatic Calculation Platform (2022.lrec-1)

Challenge:	Existing open-source multi-modal large language models (MLLMs) focus on enhancing foundational capabilities, leaving a significant gap in human preference alignment.
Approach:	They propose a dataset of 200K high-quality training samples featuring diverse images, complex questions, and varied response formats to improve MLLMs’ alignment with human preferences.
Outcome:	The proposed dataset of 200K high-quality training samples improves human preference alignment while maintaining or enhancing performance on standard VQA benchmarks.

Copied to clipboard

Yue Cui, Junhui Zhu, Liner Yang, Xuezhi Fang, Xiaobin Chen, Yujie Wang, Erhong Yang