Papers by Lei Shi

Dolphin: Moving Towards Closed-loop Auto-research through Thinking, Practice, and Feedback (2025.acl-long)

Challenge:	Existing methods for automatic prompt optimization face two challenges: lack of diversity and semantic drift.
Approach:	They propose a framework for automatic prompt optimization that iteratively refines prompts through text gradients and selects the best prompt using perplexity.
Outcome:	The proposed framework outperforms existing prompt optimization methods and manual prompting on commonsense, mathematical, logical, temporal, and semantic reasoning benchmarks.

Copied to clipboard

Jiakang Yuan, Xiangchao Yan, Bo Zhang, Tao Chen, Botian Shi, Wanli Ouyang, Yu Qiao, Lei Bai, Bowen Zhou

Challenge:	Recent studies show that AI-assisted research methods can improve research efficiency . a closed-loop framework is used to enhance the automation level of scientific research .
Approach:	They propose a closed-loop LLM-driven framework to enhance the automation level of scientific research.
Outcome:	The proposed framework improves the efficiency of scientific research by improving data analysis, accelerating computation, and fostering novel idea generation.

ParaMac: A General Unsupervised Paraphrase Generation Framework Leveraging Semantic Constraints and Diversifying Mechanisms (2022.findings-emnlp)

Copied to clipboard

Jinxin Liu, Jiaxin Shi, Ji Qi, Lei Hou, Juanzi Li, Qi Tian

Text Generation

Challenge:	Existing unsupervised methods for paraphrase generation are weak in semantic equivalence or expression diversity.
Approach:	They propose a framework for unsupervised paraphrase generation that employs multi-aspect equivalence constraints and multi-granularity diversifying mechanisms to achieve good semantic equvalence and expressive diversity.
Outcome:	The proposed framework achieves 9.1% and 3.3% absolute gains over previous SOTA on Quora and MSCOCO and can improve to 18.0% and 4.6% on GLUE.

FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data (2024.emnlp-industry)

Copied to clipboard

Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, null Supryadi, Menglong Cui, Jiangcun Du, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, Shaolin Zhu, Deyi Xiong

TWAG: A Topic-Guided Wikipedia Abstract Generator (2021.acl-long)

Challenge:	Large language models exhibit significant performance discrepancies between high- and low-resource languages.
Approach:	They present an open-source multilingual LLM with 8 billion parameters and a multilingual instruction dataset.
Outcome:	The proposed model achieves consistent multilingual representations across languages.

Copied to clipboard

Fangwei Zhu, Shangqing Tu, Jiaxin Shi, Juanzi Li, Lei Hou, Tong Cui

Text Generation

Challenge:	Existing models view Wikipedia abstract as plain text, ignoring that it is a description of a certain entity and can be decomposed into different topics.
Approach:	They propose a model that guides Wikipedia abstract generation with topical information.
Outcome:	The proposed model outperforms baselines and is capable of generating comprehensive abstracts.

Hashing based Efficient Inference for Image-Text Matching (2021.findings-acl)

Copied to clipboard

Rong-Cheng Tu, Lei Ji, Huaishao Luo, Botian Shi, Heyan Huang, Nan Duan, Xian-Ling Mao

EventRAG: Enhancing LLM Generation with Event Knowledge Graphs (2025.acl-long)

Challenge:	Recent work on image-text matching has focused on exploring interactions between images and sentences to improve performance without considering inference efficiency.
Approach:	They propose a hashing-based efficient inference module which can be plugged into existing frameworks to speed up inference step without reducing retrieval performance.
Outcome:	The proposed module can be plugged into existing framework to speed up inference step without reducing retrieval performance.

Copied to clipboard

Zairun Yang, Yilin Wang, Zhengyan Shi, Yuan Yao, Lei Liang, Keyan Ding, Emine Yilmaz, Huajun Chen, Qiang Zhang

DARL: Encouraging Diverse Answers for General Reasoning without Verifiers (2026.findings-acl)

Challenge:	Existing approaches to text generation often neglect event structures that shape real-world narratives.
Approach:	They propose a framework that integrates structured event semantics with iterative retrieval and inference to enhance text generation.
Outcome:	Experiments on UltraDomain and MultiHopRAG show that the proposed framework outperforms baseline RAG systems in generation effectiveness, logical consistency, and multi-hop reasoning accuracy.

Copied to clipboard

Chongxuan Huang, Lei Lin, Xiaodong Shi, Wenping Hu, Ruiming Tang

Reasoning

Challenge:	Recent efforts such as RLPR have extended RLVR to general domains, enabling training on broader datasets and achieving improvements over RL PR.
Approach:	They propose a framework that encourages the generation of diverse answers within a controlled deviation range from the reference while preserving alignment with it.
Outcome:	Extensive experiments on 13 benchmarks show that DARL surpasses RLPR in both reasoning accuracy and output diversity.

Auto-Dialabel: Labeling Dialogue Data with Unsupervised Learning (D18-1)

Copied to clipboard

Chen Shi, Qi Chen, Lei Sha, Sujian Li, Xu Sun, Houfeng Wang, Lintao Zhang

Dialogue Systems & Conversational Agents

Challenge:	Existing dialog datasets rely on human labeling, which is expensive, limited in size, and in low coverage.
Approach:	They propose a framework to automatically cluster dialogue intents and slots . they collect context features, leverage an autoencoder for feature assembly, and adapt a dynamic hierarchical clustering method for intent and slot labeling.
Outcome:	The proposed framework can promote human labeling cost to a great extent and achieve good intent clustering accuracy (84.1%) it also provides reasonable and instructive slot labeling results.

Program Transfer for Answering Complex Questions over Knowledge Bases (2022.acl-long)

Copied to clipboard

Shulin Cao, Jiaxin Shi, Zijun Yao, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Zhiyuan Liu, Jinghui Xiao

Knowledge Representation

Challenge:	Program induction for complex questions over knowledge bases relies on a large number of parallel question-program pairs for the given KB, but the gold program annotations are usually lacking, making learning difficult.
Approach:	They propose an approach to leverage program annotations on rich KBs as external supervision signals to aid program induction for low-resourced KB.
Outcome:	The proposed approach outperforms SOTA methods on ComplexWebQuestions and WebQuestionSP.

Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models (2024.findings-naacl)

Copied to clipboard

Fobo Shi, Peijun Qing, Dong Yang, Nan Wang, Youbo Lei, Haonan Lu, Xiaodong Lin, Duantengchuan Li

On Tree-Based Neural Sentence Modeling (D18-1)

Challenge:	Prompt engineering is an essential technique for enhancing the abilities of large language models (LLMs) by providing explicit and specific instructions.
Approach:	They propose a new approach that uses text embeddings to obtain basis vectors by matrix decomposition and constructs a space for representing all prompts.
Outcome:	The proposed approach significantly outperforms state-of-the-art prompt paradigms on ten public reasoning benchmarks.

Copied to clipboard

Haoyue Shi, Hao Zhou, Jiaze Chen, Lei Li

Syntactic Text Processing

Challenge:	Existing tree-based sentence modeling approaches adopt syntactic parsing trees as the explicit structure prior.
Approach:	They replace parsing trees with trivial trees to study their effectiveness . they found that tree-based sentence modeling gives better results when crucial words are closer to the final representation .
Outcome:	The proposed tree-based sentences have shown better results on many downstream tasks.

ERRV: Eliciting Efficient Reasoning through Reasoning Vectors for Policy Optimization in Large Language Models (2026.findings-acl)

Copied to clipboard

Zhuowen Han, Lei Yang, Renren Jin, Dan Shi, Chenxi Sun, Deyi Xiong

Neural Machine Translation for Agglutinative Languages via Data Rejuvenation (2025.acl-srw)

Challenge:	Existing efforts to improve reasoning efficiency of large language models focus on modifying the reinforcement learning reward, such as adding length penalties.
Approach:	They propose a training framework that elicits efficient reasoning through reasoning vectors and a framework that allows the model to generate high-quality responses during reinforcement learning.
Outcome:	The proposed framework reduces reasoning length by 30% while maintaining stability, while retaining high accuracy.

Copied to clipboard

Chen Zhao, Yatu Ji, Ren Qing-Dao-Er-Ji, Nier Wu, Lei Shi, Fu Liu, Yepai Jia

Multilinguality

Challenge:	Recent years, advances in Neural Machine Translation (NMT) heavily rely on large-scale parallel corpora.
Approach:	They propose to combine fine-grained inactive sample identification with target-side rejuvenation to improve translation quality from agglutinative languages.
Outcome:	The proposed framework improves on four low-resource agglutinative language tasks.

E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning (2022.findings-acl)

Copied to clipboard

Jiangjie Chen, Rui Xu, Ziquan Fu, Wei Shi, Zhongqiao Li, Xinbo Zhang, Changzhi Sun, Lei Li, Yanghua Xiao, Hao Zhou

Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models (2026.acl-long)

Challenge:	Existing benchmarks to test word analogy do not reveal the underneath process of analogical reasoning of neural models.
Approach:	They propose an explanation benchmark for analogical reasoning using a Civil Service exam . they use a free-text explanation scheme to explain whether an analogy should be drawn .
Outcome:	The proposed benchmark is very challenging for state-of-the-art models, it is found.

Copied to clipboard

Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim

RACQC: Advanced Retrieval-Augmented Generation for Chinese Query Correction (2025.findings-emnlp)

Challenge:	Incomplete learning is widespread and heterogeneous in large language models . authors identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between SFT supervision and pre-training knowledge, internal inconsistencies within SFT data, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns.
Approach:	They propose a diagnostic-first framework that maps incomplete learning to causes . they identify five recurrent sources of incomplete learning: missing prerequisite knowledge, conflicts between supervision and pre-training knowledge, internal inconsistencies, left-side forgetting during sequential fine-tuning, and insufficient optimization for rare or complex patterns.
Outcome:	The proposed framework maps incomplete learning to causes using observable training and inference signals.

Copied to clipboard

Jinbo Su, Lingzhe Gao, Wei Li, Shihao Liu, Haojie Lei, Xinyi Wang, Yuanzhao Guo, Ke Wang, Daiting Shi, Dawei Yin

Information Retrieval

Challenge:	Large language models (LLMs) exhibit remarkable capabilities across many tasks, but face critical challenges in the CSC scenario: (1) poor generalization to rare entities in open-domain searches; and (2) failure to adapt to temporal entity variations due to static parameters, resulting in serious over-correction issues.
Approach:	They propose a Chinese Spelling Check system with RAG and multi-task learning that integrates dynamic knowledge retrieval and entity-centric RAG to address rare entities.
Outcome:	The proposed system outperforms existing baselines in the CSC task and achieves a maximum improvement of +9.92% on the search scenario benchmark and +3.2% on the general-domain dataset.

KwaiChat: A Large-Scale Video-Driven Multilingual Mixed-Type Dialogue Corpus (2025.findings-naacl)

Copied to clipboard

Xiaoming Shi, Zeming Liu, Yiming Lei, Chenkai Zhang, Haitao Leng, Chuan Wang, Qingjie Liu, Wanxiang Che, Yunhong Wang

Probabilistic Tree-of-thought Reasoning for Answering Knowledge-intensive Complex Questions (2023.findings-emnlp)

Challenge:	Currently, video-based dialogue systems rely on a single dialogue type, hindering their versatility in practical applications.
Approach:	They propose to generate video-driven multilingual mixed-type dialogues using KwaiChat . they propose to create a video-based multilingual mix of 4 dialogue types, 30 domains, 4 languages, 13 topics .
Outcome:	The proposed model performs best on KwaiChat but is not perfect in this situation.

Copied to clipboard

Shulin Cao, Jiajie Zhang, Jiaxin Shi, Xin Lv, Zijun Yao, Qi Tian, Lei Hou, Juanzi Li

Reasoning

Challenge:	Large language models (LLMs) are capable of answering knowledge-intensive complex questions with chain-of-thought reasoning.
Approach:	They propose a method to solve complex questions with a tree-of-thought approach using parametric knowledge and retrieved external knowledge to augment CoT reasoning.
Outcome:	The proposed approach outperforms SOTA methods on three Complex QA datasets under the open-domain setting.

RAAMove: A Corpus for Analyzing Moves in Research Article Abstracts (2024.lrec-main)

Copied to clipboard

Hongzheng Li, Ruojin Wang, Ge Shi, Xing Lv, Lei Lei, Chong Feng, Fang Liu, Jinkun Lin, Yangguang Mei, Linnan Xu

Think-Search-Patch: A Retrieval-Augmented Reasoning Framework for Repository-Level Code Repair (2025.emnlp-industry)

Challenge:	RAAMove is a comprehensive multi-domain corpus dedicated to the annotation of move structures in Research Article (RA) abstracts.
Approach:	They propose a multi-domain corpus dedicated to the annotation of move structures in RA abstracts.
Outcome:	The proposed corpus is based on a human-annotated dataset and a BERT-based model to verify its effectiveness.

Copied to clipboard

Bojian Xiong, Yikun Lei, Xikai Liu, Shaowei Zhang, Pengyun Zhu, Yan Liu, Yongqi Leng, Ling Shi, Meizhi Zhong, Yurong Zhang, Yan Gao, null Yiwu, Yao Hu, Deyi Xiong

Information Retrieval

Challenge:	Large language models suffer from multiple-file coding scenarios with strong inter-file dependencies . experimental results show that large language models exhibit inadequate performance in multi-file scenarios .
Approach:	They propose a retrieval-augmented reasoning framework for repository-level code repair . they use a dataset to generate standardized patches based on the key snippets .
Outcome:	The proposed framework improves retrieval accuracy and repair success on SWE-bench Lite . it surpasses models with larger size in managing extensive code contexts and fixing bugs spanning across multiple files.

SiMFy: A Simple Yet Effective Approach for Temporal Knowledge Graph Reasoning (2023.findings-emnlp)

Copied to clipboard

Zhengtao Liu, Lei Tan, Mengfan Li, Yao Wan, Hai Jin, Xuanhua Shi

Structured Data in NLP

Challenge:	Existing models for temporal knowledge graph reasoning suffer from low training efficiency and insufficient generalization ability.
Approach:	They propose a temporal knowledge graph reasoning approach that uses multilayer perceptron to model the structural dependencies of events and adopts a fixed-frequency strategy to incorporate historical frequency during inference.
Outcome:	The proposed model achieves state-of-the-art performance with faster convergence speed and better generalization ability.

SocialEval: Evaluating Social Intelligence of Large Language Models (2025.acl-long)

Copied to clipboard

Jinfeng Zhou, Yuxuan Chen, Yihan Shi, Xuanming Zhang, Leqi Lei, Yi Feng, Zexuan Xiong, Miao Yan, Xunzhi Wang, Yaru Cao, Jianing Yin, Shuai Wang, Quanyu Dai, Zhenhua Dong, Hongning Wang, Minlie Huang

Praetor: A Fine-Grained Generative LLM Evaluator with Instance-Level Customizable Evaluation Criteria (2025.acl-long)

Challenge:	Existing work on LLMs does not address their social intelligence (SI) and their discrepancy with humans.
Approach:	They propose a script-based bilingual SI benchmark that integrates outcome-oriented goal achievement evaluation and process-oriented interpersonal ability evaluation by manually crafting narrative scripts.
Outcome:	The proposed model is based on a script-based bilingual evaluation paradigm that integrates outcome- and process-oriented evaluation by manually crafting narrative scripts.

Copied to clipboard

Yongqi Leng, Renren Jin, Yue Chen, Zhuowen Han, Ling Shi, Jianxiang Peng, Lei Yang, Juesi Xiao, Deyi Xiong

Challenge:	Existing evaluation methods are inadequate to evaluate large language models (LLMs).
Approach:	They propose a fine-grained generative LLM evaluator with instance-level customazable evaluation criteria that can be used to evaluate large language models.
Outcome:	The proposed model outperforms existing LLM evaluators and instruction-tuned LLMs on multiple benchmarks and sets new SOTA results.

Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty (2026.findings-acl)

Copied to clipboard

Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan (2026.acl-long)

Challenge:	Existing approaches to generating reward models rely on voting-based mechanisms to evaluate CoT outputs.
Approach:	They propose an efficient generative reward modeling framework grounded in model-internal uncertainty.
Outcome:	The proposed framework reduces inference cost while improving answer accuracy.

Copied to clipboard

Lei Yang, Leiyu Pan, Bojian Xiong, Renren Jin, Shaowei Zhang, Yue Chen, Ling Shi, Jiang Zhou, Junru Wu, Zhen Wang, Jianxiang Peng, Juesi Xiao, Tianyu Dong, Zhuowen Han, Zhuo Chen, Yuqi Ren, Deyi Xiong

LLM-ForcedAligner: A Non-Autoregressive and Accurate LLM-Based Forced Aligner for Multilingual and Long-Form Speech (2026.acl-long)

Challenge:	Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages.
Approach:	They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English.
Outcome:	The proposed model outperforms open-source and Tibetan-focused models on diverse tasks.

Copied to clipboard

Bingshen Mu, Xian Shi, Xiong Wang, Hexin Liu, Jin Xu, Lei Xie

A Speaker-Aware Co-Attention Framework for Medical Dialogue Information Extraction (2022.emnlp-main)

Challenge:	Existing methods for forcing alignment are language-specific and prone to temporal shifts.
Approach:	They propose a slot-filling paradigm that uses time indices to predict slot positions.
Outcome:	The proposed method reduces accumulated temporal shifts by 69% compared with prior methods.

Copied to clipboard

Yuan Xia, Zhenhui Shi, Jingbo Zhou, Jiayu Xu, Chao Lu, Yehui Yang, Lei Wang, Haifeng Huang, Xia Zhang, Junwei Liu

RAP-ID: Mechanistic Prompt Injection Detection via Impostor Behavior Analysis (2026.findings-acl)

Challenge:	With the development of medical digitization, the extraction and structuring of electronic medical records (EMRs) have become challenging but fundamental tasks.
Approach:	They propose a speaker-aware dialogue encoder with multi-task learning which takes the speaker's identity into account and a co-attention fusion network to aggregate the utterance information.
Outcome:	The proposed framework outperforms the state-of-the-art methods on the public medical dialogue extraction datasets to demonstrate its superiority.

Copied to clipboard

Yuchen Yang, Lei Peng, Yujie He, Yang yu, Zhongxin Wu, Yanlei Shi

R3: End-to-End Reasoning-based Planning for Multi-step Retrosynthesis via Reinforcement Learning (2026.acl-long)

Challenge:	Existing defenses rely on externally deployed guardrail models or response inspection . current defenses depend on external guardrails or response inspecting .
Approach:	They propose a mechanistic, train-free detection framework that operates exclusively on internal state dynamics during the initial forward pass.
Outcome:	The proposed framework achieves competitive performance with significant overall improvements compared to heuristic methods.

Copied to clipboard

YiFei Wang, Qizhi Pei, Jiangtao Feng, Yuntian Shi, Yi Duan, Lihao Wang, Lei Bai, Lijun Wu, Wei-Ying Ma, Hao Zhou

Reasoning

Challenge:	Experimental results show that R3 is a superior alternative to traditional search algorithms for multistep retrosynthesis planning.
Approach:	They propose a framework that reformulates multistep retrosynthetic planning as a generative reasoning task.
Outcome:	The proposed framework achieves state-of-the-art Top-1 accuracy of 43.7% on retrobench . it leverages Large Language Models to reformulate multistep retrosynthesis as a generative reasoning task.

GeoRA: Geometry-Aware Low-Rank Adaptation for RLVR (2026.acl-long)

Copied to clipboard

Jiaying Zhang, Lei Shi, Jiguo Li, Jun Xu, Jiuchong Gao, Jinghua Hao, Renqing He

Nature-Inspired Population-Based Evolution of Large Language Models (2026.acl-long)

Challenge:	Existing parameter-efficient methods for RLVR face limitations . low-rank adaptation methods do not account for the distinct optimization dynamics .
Approach:	They propose a low-rank adaptation method tailored for RLVR that exploits the anisotropic structure of RL update subspace and extracts its principal directions via Singular Value Decomposition (SVD).
Outcome:	Experiments on large reasoning models show that GeoRA outperforms strong low-rank baselines across RLVR settings while showing stronger generalization and less forgetting on out-of-domain tasks.

Copied to clipboard

Yiqun Zhang, Peng Ye, Xiaocui Yang, Shi Feng, Shufei Zhang, Lei Bai, Wanli Ouyang, Shuyue Hu

Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model (D19-1)

Challenge:	a new framework for population-based evolution of large language models is emerging . a population-driven evolution of LLMs is a key component of evolution, authors say .
Approach:	They propose a framework that allows for population-based evolution of large language models . they start with a population of parent LLMs and allow this population to evolve .
Outcome:	The proposed framework outperforms existing methods on 12 datasets.

Copied to clipboard

Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, Tat-Seng Chua

MARS: Unleashing the Power of Speculative Decoding via Margin-Aware Verification (2026.findings-acl)

Challenge:	Entity alignment aims at integrating complementary knowledge graphs (KGs) from different sources or languages.
Approach:	They propose a semi-supervised entity alignment method by joint Knowledge Embedding model and Cross-Graph model to make better use of seed alignments to propagate over the entire graphs with KG-based constraints.
Outcome:	The proposed method can make better use of seed alignments to propagate over entire graphs with KG-based constraints.

Copied to clipboard

Jingwei Song, Xinyu Wang, Hanbin Wang, Xiaoxuan Lei, Tianyu Shi, Shixin Han, Eric Yang, Xiao-Wen Chang, Lynn Ai

Challenge:	Autoregressive large language models suffer from high inference latency due to memorybandwidth constraints.
Approach:	They propose a method that decouples generation and verification by decoupling tokens and a lightweight draft model.
Outcome:	The proposed method delivers consistent and significant speedups over state-of-the-art baselines while preserving generation quality across diverse benchmarks.

Reasoning over Hierarchical Question Decomposition Tree for Explainable Question Answering (2023.acl-long)

Copied to clipboard

Jiajie Zhang, Shulin Cao, Tingjian Zhang, Xin Lv, Juanzi Li, Lei Hou, Jiaxin Shi, Qi Tian

Question Answering

Challenge:	Existing XQA methods focus on reasoning on a single knowledge source, e.g., structured knowledge bases, unstructured corpora, etc. Existing work in XQA focuses on integrating information from heterogeneous knowledge sources.
Approach:	They propose to leverage question decomposing for heterogeneous knowledge integration by breaking down a complex question into simpler ones and selecting the appropriate knowledge source for each sub-question.
Outcome:	The proposed framework outperforms SOTA methods on complex QA datasets.

Non-Autoregressive Math Word Problem Solver with Unified Tree Structure (2023.emnlp-main)

Copied to clipboard

Yi Bin, Mengqun Han, Wenhao Shi, Lei Wang, Yang Yang, See-Kiong Ng, Heng Shen

Numerical Reasoning

Challenge:	Existing MWP solvers do not handle variants that can be derived via mathematical manipulation.
Approach:	They propose a non-autoregressive solver to present a solution expression and decode it from a given problem description.
Outcome:	The proposed solver is able to decode multiple expression variants and correct them . it is based on a unified tree structure and is available on Math23K and MAWPS.

How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answering (2024.findings-acl)

Copied to clipboard

Jinxin Liu, Shulin Cao, Jiaxin Shi, Tingjian Zhang, Lunyiu Nie, Linmei Hu, Lei Hou, Juanzi Li

FlowSearch: Advancing Deep Research with Dynamic Structured Knowledge Flow (2026.acl-long)

Challenge:	Recent studies have validated that large language models (LLMs) are capable of solving some KBQA problems, but there has been little discussion on the differences in LLMs’ proficiency in formal languages used in semantic parsing.
Approach:	They propose to evaluate the understanding and generation ability of large language models (LLMs) to deal with differently structured logical forms by examining the inter-conversion of natural and formal language through in-context learning of LLMs.
Outcome:	The proposed model can understand formal languages as well as humans, but generating correct logical forms remains a challenge.

Copied to clipboard

Yusong Hu, Runmin Ma, Yue Fan, Jinxin Shi, Zongsheng Cao, Yuhao Zhou, Jiakang Yuan, Shuaiyu Zhang, Shiyang Feng, Xiangchao Yan, Shufei Zhang, Wenlong Zhang, Lei Bai, Bo Zhang

Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition (2024.acl-long)

Challenge:	FlowSearch is a multi-agent framework that actively constructs and evolves a dynamic structured knowledge flow to drive subtask execution and reasoning.
Approach:	They propose a multi-agent framework that actively constructs and evolves a dynamic structured knowledge flow to drive subtask execution and reasoning.
Outcome:	The proposed framework achieves competitive performance on GAIA, HLE, GPQA and TRQA benchmarks and is available to download.

Copied to clipboard

Matteo Bortoletto, Constantin Ruhdorfer, Adnen Abdessaied, Lei Shi, Andreas Bulling

Dialogue Systems & Conversational Agents

Challenge:	Recent work on dialogue-based collaborative plan acquisition (CPA) suggests Theory of Mind (ToM) modelling can improve missing knowledge prediction in settings with asymmetric skill-sets and knowledge.
Approach:	They propose to use task-specific constraints to represent plans as graphs and exploit task-related constraints to improve missing knowledge prediction in CPA.
Outcome:	The proposed model improves missing knowledge prediction in contexts with asymmetric skill-sets and knowledge, but the improvements diminish . the proposed model is compared with baseline models and found to be more effective than existing models.

From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning (2025.findings-emnlp)

Copied to clipboard

Zihan Chen, Song Wang, Xingbo Fu, Chengshuai Shi, Zhenyu Lei, Cong Shen, Jundong Li

Low-Resource NLP

Challenge:	In-context learning (ICL) enables large language models to perform novel tasks without parameter updates by conditioning on a few input-output examples.
Approach:	They propose a cost-efficient two-stage pipeline that reduces reliance on LLMs for data labeling.
Outcome:	The proposed pipeline reduces reliance on LLMs for data labeling . it leverages readily available cross-task examples to prompt an LLM and pseudo-label a small set of target task instances.

GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representation (2022.emnlp-main)

Copied to clipboard

Lunyiu Nie, Shulin Cao, Jiaxin Shi, Jiuding Sun, Qi Tian, Lei Hou, Juanzi Li, Jidong Zhai

Dense Procedure Captioning in Narrated Instructional Videos (P19-1)

Challenge:	Existing approaches to neural semantic parsing are limited by the semantic gap between natural and formal languages.
Approach:	They propose a unified intermediate representation for graph query languages, named GraphQ IR, which has a natural-language-like expression that bridges the semantic gap and formally defined syntax that maintains the graph structure.
Outcome:	The proposed representation can convert user queries into graphQ IR, which can later be losslessly compiled into various downstream graph query languages.

Copied to clipboard

Botian Shi, Lei Ji, Yaobo Liang, Nan Duan, Peng Chen, Zhendong Niu, Ming Zhou

Debiasing with Sufficient Projection: A General Theoretical Framework for Vector Representations (2024.naacl-long)

Challenge:	Existing models for video dense captioning learn video segments and generate captions without considering transcripts.
Approach:	They propose a model to generate procedure captions from narrated instructional videos . they extract procedures by a cross-modality module and generate captions by encoding video frames and transcripts within each extracted procedure.
Outcome:	The proposed model can extract procedures from narrated instructional videos and generate procedure captions by encoding video frames and transcripts.

Copied to clipboard

Enze Shi, Lei Ding, Linglong Kong, Bei Jiang

TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph (2021.emnlp-main)

Challenge:	Pre-trained vector representations can inadvertently encode undesirable social biases.
Approach:	They propose a framework for reducing bias by transforming vector representations to an unbiased subspace using sufficient projection.
Outcome:	The proposed framework mitigates bias across debiasing and fairness tasks and across various vector representation types, including word embeddings and output representations of transformer models.

Copied to clipboard

Jiaxin Shi, Shulin Cao, Lei Hou, Juanzi Li, Hanwang Zhang

Explainability & Interpretability in NLP

Challenge:	Existing models infer the answer by predicting the sequential relation path or aggregating the hidden graph features.
Approach:	They propose a model which jumps between entities at multiple steps . they demonstrate that TransferNet surpasses state-of-the-art models by a large margin .
Outcome:	The proposed model surpasses state-of-the-art models on MetaQA and on other datasets.

Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models (2025.findings-emnlp)

Copied to clipboard

Matteo Bortoletto, Constantin Ruhdorfer, Lei Shi, Andreas Bulling

Dependency Parsing via Sequence Generation (2022.findings-emnlp)

Challenge:	Despite growing interest in Theory of Mind (ToM) tasks for evaluating language models, little is known about how LMs internally represent mental states of self and others.
Approach:	They propose to investigate how LMs internally represent mental states of self and others .
Outcome:	The proposed model size and finetuning significantly improve LMs’ internal representations of others’ beliefs, which are structured - not mere by-products of spurious correlations - yet brittle to prompt variations.

Copied to clipboard

Boda Lin, Zijun Yao, Jiaxin Shi, Shulin Cao, Binghao Tang, Si Li, Yong Luo, Juanzi Li, Lei Hou

Syntactic Text Processing

Challenge:	Existing methods for dependency parsing are transition-based, graph-based and sequence-to-sequence method.
Approach:	They propose to achieve dependency parsing (DP) via Sequence Generation (SG) by utilizing only the pre-trained language model without any auxiliary structures.
Outcome:	The proposed method performs well on DP benchmarks including PTB, UD2.2, SDP15 and SemEval16.

ALLabel: Three-stage Active Learning for LLM-based Entity Recognition using Demonstration Retrieval (2025.emnlp-main)

Copied to clipboard

Zihan Chen, Lei Shi, Weize Wu, Qiji Zhou, Yue Zhang

Information Extraction & Text Mining

Challenge:	Large language models (LLMs) are increasingly used to solve the entity recognition task.
Approach:	They propose a framework to select the most informative and representative samples for LLM in-context learning.
Outcome:	The proposed framework outperforms baselines on three specialized domain datasets.

Mis-prompt: Benchmarking Large Language Models for Proactive Error Handling (2025.acl-long)

Copied to clipboard

Jiayi Zeng, Yizhe Feng, Mengliang He, Wenhui Lei, Wei Zhang, Zeming Liu, Xiaoming Shi, Aimin Zhou

ATM: Adversarial Tuning Multi-agent System Makes a Robust Retrieval-Augmented Generator (2024.emnlp-main)

Challenge:	Current error-handling works are performed in a passive manner, with explicit error- handling instructions.
Approach:	They propose a new benchmark to analyze LLMs' performance on a mis-prompt benchmark and a dataset to promote further research.
Outcome:	The proposed benchmark shows that current LLMs show poor performance on proactive error handling, and that SFT improves on error handling instances.

Copied to clipboard

Junda Zhu, Lingyong Yan, Haibo Shi, Dawei Yin, Lei Sha

Robustness in NLP

Challenge:	Large language models (LLMs) are proven to benefit a lot from retrieval-augmented generation (RAG) due to noisy and fabricating content, it is inevitable that RAG systems are vulnerable to these noises and prone to respond incorrectly.
Approach:	They propose to optimize retrieval-augmented generation (RGG) with an Adversarial Tuning Multi-agent system (ATM) ATM steers the Generator to have a robust perspective of useful documents for question answering with the help of an auxiliary Attacker agent.
Outcome:	The proposed system improves the retrieval-augmented generator with an auxiliary Attacker agent and can discriminate useful documents amongst fabrications.

KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base (2022.acl-long)

Copied to clipboard

Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, Hanwang Zhang

MTRouter: Cost-Aware Multi-Turn LLM Routing with History–Model Joint Embeddings (2026.acl-long)

Challenge:	Existing benchmarks for Complex KBQA lack compositional reasoning capabilities . Existing methods for Complex questions are poor in diversity or scale .
Approach:	They propose a compositional programming language to represent the reasoning process of complex questions.
Outcome:	The proposed dataset includes around 120K diverse natural language questions . it provides a compositional and interpretable programming language to represent the reasoning process of complex questions based on the proposed model .

Copied to clipboard

Yiqun Zhang, Hao Li, Zihan Wang, Shi Feng, Xiaocui Yang, Daling Wang, Bo Zhang, Lei Bai, Shuyue Hu

AI2Agent: An End-to-End Framework for Deploying AI Projects as Autonomous Agents (2025.acl-demo)

Challenge:	Multi-turn, long-horizon tasks require dozens of sequential model calls per episode.
Approach:	They propose a cost-aware multi-turn LLM routing tool which encodes interaction history and candidate models into joint history–model embeddings and learns an outcome estimator from logged trajectories to predict turn-level model utility.
Outcome:	The proposed model reduces cost and performance by 58.7% on ScienceWorld and on Humanity’s Last Exam (HLE) and even reduces costs for held-out tasks.

Copied to clipboard

Jiaxiang Chen, Jingwei Shi, Lei Gan, Jiale Zhang, Qingyu Zhang, Dongqian Zhang, Pang Xin, Zhucong Li, Xu Yinghui

Online Iterative Self-Alignment for Radiology Report Generation (2025.acl-long)

Challenge:	a new framework automates deployment and debugging of AI projects . complexity of environment configurations, dependency conflicts, and debuggering issues hinder scalability and adoption.
Approach:	They propose an end-to-end framework that automates AI project deployment . they conducted experiments on 30 AI deployment cases to evaluate its effectiveness .
Outcome:	The proposed framework reduces deployment time and improves success rates by reducing human intervention.

Copied to clipboard

Ting Xiao, Lei Shi, Yang Zhang, HaoFeng Yang, Zhe Wang, Chenjia Bai

Text Generation

Challenge:	Existing methods for RRG rely on supervised fine-tuning based on data pairs of radiological images and corresponding radiologist-annotated reports.
Approach:	They propose a method that performs supervised fine-tuning on data pairs of radiological images and corresponding radiologist-annotated reports.
Outcome:	The proposed method surpasses existing methods and achieves state-of-the-art performance across multiple evaluation metrics.

What Factors Influence LLMs’ Judgments? A Case Study on Question Answering (2024.lrec-main)

Copied to clipboard

Lei Chen, Bobo Li, Li Zheng, Haining Wang, Zixiang Meng, Runfeng Shi, Hao Fei, Jun Zhou, Fei Li, Chong Teng, Donghong Ji

Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting (2025.findings-acl)

Challenge:	Existing studies indicate that Large Language Models perform at a level comparable to humans with advantages of speed and cost-effectiveness in different fields.
Approach:	They propose to introduce four unexplored factors and a new dimension of question difficulty to provide a more comprehensive understanding of LLMs’ judgments across varying question intricacies.
Outcome:	The proposed dimensions of question difficulty and answer quantity provide valuable insights into optimizing LLMs’ performance as judges.

Copied to clipboard

Hao Feng, Shu Wei, Xiang Fei, Wei Shi, Yingdong Han, Lei Liao, Jinghui Lu, Binghong Wu, Qi Liu, Chunhui Lin, Jingqun Tang, Hao Liu, Can Huang

Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization (2023.findings-emnlp)

Challenge:	Current document image parsing solutions rely on specialized models or generate content autoregressively.
Approach:	They propose a multimodal document image parsing model that integrates specialized models with autogeneous content generation.
Outcome:	The proposed model achieves state-of-the-art performance across diverse page-level and element-level settings while ensuring superior efficiency.

Copied to clipboard

Lei Lin, Shuangtao Li, Yafang Zheng, Biao Fu, Shan Liu, Yidong Chen, Xiaodong Shi

LEGENT: Open Platform for Embodied Agents (2024.acl-demos)

Challenge:	Recent studies show that sequence-to-sequence (seq2sequ) models struggle with compositional generalization (CG) a crucial property of human language learning is its compositional globalization (GC), the algebraic ability to understand and produce a potentially infinite number of novel combinations from known components.
Approach:	They propose a sequence-to-sequence (seq2sequ) extension which learns to compose representations of different encoder layers dynamically for different tasks.
Outcome:	The proposed model achieves competitive results on two comprehensive and realistic benchmarks, which empirically demonstrates the effectiveness of the proposed model.

Copied to clipboard

Zhili Cheng, Zhitong Wang, Jinyi Hu, Shengding Hu, An Liu, Yuge Tu, Pengkai Li, Lei Shi, Zhiyuan Liu, Maosong Sun