Papers by Ji-Rong Wen

Learning to Imagine: Visually-Augmented Natural Language Generation (2023.acl-long)

Challenge:	End-to-end generative retrieval models produce document identifiers in response to a query . however, this approach has two challenges: an overemphasis on top-1 results at the expense of overall ranking quality.
Approach:	They propose a generative retrieval model with reinforcement learning from relevance feedback to align token-level docid generation with document-level relevance estimation.
Outcome:	The proposed model aligns token-level docid generation with document-level relevance estimation.

Copied to clipboard

Tianyi Tang, Yushuo Chen, Yifan Du, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen

Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models (2024.acl-long)

Challenge:	Existing methods for natural language generation are pre-trained on text-only corpora, resulting in visual commonsense.
Approach:	They propose a method that makes pre-trained language models learn to imagine for visually-augmented natural language generation.
Outcome:	The proposed method is compatible with Transformer-based architecture.

Copied to clipboard

Changyu Chen, Xiting Wang, Ting-En Lin, Ang Lv, Yuchuan Wu, Xin Gao, Ji-Rong Wen, Rui Yan, Yongbin Li

There Are a Thousand Hamlets in a Thousand People’s Eyes: Enhancing Knowledge-grounded Dialogue with Personal Memory (2022.acl-long)

Challenge:	Despite the advances in large language models, they still face difficulties with multi-step reasoning tasks.
Approach:	They propose a method that randomly masks certain tokens within the chain of thought to improve model accuracy by 5% over standard supervised fine-tuning.
Outcome:	The proposed method improves accuracy and accuracy by 5% over standard fine-tuning with a few codes modified.

Copied to clipboard

Tingchen Fu, Xueliang Zhao, Chongyang Tao, Ji-Rong Wen, Rui Yan

CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmented Generation (2025.findings-naacl)

Challenge:	Existing methods for knowledge selection focus on relevance between knowledge and dialogue context, ignoring personal preference for knowledge.
Approach:	They propose to introduce personal memory into knowledge selection in chatbots to address personalization issue by integrating personal memory and inverse mapping into a closed loop.
Outcome:	The proposed method outperforms existing methods significantly on automatic evaluation and human evaluation.

Copied to clipboard

Yiruo Cheng, Kelong Mao, Ziliang Zhao, Guanting Dong, Hongjin Qian, Yongkang Wu, Tetsuya Sakai, Ji-Rong Wen, Zhicheng Dou

Context-Tuning: Learning Contextualized Prompts for Natural Language Generation (2022.coling-1)

Challenge:	Existing research focuses on single-turn RAG, leaving a gap in addressing multi-turn conversations . a new benchmark is designed to assess RAG systems in realistic multi-turned conversations based on Wikipedia .
Approach:	They propose a large-scale benchmark to assess RAG systems in multi-turn contexts . CORAL includes diverse information-seeking conversations automatically derived from Wikipedia . authors propose unified framework to standardize various conversational RAG methods .
Outcome:	The proposed framework supports three core tasks of conversational RAG: passage retrieval, response generation, and citation labeling.

Copied to clipboard

Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen

EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesis (2026.findings-acl)

Challenge:	Recent studies have shown that pretrained language models (PLMs) lack sufficient consideration of input semantics to generate natural language.
Approach:	They propose a continuous prompting approach to fine-tune PLMs for natural language generation by modeling an inverse generation process from output to input.
Outcome:	The proposed method fine-tunes only 0.12% of the parameters while maintaining good performance.

Copied to clipboard

Xiaoshuai Song, Haofei Chang, Guanting Dong, Yutao Zhu, Ji-Rong Wen, Zhicheng Dou

A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations (2021.acl-long)

Challenge:	Large language models (LLMs) are expected to be trained to act as agents in various real-world environments, but access to real systems is often restricted and manually built sandboxes are hard to scale.
Approach:	They propose an automated framework for scalable tool-interaction environments via programmatic synthesis that synthesizes 191 environments and about 7K scenarios and applies them to Supervised Fine-Tuning and Reinforcement Learning for Qwen3 series models.
Outcome:	The proposed framework significantly improves LLMs’ ability to solve tasks in complex environments involving multi-turn, multi-tool interactions.

Copied to clipboard

Chongyang Tao, Changyu Chen, Jiazhan Feng, Ji-Rong Wen, Rui Yan

Enhancing Parameter-efficient Fine-tuning with Simple Calibration Based on Stable Rank (2024.lrec-main)

Challenge:	Existing methods to train retrieval-based dialogue systems rely on crowd-sourced data . however, it is difficult to collect large-scale dialogues that are grounded on background knowledge .
Approach:	They propose to decompose training of knowledge-grounded response selection into three tasks . they propose to combine query-passage matching task with query-dialogue history matching task .
Outcome:	Experimental results show that the proposed model can perform comparable to existing methods . the retrieval-based system can leverage background knowledge when conversing with humans .

Copied to clipboard

Peiyu Liu, Ze-Feng Gao, Xiao Zhang, Wayne Xin Zhao, Ji-Rong Wen

TOME: A Two-stage Approach for Model-based Retrieval (2023.acl-long)

Challenge:	Existing methods for lightweight fine-tuning are ineffective in low-resource settings but fail in high-resourced settings, leading to unreliable outcomes.
Approach:	They propose a calibration strategy that takes into account the inherent variance of generalization ability in model components and potential changes during the fine-tuning process.
Outcome:	The proposed calibration improves GLUE score by 3.1 points over the previous calibration method.

Copied to clipboard

Ruiyang Ren, Wayne Xin Zhao, Jing Liu, Hua Wu, Ji-Rong Wen, Haifeng Wang

A Joint Model for Dropped Pronoun Recovery and Conversational Discourse Parsing in Chinese Conversational Speech (2021.acl-long)

Challenge:	Recent research has focused on model-based retrieval, which discards the index in the traditional retrieval model and memorizes the candidate corpora using model parameters.
Approach:	They propose a model-based retrieval approach that discards the index in the traditional retrieval model and memorizes the candidate corpora using model parameters.
Outcome:	The proposed approach eliminates the index in the traditional retrieval model and memorizes the candidate corpora using model parameters.

Copied to clipboard

Jingxuan Yang, Kerui Xu, Jun Xu, Si Li, Sheng Gao, Jun Guo, Nianwen Xue, Ji-Rong Wen

Generating Long and Informative Reviews with Aspect-Aware Coarse-to-Fine Decoding (P19-1)

Challenge:	Existing work regards dropped pronoun recovery and conversational discourse parsing as two separate tasks and tackles them separately.
Approach:	They propose a neural model for dropped pronoun recovery and conversational discourse parsing in Chinese conversational speech.
Outcome:	The proposed model outperforms the state-of-the-art models on a new dataset . the proposed model is based on linguistic and semantic information from Chinese conversational speech .

Copied to clipboard

Junyi Li, Wayne Xin Zhao, Ji-Rong Wen, Yang Song

Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation (2024.eacl-long)

Challenge:	Existing methods for review generation lack topical and syntactic characteristics of natural languages.
Approach:	They propose a review generation model that uses aspect semantics, syntactic sketch, and context information to generate a sentence and corresponding words.
Outcome:	The proposed model can generate long and informative review text for users given a product and her/his rating on it.

Copied to clipboard

Kun Zhou, Yifan Li, Xin Zhao, Ji-Rong Wen

MISC: A Mixed Strategy-Aware Model integrating COMET for Emotional Support Conversation (2022.acl-long)

Challenge:	Existing non-autoregressive (NAR) text-to-text generation methods are unable to generate coherent and fluent texts due to discrete nature of text.
Approach:	They propose to integrate discrete diffusion models (DDM) into NAR text-to-text generation and integrate BART to improve the performance.
Outcome:	The proposed method outperforms competing methods and surpasses autoregressive methods on 7 datasets.

Copied to clipboard

Quan Tu, Yanran Li, Jianwei Cui, Bin Wang, Ji-Rong Wen, Rui Yan

Debiased Contrastive Learning of Unsupervised Sentence Representations (2022.acl-long)

Challenge:	Existing methods for emotional support conversation are too coarse-grained to capture user’s instant mental state and focus on expressing empathy in the response rather than gradually reducing user’ s distress.
Approach:	They propose a model which firstly infers the user’s fine-grained emotional status and then responds skillfully using a mixture of strategy.
Outcome:	The proposed model infers the user’s fine-grained emotional status and responds skillfully using mixed-up strategy modeling.

Copied to clipboard

Kun Zhou, Beichen Zhang, Xin Zhao, Ji-Rong Wen

Evaluating Object Hallucination in Large Vision-Language Models (2023.emnlp-main)

Challenge:	Recent studies have shown that contrastive learning improves pre-trained language models to derive high-quality sentence representations.
Approach:	They propose a framework to punish false negatives and generate noise-based negatives to guarantee the uniformity of the representation space.
Outcome:	The proposed framework improves pre-trained language models while pushing apart irrelevant negatives to guarantee the uniformity of the representation space.

Copied to clipboard

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, Ji-Rong Wen

DAWN-ICL: Strategic Planning of Problem-solving Trajectories for Zero-Shot In-Context Learning (2025.naacl-long)

Challenge:	Large vision-language models (LVLMs) suffer from object hallucinations, i.e., they tend to generate objects inconsistent with the target images in the descriptions.
Approach:	They propose to integrate powerful large vision-language models (LVLMs) they propose a polling-based query method to evaluate object hallucination .
Outcome:	The proposed model can evaluate object hallucination in a more stable and flexible way.

Copied to clipboard

Xinyu Tang, Xiaolei Wang, Xin Zhao, Ji-Rong Wen

StructGPT: A General Framework for Large Language Model to Reason over Structured Data (2023.emnlp-main)

Challenge:	Existing methods to conduct in-context learning without using human-annotated demonstrations are unreliable and lead to error accumulation.
Approach:	They propose a method to conduct in-context learning without using human-annotated demonstrations.
Outcome:	The proposed method outperforms existing methods using human-annotated demonstrations.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Xin Zhao, Ji-Rong Wen

Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models (2024.findings-emnlp)

Challenge:	Experiments conducted on three types of structured data show that StructGPT greatly improves the performance of LLMs.
Approach:	They propose an iterative Reading-then-Reasoning framework to solve question answering tasks based on structured data.
Outcome:	The proposed framework improves the reasoning ability of large language models over structured data under the few-shot and zero-shot settings.

Copied to clipboard

Jie Chen, Yupeng Zhang, Bingning Wang, Xin Zhao, Ji-Rong Wen, Weipeng Chen

Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration (2024.findings-acl)

Challenge:	acquiring large amounts of high-quality data can be challenging due to data scarcity, privacy concerns, and high costs.
Approach:	They propose a method which reverses instruction-following issues caused by uniform format of synthetic data and proposes unlearning techniques to mitigate these flaws.
Outcome:	The proposed method reverses instruction-following issues caused by pattern overfitting without compromising performance on benchmarks at relatively low cost.

Copied to clipboard

Sunhao Dai, Weihao Liu, Yuqi Zhou, Liang Pang, Rongju Ruan, Gang Wang, Zhenhua Dong, Jun Xu, Ji-Rong Wen

Small Pre-trained Language Models Can be Fine-tuned as Large Models via Over-Parameterization (2023.acl-long)

Challenge:	Large Language Models (LLMs) have led to an influx of AI-generated content on the internet, transforming corpus of Information Retrieval (IR) systems from human-written to a coexistence with LLM-generated contents.
Approach:	They propose a benchmark named Cocktail that compares IR models with LLMs to find relevant documents and passages from a corpus.
Outcome:	The proposed benchmark aims to evaluate IR models in the mixed-sourced data landscape of the LLM era.

Copied to clipboard

Ze-Feng Gao, Kun Zhou, Peiyu Liu, Wayne Xin Zhao, Ji-Rong Wen

Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models (2026.acl-long)

Challenge:	Large pre-trained language models (PLMs) have shown remarkable performance in various natural language processing tasks, outperforming small PLMs by a large margin.
Approach:	They propose to scale up parameters of pre-trained language models only during fine-tuning to benefit from over-parameterization.
Outcome:	The proposed approach can significantly boost the fine-tuning performance of small PLMs and even help small PDMs outperform 3 parameterized larger ones.

Copied to clipboard

Jia Deng, Junyi Li, Xin Zhao, Jinpeng Wang, Hongyu Lu, Ji-Rong Wen

INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning (2024.acl-long)

Challenge:	Existing methods for full-attention dLLMs rely on random masking strategies that overlook intrinsic token dependencies.
Approach:	They propose an attention-guided denoising and optimization framework that aligns training and optimization with attention-derived dependencies.
Outcome:	The proposed framework outperforms state-of-the-art methods on mathematical and coding benchmarks.

Copied to clipboard

Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zheng Liu, Ji-Rong Wen, Zhicheng Dou

KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph (2025.acl-long)

Challenge:	Large language models (LLMs) have demonstrated impressive capabilities in various natural language processing tasks, but their application to information retrieval tasks is still challenging due to the infrequent occurrence of many IR-specific concepts in natural language.
Approach:	They propose to use instruction tuning to enhance LLMs' proficiency in IR tasks by combining a dataset with manually written templates to analyze the effects of instruction design, template diversity, few-shot demonstrations, and the volume of instructions.
Outcome:	The proposed model can be used to perform query understanding, document understanding, and query-document relationship understanding tasks.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Xin Zhao, Yang Song, Chen Zhu, Hengshu Zhu, Ji-Rong Wen

Knowledge Representation

Challenge:	Existing methods to design the interaction strategy between large language models and knowledge graphs (KGs) are not effective for large language model (LLM)s to solve complex tasks due to the large volume and structured format of KG data.
Approach:	They propose an LLM-based agent framework that enables small LLMs to actively make decisions over knowledge graphs.
Outcome:	The proposed framework outperforms existing methods on in-domain and out-domain datasets using 10K samples.

Enabling Lightweight Fine-tuning for Pre-trained Language Model Compression based on Matrix Product Operators (2021.acl-long)

Copied to clipboard

Peiyu Liu, Ze-Feng Gao, Wayne Xin Zhao, Zhi-Yuan Xie, Zhong-Yi Lu, Ji-Rong Wen

Large Language Model-based Human-Agent Collaboration for Complex Task Solving (2024.findings-emnlp)

Challenge:	Existing methods for pre-trained language models (PLMs) use parameter reduction techniques.
Approach:	They propose a pre-trained language model compression approach based on the matrix product operator from quantum many-body physics.
Outcome:	The proposed approach can decompose an original matrix into central tensors and auxiliary tenses . it can be applied to the original or compressed PLMs in a general way, with a lighter network .

Copied to clipboard

Xueyang Feng, Zhi-Yuan Chen, Yujia Qin, Yankai Lin, Xu Chen, Zhiyuan Liu, Ji-Rong Wen

A Neural Citation Count Prediction Model based on Peer Review Text (D19-1)

Challenge:	Recent advances in large language models have led to the development of LLM-based autonomous agents.
Approach:	They propose a Reinforcement Learning-based Human-Agent Collaboration method which trains a policy model to determine the most opportune stages for human intervention within the task-solving process.
Outcome:	The proposed method improves human-agent collaboration significantly through well-planned, limited human intervention.

Copied to clipboard

Siqing Li, Wayne Xin Zhao, Eddy Jing Yin, Ji-Rong Wen

TrendSim: Simulating Trending Topics in Social Media Under Poisoning Attacks with LLM-based Multi-agent System (2025.findings-naacl)

Challenge:	Recent studies have focused on extracting or mining useful features from the paper itself or the associated authors.
Approach:	They propose to utilize peer review data for the CCP task with a neural prediction model to learn a comprehensive semantic representation for peer review text.
Outcome:	The proposed model improves on the peer review data and hand-crafted features.

Copied to clipboard

Zeyu Zhang, Jianxun Lian, Chen Ma, Yaning Qu, Ye Luo, Lei Wang, Rui Li, Xu Chen, Yankai Lin, Le Wu, Xing Xie, Ji-Rong Wen

Robustness in NLP

Challenge:	Trending topics bring in a new channel for poisoning attacks, resulting in negative impacts on society.
Approach:	They propose an LLM-based multi-agent system to simulate trending topics in social media . they propose a time-aware interaction mechanism, centralized message dissemination, and an interactive system .
Outcome:	The proposed system simulates trending topics under poisoning attacks on social media platforms.

MVP: Multi-task Supervised Pre-training for Natural Language Generation (2023.findings-acl)

Copied to clipboard

Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen

Low-Resource NLP

Challenge:	Pre-trained language models (PLMs) have achieved remarkable success in natural language generation tasks.
Approach:	They propose to use a large-scale natural language generation corpus to pre-train a text generation model MVP in a supervised manner.
Outcome:	The proposed model outperforms BART and Flan-T5 on 13 out of 17 datasets and outperformed BART by 9.3% and FlaN-T5.

AuriSRec: Adversarial User Intention Learning in Sequential Recommendation (2024.findings-emnlp)

Copied to clipboard

Junjie Zhang, Ruobing Xie, Wenqi Sun, Leyu Lin, Xin Zhao, Ji-Rong Wen

Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models (2022.coling-1)

Challenge:	Existing work focuses on capturing user implicit preferences from historical interactions and matching them with the next behavior, instead of predicting user explicit intentions.
Approach:	They propose an adversarial user intention learning approach for sequential recommendaiton . the approach explicitly predicts user current intentions by taking historical reviews as inputs .
Outcome:	The proposed approach explicitly predicts user intentions by inferring their decision-making process as explained in target reviews.

Copied to clipboard

Ze-Feng Gao, Peiyu Liu, Wayne Xin Zhao, Zhong-Yi Lu, Ji-Rong Wen

Great~Truths~are ~Always ~Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models (2022.findings-naacl)

Challenge:	Recent results show that the mix-of-experts architecture is parameter inefficient . large-scale pre-trained language models can achieve excellent performance in many NLP tasks.
Approach:	They propose to build a parameter-efficient mix-of-experts architecture by sharing information across experts.
Outcome:	The proposed architecture increases model capacity without increasing computation costs.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Ji-Rong Wen, Xin Zhao

Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models (2021.emnlp-main)

Challenge:	Existing approaches to enhance pre-trained language models (PTMs) with a knowledge-aware graph neural network (GNN) encoder that models a commonsense knowledge graph (CSKG) can't explain how external knowledge resources improve the reasoning capacity of PTMs.
Approach:	They propose to use relation features from CSKGs to enhance the reasoning capacity of pre-trained language models (PTMs) by encoding a commonsense knowledge graph (CSKG)
Outcome:	The proposed approach reduces the parameters for encoding CSKGs and improves on five benchmarks.

Copied to clipboard

Kun Zhou, Wayne Xin Zhao, Sirui Wang, Fuzheng Zhang, Wei Wu, Ji-Rong Wen

Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models (2026.acl-long)

Challenge:	Recent studies have shown that powerful pre-trained language models can be fooled by small perturbations or intentional attacks.
Approach:	They propose a framework for fine-tuning PLMs using a masked language model and Gaussian noise to augment semantically relevant examples with sufficient diversity.
Outcome:	The proposed framework improves the robustness of pre-trained language models and alleviates performance degradation under adversarial attacks.

Copied to clipboard

Haoxiang Sun, Yingqian Min, Zhipeng Chen, Xin Zhao, Ji-Rong Wen

Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains (2020.emnlp-main)

Challenge:	Existing evaluation frameworks for large reasoning models are saturated by a lack of reliable and verifiable benchmarks.
Approach:	They propose a rigorously curated, Olympiad-level math benchmark comprising 350 problems, each with parallel English and Chinese versions.
Outcome:	The proposed benchmark unifies two evaluation paradigms and offers 150 problems formalized in Lean 4 for rigorous process-level evaluation.

Copied to clipboard

Weijie Yu, Chen Xu, Jun Xu, Liang Pang, Xiaopeng Gao, Xiaozhao Wang, Ji-Rong Wen

Representation Learning

Challenge:	Asymmetrical text matching is a fundamental problem in information retrieval and natural language processing.
Approach:	They propose a method that regularizes features vectors projected from different domains . WD-Match can be used to improve different text matching methods .
Outcome:	The proposed method outperforms existing methods and benchmarks on four datasets.

Enhancing LLM Text Detection with Retrieved Contexts and Logits Distribution Consistency (2025.emnlp-main)

Copied to clipboard

Zhaoheng Huang, Yutao Zhu, Ji-Rong Wen, Zhicheng Dou

CRSLab: An Open-Source Toolkit for Building Conversational Recommender System (2021.acl-demo)

Challenge:	Existing methods for detecting large language models (LLMs) generate fluent text, but they only use a few tokens due to the short length or insufficient information in some texts.
Approach:	They propose a method that leverages external text corpora to evaluate the difference in logit distribution of input text under retrieved human-written and LLM-rewritten contexts.
Outcome:	The proposed method achieves state-of-the-art performance in AUROC on five public datasets with three widely-used source LLMs.

Copied to clipboard

Kun Zhou, Xiaolei Wang, Yuanhang Zhou, Chenzhan Shang, Yuan Cheng, Wayne Xin Zhao, Yaliang Li, Ji-Rong Wen

A Multi-Modal Knowledge Graph for Classical Chinese Poetry (2022.findings-emnlp)

Challenge:	Existing studies on conversational recommender systems lack a unified and standardized implementation or comparison.
Approach:	They propose to use a unified framework and highly-decoupled modules to develop CRSs.
Outcome:	The proposed framework collects 6 commonly used human-annotated CRS datasets and implements 19 models that include advanced techniques such as graph neural networks and pre-training models.

Copied to clipboard

Yuqing Li, Yuxin Zhang, Bin Wu, Ji-Rong Wen, Ruihua Song, Ting Bai

ICLEval: Evaluating In-Context Learning Ability of Large Language Models (2025.coling-main)

Challenge:	Existing studies in classical Chinese poetry area focus on generation and analysis of poetry.
Approach:	They propose to integrate the visual information of words in classical Chinese poetry into a multi-modal knowledge graph.
Outcome:	The proposed model bridges the semantic gap between two modalities and achieves state-of-the-art performance on the poetry-image retrieval task.

Copied to clipboard

Wentong Chen, Yankai Lin, ZhenHao Zhou, HongYun Huang, YanTao Jia, Zhao Cao, Ji-Rong Wen

PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval (2021.findings-acl)

Challenge:	Existing evaluation frameworks focus on language abilities and knowledge, often overlooking the assessment of ICL ability.
Approach:	They propose to evaluate the ICL ability of Large Language Models (LLMs) using the ICLEval benchmark.
Outcome:	The proposed benchmark demonstrates that ICL ability is universally present in different LLMs and model size is not the sole determinant of ICL efficacy.

Copied to clipboard

Ruiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, Ji-Rong Wen

Can Large Language Models Mine Interpretable Financial Factors More Effectively? A Neural-Symbolic Factor Mining Agent Model (2024.findings-acl)

Challenge:	Recent studies only consider query-centric similarity relation when learning the dual-encoder retriever.
Approach:	They propose a query-centric and PAssage-centric approach to capture more comprehensive similarity relations for dense passage retrieval.
Outcome:	The proposed approach significantly outperforms existing models on both MSMARCO and Natural Questions datasets.

Copied to clipboard

Zhiwei Li, Ran Song, Caihong Sun, Wei Xu, Zhengtao Yu, Ji-Rong Wen

ViFT: Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models (2025.findings-emnlp)

Challenge:	Existing factor mining models are inefficient and inefficient, resulting in a significant challenge to extract interpretable factors.
Approach:	They propose a model that integrates the strengths of both neural and symbolic models for factor mining.
Outcome:	The proposed model surpasses the SOTA RankIC and RankICIR in predicting S&P 500 returns on real-world stock market data.

Copied to clipboard

Zikang Liu, Kun Zhou, Xin Zhao, Dawei Gao, Yaliang Li, Ji-Rong Wen

Finding the Dominant Winning Ticket in Pre-Trained Language Models (2022.findings-acl)

Challenge:	Visual instruction tuning is the predominant technology in eliciting multimodal task-solving capabilities of large vision-language models.
Approach:	They propose a visual instruction-free fine-tuning framework for large vision-language models . they require only text-only instructions and image caption data during training .
Outcome:	The proposed framework is based on visual instruction tuning, but requires images as input . it can achieve state-of-the-art performance on several downstream benchmarks with less training data.

Copied to clipboard

Zhuocheng Gong, Di He, Yelong Shen, Tie-Yan Liu, Weizhu Chen, Dongyan Zhao, Ji-Rong Wen, Rui Yan

MMATH: A Multilingual Benchmark for Mathematical Reasoning (2025.findings-emnlp)

Challenge:	Existing studies on pre-trained language models show that they can fine-tune parameters but achieve good downstream performance.
Approach:	They find that a dominant winning ticket takes up 0.05% of the parameters and is transferable across different tasks.
Outcome:	The proposed model can achieve comparable performance with the full-parameter model, the authors show . the dominant winning ticket takes up 0.05% of the parameters, and the model is transferable across tasks, they show - the authors conclude .

Copied to clipboard

Wenyang Luo, Xin Zhao, Jing Sha, Shijin Wang, Ji-Rong Wen

Transformer-GCRF: Recovering Chinese Dropped Pronouns with General Conditional Random Fields (2020.findings-emnlp)

Challenge:	a benchmark for multilingual complex reasoning spans 374 high-quality math problems across 10 typologically diverse languages.
Approach:	They propose a benchmark for multilingual complex reasoning across 10 languages . they show reasoning in English and answering in target languages can enhance performance .
Outcome:	The proposed benchmark demonstrates that models with high-quality reasoning can perform in multiple languages.

Copied to clipboard

Jingxuan Yang, Kerui Xu, Jun Xu, Si Li, Sheng Gao, Jun Guo, Ji-Rong Wen, Nianwen Xue

Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression (2024.acl-long)

Challenge:	Existing approaches to recover dropped pronouns ignore the dependencies between pronounes in neighboring utterances.
Approach:	They propose a framework that combines Transformer network and General Conditional Random Fields to model the dependencies between pronouns in neighboring utterances.
Outcome:	The proposed framework outperforms state-of-the-art models on three Chinese conversation datasets showing that it captures the dependencies between pronouns in neighboring utterances.

Copied to clipboard

Peiyu Liu, Ze-Feng Gao, Xin Zhao, Yipeng Ma, Tao Wang, Ji-Rong Wen

Green & Sustainable NLP

Challenge:	Existing methods to compress KV cache compromise precision or require extra data for calibration, limiting their practicality in LLM deployment.
Approach:	They propose a low-bit quantization technique based on tensor decomposition to effectively compress KV cache.
Outcome:	The proposed method reduces memory footprint and performance by 75% . it is compared with existing methods that compromise precision or require extra data for calibration .

DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy (2024.acl-long)

Copied to clipboard

Hongda Sun, Weikai Xu, Wei Liu, Jian Luan, Bin Wang, Shuo Shang, Ji-Rong Wen, Rui Yan

Distilling Rule-based Knowledge into Large Language Models (2025.coling-main)

Challenge:	Recent advances in large language models (LLMs) have revolutionized the landscape of reasoning tasks.
Approach:	They propose a new approach that rethinks the reasoning process as an evolution from indeterminacy to determinacy.
Outcome:	The proposed model surpasses all baselines on various logical reasoning benchmarks.

Copied to clipboard

Wenkai Yang, Yankai Lin, Jie Zhou, Ji-Rong Wen

Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Models (2025.emnlp-main)

Challenge:	Recent advances in large language models have broadened their applicability across diverse realworld scenarios.
Approach:	They propose to encode rule-based knowledge into large language models by using strong in-context abilities to extract the knowledge from the textual rules and then explicitly encode it into the parameters of LLMs.
Outcome:	The proposed learning paradigm is much more efficient than example-based learning in both sample size and generalization ability.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Liang Song, Xin Zhao, Bingning Wang, Weipeng Chen, Ji-Rong Wen

YuLan-Mini: Pushing the Limits of Open Data-efficient Language Model (2025.acl-long)

Challenge:	Existing work relies on training with multi-lingual ability-related data, which may not be available for low-resource languages.
Approach:	They propose a multi-lingual ability-enhanced LLM that extracts language-agnostic ability-related weights from LLMs and combine them across different languages by simple addition and subtraction operations without training.
Outcome:	The proposed approach extracts language-agnostic ability-related weights from LLMs and combine them across different languages without training.

Copied to clipboard

Hu Yiwen, Huatong Song, Jie Chen, Jia Deng, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Zican Dong, Yang Lu, Xu Miao, Xin Zhao, Ji-Rong Wen

BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models (2024.lrec-main)

Challenge:	prevailing pre-training approaches for large language models involve several complexities.
Approach:	They propose a low-cost training recipe and a robust optimization approach to mitigate training instability . they also propose synthesis, curriculum, and data selection pipelines to integrate data .
Outcome:	The proposed model achieves top-tier performance among models with similar parameter scale . it is comparable to industry-leading models that require significantly more data .

Copied to clipboard

Zican Dong, Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen

Learning Dynamic Multi-attribute Interest for Personalized Product Search (2024.findings-emnlp)

Challenge:	Existing long context models suffer from performance decline when the input text exceeds their length limit.
Approach:	They propose a multi-task long context benchmark to evaluate LLMs' long context ability using 10 datasets from 5 different NLP tasks.
Outcome:	The proposed model covers 5 domains and core capacities of large language models.

Copied to clipboard

Yutong Bai, Zhicheng Dou, Ji-Rong Wen

Zero-shot Visual Question Answering with Language Model Feedback (2023.findings-acl)

Challenge:	Existing methods to capture valuable features for Personalized product search ignore that the user’s attention varies on product attributes.
Approach:	They propose a dynamic multi-attribute interest learning model to tackle the influences from attributes to user interests.
Outcome:	The proposed model significantly improves existing methods on large-scale datasets.

Copied to clipboard

Yifan Du, Junyi Li, Tianyi Tang, Wayne Xin Zhao, Ji-Rong Wen

Web Sitemap Knowledge Can Enhance Autonomous Browsing (2026.findings-acl)

Challenge:	Existing methods for knowledge-based visual question answering are based on pre-trained language models.
Approach:	They propose a language model guided captioning approach that leverages a pre-trained language model to generate captions for an image to help answer a visual question.
Outcome:	The proposed method outperforms several competing methods on the knowledge-based VQA task and achieves comparable results to a fine-tuned VLP model.

Copied to clipboard

Yuyao Zhang, Hongyu Lu, Jiajie Jin, Hongjin Qian, Shiyu Li, Zhao Yang, Yutao Zhu, Ji-Rong Wen, Zhicheng Dou

Semantic Sentence Matching via Interacting Syntax Graphs (2022.coling-1)

Challenge:	Existing web agents suffer from limited robustness, efficiency and task success due to lack of structural understanding of websites and lack of browsing priors in pre-trained models.
Approach:	They propose an agent-oriented sitemap protocol that integrates structured website knowledge into web agents.
Outcome:	The proposed agent-oriented sitemap improves robustness, efficiency and effectiveness without extra training.

Copied to clipboard

Chen Xu, Jun Xu, Zhenhua Dong, Ji-Rong Wen

The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)

Challenge:	Extensive research efforts have been devoted to the task of matching two natural language sentences.
Approach:	They propose to embed syntactic structures into an embedding vector and combine them with other features to predict matching scores.
Outcome:	The proposed method outperforms the state-of-the-art methods on three public datasets and can interpret sentences in interpretable way.

Copied to clipboard

Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework (2025.emnlp-main)

Challenge:	a growing number of researchers are studying the hallucination issue in large language models.
Approach:	They propose a hallucination detection benchmark and a method to detect hallucines in LLMs.
Outcome:	The proposed method detects hallucinations and mitigates them using different training stages.

Copied to clipboard

Jie Chen, Jinhao Jiang, Yingqian Min, Zican Dong, Shijie Wang, Xin Zhao, Ji-Rong Wen

RAG-Critic: Leveraging Automated Critic-Guided Agentic Workflow for Retrieval Augmented Generation (2025.acl-long)

Challenge:	Large reasoning models have exhibited strong performance on complex reasoning tasks, but current test-time scaling methods rely on redundant sampling and ignore historical experience utilization.
Approach:	They propose a test-time scaling framework that coordinates three collaborative LRMs to iteratively explore and refine solutions guided by historical attempts.
Outcome:	The proposed framework surpasses strong baselines on three mathematical reasoning benchmarks, including AIME-24, AIME-25, and OlymMATH.

Copied to clipboard

Guanting Dong, Jiajie Jin, Xiaoxi Li, Yutao Zhu, Zhicheng Dou, Ji-Rong Wen

Smart-Searcher: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning (2025.findings-emnlp)

Challenge:	Recent advances in large language models (LLMs) have demonstrated remarkable performance across a wide range of downstream tasks.
Approach:	They propose a framework that leverages a critic-guided agentic workflow to improve RAG capabilities autonomously.
Outcome:	The proposed framework improves RAG capabilities autonomously by leveraging a critic-guided agentic workflow.

Copied to clipboard

Huatong Song, Jinhao Jiang, Wenqing Tian, Zhipeng Chen, Yuhuan Wu, Jiahao Zhao, Yingqian Min, Xin Zhao, Lei Fang, Ji-Rong Wen

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs (2024.acl-long)

Challenge:	Large Language Models (LLMs) are powerful but prone to hallucinations due to static knowledge. Retrieval-augmented generation (RAG) helps by injecting external information, but current methods are costly, generalize poorly, or ignore the model’s internal knowledge.
Approach:	They propose a framework to train large language models to leverage both internal and external knowledge sources.
Outcome:	The proposed framework outperforms existing methods and achieves efficient retrieval-augmented reasoning.

Copied to clipboard

Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, Ji-Rong Wen

Single LLM, Multiple Roles: A Unified Retrieval-Augmented Generation Framework Using Role-Specific Token Optimization (2025.emnlp-main)

Challenge:	Existing methods to determine the knowledge an LLM already possesses and the knowledge that requires the help of a search engine are expensive and require excessive computational costs.
Approach:	They propose a slim proxy model that detects missing knowledge in LLMs with a proxy model and use it to perform retrieval for the missing knowledge.
Outcome:	The proposed approach detects missing knowledge in LLMs with a slim proxy model and takes its answers as heuristic answers.

Copied to clipboard

Yutao Zhu, Jiajie Jin, Hongjin Qian, Zheng Liu, Zhicheng Dou, Ji-Rong Wen

LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models (2026.acl-long)

Challenge:	Existing studies have optimized retrieval-augmented generation (RAG) across sub-tasks, but integrating these optimizations into a unified framework remains challenging.
Approach:	They propose a unified retrieval-augmented generation framework that optimizes role-specific tokens for multi-task processing.
Outcome:	The proposed framework achieves efficient multi-task processing through role-specific token optimization.

Copied to clipboard

Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Zhou, Yankai Lin, Ji-Rong Wen, Chongxuan Li

RLSeek: Evidence-Grounded Reasoning for RAG Hallucination Detection (2026.acl-long)

Challenge:	Masked diffusion language models have achieved significant progress in language modeling . however, the systematic analysis and empirical validation of their alignment on general tasks remains underexplored.
Approach:	They propose a framework that analyzes the bias and variance of preference optimization loss and gradient based on Direct Preference Optimization.
Outcome:	The proposed model outperforms its SFT-only predecessor on general benchmarks . it consistently outperformed other strong language models and ARMs on general tasks .

Copied to clipboard

Zhaoheng Huang, Dacheng Wen, Yutao Zhu, Xiaoying Lian, Yushi Liang, Kai Hao, Nan Li, Liangjie Zhang, Qi Zhang, Ji-Rong Wen, Zhicheng Dou, Fangzhao Wu

Towards Effective and Efficient Continual Pre-training of Large Language Models (2025.acl-long)

Challenge:	Recent work addresses this problem by training span-level hallucination detectors using reinforcement learning and chain-of-thought reasoning.
Approach:	They propose a framework that explicitly enforces active evidence seeking during CoT reasoning by requiring quotation of relevant source segments at each verification step.
Outcome:	The proposed framework improves hallucination span detection performance with limited reasoning overhead and improved robustness in out-of-domain settings.

Copied to clipboard

Jie Chen, Zhipeng Chen, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Yingqian Min, Xin Zhao, Zhicheng Dou, Jiaxin Mao, Yankai Lin, Ruihua Song, Jun Xu, Xu Chen, Rui Yan, Zhewei Wei, Di Hu, Wenbing Huang, Ji-Rong Wen

An Analysis and Mitigation of the Reversal Curse (2024.emnlp-main)

Challenge:	Continual pre-training (CPT) has been an important approach for adapting language models to specific domains or tasks.
Approach:	They propose a Continual pre-training method that can greatly improve Chinese language ability and scientific reasoning ability of LLMs.
Outcome:	The proposed method can greatly improve Chinese language ability and scientific reasoning ability of LLMs.

Copied to clipboard

Ang Lv, Kaiyi Zhang, Shufang Xie, Quan Tu, Yuhan Chen, Ji-Rong Wen, Rui Yan

Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation (2022.naacl-main)

Challenge:	Recent research observes a phenomenon in large language models called the "reversal curse" when dealing with two entities, LLMs excel in handling sequences in the form of "aRb" but when asked "who is Mary Lee Pfeiffer's son?" the LLM exhibits considerable confusion and fails to provide a as the answer .
Approach:	They conduct the first-ever study of how the reversal curse happens in large language models . they find that LLMs excel in handling sequences in the form of "aRb" but struggle to provide a satisfactory answer when asked "who is Mary Lee Pfeiffer's son?"
Outcome:	The proposed study shows that the reversal curse can stem from specific training objectives . the study also shows that a reverse query can be difficult to understand .

Copied to clipboard

Hanxun Zhong, Zhicheng Dou, Yutao Zhu, Hongjin Qian, Ji-Rong Wen

Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation (2025.coling-main)

Challenge:	Existing personalized dialogue systems extract user profiles from dialogue history to guide personalized response generation.
Approach:	They propose to refine the user dialogue history on a large scale to obtain more persona information from the dialogue history and leverage other similar users' data to enhance personalization.
Outcome:	The proposed model can handle more dialogue history and obtain more abundant and accurate persona information.

Copied to clipboard

Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hua Wu, Ji-Rong Wen, Haifeng Wang

OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain (2025.emnlp-main)

Challenge:	Large language models (LLMs) have shown impressive prowess in solving a wide range of tasks with world knowledge, but it remains unclear how well they perceive their factual knowledge boundaries.
Approach:	They propose to use a retrieval augmentation approach to enhance LLMs' awareness of factual knowledge boundaries to analyze factual and factual information in open-domain question answering (QA)
Outcome:	The proposed method improves LLMs’ QA and judgemental capabilities by integrating supporting documents with the questions.

Copied to clipboard

Shuting Wang, Jiejun Tan, Zhicheng Dou, Ji-Rong Wen

Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study (2024.lrec-main)

Challenge:	a new benchmark for RAG is developed for the financial domain . omnidirectional and automatic benchmarks are difficult to build in vertical domains .
Approach:	They propose an omnidirectional and automatic RAG benchmark for the financial domain . they categorize RAG scenarios by task classes and 16 financial topics .
Outcome:	The proposed benchmark achieves an 87.47% acceptance ratio in human evaluations of generated instances.

Copied to clipboard

Peiyu Liu, Zikang Liu, Ze-Feng Gao, Dawei Gao, Wayne Xin Zhao, Yaliang Li, Bolin Ding, Ji-Rong Wen

RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking (2021.emnlp-main)

Challenge:	Large Language Models (LLMs) require significant computational resources for deployment and use.
Approach:	They propose to use low-bit quantization methods to reduce memory footprint and increase inference rate to improve performance of Large Language Models.
Outcome:	The proposed methods can reduce the memory footprint and increase the inference rate of LLMs.

Copied to clipboard

Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, Ji-Rong Wen

Enhancing Scalability of Pre-trained Language Models via Efficient Parameter Sharing (2023.findings-emnlp)

Challenge:	Recent studies show that passage retrieval and passage reranking are important for achieving mutual improvement.
Approach:	They propose a unified listwise training approach for passage retrieval and passage reranking that incorporates a retrieval procedure and a hybrid data augmentation strategy.
Outcome:	The proposed approach improves on both MSMARCO and Natural Questions datasets.

Copied to clipboard

Peiyu Liu, Ze-Feng Gao, Yushuo Chen, Xin Zhao, Ji-Rong Wen

Experience-Driven Reflective Co-Evolution of Prompts and Heuristics for Autonomous Algorithm Design (2026.findings-acl)

Challenge:	Existing approaches to scale pre-trained language models to a deeper model depth share all parameters or use extra blocks.
Approach:	They propose a parameter-efficient approach to scaling pre-trained language models to a deeper model depth using matrix product operator.
Outcome:	The proposed model scales pre-trained language models to a deeper model depth by 4x and achieves 0.1 points higher than BERT-large for GLUE score.

Copied to clipboard

Yihong Liu, Junyi Li, Hongyu Lu, Xin Zhao, Ji-Rong Wen

Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network (2022.acl-long)

Challenge:	Combinatorial optimization has long been dominated by manually engineered heuristics, which require substantial expert intuition and implementation overhead.
Approach:	They propose a framework that couples an island migration model with elite selection to maintain population diversity.
Outcome:	The proposed framework achieves superior accuracy on the Traveling Salesman and Bin Packing Problems.

Copied to clipboard

Zheng Gong, Kun Zhou, Xin Zhao, Jing Sha, Shijin Wang, Ji-Rong Wen

LLM-Generated Text May Harm Your Retrieval! A Robust Detection Strategy for Retrieval-Augmented Generation (2026.acl-long)

Challenge:	a fundamental challenge in modeling math problems is how to fuse semantics of textual description and formulas.
Approach:	They propose a method to continually pre-train language models for improving understanding of math problems with syntax-aware memory networks.
Outcome:	The proposed approach outperforms competitive baselines on four math tasks.

Copied to clipboard

Zhaoheng Huang, Yutao Zhu, Ji-Rong Wen, Zhicheng Dou

Optimal Partial Transport Based Sentence Selection for Long-form Document Matching (2022.coling-1)

Challenge:	Retrieval-augmented generation (RAG) improves accuracy and timeliness of large language models, but external corpora may become contaminated with LLM-generated texts.
Approach:	They propose a method that integrates external knowledge retrieved from external sources into RAG to filter out LLM-generated texts from retrieved results.
Outcome:	The proposed method mitigates performance degradation and improves stability of RAG systems.

Copied to clipboard

Weijie Yu, Liang Pang, Jun Xu, Bing Su, Zhenhua Dong, Ji-Rong Wen

Challenge:	Existing methods for document matching are limited by the partial nature of the sentence-level matching signals.
Approach:	They propose a matching approach that equips existing document matching models with an Optimal Partial Transport component, namely OPT-Match, which selects the key sentences that play a major role in matching.
Outcome:	The proposed approach outperforms existing models on four publicly available datasets and the key sentences selected by it are consistent with human-provided rationales.

Learning to Transfer Prompts for Text Generation (2022.naacl-main)

Copied to clipboard

Junyi Li, Tianyi Tang, Jian-Yun Nie, Ji-Rong Wen, Xin Zhao

Towards Topic-Guided Conversational Recommender System (2020.coling-main)

Challenge:	Pretrained language models (PLMs) have made remarkable progress in text generation tasks via fine-tuning.
Approach:	They propose a prompt-based method that learns source prompts and transfers them as target prompts to perform target generation tasks.
Outcome:	The proposed method can be used to perform text generation tasks in a transferable setting.

Copied to clipboard

Kun Zhou, Yuanhang Zhou, Wayne Xin Zhao, Xiaoke Wang, Ji-Rong Wen

REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering (2024.emnlp-main)

Challenge:	Existing CRS datasets focus on immediate requests from users, while lack proactive guidance to the recommendation scenario.
Approach:	They propose a topic-guided conversational recommendation dataset . it incorporates topic threads to enforce natural semantic transitions towards the recommendation scenario .
Outcome:	The proposed approach is more reasonable and controllable than previous approaches.

Copied to clipboard

Yuhao Wang, Ruiyang Ren, Junyi Li, Xin Zhao, Jing Liu, Ji-Rong Wen

Domain Adaptation for Person-Job Fit with Transferable Deep Global Match Network (D19-1)

Challenge:	Existing methods to extend knowledge scope of large language models (LLMs) lack internal parametric knowledge, resulting in misusing external knowledge.
Approach:	They propose a retrieval-augmented approach that provides LLMs with potentially relevant documents through a module.
Outcome:	The proposed approach outperforms existing methods on four open-domain QA tasks.

Copied to clipboard

Shuqing Bian, Wayne Xin Zhao, Yang Song, Tao Zhang, Ji-Rong Wen

Improving Retrospective Language Agents via Joint Policy Gradient Optimization (2025.naacl-long)

Challenge:	Existing methods to solve person-job fit in single-domain setting are limited by labeled data.
Approach:	They propose a deep global match network for capturing the global semantic interactions between two sentences from a job posting and a candidate resume respectively.
Outcome:	The proposed model is effective when there is not enough labeled data.

Copied to clipboard

Xueyang Feng, Bo Lan, Quanyu Dai, Lei Wang, Jiakai Tang, Xu Chen, Zhenhua Dong, Ji-Rong Wen

Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models (2023.emnlp-main)

Challenge:	Recent advances in large language models have sparked interest in creating autonomous agents.
Approach:	They propose a framework that jointly optimizes both task-planning and self-reflective evolution capabilities in language agents.
Outcome:	The proposed framework improves task planning and self-reflective evolution capabilities in language agents.

Copied to clipboard

Xiaolei Wang, Xinyu Tang, Xin Zhao, Jingyuan Wang, Ji-Rong Wen

ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models (2023.findings-emnlp)

Challenge:	Existing evaluation protocols for large language models (LLMs) are inadequate for conversational recommender systems.
Approach:	They propose an evaluation approach based on LLMs that harnesses LLM-based user simulators to evaluate ChatGPT's performance.
Outcome:	The proposed evaluation approach can simulate various system-user interaction scenarios.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Beichen Zhang, Zheng Gong, Xin Zhao, Ji-Rong Wen

Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models (2024.acl-long)

Challenge:	Large language models (LLMs) have excellent performance in evaluation benchmarks, but struggle in complex reasoning tasks.
Approach:	They propose a tool-augmented chain-of-thought reasoning framework for chat-based LLMs . they model chain- of-thoughting reasoning as multi-turn conversations to utilize tools .
Outcome:	The proposed framework can outperform state-of-the-art models on complex reasoning tasks.

Copied to clipboard

Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, Ji-Rong Wen

Neuro-Symbolic Query Compiler (2025.findings-acl)

Challenge:	Despite the impressive multilingual capabilities demonstrated by LLMs, the understanding of how these abilities develop and function remains nascent.
Approach:	They propose a novel detection method to pinpoint language-specific neurons within LLMs by selectively activating or deactivating these neurons.
Outcome:	The proposed method can “steer” the output language of LLMs by selectively activating or deactivating language-specific neurons.

Copied to clipboard

Yuyao Zhang, Zhicheng Dou, Xiaoxi Li, Jiajie Jin, Yongkang Wu, Zhonghua Li, Ye Qi, Ji-Rong Wen

A Thorough Examination on Zero-shot Dense Retrieval (2023.findings-emnlp)

Challenge:	Retrieval-Augmented Generation (RAG) systems are limited in their ability to process information in open-source environments.
Approach:	They propose a neuro-symbolic framework inspired by linguistic grammar rules and compiler design to formalize complex queries using a minimal yet sufficient Backus-Naur Form grammar.
Outcome:	The proposed framework is based on a backus-naur form grammar and compiler design that maintains completeness while minimizing redundancy.

Copied to clipboard

Ruiyang Ren, Yingqi Qu, Jing Liu, Xin Zhao, Qifei Wu, Yuchen Ding, Hua Wu, Haifeng Wang, Ji-Rong Wen

HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023.emnlp-main)

Challenge:	Recent advances in dense retrieval (DR) models have been shown to be not as competitive as traditional sparse retrieval models in a zero-shot retrieval setting.
Approach:	They propose to examine the zero-shot capability of DR models by analyzing key factors related to source training set and potential bias from target dataset.
Outcome:	The proposed model is not as competitive as sparse retrieval models in a zero-shot retrieval setting.

Copied to clipboard

Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Progressive Multimodal Reasoning via Active Retrieval (2025.acl-long)

Challenge:	Existing large language models (LLMs) are prone to generate hallucinations . a recent study shows that LLMs are able to generate content that conflicts with the source or cannot be verified by factual knowledge.
Approach:	They propose a framework to evaluate the performance of large language models (LLMs) they propose to use a sample of generated and human-annotated hallucinated samples to evaluate their performance .
Outcome:	The proposed framework generates and annotates hallucinated samples from ChatGPT . the results show that existing LLMs face great challenges in recognizing hallucines .

Copied to clipboard

Guanting Dong, Chenghao Zhang, Mengjie Deng, Yutao Zhu, Zhicheng Dou, Ji-Rong Wen

LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)

Challenge:	Existing approaches to improve multimodal large language models' reasoning performance are limited.
Approach:	They propose a framework to progressively improve multimodal reasoning capabilities . they propose active retrieval and Monte Carlo tree search to improve MLLMs' reasoning .
Outcome:	The proposed framework improves multimodal reasoning capabilities in multimodal large language models.

Copied to clipboard

Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen

BASES: Large-scale Web Search User Simulation with Large Language Model based Agents (2024.findings-emnlp)

Challenge:	a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented.
Approach:	They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs).
Outcome:	The proposed library is based on extensive experiments in a variety of evaluation settings.

Copied to clipboard

Ruiyang Ren, Peng Qiu, Yingqi Qu, Jing Liu, Xin Zhao, Hua Wu, Ji-Rong Wen, Haifeng Wang

The Web Can Be Your Oyster for Improving Language Models (2023.findings-acl)

Challenge:	Existing research on web search rely on real-user experiments, which can be costly to scale up.
Approach:	They propose a user simulation framework with LLM-based agents that can generate unique user profiles at scale.
Outcome:	The proposed framework can generate unique user profiles at scale, leading to diverse search behaviors.

Copied to clipboard

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jingyuan Wang, Jian-Yun Nie, Ji-Rong Wen

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph (2023.emnlp-main)

Challenge:	Pretrained language models encode a large amount of knowledge, but knowledge is frozen at the time of training, and the models become static and limited by training data.
Approach:	They propose an adaptive search engine assisted learning method that can self-evaluate the confidence level of PLM’s predictions and adaptively determine when to refer to the web for more data.
Outcome:	The proposed model outperforms retrieval-augmented methods on 16 knowledge-intensive tasks on a wide range of knowledge-related tasks.

Copied to clipboard

Jinhao Jiang, Kun Zhou, Xin Zhao, Yaliang Li, Ji-Rong Wen

What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning (2025.coling-main)

Challenge:	Question Answering over Knowledge Graph (KGQA) aims to find answer entities for natural language questions based on knowledge graphs.
Approach:	They propose a subgraph-aware self-attention mechanism to imitate the graph neural network (GNN) based module to perform multi-hop reasoning on KG.
Outcome:	The proposed method surpasses state-of-the-art models by a large margin even with fewer updated parameters and less training data.

Copied to clipboard

Yifan Du, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Jinpeng Wang, Chuyuan Wang, Mingchen Cai, Ruihua Song, Ji-Rong Wen

SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis (2025.findings-emnlp)

Challenge:	Experimental results show that visual instruction tuning improves performance of Multi-modal Large Language Models (MLLMs) to extend the application scope of Large Language Modells, a surge of work augments LLMs with vision encoders to endow the ability of multi-modal cognition and reasoning.
Approach:	They propose a systematic approach to create high-quality visual reasoning instructions using a synthesize-complicate-reformulate paradigm.
Outcome:	The proposed method improves performance of MLLMs by 27.86% and 27.60% on MME-Perception and MME Cognition.

Copied to clipboard

Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Fei Bai, Jia Deng, Xin Zhao, Zheng Liu, Lei Fang, Zhongyuan Wang, Ji-Rong Wen

ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting (2024.lrec-main)

Challenge:	Existing approaches to deep search training lack high-quality training trajectories, prohibitive computational costs and lack of high-fidelity training data.
Approach:	They propose a framework that synthesizes high-quality training data by simulating real user interactions in live web search environments.
Outcome:	The proposed framework synthesizes high-quality training data by simulating user interactions in live web search environments.

Copied to clipboard

Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen

Visually-augmented pretrained language models for NLP tasks without images (2023.acl-long)

Challenge:	Existing CoT synthesis approaches focus on simpler reasoning tasks and result in inconsistent CoT prompts.
Approach:	They propose a framework for automatic generation of superior CoT prompts based on three major evolution strategies . they propose 'step-level debating' method where multiple debaters discuss each reasoning step to arrive at the correct answer.
Outcome:	The proposed framework can generate superior CoT prompts from a CoT dataset.

Copied to clipboard

Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Qinyu Zhang, Ji-Rong Wen

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint (2024.findings-acl)

Challenge:	Existing approaches to improve pre-trained language models lack visual commonsense and semantics.
Approach:	They propose a visual-augmented approach to fine-tune pre-trained language models by using retrieved or generated images instead of relying on explicit images.
Outcome:	The proposed approach outperforms baselines on ten tasks and consistently outperformed other approaches.

Copied to clipboard

Zhipeng Chen, Kun Zhou, Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, Ji-Rong Wen

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment (2024.emnlp-main)

Challenge:	Existing reinforcement learning methods do not provide fine-grained supervision for complex reasoning tasks.
Approach:	They propose a reinforcement learning method that incorporates a generative model as the reward model and a token-level supervision model for RL training.
Outcome:	Experiments on 8 tasks show the proposed method is effective .

Copied to clipboard

Zhipeng Chen, Kun Zhou, Xin Zhao, Jingyuan Wang, Ji-Rong Wen

Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking (2025.findings-acl)

Challenge:	Experimental results show that large language models are struggling to align with human preference in complex tasks and scenarios.
Approach:	They propose a low-redundant alignment method that selects the top-10% most updated parameters in LLMs for alignment training.
Outcome:	The proposed method improves on 10 datasets and shows that it is redundant . it can be used to train LLMs on QA and ECQA datasets, but it is not feasible to test it on a large dataset.

Copied to clipboard

Xiaoxue Cheng, Junyi Li, Xin Zhao, Ji-Rong Wen

SSP: Self-Supervised Post-training for Conversational Search (2023.findings-acl)

Challenge:	Existing studies have focused on the issue of hallucination in large language models.
Approach:	They propose a framework that allows an explicit slow thinking generation process for mitigating hallucinations during inference.
Outcome:	The proposed framework outperforms baseline approaches on English and Chinese datasets.

Copied to clipboard

Quan Tu, Shen Gao, Xiaolong Wu, Zhao Cao, Ji-Rong Wen, Rui Yan

Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector (2024.emnlp-main)

Challenge:	Existing methods to search for information are limited by data scarcity and lack contextual semantic understanding.
Approach:	They propose a new post-training paradigm with three self-supervised tasks to initialize the conversational search model to enhance dialogue structure and contextual semantic understanding.
Outcome:	The proposed method can initialize the conversational search model to enhance dialogue structure and contextual semantic understanding.

Copied to clipboard

Xiaoxue Cheng, Junyi Li, Xin Zhao, Hongzhi Zhang, Fuzheng Zhang, Di Zhang, Kun Gai, Ji-Rong Wen

Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models (2021.findings-acl)

Challenge:	Existing studies on hallucination detection rely heavily on closed-source LLMs such as GPT-4.
Approach:	They propose an LLM-based agent framework called HaluAgent that integrates LLMs, multi-functional toolbox and a memory mechanism for hallucination detection.
Outcome:	The proposed framework integrates the LLM, multi-functional toolbox, and can detect hallucinations on Chinese and English datasets.

Copied to clipboard

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Zhicheng Wei, Nicholas Jing Yuan, Ji-Rong Wen

TextBox: A Unified, Modularized, and Extensible Framework for Text Generation (2021.acl-demo)

Challenge:	Existing models for KG-to-text generation are based on pretrained language models.
Approach:	They propose to automatically generate a text that describes the facts in knowledge graph (KG) they leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation.
Outcome:	The proposed model outperforms all comparison methods on fully-supervised and fewshot settings.

Copied to clipboard

Junyi Li, Tianyi Tang, Gaole He, Jinhao Jiang, Xiaoxuan Hu, Puzhao Xie, Zhipeng Chen, Zhuohao Yu, Wayne Xin Zhao, Ji-Rong Wen

Text Generation

Challenge:	TextBox is an open-source text generation framework that is modularized and extensible.
Approach:	They propose to provide a unified, modularized, and extensible text generation framework that implements 21 text generation models on 9 benchmark datasets.
Outcome:	The proposed framework implements 21 models on 9 benchmark datasets and is available under the Apache License 2.0 license.

ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models (2022.naacl-main)

Copied to clipboard

Junyi Li, Tianyi Tang, Zheng Gong, Lixin Yang, Zhuohao Yu, Zhipeng Chen, Jingyuan Wang, Xin Zhao, Ji-Rong Wen

ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation (2022.emnlp-main)

Challenge:	Recent years have featured a trend towards Transformer based pretrained language models (PLMs) in natural language processing systems.
Approach:	They propose to use four evaluation dimensions to evaluate ten widely-used PLMs . they find that pretrained language models are good at different ability tests .
Outcome:	The results show that pretrained language models are good at different ability tests and have excellent transferability between tasks.

Copied to clipboard

Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen