Papers by Lingpeng Kong

Large Language Models are not Fair Evaluators (2024.acl-long)

Challenge:	Existing approaches to attention with bounded-memory control (ABC) have a quadratic complexity in sequence lengths, making it prohibitive for long sequences.
Approach:	They propose a new abstraction that bounds memory size to improve efficiency . they propose bounded-memory control, which connects several efficient attention variants .
Outcome:	The proposed approach outperforms existing approaches on language modeling, machine translation, and masked language model finetuning.

Copied to clipboard

Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, Zhifang Sui

How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models (2025.findings-naacl)

Challenge:	Existing evaluation frameworks that use large language models as referees are insufficient for accurately assessing their alignment with human intent.
Approach:	They propose a calibration framework to address positional bias in large language models as evaluators by manually annotating the “win/tie/lose” outcomes of responses from ChatGPT and Vicuna-13B in the Vicun A Benchmark’s question prompt.
Outcome:	The proposed framework alleviates evaluation bias, resulting in closer alignment with human judgments.

Copied to clipboard

Jiyue Jiang, Pengan Chen, Liheng Chen, Sheng Wang, Qinghang Bao, Lingpeng Kong, Yu Li, Chuan Wu

DetGPT: Detect What You Need via Reasoning (2023.emnlp-main)

Challenge:	Cantonese has scant representation in NLP research, especially compared to other languages from similarly developed regions.
Approach:	They propose to evaluate Cantonese LLM performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantonesian.
Outcome:	The proposed models will evaluate Cantonese's performance in factual generation, mathematical logic, complex reasoning, and general knowledge in Cantone.

Copied to clipboard

Renjie Pi, Jiahui Gao, Shizhe Diao, Rui Pan, Hanze Dong, Jipeng Zhang, Lewei Yao, Jianhua Han, Hang Xu, Lingpeng Kong, Tong Zhang

Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering (2023.acl-long)

Challenge:	Recent advances in the field of computer vision have enabled more effective and sophisticated interactions between humans and machines.
Approach:	They propose a reasoning-based object detection paradigm that leverages state-of-the-art multi-modal models and open-vocabulary object detectors to perform reasoning within the context of the user’s instructions and the visual scene.
Outcome:	The proposed method enables users to interact with the system using natural language instructions, allowing for a higher level of interactivity.

Copied to clipboard

Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng Kong

Low-Resource NLP

Challenge:	In-context learning is a common practice to randomly sample examples to serve as context.
Approach:	They propose a new principle for in-context learning that helps each sample find an in-constitut example organization that can derive the correct prediction.
Outcome:	The proposed method achieves 40% relative improvement over the common practice setting.

VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment (2024.emnlp-main)

Copied to clipboard

Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, Lingpeng Kong, Qi Liu

Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models (2025.findings-emnlp)

Challenge:	Large vision-language models (LVLMs) are evolving rapidly and require data with human supervision to achieve better alignment.
Approach:	They introduce VLFeedback, the first large-scale vision-language feedback dataset . they train Silkie, an LVLM fine-tuned via direct preference optimization .
Outcome:	The proposed model outperforms its base model in helpfulness, visual faithfulness, and safety metrics and exhibits enhanced resilience against red-teaming attacks.

Copied to clipboard

Jiyue Jiang, Alfred Kar Yin Truong, Yanyu Chen, Qinghang Bao, Sheng Wang, Pengan Chen, Jiuming Wang, Lingpeng Kong, Yu Li, Chuan Wu

Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning (2025.acl-long)

Challenge:	Cantonese is considered a low-resource language due to the dominance of Mandarin . rich colloquial vocabulary of Cantone, English loanwords, and code-switching characteristics add to the complexity of corpus collection and processing.
Approach:	We collect Cantonese texts from open source corpora, Hong Kong-specific forums, Wikipedia . we refine the model through supervised fine-tuning on curated Cantonesian tasks .
Outcome:	The model achieves state-of-the-art (SOTA) performance on four Cantonese benchmarks.

Copied to clipboard

Lei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan, Xiachong Feng, Yuxuan Gu, Yangfan Ye, Liang Zhao, Weihong Zhong, Baoxin Wang, Dayong Wu, Guoping Hu, Lingpeng Kong, Tong Xiao, Ting Liu, Bing Qin

BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models (2024.findings-acl)

Challenge:	Large language models (LLMs) suffer from severe hallucination issues due to the knowledge misalignment between the pre-training stage and the supervised fine-tuning stage.
Approach:	They propose a training objective with an abstention mechanism that selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.
Outcome:	The proposed model selectively rejects tokens that misalign with the desired knowledge distribution via a special [REJ] token.

Copied to clipboard

Xueliang Zhao, Xinting Huang, Tingchen Fu, Qintong Li, Shansan Gong, Lemao Liu, Wei Bi, Lingpeng Kong

A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairment (2023.acl-long)

Challenge:	Multimodal reasoning is a key capability for large vision-language models . however, the vanilla Chain-of-Thought method fails to address critical steps in multi-step reasoning tasks.
Approach:	They propose a bi-modal Behavioral Alignment method to augment multimodal reasoning . they use domain-specific language to integrate multimodal information into a precise alternative form .
Outcome:	The proposed method significantly improves GPT-4V(ision) on geometry problem solving, chess positional advantage prediction and molecular property prediction.

Copied to clipboard

Jiyue Jiang, Sheng Wang, Qintong Li, Lingpeng Kong, Chuan Wu

Dialogue Systems & Conversational Agents

Challenge:	Existing cognitive stimulation systems lack data on how to integrate emotional support and therapy principles into chit-chat dialogue systems.
Approach:	They propose a multi-source knowledge fusion method for CS dialogue to generate open-ended responses guided by the therapy principle and emotional support strategy.
Outcome:	The proposed method generates open-ended responses guided by the therapy principle and emotional support strategy of the target response.

UNComp: Can Matrix Entropy Uncover Sparsity? — A Compressor Design from an Uncertainty-Aware Perspective (2025.emnlp-main)

Copied to clipboard

Jing Xiong, Jianghan Shen, Fanghua Ye, Chaofan Tao, Zhongwei Wan, Jianqiao Lu, Xun Wu, Chuanyang Zheng, Zhijiang Guo, Min Yang, Lingpeng Kong, Ngai Wong

Better Document-Level Machine Translation with Bayes’ Rule (2020.tacl-1)

Challenge:	Deploying large language models (LLMs) for long-context inference remains challenging due to their substantial memory and computational demands.
Approach:	They propose an uncertainty-aware framework that leverages truncated matrix entropy to identify areas of low information content.
Outcome:	The proposed framework reduces the KV cache size to 4.74% of the original and achieves a 6% speedup.

Copied to clipboard

Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom, Chris Dyer

Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks (2025.coling-main)

Challenge:	Existing document translation models are based on autoregressive language models, but they are not able to be learned from monolingual documents.
Approach:	They propose to use Bayes' rule to create document translation models that can be learned from only parallel sentences and monolingual documents.
Outcome:	The proposed model outperforms existing document translation approaches and is based on a novel left-to-right beam-search algorithm.

Copied to clipboard

Qintong Li, Leyang Cui, Lingpeng Kong, Wei Bi

An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks (2022.emnlp-main)

Challenge:	Existing work uses large language models (LLMs) to evaluate natural language process tasks, but there are shortcomings in current LLMs.
Approach:	They examine the alignment between LLM evaluators and human annotators by comparing conventional and alignment tasks with different evaluation criteria.
Outcome:	The proposed models excel in general criteria, such as fluency, but face challenges with complex criteria, including numerical reasoning.

Copied to clipboard

Changlong Yu, Tianyi Xiao, Lingpeng Kong, Yangqiu Song, Wilfred Ng

FactTrack: Time-Aware World State Tracking in Story Outlines (2025.naacl-long)

Challenge:	Recent work attempts to explicitly incorporate human-defined linguistic priors into fine-tuning tasks.
Approach:	They replace parsed graphs or trees with trivial ones to investigate linguistic priors . they propose to use trivial graphs as baselines to design advanced knowledge fusion methods .
Outcome:	The use of trivial graphs improves performance in fully-supervised and few-shot settings.

Copied to clipboard

Zhiheng Lyu, Kevin Yang, Lingpeng Kong, Dan Klein

LoRA Meets Dropout under a Unified Framework (2024.findings-acl)

Challenge:	Existing language models still struggle to reason over long context windows . et al., 2022, show that long context generation is a challenge for LLMs .
Approach:	They propose a method for tracking atomic facts and addressing factual contradictions . they use a four-step pipeline to update a world state data structure for each new event .
Outcome:	The proposed method outperforms a baseline and fair method on story outlines.

Copied to clipboard

Sheng Wang, Liheng Chen, Jiyue Jiang, Boyang Xue, Lingpeng Kong, Chuan Wu

Green & Sustainable NLP

Challenge:	Parameter-efficientfinetuning (PEFT) has gained popularity as a lightweight approach for model customization.
Approach:	They propose a parameter-efficient dropout method that is overfitting-prone and parameter-freezed.
Outcome:	The proposed method is superior to existing methods and compares with transformer-specific methods.

Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation (2023.findings-acl)

Copied to clipboard

Fei Yuan, Yinquan Lu, Wenhao Zhu, Lingpeng Kong, Lei Li, Yu Qiao, Jingjing Xu

Multilinguality

Challenge:	Existing monolithic models for multilingual neural machine translation encounter parameter interference and inefficient inference for large models.
Approach:	They propose a detachable multi-way model that assigns each language to an individual branch . they use data from OPUS to build a translation benchmark covering 433 languages .
Outcome:	The proposed model outperforms existing models in OPUS and is faster than existing models.

Adaptive Semiparametric Language Models (2021.tacl-1)

Copied to clipboard

Dani Yogatama, Cyprien de Masson d’Autume, Lingpeng Kong

GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers (2024.acl-long)

Challenge:	Existing language models that use a large parametric neural network with episodic memory are not efficient.
Approach:	They propose a language model that combines a large parametric neural network with a non-parametric episodic memory component in an integrated architecture.
Outcome:	The proposed model can predict local context, short-term memory, or long-term memories on an ad hoc basis depending on the context.

Copied to clipboard

Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, Wei Bi

Long Chain-of-Thought Fine-tuning via Understanding-to-Reasoning Transition (2025.emnlp-main)

Challenge:	Large language models (LLMs) have demonstrated impressive performance across various mathematical reasoning benchmarks.
Approach:	They introduce an adversarial grade school math dataset and explore whether LLMs can be more robust when questions are slightly changed.
Outcome:	The proposed method generates and verifies each intermediate thought based on its reasoning goal and calculation result.

Copied to clipboard

Chenxin An, Zhihui Xie, Xiaonan Li, Ming Zhong, Shansan Gong, Lei Li, Jun Zhang, Jingjing Xu, Lingpeng Kong

Recipe2Plan: Evaluating Planning Abilities of LLMs for Efficient and Feasible Multitasking with Time Constraints Between Actions (2025.findings-emnlp)

Challenge:	Existing research on long-context scaling in language models has focused on managing lengthy input prompts instead of producing long outputs.
Approach:	They propose a sequence-level curriculum learning framework that shifts a model’s focus from interpreting long chain-of-thoughts to generating them.
Outcome:	Experiments on rigorous reasoning benchmarks, including AIME24 and GPQA Diamond, show that the proposed approach surpasses standard fine-tuning by over 10% while maintaining robust performance on understanding tasks.

Copied to clipboard

Zirui Wu, Xiao Liu, Jiayi Li, Lingpeng Kong, Yansong Feng

QSpec: Speculative Decoding with Complementary Quantization Schemes (2025.emnlp-main)

Challenge:	Existing evaluation benchmarks focus on single task performance, ignoring multitask planning and execution efficiency.
Approach:	They propose a benchmark framework based on real-world cooking scenarios . recipe2plan challenges agents to optimize cooking time through parallel task execution .
Outcome:	The proposed benchmarks highlight the need for improved temporal awareness and global multitasking capabilities in large language models.

Copied to clipboard

Juntao Zhao, Wenhao Lu, Sheng Wang, Lingpeng Kong, Chuan Wu

Green & Sustainable NLP

Challenge:	Quantization is widely adopted to accelerate inference and reduce memory consumption in large language models.
Approach:	They propose a quantization paradigm that decouples efficiency from quality by integrating two complementary schemes via speculative decoding.
Outcome:	The proposed approach achieves 1.64x speedup without quality degradation and outperforms state-of-the-art speculative decoding methods by 1.55x in batched settings.

Syntactic Structure Distillation Pretraining for Bidirectional Encoders (2020.tacl-1)

Copied to clipboard

Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, Phil Blunsom

Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation (2021.acl-long)

Challenge:	Textual representation learners trained on large amounts of data have been successful on downstream tasks.
Approach:	They propose a knowledge distillation strategy for injecting syntactic biases into BERT pretraining by distilling the approximate marginal distribution over words in context from the syntaktic LM.
Outcome:	The proposed method reduces relative error by 2–21% on a diverse set of structured prediction tasks.

Copied to clipboard

Zhiyong Wu, Lingpeng Kong, Wei Bi, Xiang Li, Ben Kao

Multimodality

Challenge:	Recent studies report improvements when equipping models with multimodal information, but it remains unclear whether such improvements actually come from the multimodal part.
Approach:	They propose to extend conventional text-only translation models with multimodal information by extending them with visual input.
Outcome:	The proposed models replicate similar gains as recently developed multimodal-integrated systems achieved, but learn to ignore multimodal information.

ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom (2025.emnlp-main)

Copied to clipboard

Jingqi Zhou, Sheng Wang, Jingwei Dong, Kai Liu, Lei Li, Jiahui Gao, Jiyue Jiang, Lingpeng Kong, Chuan Wu

Multimodality

Challenge:	Large vision-language models often prioritize language knowledge over image information on visual reasoning tasks, incurring performance degradation.
Approach:	They propose a visual reasoning framework that decouples vision-reasoning capabilities and multi-run proactive perception.
Outcome:	The proposed framework outperforms existing models on benchmarks for open-source and closed-source models with 13.2% performance gain.

INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation (2023.acl-long)

Copied to clipboard

Wenhao Zhu, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen

Machine Translation

Challenge:	Neural machine translation models induce a non-smooth representation space, which harms its generalization results.
Approach:	They propose a framework to smooth the representation space by adjusting neighbor representations with a small number of new parameters.
Outcome:	The proposed framework outperforms the state-of-the-art kNN-MT system with average gains of 1.99 COMET and 1.0 BLEU on four benchmark datasets.

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis (2024.findings-naacl)

Copied to clipboard

Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li

Multilinguality

Challenge:	Existing studies show that large language models (LLMs) can handle multilingual machine translation (MMT) However, the multilingual translation ability of LLMs remains under-explored.
Approach:	They evaluate eight popular LLMs including ChatGPT and GPT-4 to determine their performance in multilingual machine translation.
Outcome:	The proposed model can generate moderate translation even on zero-resource languages and cross-lingual exemplars can provide better task guidance for low-resourced translation than exemplar in the same language pairs.

Explanation Regeneration via Information Bottleneck (2023.findings-acl)

Copied to clipboard

Qintong Li, Zhiyong Wu, Lingpeng Kong, Wei Bi

Achilles-Bench: A Challenging Benchmark for Low-Resource Evaluation (2024.findings-acl)

Challenge:	Recent work builds on prompt engineering to generate free-text explanations without specific training, but they lack sufficiency and conciseness due to the prompt complexity and hallucination issues.
Approach:	They propose to generate explanations via the information bottleneck theory by polishing the single-pass output of large pretrained language models but retaining the information that supports the contents being explained by balancing two information bottle neck objectives.
Outcome:	The proposed explanations are based on the information bottleneck theory . they are able to explain black-box predictions naturally and accurately .

Copied to clipboard

Yudong Wang, Chang Ma, Qingxiu Dong, Zhifang Sui, Lingpeng Kong, Jingjing Xu

The Devil in Linear Transformer (2022.emnlp-main)

Challenge:	Existing low-resource datasets that challenge neural networks cause over-estimated performance, despite promising yet saturated results in high-res settings.
Approach:	They propose a benchmark Achilles-Bench to better evaluate the learning ability of neural networks in low-resource settings.
Outcome:	The proposed benchmarks show that even pre-trained language models show performance drops on NLP tasks.

Copied to clipboard

Zhen Qin, Xiaodong Han, Weixuan Sun, Dongxu Li, Lingpeng Kong, Nick Barnes, Yiran Zhong

Retrieved Sequence Augmentation for Protein Representation Learning (2024.emnlp-main)

Challenge:	Existing linear transformers suffer from performance degradations on various tasks and corpus.
Approach:	They propose a new linear attention that replaces scaling with a normalization to stabilize gradients and confine attention to neighbouring tokens in early layers.
Outcome:	The proposed model outperforms vanilla transformers on the long-range arena benchmark while being significantly more space-time efficient.

Copied to clipboard

Chang Ma, Haiteng Zhao, Lin Zheng, Jiayi Xin, Qintong Li, Lijun Wu, Zhihong Deng, Yang Lu, Qi Liu, Sheng Wang, Lingpeng Kong

PromptCoT: Synthesizing Olympiad-level Problems for Mathematical Reasoning in Large Language Models (2025.findings-acl)

Challenge:	Using multiple sequence alignments (MSA) to extract evolutionary knowledge is limited.
Approach:	They propose to use multiple sequence alignments to augment protein representations . they propose to employ Retrieved Sequence Augmentation to enhance protein representation learning .
Outcome:	The proposed method surpasses MSA Transformer by 5% in structural and property prediction tasks while being 373 times faster.

Copied to clipboard

Xueliang Zhao, Wei Wu, Jian Guan, Lingpeng Kong

OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows (2026.acl-long)

Challenge:	PromptCoT is a novel approach for synthesizing Olympiad-level math problems . it integrates rationale generation and mathematical concepts to generate complex problems based on concepts and rationale behind problem construction.
Approach:	They propose a method for automatically generating high-quality Olympiad-level math problems . they use mathematical concepts and the rationale behind problem construction to synthesize complex problems based on mathematical concepts .
Outcome:	The proposed method outperforms existing problem generation methods on benchmarks including GSM8K, MATH-500, and AIME2024.

Copied to clipboard

Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong

Challenge:	Existing methods for detecting unsafe mobile GUI agents are underexplored.
Approach:	They propose a mobile agent safety detection framework that integrates a formal verifier and a VLM-based contextual judge to detect system-level violations.
Outcome:	The proposed framework achieves 10%–30% improvements over existing approaches across multiple metrics.

ZeroGen: Efficient Zero-shot Learning via Dataset Generation (2022.emnlp-main)

Copied to clipboard

Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, Lingpeng Kong

ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models (2026.acl-long)

Challenge:	Existing approaches to generate training data with pre-trained language models have been found effective in various scenarios.
Approach:	They propose an unsupervised zero-shot learning method that generates a dataset from scratch and trains a tiny task model under supervision of the synthesized dataset.
Outcome:	The proposed method is annotated-free and efficient, but can provide useful insights from the perspective of data-free model-agnostic knowledge distillation and unreferenced text generation evaluation.

Copied to clipboard

Chonghan Qin, Xiachong Feng, Weitao Ma, Xiaocheng Feng, Lingpeng Kong

Generating Data for Symbolic Language with Large Language Models (2023.emnlp-main)

Challenge:	Existing memory benchmarks for LLMs evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval.
Approach:	They propose a benchmark that evaluates implicit memory using three constructs from non-declarative memory.
Outcome:	The new benchmark reframes evaluation from "what agents recall" to "what they automatically enact" no model exceeds 66% overall, with top performers far below human baselines .

Copied to clipboard

Jiacheng Ye, Chengzu Li, Lingpeng Kong, Tao Yu

Lexical Knowledge Internalization for Neural Dialog Generation (2022.acl-long)

Challenge:	Large language models (LLMs) bring performance and complexity, but they incur a large computational cost in practice.
Approach:	They propose a task-based model which uses large language models to generate symbolic language data by an informative prompt and agreement-based verifier.
Outcome:	The proposed model can generate symbolic language data with a few human demonstrations and saves a considerable amount of inference effort.

Copied to clipboard

Zhiyong Wu, Wei Bi, Xiang Li, Lingpeng Kong, Ben Kao

Dialogue Systems & Conversational Agents

Challenge:	Existing knowledge-grounded dialog models ignore the knowledge that resides in people's minds during a conversation.
Approach:	They propose to integrate lexical knowledge internally into the model's parameters instead of further conditioning them on external knowledge . they adopt contrastive learning approach and use a dictionary-based token-level lexicon retriever that requires only weak supervision.
Outcome:	The proposed model can relate J.K Rowling to Khalsa Aid with the knowledge retrieved from Wikipedia.

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022.emnlp-main)

Copied to clipboard

Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu

Cascaded Head-colliding Attention (2021.acl-long)

Challenge:	Structured knowledge grounding (SKG) uses structured knowledge to complete user requests . since inputs and outputs of SKG tasks are heterogeneous, they have been studied separately .
Approach:	They propose a framework that unifies 21 SKG tasks into a text-to-text format . they use unifiedSKG to benchmark T5 with different sizes .
Outcome:	The proposed framework unifies 21 SKG tasks into a text-to-text format . it achieves state-of-the-art performance on almost all of the 21 tasks, the authors show .

Copied to clipboard

Lin Zheng, Zhiyong Wu, Lingpeng Kong

Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play (2026.acl-long)

Challenge:	Existing frameworks for natural language processing ignore interactions among different heads, which wastes the capacity of the model.
Approach:	They propose a model which explicitly models interactions between attention heads through a hierarchical variational distribution.
Outcome:	The proposed model outperforms the baseline model on Wikitext-103 and WMT14 EN-DE on language modeling and translation tasks.

Copied to clipboard

Xiachong Feng, Deyi Yin, Xiaocheng Feng, Yi Jiang, Libo Qin, Yangfan Ye, Lei Huang, Weitao Ma, Qiming Li, Yuxuan Gu, Bing Qin, Lingpeng Kong

Reasoning Does Not Necessarily Improve Role-Playing Ability (2025.findings-acl)

Challenge:	Existing self-play approaches to developing general reasoning in language models rely on terminal game outcomes.
Approach:	They propose a game-based reasoning transfer model that addresses two barriers to reasoning transfer.
Outcome:	The proposed model improves mathematical reasoning, general reasoning, and code generation benchmarks.

Copied to clipboard

Xiachong Feng, Longxu Dou, Lingpeng Kong

SORTIE: Dependency-Aware Symbolic Reasoning for Logical Data-to-text Generation (2023.findings-acl)

Challenge:	a study compares zero-shot role-playing, reasoning-optimized LLMs, and reasoning-based LLM.
Approach:	They propose to use reasoning-optimized LLMs to improve role-playing performance . they propose to develop a chain-of-thought-based learning system that can be used to improve LLM performance if reasoning is used .
Outcome:	The proposed research compares zero-shot role-playing, role-playering with Chain-of-Thought, and reasoning-optimized LLMs.

Copied to clipboard

Xueliang Zhao, Tingchen Fu, Lemao Liu, Lingpeng Kong, Shuming Shi, Rui Yan

Multimodality

Challenge:	Existing studies on logical data-to-text generation rely on neural language models to generate the final table description, but they have difficulty working out key entities in the description.
Approach:	They propose a symbolic reasoning framework that reasons out each entity in the table description with a table-compatible programming language.
Outcome:	The proposed framework outperforms existing methods on three datasets and three backbones with an absolute improvement of 5.7%11.5% on SP-Acc.

Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling (2022.naacl-main)

Copied to clipboard

Jakob Prange, Nathan Schneider, Lingpeng Kong

DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models (2023.findings-emnlp)

Challenge:	Existing models of language understanding are based on explicit representations of hierarchical structure, but there are good reasons to doubt that they can be said to understand language in any meaningful way.
Approach:	They examine whether syntactic and semantic graph representations can complement and improve neural language modeling.
Outcome:	The proposed model outperforms pretrained models on English WSJ in perplexity and other metrics.

Copied to clipboard

Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, Lingpeng Kong

ImgTrojan: Jailbreaking Vision-Language Models with ONE Image (2025.naacl-long)

Challenge:	Existing approaches to text generation use discrete text within a continuous diffusion space, which incurs substantial computational overhead during training and results in slower sampling speeds.
Approach:	They propose a soft absorbing state that facilitates diffusion models in learning to reconstruct discrete mutations based on the underlying Gaussian space.
Outcome:	The proposed method accelerates training convergence by 4x and generates samples of similar quality 800x faster, rendering it closer to practical application.

Copied to clipboard

Xijia Tao, Shuai Zhong, Lei Li, Qi Liu, Lingpeng Kong

SEGO: Sequential Subgoal Optimization for Mathematical Problem-Solving (2024.acl-long)

Challenge:	Existing studies on the safety of large language models (LLMs) with human values have focused on the integration of multi-modal user input into these models.
Approach:	They propose a method to bypass safety constraints of large language models by using poisoned images instead of original textual captions.
Outcome:	The proposed attack bypasses safety constraints of large language models (VLMs) by replacing the original textual captions with malicious jailbreak prompts.

Copied to clipboard

Xueliang Zhao, Xinting Huang, Wei Bi, Lingpeng Kong

Numerical Reasoning

Challenge:	Large Language Models (LLMs) have demonstrated impressive capabilities across a wide range of tasks, including mathematical problem-solving.
Approach:	They propose a framework that connects the subgoal breakdown process and the probability of solving problems by identifying better subgoals with theoretical guarantees.
Outcome:	The proposed framework outperforms existing methods on two benchmarks, GSM8K and MATH, highlighting the potential of SEGO in AI-driven mathematical problem-solving.

SAVOIR: Learning Social Savoir-Faire via Shapley-based Reward Attribution (2026.findings-acl)

Copied to clipboard

Xiachong Feng, Yi Jiang, Xiaocheng Feng, Deyi Yin, Libo Qin, Yangfan Ye, Lei Huang, Weitao Ma, Yuxuan Gu, Chonghan Qin, Bing Qin, Lingpeng Kong

Can Language Models Understand Physical Concepts? (2023.emnlp-main)

Challenge:	Existing approaches to improve social intelligence of AI systems employ retrospective attributions and lack theoretical grounding.
Approach:	They propose a framework that uses Shapley values to ensure fair credit distribution with axiomatic guarantees of efficiency, symmetry, and marginality.
Outcome:	The proposed framework matches or exceeds proprietary models including GPT-4o and Claude-3.5-Sonnet.

Copied to clipboard

Lei Li, Jingjing Xu, Qingxiu Dong, Ce Zheng, Xu Sun, Lingpeng Kong, Qi Liu