Papers by Ke Tang
CoSQA: 20,000+ Web Queries for Code Search and Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Using deep neural networks to find codes is difficult . we present a dataset that includes 20,604 labels for natural language queries and codes . |
| Approach: | They introduce a contrastive learning method to enhance text-code matching . they find that CoSQA improves the accuracy of code question answering by 5.1% . |
| Outcome: | The proposed method improves the accuracy of code question answering by 5.1% and improves by 10.5% on a CodeBERT model. |
TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Mainstream approaches to aligning large language models heavily rely on human preference data. |
| Approach: | They propose a framework that fine-tunes a policy model using pairwise feedback data automatically mined from its outputs. |
| Outcome: | The proposed framework outperforms the base model with an average win rate of 69.7% across seven conversational or instruction-following datasets. |
xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, human evaluation is the most reliable way to holistically judge the quality of the dialogue. |
| Approach: | They propose to use English dialogue evaluation metrics to generalize them to other languages. |
| Outcome: | The proposed metrics outperform OpenAI’s ChatGPT in terms of average Pearson correlations over all datasets and languages. |
Towards Bridging the Reward-Generation Gap in Direct Alignment Algorithms (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, including instruction following, mathematical problem solving, and coding generation. |
| Approach: | They propose a method that truncates both preferred and dispreferred responses to match the shorter one’s length. |
| Outcome: | The proposed approach improves over standard implementations and achieves 11.8 points in AlpacaEval 2 and overall improvements across downstream tasks. |
AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models (2024.findings-emnlp)
Copied to clipboard
Jiale Cheng, Yida Lu, Xiaotao Gu, Pei Ke, Xiao Liu, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Large Language Models (LLMs) exhibit significant but subtle weaknesses, such as mistakes in instruction-following or coding tasks. |
| Approach: | They propose a framework to automatically expose weaknesses in Large Language Models (LLMs) they use three LLM-powered agents to perform comprehensive weakness identification . |
| Outcome: | The proposed framework shows that it is more effective than untargeted data augmentation methods like Self-Instruct to identify weaknesses in LLMs. |
Response-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding (2026.acl-long)
Copied to clipboard
Ke Ma, Jiaqi Tang, Bin Guo, Xueting Han, Ruonan Xu, Qingfeng He, Ziheng Wang, Xu Wang, Qifeng Chen, Zhiwen Yu, Yunhao Liu
| Challenge: | Existing methods for streaming video understanding are query-agnostic and implicitly model video evidence. |
| Approach: | They propose a framework that establishes explicit, structured alignment between the accumulated video evidence and the query’s expected response conditions via scene graphs. |
| Outcome: | The proposed model achieves more interpretable and accurate response timing decisions on both proactive and reactive tasks. |
MTP-RL: Acceleration of Reinforcement Learning Rollouts with Policy-Aligned Multi-Token Prediction (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) is widely applied to boost the performance of pretrained models, yet its training efficiency is severely constrained by rollout generation. |
| Approach: | They propose a framework that accelerates the rollout phase for diverse models by equipping a pipeline to equip the multi-layer parameter-sharing MTP for all models and an advantage-aware MTP optimization strategy. |
| Outcome: | The proposed framework achieves stable growth of acceptance length during RL training, and also accelerates RL rollouts, achieving an average 23.1%–55.3% reduction in rollout time compared to baselines. |
From Implicit Graph Encoding to Explicit Evidence: A Training-Free LLM Framework for Temporal Knowledge Graph Reasoning (2026.findings-acl)
Copied to clipboard
Guo Tang, Ke Cheng, Huiming Fan, Heng Chang, Wenxiang Zheng, Xianhao Ou, Junjia Xiang, Ming Liu, Yujun Zhou, Li Lanyu, Bing Qin
| Challenge: | Existing Large Language Models (LLMs) struggle with implicit modality alignment and suboptimal graph linearization. |
| Approach: | They propose a training-free, test-time adaptive framework that reframes TKG prediction as explicit evidence-driven reasoning. |
| Outcome: | ExE-LLM outperforms fully trained graph neural networks on four benchmarks . it achieves SOTA performance in inductive settings, significantly outperforming fully trained neural networks . |
AlignBench: Benchmarking Chinese Alignment of Large Language Models (2024.acl-long)
Copied to clipboard
Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment. |
| Approach: | They propose a multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese with 8 main categories, 683 real-scenario rooted queries and corresponding human verified references. |
| Outcome: | The benchmark uses a human-in-the-loop data curation pipeline, 683 real-scenario rooted queries and human verified references. |
Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning (2026.findings-acl)
Copied to clipboard
Haiyang Yu, Yuchuan Wu, Fan Shi, Jinghui Lu, Ke Niu, Xiaodong Ge, Minghan Zhuo, Jingqun Tang, Bin Li
| Challenge: | Existing document benchmarks focus on English printed texts or simplified Chinese . current vision-language models struggle with visual complexity and poor adaptability . |
| Approach: | They propose a benchmark to evaluate Chinese ancient documents' visual/linguistic complexity . ancient documents are valuable cultural heritage, but they face challenges in digitization and understanding . |
| Outcome: | the first benchmark for Chinese ancient documents evaluates VLMs from OCR to knowledge reasoning . ancient documents carry thousands of years of Chinese history and culture . traditional methods only scan images, while current models struggle with visual complexity . |
Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization (2024.acl-long)
Copied to clipboard
Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, Weiming Lu
| Challenge: | Large Language Models (LLMs) are designed as specific task solvers with sophisticated prompt engineering, but are inherently incapacitating to address complex dynamic scenarios. |
| Approach: | They propose an LLM-based agent with policy-level reflection and optimization that can learn from interactive experiences and progressively elevate its behavioral policy. |
| Outcome: | The proposed agent outperforms vanilla LLM and specialized models in blackjack and Texas hold’em. |
Black-Box Prompt Optimization: Aligning Large Language Models without Model Training (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models are often not well aligned with human intents, which requires additional training. |
| Approach: | They propose to use Black-Box Prompt Optimization (BPO) to perform alignments on large language models that are not well aligned with human intents. |
| Outcome: | The proposed model outperforms existing models and is model-agnostic. |
When 20 Agents Fail to Sort: The Distributed Sorting Benchmark for Scalable Multi-Agent Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | MAS-BENCH isolates coordination under explicit communication constraints . CAMOC significantly improves coordination success and efficiency across backends . |
| Approach: | They propose a distributed-sorting benchmark that isolates coordination under explicit communication constraints. |
| Outcome: | MAS-BENCH improves coordination success and efficiency across backends . CAMOC significantly improves efficiency under shared-state interaction . |
RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation (2025.emnlp-main)
Copied to clipboard
Sashuai Zhou, Weinan Gan, Qijiong Liu, Ke Lei, Jieming Zhu, Hai Huang, Yan Xia, Ruiming Tang, Zhenhua Dong, Zhou Zhao
| Challenge: | Existing methods for addressing item-level user interests are lacking in cross-domain generalization . RecBase model is domain-agnostic and can be used to enhance recommender systems' effectiveness . |
| Approach: | They propose a domain-agnostic foundational model pretrained with a recommendation-oriented objective that leverages a large-scale, heterogeneous, cross-domain corpus with unified textual representations and feature mappings to enhance cross- domain generalization. |
| Outcome: | The proposed model matches or surpasses baselines in zero-shot and cross-domain recommendation tasks on eight real-world datasets. |
HPSS: Heuristic Prompting Strategy Search for LLM Evaluators (2025.findings-acl)
Copied to clipboard
Bosi Wen, Pei Ke, Yufei Sun, Cunxiang Wang, Xiaotao Gu, Jinfeng Zhou, Jie Tang, Hongning Wang, Minlie Huang
| Challenge: | Existing efforts to optimize text evaluation prompts neglect the combinatorial impact of multiple factors, leading to insufficient optimization of the evaluation pipeline. |
| Approach: | They propose to integrate 8 key factors for evaluation prompts and integrate them into an algorithm that searches for well-behaved prompting strategies for LLM evaluators. |
| Outcome: | The proposed method outperforms existing methods and human-designed evaluation prompts on four evaluation tasks. |
CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation (2024.acl-long)
Copied to clipboard
Pei Ke, Bosi Wen, Andrew Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Existing models for NLP evaluations lack the ability to generate informative critiques in pointwise grading and pairwise comparison especially without references. |
| Approach: | They propose a method which can acquire pointwise grading critiques with pseudo references and revise these critiques via multi-path prompting to obtain informative evaluation data in different tasks and settings. |
| Outcome: | The proposed method outperforms all open-source models and even GPT-4 in system-level correlations of pointwise grading. |
CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)
Copied to clipboard
Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions. |
| Approach: | They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges. |
| Outcome: | Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4. |
Why Agents Compromise Safety Under Pressure (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on adversarial attacks, but this perspective overlooks a critical threat arising from the internal drive of the agent. |
| Approach: | They propose a new concept called Agentic Pressure which characterizes tension when compliant execution becomes infeasible. |
| Outcome: | The proposed model is able to achieve goal achievement while maintaining safety constraints. |