Papers by Tong Sun

Enhancing Self-Attention with Knowledge-Assisted Attention Maps (2022.naacl-main)

Challenge:	Pun memes combine wordplay with visual elements to create humor, irony, or other rhetorical effects.
Approach:	They propose a benchmark to assess Chinese pun memes' processing capabilities across three progressive tasks: pun meme detection, sentiment analysis, and chat-driven meme response.
Outcome:	The proposed model can detect pun memes, analyze sentiments, and respond to chats, while ignoring homophone wordplay.

Copied to clipboard

Jiangang Bai, Yujing Wang, Hong Sun, Ruonan Wu, Tianmeng Yang, Pengfei Tang, Defu Cao, Mingliang Zhang1, Yunhai Tong, Yaming Yang, Jing Bai, Ruofei Zhang, Hao Sun, Wei Shen

FPE2M2: Approaching Lossless and Efficient Quantization with Native Floating Point (2025.findings-acl)

Challenge:	Existing works of knowledge infusion depend on multi-task learning frameworks, which are inefficient and require large-scale retraining when new knowledge is considered.
Approach:	They propose a method which integrates knowledge-generated attention maps into the self-attention mechanism and integrates it into the model.
Outcome:	The proposed model outperforms existing methods on academic datasets and industry-scale ad relevance applications.

Copied to clipboard

Ke Yi, Jianwei Zhang, Zhiying Xu, Xinlong Yang, Yang Zhou, Minmin Sun, Zengke Liu, Tong Zhang, Junyang Lin, Jingren Zhou

Green & Sustainable NLP

Challenge:	Auto-regressive decoding is a memory-bound job, meaning decoding performance is limited by the bandwidth rather than the computational capabilities of the GPU.
Approach:	They propose a framework that supports lossless weight-only quantization inference and validate it on Qwen and LLaMA Models.
Outcome:	The proposed framework achieves the highest efficiency with lossless accuracy on Qwen and LLaMA Models across various modalities.

MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokens (2025.findings-naacl)

Copied to clipboard

Yongqi Fan, Hongli Sun, Kui Xue, Xiaofan Zhang, Shaoting Zhang, Tong Ruan

Challenge:	Existing benchmarks in the generic domain have evaluated long-context capabilities for LLMs.
Approach:	They propose a medical long-context benchmark with seven length levels ranging from 4K to 200K tokens.
Outcome:	The proposed benchmarks have seven length levels ranging from 4K to 200K tokens.

CogToM: A Comprehensive Theory of Mind Benchmark inspired by Human Cognition for Large Language Models (2026.acl-long)

Copied to clipboard

Haibo Tong, Zeyang Yue, Feifei Zhao, Erliang Lin, Lu Jia, Ruolin Chen, Yinqian Sun, Qian Zhang, Yi Zeng

Linguistics & Cognitive NLP

Challenge:	Existing benchmarks for Large Language Models (LLMs) are limited to false belief tasks, highlighting bottlenecks in specific dimensions.
Approach:	They propose a benchmark to evaluate Large Language Models' Theory of Mind capabilities . they evaluate 8000 bilingual instances across 46 paradigms and validated by 49 human annotators .
Outcome:	The proposed benchmark reveals performance heterogeneities and bottlenecks in 22 representative models.

ATLAS: A System for PDF-centric Human Interaction Data Collection (2024.naacl-demo)

Copied to clipboard

Alexa Siu, Zichao Wang, Joshua Hoeflich, Naman Kapasi, Ani Nenkova, Tong Sun

Challenge:	Recent advances in AI only make the importance of high-quality data more pronounced.
Approach:	They propose to use the Portable Document Format (PDF) as a data format to better support researchers in collecting rich PDF-centric datasets from users.
Outcome:	The proposed toolkit and extensible schema allows researchers to customize the data collection tasks for a variety of purposes, including annotations, drawing, and reading behavior analytics.

Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models (2021.naacl-main)

Copied to clipboard

Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, Xia Hu

Explainability & Interpretability in NLP

Challenge:	Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding.
Approach:	They propose a shortcut mitigation framework to suppress NLU models from making overconfident predictions for samples with large shortcut degree.
Outcome:	The proposed framework suppresses the model from making overconfident predictions for samples with large shortcut degree.

Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases (2021.acl-long)

Copied to clipboard

Boxi Cao, Hongyu Lin, Xianpei Han, Le Sun, Lingyong Yan, Meng Liao, Tong Xue, Jin Xu

Open-Domain Question Answering with Pre-Constructed Question Spaces (2021.naacl-srw)

Challenge:	Recent studies show that pre-trained masked language models can be factual knowledge bases.
Approach:	They conduct a rigorous study to explore the underlying predicting mechanisms of MLMs . they find that previous decent performance mainly owes to the biased prompts which overfit dataset artifacts a .
Outcome:	The proposed model improves on illustrative cases and external contexts . the results question the previous findings that MLMs can be reliable factual knowledge bases .

Copied to clipboard

Jinfeng Xiao, Lidan Wang, Franck Dernoncourt, Trung Bui, Tong Sun, Jiawei Han

Information Retrieval

Challenge:	Open-domain question answering aims at locating answers to user-generated questions in massive collections of documents.
Approach:	They propose an algorithm with a novel reader-retriever design that differs from both families of algorithms.
Outcome:	The proposed algorithm outperforms retrieval-based methods with two large-scale datasets and is state-of-the-art.

MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning (2025.emnlp-main)

Copied to clipboard

Jingyan Shen, Jiarui Yao, Rui Yang, Yifan Sun, Feng Luo, Rui Pan, Tong Zhang, Han Zhao

CogBench: Benchmarking Cognitive Alignment of Large Language Models in Educational Question Answering (2026.findings-acl)

Challenge:	Existing reward models assume a global reward function, limiting personalization and pluralistic alignment.
Approach:	They propose a framework that leverages binary preference datasets to enhance personalized preference learning.
Outcome:	The proposed framework captures diverse human preferences without fine-grained annotations and significantly improves personalized preference learning on downstream tasks.

Copied to clipboard

Tong Lu, Zhichun Wang, Yuanhao Sun, Yaoyu Zhou, Mingrui Li, Yiming Guan, Zhiyong Bai

Question Answering

Challenge:	Large language models (LLMs) possess strong capabilities in language understanding and generation, as well as remarkable problem-solving abilities.
Approach:	They propose a benchmark to assess the cognitive alignment capabilities of large language models in educational QA.
Outcome:	The proposed evaluation benchmark assesses the cognitive alignment capabilities of large language models in educational QA.

Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark (2023.acl-long)

Copied to clipboard

Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, Xing Xie

Plum: Prompt Learning using Metaheuristics (2024.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated exceptional abilities in both text understanding and generation.
Approach:	They propose an Embedding Watermark method that implants backdoors on embeddings to protect copyright of large language models.
Outcome:	The proposed method protects the copyright of large language models without compromising service quality while minimizing the adverse impact on the original embeddings’ utility.

Copied to clipboard

Rui Pan, Shuo Xing, Shizhe Diao, Wenhe Sun, Xiang Liu, KaShun Shum, Jipeng Zhang, Renjie Pi, Tong Zhang

Low-Resource NLP

Challenge:	Recent advances in prompt learning have led to a need for general prompt optimization methods.
Approach:	They propose a branch of discrete non-convex optimization methods with over 100 options as a promising approach to prompt learning.
Outcome:	The proposed methods can be used to discover more human-understandable prompts that were previously unknown in reasoning and image generation tasks.

MATSA: Multi-Agent Table Structure Attribution (2024.emnlp-demo)

Copied to clipboard

Puneet Mathur, Alexa Siu, Nedim Lipka, Tong Sun

Thinking with Reasoning Skills: Fewer Tokens, More Accuracy (2026.acl-industry)

Challenge:	Tabular data present unique challenges for attribution due to ambiguities, complex header hierarchies, and the difficulty in interpreting individual table cells without row and column context.
Approach:	They propose a task to generate row and column-level attributions supporting LLM-generated answers.
Outcome:	The proposed task outperforms baselines on tabCite and improves F1 score.

Copied to clipboard

Guangxiang Zhao, Qilong Shi, Xusen Xiao, Xiangzheng Zhang, Tong Yang, Lin Sun

Reasoning

Challenge:	Reasoning LLMs often spend tokens on long intermediate reasoning traces when solving new problems.
Approach:	They propose to store reusable reasoning skills distilled from extensive deliberation and trial-and-error exploration and retrieve these skills at inference time to guide future reasoning.
Outcome:	The proposed approach reduces reasoning tokens while improving overall performance on coding and mathematical reasoning tasks.

CTC-based Non-autoregressive Speech Translation (2023.acl-long)

Copied to clipboard

Chen Xu, Xiaoqian Liu, Xiaowen Liu, Qingxuan Sun, Yuhao Zhang, Murun Yang, Qianqian Dong, Tom Ko, Mingxuan Wang, Tong Xiao, Anxiang Ma, Jingbo Zhu

Machine Translation

Challenge:	End-to-end speech translation (E2E ST) and non-autoregressive (NAR) generation are promising in language and speech processing for their advantages of less error propagation and low latency.
Approach:	They develop a model that uses connectionist temporal classification to predict the source and target texts.
Outcome:	The proposed model achieves an average BLEU score of 29.5 with a speed-up of 5.67.

MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper (2025.emnlp-main)

Copied to clipboard

Runjia Zeng, Guangyan Sun, Qifan Wang, Tong Geng, Sohail Dianat, Xiaotian Han, Raghuveer Rao, Xueling Zhang, Cheng Han, Lifu Huang, Dongfang Liu

Noisy Positive-Unlabeled Learning with Self-Training for Speculative Knowledge Graph Reasoning (2023.findings-acl)

Challenge:	Empirical evaluations show that Mixture of Expert Prompt Tuning outperforms state-of-the-art parameter efficient baselines on SuperGLUE.
Approach:	They propose a pretrain-then-fine-tune paradigm for manifold mapping using multiple prompt experts.
Outcome:	Empirical results show that the proposed approach outperforms state-of-the-art methods on SuperGLUE while reducing activated prompts by 79.25%.

Copied to clipboard

Ruijie Wang, Baoyu Li, Yichen Lu, Dachun Sun, Jinning Li, Yuchen Yan, Shengzhong Liu, Hanghang Tong, Tarek Abdelzaher

Sinkhorn Distance Minimization for Knowledge Distillation (2024.lrec-main)

Challenge:	State-of-the-art methods fail in speculative reasoning task on knowledge graphs . state-of the-art approaches assume correctness of fact is determined by its presence in KG .
Approach:	They propose a speculative reasoning task on real-world knowledge graphs . they propose nPUGraph that estimates correctness of both collected and uncollected facts .
Outcome:	The proposed framework improves the robustness of a label posterior-aware graph encoder against false positive links and identifies missing facts to provide high-quality grounds of reasoning.

Copied to clipboard

Xiao Cui, Yulei Qin, Yuting Gao, Enwei Zhang, Zihan Xu, Tong Wu, Ke Li, Xing Sun, Wengang Zhou, Houqiang Li

MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark (2025.acl-long)

Challenge:	Existing knowledge distillation methods investigate divergence measures but fail to deliver effective supervision when few distribution overlap exists between teacher and student.
Approach:	They propose a knowledge distillation method that exploits the Sinkhorn distance to ensure a nuanced assessment of the disparity between teacher and student distributions.
Outcome:	The proposed method outperforms state-of-the-art methods on all kinds of LLMs with encoder-only, encoder decoder, and decoded architectures.

Copied to clipboard

Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, Graham Neubig

TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignment (2026.acl-long)

Challenge:	Recent advances in multimodal large language models have led to progress in tackling complex reasoning tasks that combine textual and visual information.
Approach:	They introduce a robust version of the Massive Multi-discipline Multimodal Understanding and Reasoning (MMMU) benchmark.
Outcome:	The proposed model performs lower on MMMU-Pro than on the previous benchmark, ranging from 16.8% to 26.9%.

Copied to clipboard

Zhewen Tan, Wenhan Yu, Jianfeng Si, Tongxin Liu, Kaiqi Guan, Huiyan Jin, Jiawen Tao, Xiaokun Yuan, Xiangzheng Zhang, Duohe Ma, Tong Yang, Lin Sun

Robustness in NLP

Challenge:	Existing approaches to safety alignment of large language models rely on costly manual annotations or human review.
Approach:	They propose a closed-loop reinforcement learning framework called TriPlay-RL that enables iterative collaboration among three roles with near-zero manual annotation.
Outcome:	The proposed framework achieves 20%–50% improvement in adversarial effectiveness while preserving high output diversity while achieving 10%–30% gains in safety performance without degrading general reasoning capability.

Adaptive Simultaneous Sign Language Translation with Confident Translation Length Estimation (2024.lrec-main)

Copied to clipboard

Tong Sun, Biao Fu, Cong Hu, Liang Zhang, Ruiquan Zhang, Xiaodong Shi, Jinsong Su, Yidong Chen

MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding (2022.emnlp-main)

Challenge:	Existing non-simultaneous sign language translation methods suffer from inherent inference delays in real-time scenarios.
Approach:	They propose an adaptive policy for simultaneous sign language translation that progressively converts incrementally received sign video into its corresponding natural sentence.
Outcome:	The proposed policy excels in situations requiring extremely low latency.

Copied to clipboard

Zilong Wang, Jiuxiang Gu, Chris Tensmeyer, Nikolaos Barmpalios, Ani Nenkova, Tong Sun, Jingbo Shang, Vlad Morariu

Empirical Analysis of Decoding Biases in Masked Diffusion Models (2026.acl-long)

Challenge:	Existing methods learn features from word-level or region-level but fail to consider both simultaneously.
Approach:	They propose a multi-modal multi-granular pre-training framework that encodes page-level, region-level and word-level information at the same time.
Outcome:	The proposed model learns features from word-level and region-level but fails to consider both simultaneously.

Copied to clipboard

Pengcheng Huang, Tianming Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Tong Xiao, Zulong Chen, Maosong Sun

WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling (2026.findings-acl)

Challenge:	Existing MDMs employ uncertainty-based decoding strategies that limit their reasoning ability and ultimately degrade generation quality.
Approach:	They propose a framework that regularizes uncertainty-based decoding by incorporating two complementary priors to shape global decoding trajectories and promote content informativeness.
Outcome:	The proposed framework outperforms existing decoding strategies by more than 7% while achieving comparable performance to autoregressive models of similar parameter scales.

Copied to clipboard

Jiacheng Li, Jianchao Tan, Zhidong Yang, Pingwei Sun, Feiye Huo, Jiayu Qin, Xiangyu Zhang, Maoxin He, Guangming Tan, Weile Jia, Xunliang Cai, Tong Zhao

MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses (2025.emnlp-main)

Challenge:	Recent advances in training optimization for Transformer-based large language models lack systematic optimization of weight patterns during training.
Approach:	They propose a Weight Scaling method that rescales weights while preserving model outputs to improve model training efficiency and model quality.
Outcome:	The proposed method significantly improves convergence quality and loss reduction in LLMs with Grouped Query Attention architectures and LoRA fine-tuning tasks.

Copied to clipboard

Tong Chen, Zimu Wang, Yiyi Miao, Haoran Luo, Sun Yuanfei, Wei Wang, Zhengyong Jiang, Procheta Sen, Jionglong Su

E-ABSA20K: A Dataset and Propose-and-Verify for Aspect-Based Sentiment Analysis in Long E-commerce Reviews (2026.findings-acl)

Challenge:	Existing medical fact-checking datasets focus on human-generated content, leaving the verification of content generated by large language models (LLMs) relatively unexplored.
Approach:	They propose to use Chinese medical fact-checking datasets to verify LLM-generated medical content by combining in-context learning and fine-tuning.
Outcome:	The first evidence-based Chinese medical fact-checking dataset of LLM-generated medical content consists of 1,321 questions and 7,409 claims .

Copied to clipboard

Tong Sun, Mingyang Ma, Cheng Yu

Sentiment Analysis

Challenge:	Aspect-based Sentiment Analysis (ABSA) is critical for extracting actionable product insights from e-commerce reviews.
Approach:	They propose a framework that decomposes ABSA into two stages to extract review-level quadruple reviews from 20K reviews from four product categories.
Outcome:	The proposed framework outperforms existing benchmarks and single-stage prompting and competitive ABSA extraction baselines.

MoDS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collections (2025.naacl-long)

Copied to clipboard

Nishant Balepur, Alexa Siu, Nedim Lipka, Franck Dernoncourt, Tong Sun, Jordan Lee Boyd-Graber, Puneet Mathur

Text Generation

Challenge:	Query-focused summarization (QFS) gives an overview of documents to answer a query, ignoring debatable ones.
Approach:	They propose a multi-LLM framework that uses a Query-focused summarization approach to create balanced summaries that answer debatable queries.
Outcome:	The proposed framework beats SOTA by 38-59% in topic paragraph coverage and balance, based on new citation metrics.

DocPilot: Copilot for Automating PDF Edit Workflows in Documents (2024.acl-demos)

Copied to clipboard

Puneet Mathur, Alexa Siu, Varun Manjunatha, Tong Sun

Challenge:	Document workflow copilot system that can understand user intent and execute tasks accordingly to help users streamline their workflows.
Approach:	They propose an AI-assisted document workflow copilot system capable of understanding user intent and executing tasks accordingly.
Outcome:	The proposed system can understand user intent and execute tasks accordingly to help users streamline their workflows.

Text-to-ES Bench: A Comprehensive Benchmark for Converting Natural Language to Elasticsearch Query (2025.acl-long)

Copied to clipboard

DonggeXue DonggeXue, Zhili Pu, Zhentao Xia, Hongli Sun, Ruihui Hou, Guangya Yu, Yupian Lin, Yongqi Fan, Jingping Liu, Tong Ruan

Information Retrieval

Challenge:	Recent research on text-to-Query has explored using large language models to convert user query intent to executable code.
Approach:	They propose a novel semantic parsing task that leverages large language models to generate domain-specific language and post-processing code to support multi-index Elasticsearch queries.
Outcome:	The proposed model outperforms DeepSeek-R1 on the large Elasticsearch Dataset (LED) and BirdES datasets.

Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns (2022.findings-acl)

Copied to clipboard

Zihan Wang, Jiuxiang Gu, Jason Kuen, Handong Zhao, Vlad Morariu, Ruiyi Zhang, Ani Nenkova, Tong Sun, Jingbo Shang

A Critical Analysis of Document Out-of-Distribution Detection (2023.findings-emnlp)

Challenge:	Adaptive Axis Attention learns different attention patterns for each task and model layer . sparse attention patterns do not improve the run time of the models but they reduce model memory requirements .
Approach:	They propose a method that learns different attention patterns for each Transformer layer . they propose 'adaptive axis attention' method that identifies important tokens .
Outcome:	The proposed method does not require pre-training to accommodate sparse attention patterns.

Copied to clipboard

Jiuxiang Gu, Yifei Ming, Yi Zhou, Jason Kuen, Vlad Morariu, Handong Zhao, Ruiyi Zhang, Nikolaos Barmpalios, Anqi Liu, Yixuan Li, Tong Sun, Ani Nenkova

Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation (2021.acl-long)

Challenge:	Existing document understanding models focus on single-modal inputs such as images or texts.
Approach:	They propose to use a spatial-aware adapter to adapt transformer-based language models to document domain to exploit multi-modal information.
Outcome:	The proposed model significantly improves the OOD detection performance compared to using a standard language model and to competitive baselines.

Copied to clipboard

Tong Zhang, Long Zhang, Wei Ye, Bo Li, Jinan Sun, Xiaoyu Zhu, Wen Zhao, Shikun Zhang

Multilinguality

Challenge:	Existing approaches to incorporate bilingual dictionaries into Neural Machine Translation (NMT) models have been criticized for lack of integration of bilingual lexical information into the neural architecture.
Approach:	They propose a neural architecture to incorporate bilingual dictionaries into Neural Machine Translation models by introducing three new components: Pointer, Disambiguator, and Copier.
Outcome:	The proposed method achieves the following merits inherently compared with previous efforts: (1) Pointer leverages the semantic information from bilingual dictionaries, for the first time, to better locate source words whose translation in dictionary can potentially be used; (2) Disambiguator synthesizes contextual information from source view and target view, both of which contribute to distinguishing translation of a specific source word from multiple candidates in dicaries; (3) Copier systematically connects Pointer and Disambiguators based on a hierarchical

Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training (2026.acl-long)

Copied to clipboard

Changhao Jiang, Ming Zhang, Yifei Cao, Junjie Ye, Xiaoran Fan, Shihan Dou, Zhiheng Xi, Jiajun Sun, Yi Dong, Yujiong Shen, Jingqi Tong, Baoyu Fan, Tao Gui, Qi Zhang, Xuanjing Huang