POST-TRAINING

On Domain-Adaptive Post-Training for Multimodal Large Language Models (2025.findings-emnlp)

Challenge:	Recent advances in large language model pruning have shown high predictive performance in post-training settings.
Approach:	They conduct an empirical study on the performance and internal representation changes associated with pruning multilingual models for monolingual applications.
Outcome:	The proposed pruning methods retain perplexity and yield high signal-to-noise ratios, but not consistently improve downstream tasks.

Copied to clipboard

Daixuan Cheng, Shaohan Huang, Ziyu Zhu, Xintong Zhang, Xin Zhao, Zhongzhi Luan, Bo Dai, Zhenliang Zhang

Post-Training with Interrogative Sentences for Enhancing BART-based Korean Question Generator (2022.aacl-short)

Challenge:	Adapting general multimodal large language models to specific domains is important for practical applications.
Approach:	They investigate domain adaptation of multimodal large language models via post-training . they develop a generate-then-filter pipeline that curates diverse visual instruction tasks .
Outcome:	The proposed model outperforms existing models in domain adaptation by combining data from open-source models with training pipelines.

Copied to clipboard

Gyu-Min Park, Seong-Eun Hong, Seong-Bae Park

A Dual-Phase Self-Evolution Framework for Large Language Models (2026.findings-acl)

Challenge:	Existing pre-trained language models fail to generate perfect interrogative sentences in Korean question generation.
Approach:	They propose to add question infilling objective to KoBART to enhance it for Korean question generation.
Outcome:	The proposed post-training improves KoBART for Korean question generation.

Copied to clipboard

Haoran Sun, Zekun Zhang, Shaoning Zeng

Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems (2024.naacl-long)

Challenge:	Existing strategies to optimize LLMs through pretraining fail to enhance domain cognition.
Approach:	They propose a dual-phase self-evolution framework that integrates user preference adaptation and domain-specific competence to optimize LLMs.
Outcome:	The proposed framework outperforms Supervised Fine-Tuning, Preference Optimization, and Memory-Augmented baselines on general NLP benchmarks and long-term dialogue tasks.

Copied to clipboard

Zhenpeng Su, Xing W, Wei Zhou, Guangyuan Ma, Songlin Hu

TARo: Token-level Adaptive Routing for LLM Test-time Alignment (2026.findings-acl)

Challenge:	Existing studies on dialogue response selection focus on post-training and fine-tuning for cross-encoders.
Approach:	They propose a post-training technique tailored for dense encoders in dialogue response selection . they propose 'Dialogue Contextual Masking Auto-Encoder' which compresses dialogue semantics into dense vectors .
Outcome:	The proposed technique achieves state-of-the-art on two commonly evaluated benchmarks.

Copied to clipboard

Arushi Rai, Qiang Zhang, Hanqing Zeng, Yunkai Zhang, Dipesh Tamboli, Xiangjun Fan, Zhuokai Zhao, Lizhu Zhang

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph (2020.emnlp-main)

Challenge:	Large language models (LLMs) exhibit strong reasoning capabilities but typically require expensive post-training to reach high performance.
Approach:	They propose to use token-level Adaptive Routing to steer frozen LLMs toward structured reasoning entirely at inference time.
Outcome:	Extensive experiments show that TARo significantly improves reasoning performance by up to +22.4% over base model and +8.4% .

Copied to clipboard

Haozhe Ji, Pei Ke, Shaohan Huang, Furu Wei, Xiaoyan Zhu, Minlie Huang

Adaptive LLM-Symbolic Reasoning via Dynamic Logical Solver Composition (2026.eacl-long)

Challenge:	Existing approaches that integrate commonsense knowledge into pre-trained language models simply transfer relational knowledge while ignoring rich connections within the knowledge graph.
Approach:	They propose a method that leverages structural and semantic information of the knowledge graph to generate commonsense-aware text.
Outcome:	The proposed method outperforms baseline models on three text generation tasks that require reasoning over commonsense knowledge.

Copied to clipboard

Lei Xu, Pierre Beckmann, Marco Valentino, Andre Freitas

Comprehensive Study of Bilingual and Multi-category Instruction Pre-training (2026.findings-eacl)

Challenge:	Existing approaches to NLP are static and require manual formalization.
Approach:	They propose an adaptive, multi-paradigm, neuro-symbolic inference framework that automatically identifies formal reasoning strategies from problems expressed in natural language and dynamically selects and applies specialized formal logical solvers.
Outcome:	The proposed framework outperforms baselines on individual and multi-paradigm reasoning tasks by 17% and 6%.

Copied to clipboard

Takashi Kodama, Yusuke Oda

AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation (2023.findings-acl)

Challenge:	Instruction pre-training (IPT) has recently gained attention as an intermediate stage between pre- and post-training for large language models.
Approach:	They study the optimal balance between raw and instruction-response data, languages, and task categories in an LLM instruction-respondence dataset.
Outcome:	The proposed model improves on English-centric and bilingual models using bilingual instruction-response datasets.

Copied to clipboard

Chujie Zheng, Sahand Sabour, Jiaxin Wen, Zheng Zhang, Minlie Huang

Dialogue Systems & Conversational Agents

Challenge:	Crowdsourced dialogue corpora are limited in scale and topic coverage due to the expensive cost of data curation.
Approach:	They construct an augmented dataset for the emotional support conversation task using large language models for dialogue augmentation.
Outcome:	The proposed approach outperforms baselines of dialogue augmentation and improves the model's generalization ability to open-domain topics.

CARMO: Dynamic Criteria Generation for Context Aware Reward Modelling (2025.findings-acl)

Copied to clipboard

Taneesh Gupta, Shivam Shandilya, Xuchao Zhang, Rahul Madhavan, Supriyo Ghosh, Chetan Bansal, Huaxiu Yao, Saravan Rajmohan

Challenge:	Reward modeling in large language models is susceptible to reward hacking . flawed reward signals often lead to outputs that optimize for spurious correlates .
Approach:	They propose a new approach that generates dynamic, context-relevant criteria to ground the reward model prior to producing reward scores.
Outcome:	The proposed approach generates dynamic, context-relevant criteria to ground the model prior to producing reward scores.

ComfyFlow: Benchmarking LLMs for AIGC Workflow Generation (2026.findings-acl)

Copied to clipboard

Zhenran Xu, Yiyu Wang, Yunxin li, Muyang Ye, null Yangxue, Kai Chen, Longyue Wang, Weihua Luo, Baotian Hu, Min Zhang

Bi-Granularity Contrastive Learning for Post-Training in Few-Shot Scene (2021.findings-acl)

Challenge:	Large language models (LLMs) have shown promising advances in tackling human-level tasks, but generating workflows for collaborative AI systems remains a critical and challenging step.
Approach:	They propose a benchmark to evaluate LLMs’ ability to generate executable and instruction-following AIGC workflows in ComfyUI.
Outcome:	The proposed benchmarks show that LLMs can generate executable and instruction-following AIGC workflows in ComfyUI.

Copied to clipboard

Ruikun Luo, Guanhuan Huang, Xiaojun Quan

MIDLM: Multi-Intent Detection with Bidirectional Large Language Models (2025.coling-main)

Challenge:	Existing approaches to fine-tune pre-trained models to downstream tasks are limited by labeled examples.
Approach:	They propose to apply post-training on unlabeled task data before fine-tuning by contrastive learning that considers either token-level or sequence-level similarity.
Outcome:	Empirical results show that contrastive masked language modeling surpasses other methods in few-shot settings without the need for data augmentation.

Copied to clipboard

Shangjian Yin, Peijie Huang, Yuhong Xu

Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective (2025.emnlp-main)

Challenge:	Existing models that use autoregressive architectures restrict the sharing of token information within a sentence.
Approach:	They propose a framework that integrates intent number detection and multi-intent selection to enable autoregressive LLMs to leverage bidirectional information awareness through post-training.
Outcome:	The proposed framework outperforms existing models and pretrained baselines in the multi-intent detection task.

Copied to clipboard

Siyue Zhang, Yilun Zhao, Liyuan Geng, Arman Cohan, Anh Tuan Luu, Chen Zhao

Transferable Post-training via Inverse Value Learning (2025.naacl-long)

Challenge:	Large language model (LLM)-based embedding models surpass BERT and T5 on general-purpose text embeddable tasks.
Approach:	They propose to adopt diffusion language models for text embeddings to overcome limitations in unidirectional attention used during autoregressive pre-training.
Outcome:	The proposed model outperforms the existing LLM-based embedding model on reasoning tasks by 20% and 2% on traditional embeddable benchmarks.

Copied to clipboard

Xinyu Lu, Xueru Wen, Yaojie Lu, Bowen Yu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han, Yongbin Li

BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis (N19-1)

Challenge:	Existing algorithms for post-training large datasets are requiring a large computational effort.
Approach:	They propose to model the changes at logits level during post-training using a separate neural network . they demonstrate that the value network can be seamlessly integrated with another pre-trained model .
Outcome:	The proposed model can be integrated with another pre-trained model during inference, enabling similar capability enhancements.

Copied to clipboard

Hu Xu, Bing Liu, Lei Shu, Philip Yu

P2 Law: Scaling Law for Post-Training After Model Pruning (2025.acl-long)

Challenge:	Existing work on question-answering has limited training examples for RRC . question-announced questions are a key component of online commerce .
Approach:	They propose to turn customer reviews into a large source of knowledge that can be exploited to answer user questions.
Outcome:	The proposed approach improves review reading comprehension on popular language model BERT . it also improves aspect extraction and aspect sentiment classification tasks .

Copied to clipboard

Xiaodong Chen, Yuxuan Hu, Xiaokang Zhang, Yanling Wang, Cuiping Li, Hong Chen, Jing Zhang

Challenge:	Pruning has become a widely adopted technique for reducing the hardware requirements of large language models (LLMs).
Approach:	They propose to use model pruning techniques to maintain high performance while reducing hardware requirements for large language models (LLMs).
Outcome:	The proposed model pruning law can be generalized to larger dataset sizes, larger model sizes, and higher pruning rates, offering valuable insights for resource allocation in pruned LLMs.

Tackling Distractor Documents in Multi-Hop QA with Reinforcement and Curriculum Learning (2026.findings-eacl)

Copied to clipboard

Jerry Huang, Siddarth Madala, Risham Sidhu, Cheng Niu, Hao Peng, Julia Hockenmaier, Tong Zhang

Information Retrieval

Challenge:	Existing work on retrieval-augmented generation systems has shown that retrievers exhibit imperfect recall and precision, limiting downstream performance.
Approach:	They propose a retrieval-augmented generation model that generates answers from larger sets of retrieved contexts.
Outcome:	The proposed model generates answers and cites relevant information from larger sets of retrieved contexts.

Improving Attributed Text Generation of Large Language Models via Preference Learning (2024.findings-acl)

Copied to clipboard

Dongfang Li, Zetian Sun, Baotian Hu, Zhenyu Liu, Xinshuo Hu, Xuebo Liu, Min Zhang

LaCo: Large Language Model Pruning via Layer Collapse (2024.findings-emnlp)

Challenge:	Large language models have been widely adopted in natural language processing, yet they produce unreliable content.
Approach:	They propose to model the attribution task as preference learning and introduce an automatic preference optimization framework that synthesizes attribution preference data.
Outcome:	The proposed method achieves state-of-the-art citation F1 with higher answer quality than existing methods.

Copied to clipboard

Yifei Yang, Zouying Cao, Hai Zhao

Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models (2025.acl-long)

Challenge:	Existing methods for model quantization, knowledge distillation, and model pruning are limited by hardware support limitations and the need for extensive training.
Approach:	They propose a layer-wise structured pruner that collapses rear model layers into a prior layer and enables a rapid reduction in model size while preserving the model structure.
Outcome:	The proposed pruner outperforms state-of-the-art pruning methods at pruning ratios of 25-30% and maintains an average task performance of over 80% at different pruning ratio.

Copied to clipboard

Zixiang Xu, Yanbo Wang, Yue Huang, Xiuying Chen, Jieyu Zhao, Meng Jiang, Xiangliang Zhang

TT-SI: Self-Improving LLM Agents with Test-Time Training (2026.findings-acl)

Challenge:	Large language models have achieved remarkable success in Natural Language Processing, yet their cross-lingual consistency remains a significant challenge.
Approach:	They propose a method to identify cross-lingual weaknesses in Large Language Models . they construct bilingual question pairs that expose performance discrepancies between English and target languages .
Outcome:	The proposed method uncovers over 50% accuracy drops in target languages across models.

Copied to clipboard

Emre Can Acikgoz, Cheng Qian, Heng Ji, Dilek Hakkani-Tür, Gokhan Tur

Green & Sustainable NLP

Challenge:	Existing methods for language model fine-tuning are expensive and inefficient . existing methods rarely assess whether a training sample provides novel information .
Approach:	They propose a test-time self-improvement algorithm that generates a sample that model struggles with . they also explore Test-Time Distillation, which leverages 'stronger supervisors'
Outcome:	The proposed algorithm improves performance with +5.48% absolute accuracy gain on average across benchmarks.

A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations (2021.emnlp-main)

Copied to clipboard

Ziyi Yang, Yinfei Yang, Daniel Cer, Eric Darve

Multilinguality

Challenge:	Language agnostic and semantic-language information isolation is an emerging research direction for multilingual representations models.
Approach:	They propose a method that factors out language identity information from semantic related components in multilingual representations pre-trained on monolingual data.
Outcome:	The proposed method improves cross-lingual transfer performance on weak alignment models.

Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch (2025.findings-emnlp)

Copied to clipboard

Yirong Zeng, Xiao Ding, Yutai Hou, Yuxian Wang, Li Du, Juyi Dai, Qiuyang Ding, Duyu Tang, Dandan Tu, Weiwen Liu, Bing Qin, Ting Liu

Universal Sentence Representation Learning with Conditional Masked Language Model (2021.emnlp-main)

Challenge:	Experimental results demonstrate that our models achieve over 7% performance improvement compared to both SFT and RL-with-SFT models under the same experimental settings.
Approach:	They propose a dynamic generalization-guided reward design for rule-based RL that shifts rewards from exploratory to exploitative tool-use patterns.
Outcome:	The proposed model achieves over 7% performance improvement compared to SFT and RL-with-SFT models under the same experimental settings.

Copied to clipboard

Ziyi Yang, Yinfei Yang, Daniel Cer, Jax Law, Eric Darve

Reinforcement Learning on Pre-Training Data (2026.acl-long)

Challenge:	Existing methods to learn sentence representations on unlabeled corpora are difficult and expensive to obtain, making it hard to cover many domains and languages.
Approach:	They propose a method to train sentence representations on large unlabeled corpora by conditioning on the encoded vectors of adjacent sentences.
Outcome:	The proposed method outperforms existing models on SentEval and can be extended to a broad range of languages and domains.

Copied to clipboard

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang

AdamS: Momentum Itself Can Be A Normalizer for LLM Pretraining and Post-training (2025.emnlp-main)

Challenge:	Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach:	They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome:	Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.

Copied to clipboard

Huishuai Zhang, Bohan Wang, Luoxin Chen

Do Code Semantics Help? A Comprehensive Study on Execution Trace-Based Information for Code Large Language Models (2025.findings-emnlp)

Challenge:	Empirically, AdamS demonstrates strong performance in various tasks . et al., 2023b): AdamS is efficient, efficient, and model-agnostic.
Approach:	They propose a model-agnostic alternative to Adam for large language model pretraining and post-training.
Outcome:	The proposed method matches memory footprint of SGD with momentum while delivering superior performance.

Copied to clipboard

Jian Jornbowrl Wang, Xiaofei Xie, Qiang Hu, Shangqing Liu, Yi Li

Patches of Nonlinearity: Instruction Vectors in Large Language Models (2026.acl-long)

Challenge:	Code Large Language Models have limited ability to reason about runtime behavior and understand functionality . authors present a generic framework to support integrating semantic information to code task-relevant prompts .
Approach:	a study examines the role of trace-based semantic information in boosting supervised fine-tuning and post-phase inference of Code LLMs.
Outcome:	a new framework integrates semantic information to code task-relevant prompts . the proposed framework shows that trace-based semantic information boosts reasoning ability .

Copied to clipboard

Irina Bigoulaeva, Jonas Rohweder, Subhabrata Dutta, Iryna Gurevych

Adapting a Language Model While Preserving its General Knowledge (2022.emnlp-main)

Challenge:	Despite the success of instruction-tuned language models, little is known about how they process instructions internally.
Approach:	They propose a method to localize instruction processing in language models that is free from patching assumptions.
Outcome:	The proposed method disentangles the implicit linear assumptions of patching-based techniques.

Copied to clipboard

Zixuan Ke, Yijia Shao, Haowei Lin, Hu Xu, Lei Shu, Bing Liu

MMA: Cross-Domain Knowledge Integration via Mixture of Multi-Domain Agents (2025.findings-emnlp)

Challenge:	Existing DA-training methods do not explicitly identify what knowledge should be preserved and what should be changed by the domain corpus.
Approach:	They propose to use an unlabeled corpus of aparticular domain to train a pre-trained general-purpose language model to adapt the LM so that end-tasks in the domain can give improved performances.
Outcome:	The proposed method improves the performance of pre-trained general-purpose language models by contrasting the representations of the general and the full (both general and domain knowledge) to learn an integrated representation with both general and specific knowledge.

Copied to clipboard

Kehang Jia, Juntao Li, Xiaobo Liang, Yisheng Xiao, Yixuan Yang, Min Zhang

Challenge:	achieving synergistic improvements between generalization and domain specialization remains a challenge in pre-training and post-training.
Approach:	They propose a test-time cross-domain knowledge integration method that integrates general-purpose and domain-specific models to enhance their performance on complex, domainspecific tasks.
Outcome:	The proposed method combines the outputs of general-purpose and domain-specific models to improve their performance on complex, domainspecific tasks.

Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activation (2026.findings-acl)

Copied to clipboard

Yupu Hao, Zhuoran Jin, Huanxuan Liao, Kang Liu, Jun Zhao

Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge (2025.findings-acl)

Challenge:	Existing approaches treat tool use as a problem of prompt design, API documents specification, or supervised or unsupervised alignment.
Approach:	They propose a knowledge-augmented tool execution framework that integrates experiential knowledge with reasoning-width-expanded inference and knowledge-aware training.
Outcome:	The proposed framework improves on BFCL-V3 and AppWorld on three model scales.

Copied to clipboard

Xinyue Cui, Johnny Wei, Swabha Swayamdipta, Robin Jia

Robustness in NLP

Challenge:	Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track and verify training data ownership.
Approach:	They propose a data watermarking approach that injects coherent and plausible yet fictitious knowledge into training data using generated passages describing a fictious entity and its associated attributes.
Outcome:	The proposed method is designed to be memorized by the LLM, and that increasing their density, length, and diversity of attributes strengthens their memorization.

Com2 : A Causal-Guided Benchmark for Exploring Complex Commonsense Reasoning in Large Language Models (2025.acl-long)

Copied to clipboard

Kai Xiong, Xiao Ding, Yixin Cao, Yuxiong Yan, Li Du, Yufei Zhang, Jinglong Gao, Jiaqian Liu, Bing Qin, Ting Liu

BOSE: A Systematic Evaluation Method Optimized for Base Models (2025.findings-acl)

Challenge:	Existing works focus on complex tasks like math and code, while complex commonsense reasoning remains underexplored due to its uncertainty and lack of structure.
Approach:	They propose to build a benchmark for large language models based on complex commonsense reasoning based upon causal event graphs and causal theory.
Outcome:	The proposed benchmark combines a complex commonsense reasoning benchmark with a detective story to achieve a more challenging subset.

Copied to clipboard

Hongzhi Luan, Changxin Tian, Zhaoxin Huan, Xiaolu Zhang, Kunlong Chen, Zhiqiang Zhang, Jun Zhou

Can Post-Training Transform LLMs into Causal Reasoners? (2026.findings-acl)

Challenge:	Existing evaluation methods for large language models (LLMs) are inadequate to provide solid conclusions for key experiments such as data ablation and scaling law.
Approach:	They propose a method specifically designed to optimize the evaluation of base models by incorporating two innovations: In-Context Light-instruction Prompt and Blank-ppl for multi-choice tasks with candidate options.
Outcome:	The proposed method significantly improves stability and consistency of evaluations during pre-training and consistency between base and instruct models.

Copied to clipboard

Junqi Chen, Sirui Chen, Chaochao Lu

Robustness in NLP

Challenge:	Causal inference is a core component of human cognition and requires decision-makers to distinguish between causation and association.
Approach:	They propose a dataset comprising seven core causal tasks for training and five diverse test sets and evaluate five different post-training approaches.
Outcome:	The proposed model achieves 93.5% accuracy on the CaLM benchmark, compared to 55.4% by OpenAI o3.

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models (2025.acl-long)

Copied to clipboard

Lindia Tjuatja, Graham Neubig

Embedding Domain Knowledge for Large Language Models via Reinforcement Learning from Augmented Generation (2025.emnlp-main)

Challenge:	Existing methods for evaluating language models are brittle, corpus-level perplexities are vague, and the choice of benchmarks is endless.
Approach:	They propose a method that uses contextual embeddings to find fine-grained features of text where one model outperforms another.
Outcome:	The proposed method extracts features that demonstrate differences with respect to ease of generation between two language models.

Copied to clipboard

Chaojun Nie, Jun Zhou, Guanxiang Wang, Shisong Wu, Zichen Wang

CARVQ: Corrective Adaptor with Group Residual Vector Quantization for LLM Embedding Compression (2025.findings-emnlp)

Challenge:	Existing approaches to embed knowledge into large language models have some limitations . static nature of training data and lack of knowledge in domains create knowledge gaps .
Approach:	They propose a method that iteratively cycles between sampling generations and optimizing the model through calculated rewards.
Outcome:	The proposed method outperforms baseline approaches on medical, legal, astronomy, and current events datasets.

Copied to clipboard

Dayin Gou, Sanghyun Byun, Nilesh Malpeddi, Gabrielle De Micheli, Prathamesh Vaste, Jacob Song, Woo Seong Chung

MimicLM: Zero-Shot Voice Imitation through Autoregressive Modeling of Pseudo-Parallel Speech Corpora (2026.findings-acl)

Challenge:	Large Language Models typically rely on a large number of parameters for token embedding, leading to substantial storage requirements and memory footprints.
Approach:	They propose a corrective Adaptor with group Residual Vector Quantization that can be used to compress the embedding layer without requiring specialized hardware.
Outcome:	The proposed corrective adaptor can achieve lower average bitwidth-per-parameter while maintaining reasonable perplexity and accuracy compared to scalar quantization.

Copied to clipboard

Tao Feng, Yuxiang Wang, Yuancheng Wang, Xueyao Zhang, Dekun Chen, Chaoren Wang, Xun Guan, Zhizheng Wu

PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving (2025.emnlp-main)

Challenge:	Existing approaches to voice imitation use complex model design and a quality ceiling when synthetic speech is used as training sources.
Approach:	They propose a model that uses synthetic speech as training sources while retaining real recordings as targets.
Outcome:	The proposed model outperforms existing methods in naturalness while maintaining competitive similarity scores across speaker identity, accent, and emotion dimensions.

Copied to clipboard

Mihir Parmar, Palash Goyal, Xin Liu, Yiwen Song, Mingyang Ling, Chitta Baral, Hamid Palangi, Tomas Pfister

When Models Reason in Your Language: Controlling Thinking Language Comes at the Cost of Accuracy (2025.findings-emnlp)

Challenge:	Recent studies have shown that decomposing complex problems into simple subtasks has significantly boosted the performance of large language models (LLMs).
Approach:	They propose a unified post-training framework that distills synthetic task decompositions and fine-tunes smaller LLMs via supervised and reinforcement-learning objectives to improve complex reasoning.
Outcome:	The proposed framework outperforms strong baselines on GSM8k and MATH benchmarks and shows that it can improve generalization capabilities on out-of-domain datasets.

Copied to clipboard

Jirui Qi, Shan Chen, Zidi Xiong, Raquel Fernández, Danielle Bitterman, Arianna Bisazza

AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic (2025.findings-acl)

Challenge:	Recent Large Reasoning Models (LRMs) with thinking traces have shown strong performance on English reasoning tasks.
Approach:	They evaluate two leading LRMs with thinking traces on established benchmark XReasoning and propose directions for future research.
Outcome:	The proposed models often revert to English or produce fragmented reasoning in other languages, revealing a substantial gap in the capability of thinking in non-English languages.

Copied to clipboard

Nathaniel Romney Robinson, Shahd Abdelmoneim, Kelly Marchisio, Sebastian Ruder

Challenge:	Dialectal Arabic (DA) varieties are under-served by language technologies, particularly large language models (LLMs).
Approach:	They propose a framework that comprehensively assesses LLMs’ DA modeling capabilities across four dimensions: fidelity, understanding, quality, and diglossia.
Outcome:	The proposed framework assesses LLMs’ DA modeling capabilities across four dimensions: fidelity, understanding, quality, and diglossia.

Understanding LLMs’ Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From (2025.emnlp-main)

Copied to clipboard

Changjiang Gao, Hankun Lin, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Jiajun Chen, Shujian Huang

Cross-Lingual Transfer

Challenge:	Cross-lingual context retrieval is a fundamental aspect of cross-lingual alignment, but the performance and mechanism of it for large language models (LLMs) remains unclear.
Approach:	They evaluate cross-lingual context retrieval of over 40 large language models . they use cross-linguistic machine reading comprehension as a representative scenario .
Outcome:	The results show that open LLMs show strong cross-lingual context retrieval ability . the results also show that their oracle performances improve after training .

LORE: Continual Logit Rewriting Fosters Faithful Generation (2025.findings-emnlp)

Copied to clipboard

Charles Yu, Qingyun Wang, Yuting Hu, Jinjun Xiong, Heng Ji

Text Generation

Challenge:	Using aspect-oriented summarization as a case study, we propose LOgit REwriting, a new controlled generation paradigm which can be faithful to external knowledge and to the LLM’s intentions.
Approach:	They propose a controlled generation paradigm which can be faithful to external knowledge and to the LLM's intentions.
Outcome:	The proposed paradigm can be faithful to external knowledge and to the LLM's intentions while balancing that with accuracy.

CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation (2026.acl-long)

Copied to clipboard

Yunfan Yang, Cuiling Lan, Jitao Sang, Yan Lu

Multimodality

Challenge:	Tables contain rich structured information, but when stored as images their contents remain "locked" within pixels.
Approach:	They propose a framework that disentangles optimization across LaTeX tables components . CSPO assigns component-specific rewards and backpropagates each signal through tokens .
Outcome:	The proposed framework disentangles optimization across LaTeX tables components—structure, style, and content.

Pruning General Large Language Models into Customized Expert Models (2025.findings-acl)

Copied to clipboard

Yiran Zhao, Guizhen Chen, Kenji Kawaguchi, Lidong Bing, Wenxuan Zhang

Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification (2026.findings-acl)

Challenge:	Large language models (LLMs) require significant computational resources to maintain their general capabilities.
Approach:	They propose a Custom Pruning method to prune a large general model into a smaller lightweight expert model, positioned along the "language", "domain" and "task" dimensions.
Outcome:	The proposed method outperforms existing pruning methods and achieves minimal loss in both expert and general capabilities across models from different model families and sizes.

Copied to clipboard

Yuxuan Wan, Tianqing Fang, Zaitang LI, Yintong Huo, Wenxuan Wang, Haitao Mi, Dong Yu, Michael R. Lyu

Challenge:	Recent advances in Deep Research Agents (DRAs) are transforming automated knowledge discovery and problem-solving.
Approach:	They propose an inference-time scaling of verification wherein an agent self-improves at test time by evaluating its generated answers.
Outcome:	The proposed model outperforms vanilla agent-as-judge and LLM judge baselines by 12%–48% in meta-evaluation F1 score.

How Large Language Models Balance Internal Knowledge with User and Document Assertions (2026.findings-acl)

Copied to clipboard

Shuowei Li, Haoxin Li, Wenda Chu, Yi Fang

Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models (2025.findings-emnlp)

Challenge:	Large language models often need to balance their internal parametric knowledge with external information, such as user beliefs and content from retrieved documents, in real-world scenarios like RAG or chat-based systems.
Approach:	They propose a three-source interaction framework to evaluate 27 large language models from 3 families on 2 datasets.
Outcome:	The proposed framework systematically evaluates 27 large language models from 3 families on 2 datasets.

Copied to clipboard

Ryan Solgi, Kai Zhen, Rupak Vignesh Swaminathan, Nathan Susanj, Athanasios Mouchtaris, Siegfried Kunzmann, Zheng Zhang

Self-Training Elicits Concise Reasoning in Large Language Models (2025.findings-acl)

Challenge:	Low-rank tensor compression techniques are used for over-parameterized neural networks, but their applications to compress pre-trained LLMs for downstream tasks remain challenging due to the high-rank nature of pre-training data.
Approach:	They propose sparse augmented tensor networks to enhance low-rank tenorized LLMs . they also propose a framework that enables full model compression .
Outcome:	The proposed framework improves accuracy and efficiency in tensorized language models.

Copied to clipboard

Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, Se-Young Yun

V-RoLoRA: RLVR-Driven MoE Routing for Steerable Pluralistic Alignment (2026.findings-acl)

Challenge:	Chain-of-thought reasoning has enabled large language models to use additional computation through intermediate tokens to solve complex tasks, but current models often generate more tokens than necessary to accomplish the task, incurring extraneous inference costs.
Approach:	They propose to fine-tune models with self-generated concise reasoning paths obtained by best-of-N sampling and few-shot conditioning in task-specific settings to elicit concise reasoning.
Outcome:	The proposed method reduces output tokens by 30% on GSM8K and MATH while maintaining average accuracy.

Copied to clipboard

Jing Wang, Yaomin Wu, Yinglin Wang, Yitong Yang

MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning (2025.acl-long)

Challenge:	Current methods for steering large language models rely on prompt engineering or reasoning-time guidance.
Approach:	They propose a value-controllable pluralistic alignment framework enhanced with conditioned gating that dynamically directs the flow among multiple experts based on an input value or moral vector.
Outcome:	The proposed method outperforms prompt-based steering and multi-task PEFT benchmarks on two 8-billion-parameter backbones.

Copied to clipboard

Chanwoo Park, Seungju Han, Xingzhi Guo, Asuman E. Ozdaglar, Kaiqing Zhang, Joo-Kyung Kim

A Survey on Efficient Large Language Model Training: From Data-centric Perspectives (2025.acl-long)

Challenge:	Existing studies focus on prompting and developing workflows with frozen LLMs.
Approach:	They propose a multi-agentic framework for collaborative LLMs with reinforcement learning that leverages multi-gendered frameworks to enhance collaboration.
Outcome:	The proposed model improves collaboration performance across multiple datasets with generalization to unseen domains compared to existing models.

Copied to clipboard

Junyu Luo, Bohan Wu, Xiao Luo, Zhiping Xiao, Yiqiao Jin, Rong-Cheng Tu, Nan Yin, Yifan Wang, Jingyang Yuan, Wei Ju, Ming Zhang

How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients (2026.acl-long)

Challenge:	achieving data-efficient post-training of Large Language Models is a key research question.
Approach:	They propose a taxonomy of data-efficient LLM post-training methods from a data-centric perspective.
Outcome:	The proposed methods cover data selection, data quality enhancement, synthetic data generation, data distillation and compression, and self-evolving data ecosystems.

Copied to clipboard

Ming Li, Yanhong Li, Ziyue Li, Tianyi Zhou

Reward Yourself: Efficient Self Rewards for Trustworthy Sampling (2026.findings-acl)

Challenge:	Spectral properties of low/high-quality instruction and reasoning data are used to explain finetuning dynamics in large language models.
Approach:	They propose to analyze layer-wise gradients induced by low/high-quality instruction and reasoning data for LLM post-training.
Outcome:	The results show that higher-quality data are associated with lower nuclear norms and higher effective ranks.

Copied to clipboard

Mingjie Li, Wai Man Si, Michael Backes, Yang Zhang

Responsible & Trustworthy NLP

Challenge:	Retraining reward models to address privacy leaks and stereotypes is expensive . recent advances in large language models have led to improvements in understanding .
Approach:	They propose a lightweight intrinsic reward that can be used to prune existing LLMs to approximate an "untrust" and an ""untrust "" token distribution.
Outcome:	Experiments with two reward models and four LLMs show that selfRW improves trustworthiness with minimal impact on general utility benchmarks.

Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models (2026.findings-acl)

Copied to clipboard

Hoang Phan, Xianjun Yang, Yuanshun Yao, Jingyu Zhang, Shengjie Bi, Xiaocheng Tang, Madian Khabsa, Lijuan Liu, Deren Lei

Glyph: Scaling Context Windows via Visual-Text Compression (2026.acl-long)

Challenge:	Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning . however, the recipe introduces a significant risk of capability regression, where models forget foundational skills after prolonged training without employing regularization strategies.
Approach:	They propose a replay strategy with dynamic objective reweighting for general knowledge preservation using short-horizon signals of convergence and instability.
Outcome:	The proposed method preserves general capabilities and improves reasoning . it can be applied to existing RLVR pipelines without training additional models or tuning .

Copied to clipboard

Jiale Cheng, Yusen Liu, Xinyu Zhang, Yulin Fei, Wenyi Hong, Ruiliang Lyu, Weihan Wang, Zhe Su, Xiaotao Gu, Xiao Liu, Yushi Bai, Jie Tang, Hongning Wang, Minlie Huang

Multimodality

Challenge:	Large language models (LLMs) traditionally represent text as sequences of discrete tokens . a long-context scaling problem requires processing more tokens more efficiently .
Approach:	They propose a framework that renders long texts into compact visual pages and processes them with a vision-language model.
Outcome:	The proposed framework renders long texts into compact visual pages and processes them with a vision-language model.

Not All Citations Are Equal:Entropy-Guided Citation Selection for Noise-Resistant Medical LLM (2026.findings-acl)

Copied to clipboard

Minyu Gao, Hanlin Xiao, Ruoyu Wang, Shuai Yang, YeXuan Zhang, Xin Wu, Xingyu Liu

Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models (2026.acl-long)

Challenge:	Large language models have demonstrated extensive potential in medical applications . however, their practical deployment in healthcare faces significant challenges .
Approach:	They propose a training-free multi-turn reasoning framework and a post-training methodology that provides external knowledge support for large language models.
Outcome:	The proposed framework elicits internal thought, external thought, and fusion thought, with an entropy-based reward that encourages selective citation of beneficial external knowledge while penalizing noisy citations.

Copied to clipboard

Dan Shi, Zhuowen Han, Simon Ostermann, Renren Jin, Josef Van Genabith, Deyi Xiong