Papers by Hang Chen

APOLLO: An Optimized Training Approach for Long-form Numerical Reasoning (2024.lrec-main)

Challenge:	Large language models have demonstrated impressive performance across a wide range of tasks, but this achievement comes with the trade-off of significant computational demands.
Approach:	They propose a scaling law that decomposes the overall validation loss and assigns different importance weights to tokens to assess a specific meta-capability.
Outcome:	The proposed model can predict the loss trending of models across different levels of computation without a gap between validation loss and model's downstream capabilities.

Copied to clipboard

Jiashuo Sun, Hang Zhang, Chen Lin, Xiangdong Su, Yeyun Gong, Jian Guo

Programming Languages in NLP

Challenge:	Existing methods to generate reasoning programs that ignore the differences between facts treated all facts equally, leading to wrong punishment of programs that differed from the ground truth.
Approach:	They propose an optimized training framework for long-form numerical reasoning that incorporates a number-aware negative sampling strategy and consistency-based reinforcement learning to increase execution accuracy.
Outcome:	The proposed method improves the performance of long-form numerical reasoning on the FinQA and ConvFinQA leaderboards.

Guided Knowledge Generation with Language Models for Commonsense Reasoning (2024.findings-emnlp)

Copied to clipboard

Xiao Wei, Haoran Chen, Hang Yu, Hao Fei, Qian Liu

Reasoning

Challenge:	Large Language Models (LLMs) have achieved notable success in commonsense reasoning tasks, benefiting from extensive world knowledge acquired through extensive pretraining.
Approach:	They propose a method to generate knowledge explanations and to automatically assign labels based on the probability of correct answers.
Outcome:	The proposed method outperforms baselines on four widely-used commonsense reasoning benchmarks and shows that it can generate high quality knowledge leading to correct answers.

Ask-Before-Detection: Identifying and Mitigating Conformity Bias in LLM-Powered Error Detector for Math Word Problem Solutions (2025.acl-long)

Copied to clipboard

Hang Li, Tianlong Xu, Kaiqi Yang, Yucheng Chu, Yanling Chen, Yichi Song, Qingsong Wen, Hui Liu

Numerical Reasoning

Challenge:	Recent studies have demonstrated the potential of large language models (LLMs) for automatic error detection in math word problems (MWPs).
Approach:	They propose a framework that generates adaptive reference solutions using LLMs to enhance error detection by reducing conformity bias in MWPs.
Outcome:	The proposed framework mitigates the performance gap between conventional and alternative solutions in MWPs, especially when combined with reasoning-enhancing techniques like chain-of-thought prompting.

SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utility (2026.acl-long)

Copied to clipboard

Xuyang Zhi, Peilun Zhou, Chengqiang Lu, Hang Lv, Yiwei Liang, Rongyang Zhang, Yan Gao, null Yiwu, Yao Hu, Hongchao Gu, Defu Lian, Hao Wang, Enhong Chen

Challenge:	Large Language Models (LLMs) are shifting the focus from single verifiable tasks toward complex, open-ended real-world scenarios.
Approach:	They propose a framework that automatically adjusts reward weights and data importance to synchronize learning intent with data utility for optimal performance.
Outcome:	The proposed framework improves model capabilities across all domains and scales.

CoBa: Convergence Balancer for Multitask Finetuning of Large Language Models (2024.emnlp-main)

Copied to clipboard

Zi Gong, Hang Yu, Cong Liao, Bingchang Liu, Chaoyu Chen, Jianguo Li

Debiasing the Fine-Grained Classification Task in LLMs with Bias-Aware PEFT (2025.acl-long)

Challenge:	Existing multi-task learning approaches for large language models fall short due to computational intensive or lack of simultaneous task convergence.
Approach:	They propose a new multi-task learning approach that dynamically adjusts task weights during the training process, ensuring that the validation loss of all tasks progresses towards convergence at an even pace.
Outcome:	The proposed approach improves the performance of large language models by up to 13% compared to the second-best approaches.

Copied to clipboard

Daiying Zhao, Xinyu Yang, Hang Chen

Robustness in NLP

Challenge:	Existing methods to mitigate label biases such as retraining, post-hoc adjustment, and parameter-efficient fine-tuning fail to address prediction propensity and discriminative ability biase.
Approach:	They propose a bias-aware optimization framework that incorporates two distinct label balance constraints with a PEFT strategy targeting an intermediate layer to mitigate this issue.
Outcome:	The proposed approach outperforms or matches the performance of full-parameter fine-tuning and LoRA, achieving superior results with lower perplexity.

Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis (2022.emnlp-main)

Copied to clipboard

Shuai Fan, Chen Lin, Haonan Li, Zhenghao Lin, Jinsong Su, Hang Zhang, Yeyun Gong, JIan Guo, Nan Duan

Sentiment Analysis

Challenge:	Existing pre-trained language representation models (PLMs) capture sentiment information from word-level while under-considering sentence-level information.
Approach:	They propose a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks that enhance the PLM’s knowledge about sentiment words.
Outcome:	The proposed model achieves state-of-the-art on various sentence-level and aspect-level sentiment classification benchmarks.

BiKT: Enabling Bidirectional Knowledge Transfer Between Pretrained Models and Sequential Downstream Tasks (2024.findings-emnlp)

Copied to clipboard

Hang Zeng, Chaoyue Niu, Fan Wu, Shaojie Tang, Leihao Pei, Chengfei Lv, Guihai Chen

Document-Level Relation Extraction via Pair-Aware and Entity-Enhanced Representation Learning (2022.coling-1)

Challenge:	Existing frameworks adapt from initial pretrained model to each downstream task directly, but ignore sequential nature of downstream tasks and feedback effect on pretrained models.
Approach:	They propose a framework to enable bidirectional knowledge transfer between pretrained models and downstream tasks in rounds.
Outcome:	The proposed framework improves on 9 GLUE datasets and 6 SuperGLUEs.

Copied to clipboard

Xiusheng Huang, Hang Yang, Yubo Chen, Jun Zhao, Kang Liu, Weijian Sun, Zuyu Zhao

How to Enhance Causal Discrimination of Utterances: A Case on Affective Reasoning (2023.emnlp-main)

Challenge:	Existing document-level relation extraction methods are sparse in relational entity pairs and the representation of entity pairs is insufficient.
Approach:	They propose a Pair-Aware and Entity-Enhanced(PAEE) model to solve two challenges . they propose predicting potential relational entity pairs and assembling directional entity pairs .
Outcome:	The proposed model can obtain state-of-the-art performance on four benchmark datasets . it can predict potential relational entity pairs and assemble directional entity pairs .

Copied to clipboard

Hang Chen, Xinyu Yang, Jing Luo, Wenjing Zhu

Representation Learning

Challenge:	Existing models excel at capturing semantic correlations within utterance embeddings but fail to determine specific causal relationships.
Approach:	They propose to incorporate i.i.d. noise terms into conversation process to build a structural causal model . they propose to use unstructured conversation data to facilitate deep learning .
Outcome:	The proposed approach can be implemented in unstructured conversation data and a synthetic dataset that includes i.i.d. noise.

Collective Event Detection via a Hierarchical and Bias Tagging Networks with Gated Multi-level Attention Mechanisms (D18-1)

Copied to clipboard

Yubo Chen, Hang Yang, Kang Liu, Jun Zhao, Yantao Jia

Syntactic Text Processing

Challenge:	Existing approaches to ACE event detection treat multiple events in one sentence as independent ones and recognize them separately.
Approach:	They propose a hierarchical and bias tagging network framework to detect multiple events in one sentence collectively and a gated multi-level attention mechanism to automatically extract and fuse the sentence-level and document-level information.
Outcome:	The proposed framework outperforms state-of-the-art methods on a 2005 ACE dataset.

DCFEE: A Document-level Chinese Financial Event Extraction System based on Automatically Labeled Training Data (P18-4)

Copied to clipboard

Hang Yang, Yubo Chen, Kang Liu, Yang Xiao, Jun Zhao

Quantifying Semantic Emergence in Language Models (2025.acl-long)

Challenge:	Existing methods to extract events from documents are limited due to the high cost of labeling . Experimental results demonstrate the effectiveness of a document-level Chinese financial event extraction system.
Approach:	They propose a document-level Chinese financial event extraction framework which detects event mentions and extracts events from financial news.
Outcome:	The proposed system detects event mentions and extracts events from financial news . it can generate large scale labeled data and extract events from entire document .

Copied to clipboard

Hang Chen, Xinyu Yang, Jiaying Zhu, Wenya Wang

Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy LLMs (2026.findings-acl)

Challenge:	Existing evaluation methods for large language models (LLMs) focus on coarse-grained text, not providing interpretations for the behavior of finergrained tokens.
Approach:	They propose a quantitative metric to measure large language models’ ability to extract semantics from input tokens.
Outcome:	The proposed metric compares the entropy reduction observed for a sequence of tokens and individual tokens.

Copied to clipboard

Yue Huang, Haomin Zhuang, Jiayi Ye, Han Bao, Yanbo Wang, Hang Hua, Siyuan Wu, Pin-Yu Chen, Xiangliang Zhang

UBench: Benchmarking Uncertainty in Large Language Models with Multiple Choice Questions (2025.findings-acl)

Challenge:	prevailing taxonomies neglect robustness and honesty, yielding safer-on-paper but less useful systems.
Approach:	They propose a soft-gating pipeline where a guardian predicts a binary risk label plus a concise explanation and prepends this advice to the original query for re-inference.
Outcome:	The proposed model maintains safety while reducing over-refusal.

Copied to clipboard

Xunzhi Wang, Zhuowei Zhang, Gaonan Chen, Qiongyu Li, Bitong Luo, Zhixin Han, Haotian Wang, Zhiyu Li, Hang Gao, Mengting Hu

DRAGON: Domain-specific Robust Automatic Data Generation for RAG Optimization (2026.findings-eacl)

Challenge:	Existing methods for benchmarking the uncertainty of large language models face challenges . existing methods require internal model access, additional training, or high computational costs .
Approach:	They propose a new benchmark for evaluating the uncertainty of large language models based on confidence intervals . UBench encompasses 11,978 multiple choice questions spanning knowledge, language, understanding, and reasoning capabilities.
Outcome:	The proposed method outperforms existing methods for benchmarking the uncertainty of large language models.

Copied to clipboard

Haiyang Shen, Hang Yan, Zhongshi Xing, Mugeng Liu, Yue Li, Zhiyang Chen, Yuxiang Wang, Jiuzheng Wang, Yun Ma

Robustness in NLP

Challenge:	Existing retrieval-augmented generation paradigms rely heavily on public knowledge . Existing RAGs reliant on public information and often falter when faced with domain-specific queries.
Approach:	They propose a framework that combines a data-construction modeling approach with a scalable synthetic data-generation pipeline to optimize domain-specific retrieval performance.
Outcome:	The proposed framework optimizes domain-specific retrieval performance and bolsters retriever robustness.

HICD: Hallucination-Inducing via Attention Dispersion for Contrastive Decoding to Mitigate Hallucinations in Large Language Models (2025.findings-acl)

Copied to clipboard

Xinyan Jiang, Hang Ye, Yongxin Zhu, Xiaoying Zheng, Zikang Chen, Jun Gong

AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization via Multi-LLMs (2025.findings-naacl)

Challenge:	Large Language Models (LLMs) often generate hallucinations, producing outputs that are contextually inaccurate or factually incorrect.
Approach:	They propose a method that selects attention heads crucial to the model's prediction as inducing heads and induces hallucinations by dispersing attention of these inducers.
Outcome:	The proposed method significantly improves performance on tasks requiring contextual faithfulness, reading comprehension, and question answering.

Copied to clipboard

Jiawei Chen, Xiao Yang, Zhengwei Fang, Yu Tian, Yinpeng Dong, Zhaoxia Yin, Hang Su

LEDGER: Scaling Agentic Document Editing with Dependency-aware Graph Retrieval (2026.findings-acl)

Challenge:	Existing jailbreak research exhibits limitations in universality, validity, and efficiency . Existing methods for jailbreaking LLMs have limited validity and effectiveness .
Approach:	They propose a black-box approach that uses wordplay-guided mapping rule sampling to create universal adversarial prompts.
Outcome:	The proposed method efficiently identifies security vulnerabilities across various LLMs, achieving an average success rate of over 80% with fewer than 10 queries.

Copied to clipboard

Hang Wang, Utkarsh Garg, Reza Davari, Huitian Jiao, Hao Cheng, Baolin Peng, Si-Qing Chen, Tao Ge

Green & Sustainable NLP

Challenge:	Document editing requires full-context awareness of dependencies, but processing entire documents for each edit incurs prohibitive token costs and latency.
Approach:	a framework that constructs lightweight dependency graphs captures semantic relationships and structural hierarchies across document elements is proposed for agentic document editing . a scaLing agentic agentic framework is based on a dependency graph framework that captures dependencies and refactors function dependencies.
Outcome:	a new framework achieves 76 consistency versus 56 baseline while reducing token usage by 85 . the framework is based on a framework that captures semantic relationships and structural hierarchies across document elements . it can be used to improve document consistency, but it also reduces token costs and latency .

What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practices (2025.acl-long)

Copied to clipboard

Zhi Chen, Qiguang Chen, Libo Qin, Qipeng Guo, Haijun Lv, Yicheng Zou, Hang Yan, Kai Chen, Dahua Lin

Challenge:	Existing methods to generate long-context instruction-tuning data are limited by poor quality and fewer than 35% of samples are multi-hop .
Approach:	They propose a framework that integrates a quality verification agent, a single-hop question generation agent, and a multi-hop questions merger agent to enhance model performance.
Outcome:	The proposed framework significantly improves data quality with high-quality, multi-hop, and diverse data.

What does the language of foods say about us? (D19-62)

Copied to clipboard

Hoang Van, Ahmad Musa, Hang Chen, Stephen Kobourov, Mihai Surdeanu

Challenge:	Using a dataset of 24 million food-related tweets, we can predict if states in the United States are above the median rates for type 2 diabetes mellitus (T2DM) income, poverty, and education are important factors in predicting T2DM rates, but socioeconomic factors do not capture this information.
Approach:	They use a dataset of 24 million food-related tweets to investigate the signal contained in the language of food on social media.
Outcome:	The language of food can predict health risks, political orientation, and geographic location, and outperform previous work by 4–18%.

Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration (2026.acl-short)

Copied to clipboard

Hang Lv, Hongchao Gu, Ruiqing Yang, Liangyue Li, Zulong Chen, Defu Lian, Hao Wang, Enhong Chen

MLeVLM: Improve Multi-level Progressive Capabilities based on Multimodal Large Language Model for Medical Visual Question Answering (2024.findings-acl)

Challenge:	Existing methods for listwise reranking exhibit intrinsic position bias . existing methods are constrained by an inherent trade-off between efficiency and flexibility .
Approach:	They propose a training-free framework that mechanically decouples positional bias from ranking decisions.
Outcome:	a training-free framework decouples position bias from ranking decisions . evaluations show it outperforms training-based methods and outperformed expensive methods .

Copied to clipboard

Dexuan Xu, Yanyuan Chen, Jieyi Wang, Yue Huang, Hanpin Wang, Zhi Jin, Hongxing Wang, Weihua Yue, Jing He, Hang Li, Yu Huang

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Challenge:	Existing MVQA models ignore multi-level progressive capabilities due to unspecific data and plain architecture.
Approach:	They propose a multi-level visual language model for medical visual question answering (MVQA) which covers multi- level questions and answers as well as reasoning processes from visual clues to semantic cognition.
Outcome:	The proposed model outperforms existing medical multimodal large language models on a multi-level instruction dataset and a feature alignment module.

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

Reconstructing Event Regions for Event Extraction via Graph Attention Networks (2020.aacl-main)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Pei Chen, Hang Yang, Kang Liu, Ruihong Huang, Yubo Chen, Taifeng Wang, Jun Zhao

BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Proving (2025.acl-long)

Challenge:	Existing approaches for event extraction focus on sentence-level event extraction, but they lack a broader view of the document context.
Approach:	They build graphs with candidate event filler extractions enriched by sentential embeddings as nodes and use graph attention networks to identify event regions in a document and aggregate event information.
Outcome:	The proposed method performs well on two languages and shows that it is faster than previous methods.

Copied to clipboard

Ran Xin, Chenguang Xi, Jie Yang, Feng Chen, Hang Wu, Xia Xiao, Yifan Sun, Shen Zheng, Ming Ding

Information Retrieval

Challenge:	Existing approaches to theorem proving in large language models rely on value functions and/or Monte Carlo Tree Search (MCTS), but the potential of simpler methods like Best-First Tree Search remains underexplored.
Approach:	They propose a scalable expert iteration framework that implements strategic data filtering at each expert iteration round, excluding problems solvable via beam search node expansion to focus on harder cases.
Outcome:	The proposed framework achieves a state-of-the-art score of 72.95 on the MiniF2F test set and challenges the perceived necessity of complex tree search methods.

Red Teaming Large Reasoning Models (2026.acl-long)

Copied to clipboard

Jiawei Chen, Yang Yang, Chao Yu, Yu Tian, Zhi Cao, Xue Yang, Linghao Li, Hang Su, Zhaoxia Yin

RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery (2025.findings-acl)

Challenge:	Large Reasoning Models (LRMs) have emerged as a powerful advancement in multi-step reasoning tasks, but they introduce safety and reliability risks, such as CoT-hijacking and prompt-induced inefficiencies.
Approach:	They propose a unified benchmark to assess the trustworthiness of Large Reasoning Models.
Outcome:	The proposed benchmark evaluates truthfulness, safety and efficiency on 26 models.

Copied to clipboard

Hongchao Gu, Dexun Li, Kuicai Dong, Hao Zhang, Hang Lv, Hao Wang, Defu Lian, Yong Liu, Enhong Chen

Document-level Event Extraction via Parallel Prediction Networks (2021.acl-long)

Challenge:	Existing methods for knowledge-intensive long texts struggle with issues like hallucinations, topic incoherence, and significant latency.
Approach:	They propose a retrieval-augmented long text generation framework with writing Planning and Information to address these challenges.
Outcome:	The proposed framework outperforms state-of-the-art methods on a freshWiki-2024 dataset.

Copied to clipboard

Hang Yang, Dianbo Sui, Yubo Chen, Kang Liu, Jun Zhao, Taifeng Wang

Language Model Analysis for Ontology Subsumption Inference (2023.findings-acl)

Challenge:	Document-level event extraction (DEE) is indispensable when events are described throughout a document.
Approach:	They propose a document-level event extraction model that can extract structured events from a text in parallel.
Outcome:	The proposed model outperforms current state-of-the-art methods on a document-level event extraction task.

Copied to clipboard

Yuan He, Jiaoyan Chen, Ernesto Jimenez-Ruiz, Hang Dong, Ian Horrocks

SeaLLMs - Large Language Models for Southeast Asia (2024.acl-demos)

Challenge:	Existing studies focus on simple, triple-based, relational KBs but omit more sophisticated, logic-based conceptualised KB.
Approach:	They propose to use ontology subsumption axioms to probe LMs' knowledge of ontologies by probing datasets from atomic and complex concepts.
Outcome:	The proposed methods encode less background knowledge of Subsumption Inference (SI) than traditional Natural Language Inference but can improve on SI significantly when a small number of samples are given.

Copied to clipboard

Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Zhiqiang Hu, Chenhui Shen, Yew Ken Chia, Xingxuan Li, Jianyu Wang, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, Lidong Bing

DiMo-GUI: Advancing Test-time Scaling in GUI Grounding via Modality-Aware Visual Reasoning (2025.emnlp-main)

Challenge:	Existing large language models favor high-resource languages, such as English, at the expense of low-resourced and regional languages.
Approach:	They propose a series of language models that specifically focuses on Southeast Asian languages.
Outcome:	SeaLLM models outperform ChatGPT-3.5 in non-Latin languages by large margins . linguistic disparity impedes access to state-of-the-art AI technologies for non-English-speaking populations .

Copied to clipboard

Hang Wu, Hongkai Chen, Yujun Cai, Chang Liu, Qingwen Ye, Ming-Hsuan Yang, Yiwei Wang

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors (2026.findings-acl)

Challenge:	DiMo-GUI is a training-free framework for GUI grounding that splits input into textual elements and iconic elements, allowing the model to reason over each modality independently using general-purpose vision-language models.
Approach:	They propose a training-free framework for GUI grounding that leverages two core strategies: dynamic visual grounding and modality-aware optimization.
Outcome:	The proposed framework splits the input into textual elements and iconic elements, allowing the model to reason over each modality independently using general-purpose vision-language models.

Copied to clipboard

Hang Su, Zequn Liu, Chen Hu, Xuesong Lu, Yingce Xia, Liu Zhen

Natural Language Interfaces

Challenge:	Existing LLMs rely on surface-level similarity or manual heuristics to evaluate personalization . Existing evaluation protocols for personalization are lacking sufficient data-driven validation.
Approach:	They propose a benchmark to assess personalization by mining CIPDs to quantify individual preferences.
Outcome:	The proposed benchmark provides a more comprehensive and discriminative standard than generic metrics.

Doc-V*: Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA (2026.acl-long)

Copied to clipboard

Yuanlei Zheng, Pei Fu, Hang Li, Ziyang Wang, Yuyi Zhang, Wenyu Ruan, Xiaojin Zhang, Zhongyu Wei, Zhenbo Luo, Jian Luan, Wei Chen, Xiang Bai

GA-S3: Comprehensive Social Network Simulation with Group Agents (2025.findings-acl)

Challenge:	Existing OCR-free approaches to document visual question answering are brittle and passive.
Approach:	They propose an OCR-free agentic framework that casts multi-page DocVQA as sequential evidence aggregation.
Outcome:	The proposed framework outperforms open-source and proprietary models in five benchmarks and improves out-of-domain performance by 47.9% over baseline.

Copied to clipboard

Yunyao Zhang, Zikai Song, Hang Zhou, Wenfeng Ren, Yi-Ping Phoebe Chen, Junqing Yu, Wei Yang

Challenge:	Existing social network simulations focus on discrete events or system dynamics instead of elucidating underlying mechanisms or causal relationships.
Approach:	They propose a Social network simulation system that leverages newly designed Group Agents to make intelligent decisions regarding various online events.
Outcome:	The proposed system can make intelligent decisions regarding online events at a manageable cost.

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way (2023.emnlp-demo)

Copied to clipboard

Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search (2022.emnlp-main)

Challenge:	Large language models (LLMs) are increasingly pivotal in a wide range of tasks . however, the resources required for training these models necessitate efficient solutions .
Approach:	They propose a library that facilitates collaborative training of large language models . they use 3D parallelism, parameter-efficient fine-tuning methods and optimizers .
Outcome:	The proposed library has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios.

Copied to clipboard

Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, Nan Duan

Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm (2022.acl-long)

Challenge:	Existing code pre-training approaches often adopt (masked) language modeling as the training objective which targets on learning to predict (macked) tokens in a given code context.
Approach:	They propose a code-text contrastive learning model which learns function-level code semantic representations through large-scale code corpus.
Outcome:	The proposed model achieves new state-of-the-art with significant improvement over existing pre-trained models on eleven domain/language-specific code search tasks with six programming languages in different code granularity.

Copied to clipboard

Shaoyi Huang, Dongkuan Xu, Ian Yen, Yijue Wang, Sung-En Chang, Bingbing Li, Shiyang Chen, Mimi Xie, Sanguthevar Rajasekaran, Hang Liu, Caiwen Ding

LCO: LLM-based Constraint Optimization for Safer Agentic LLMs in Real-world Tasks (2026.findings-acl)

Challenge:	Conventional wisdom in pruning Transformer-based language models is that it reduces model expressiveness, but new research shows pruning increases risk of overfitting when performed at the fine-tuning phase.
Approach:	They propose to reduce pruning risk under pretrain-and-finetune paradigm . they propose to use knowledge distillation to improve pruning performance .
Outcome:	The proposed method outperforms the leading competitors on the GLUE benchmark.

Copied to clipboard

Jiayong Wan, Jiawei Chen, Zhaoxia Yin, Liu Shuyuan, Hang Su

GoViG: Goal-Conditioned Visual Navigation Instruction Generation via Multimodal Reasoning (2026.findings-acl)

Challenge:	Existing defense methods are insufficient to address in-context reward hacking (ICRH), where LLMs iteratively optimize their behavior to maximize proxy objectives, resulting in harmful side effects.
Approach:	They propose a framework that reduces in-context reward hacking (ICRH) through repeated interactions with the environment.
Outcome:	The proposed framework reduces ICRH without model fine-tuning while maintaining task performance.

Copied to clipboard

Fengyi Wu, Yifei Dong, Yilong Dai, Guangyu Chen, Qifeng Wu, Huiting Huang, Hang Wang, Qi Dai, Alexander G Hauptmann, Zhi-Qi Cheng