Papers by Bin Qi
Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to augment pre-trained large language models require extensive computational efforts and massive data volumes, challenging the widespread accessibility of LLM research. |
| Approach: | They propose a post-pretraining strategy of selectively enhancing shallow layers while pruning less effective deep ones to augment pretrained large language models. |
| Outcome: | The proposed approach improves performance on the corpus of code & math and a legal corpus and is widely applicable. |
Aegis:An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Aegis is an advanced LLM-based multi-agent for intelligent functional safety engineering that can perform all phases of a vehicle's lifecycle, including design, development, production, operation, and decommissioning. |
| Approach: | They introduce Aegis: An Advanced LLM-Based Multi-Agent for Intelligent Functional Safety Engineering. |
| Outcome: | The proposed solution can perform Hazard Analysis and Risk Assessment (HARA), document Functional Safety Requirements (FSR), and plan test cases for Automatic Emergency Braking (AEB) systems. |
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks focus on pairwise matching, ignoring robustness . current models exhibit frustrating degradation, with a maximum drop of 23.43 F1 score . |
| Approach: | They propose a benchmark that simulates the evaluation of open information extraction models in the real world . they perform experiments on typical models published in the last decade and a representative large language model . |
| Outcome: | The proposed model is rated robust on a knowledge-invariant clique with different syntactic and expressive forms. |
BeSimulator: A Large Language Model Powered Text-based Behavior Simulator (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing robot simulators focus on physical process modeling and realistic rendering, resulting in high computational costs and limited adaptability. |
| Approach: | They propose a modular and novel LLM-powered framework to analyze and validate robot behaviors in text-based environments. |
| Outcome: | The proposed framework can generalize across scenarios and achieve long-horizon complex simulation. |
LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods perform extensive matrix multiplications in domain specialization tasks, resulting in computational inefficiency and sub-optimal fine-tuning performance. |
| Approach: | They propose a method that localizes and optimizes critical parameters during training . they propose 'LoSiA-Pro' which reduces training latency by 27% . |
| Outcome: | The proposed method achieves minimal performance drop compared to full fine-tuning while requiring the least training time across domain specialization and common-sense reasoning tasks. |
Spatiotemporal Sycophancy: Negation-Based Gaslighting in Video Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Vid-LLMs lack robust mechanisms for maintaining grounded spatiotemporal beliefs under conversational feedback. |
| Approach: | They propose a negation-based gaslighting evaluation framework and introduce a benchmark to investigate spatiotemporal sycophancy. |
| Outcome: | The proposed framework evaluates state-of-the-art Vid-LLMs across video understanding tasks. |
Improving Robustness of Language Models from a Geometry-aware Perspective (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have found that removing the norm-bounded projection and increasing search steps in adversarial training can significantly improve robustness. |
| Approach: | They propose friendly adversarial data augmentation and geometry-aware adversarial training to achieve stronger robustness using fewer search steps. |
| Outcome: | The proposed method can obtain stronger robustness using fewer steps than existing methods. |
VerIF: Verification Engineering for Reinforcement Learning in Instruction Following (2025.emnlp-main)
Copied to clipboard
| Challenge: | Best practices for RL in instruction following remain underexplored. |
| Approach: | They propose a verification method that combines rule-based code verification with LLM-based verification from a large reasoning model. |
| Outcome: | The proposed method achieves state-of-the-art performance among models of comparable size and generalizes well to unseen constraints. |
A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to accelerate inference speed of pre-trained language models are limited to local representations of exit layer . current models are associated with large memory requirement and high computational cost, which slow down inference and further encumber the application of PLMs. |
| Approach: | They propose a method to exit early without passing through all inference layers . they take into consideration all the linguistic information embedded in the past layers a global perspective . |
| Outcome: | The proposed method outperforms existing methods by a large margin . it uses linguistic information embedded in the past layers and future features . the proposed method is scalable and cost-effective . |
Program Translation via Code Distillation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Software version migration and program translation are costly parts of the lifecycle of large codebases. |
| Approach: | They propose a model that captures semantic and structural equivalence of code in a language agnostic intermediate representation. |
| Outcome: | The proposed model achieves state-of-the-art performance on CodeXGLUE and TransCoder GeeksForGeeks translation benchmarks. |
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)
Copied to clipboard
Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang
| Challenge: | Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases. |
| Approach: | They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs. |
| Outcome: | The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs. |
ADELIE: Aligning Large Language Models on Information Extraction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) struggle to follow complex instructions of IE tasks due to not being aligned with humans. |
| Approach: | They propose an aligned large language moDEL that effectively solves various IE tasks including closed IE, open IE and on-demand IE. |
| Outcome: | The proposed model achieves state-of-the-art (SoTA) performance among open-source models. |
V-VAE: A Variational Auto Encoding Framework Towards Fine-Grained Control over Human-Like Chat (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing role-play and persona-based chat approaches rely on static role descriptions, coarse-grained signal space, and low-quality synthetic data. |
| Approach: | They propose a Verbal Variational Auto-Encoding framework which dynamically adapts dialogue behaviour based on latent variables across talking style, interaction patterns, and personal attributes. |
| Outcome: | The proposed framework outperforms baselines on HumanChatBench and DialogBench to address the scarcity of high-quality data in the human-like domain. |
TEACH: A Contrastive Knowledge Adaptive Distillation Framework for Classical Chinese Understanding (2025.acl-long)
Copied to clipboard
| Challenge: | Traditional methods for processing classical Chinese segment language understanding into discrete tasks, which overlook crucial background information and reduce user engagement. |
| Approach: | They propose a framework that integrates word sense disambiguation with sentence translation to minimize hallucinations and improve semantic analysis. |
| Outcome: | The proposed framework integrates word sense disambiguation with sentence translation to minimize hallucinations and improve semantic analysis. |
Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations. |
| Approach: | They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models. |
| Outcome: | Extensive experiments show that the proposed framework can improve results over existing models. |
Neural Network Surgery: Injecting Data Patterns into Pre-trained Models with Minimal Instance-wise Side Effects (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing neural network tuning methods cause instance-wise side effects . et al., 2018: a new approach to perform neural network surgery . |
| Approach: | They propose to perform neural network surgery by only changing 10-5 parameters . they propose to use a dynamic selecting method to achieve the best overall performance . |
| Outcome: | The proposed method achieves the best overall performance and induces fewer instance-wise side effects by changing only 10-5 of the parameters. |
TRAC: Teacher-Guided Token Reward with Adaptive Calibration for Robust Policy Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Current reward models for reinforcement learning (RL) rely on outcome rewards that propagate a single scalar value across all tokens based on final correctness. |
| Approach: | They propose a framework that derives dense token-level supervision from LLMs . they use a multi-granularity calibration mechanism to modulate teacher influence . |
| Outcome: | The proposed framework evaluates teacher reliability across problem-level expertise, trajectory-level discrimination, and token-level confidence. |
SUT: Active Defects Probing for Transcompiler Models (2023.emnlp-main)
Copied to clipboard
Mengnan Qi, Yufan Huang, Maoquan Wang, Yongqiang Yao, Zihan Liu, Bin Gu, Colin Clement, Neel Sundaresan
| Challenge: | Existing datasets are often criticized for their lack of granularity, which can mask deficiencies in basic syntactic elements that humans care about. |
| Approach: | They propose a new program translation metrics that address basic syntax errors . they propose BLUE, CodeBLUE and computation accuracy metrics which address these errors based on a highly interpretable evaluation harness. |
| Outcome: | The proposed model passes the unit tests with a 26.15% pass rate compared to previous models . |
Syntactically Robust Training on Partially-Observed Data for Open Information Extraction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Open Information Extraction models have shown promising results with sufficient supervision, but the syntactic distribution of training data is partially observable in comparison to the real world. |
| Approach: | They propose a syntactically robust training framework that enables models to be trained on a multi-paraphrase distribution based on diverse paraphrase generation. |
| Outcome: | The proposed framework can be applied to other syntactic partial observable domains. |
Exploring Question Guidance and Answer Calibration for Visually Grounded Video Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for videoQA lack temporal localization labels, leading to inaccurate localization. |
| Approach: | They propose a Question-Guided and Answer-Calibrated TRansformer which guides and calibrates localization using question and option texts without localization labels. |
| Outcome: | The proposed model achieves comparable accuracy to large-scale pretrained models and leads in localization aspects. |
Towards Economical Inference: Enabling DeepSeek’s Multi-Head Latent Attention in Any Transformer-based LLMs (2025.acl-long)
Copied to clipboard
Tao Ji, Bin Guo, Yuanbin Wu, Qipeng Guo, Shenlixing Shenlixing, Chenzhan Chenzhan, Xipeng Qiu, Qi Zhang, Tao Gui
| Challenge: | Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference by significantly compressing the Key-Value (KV) cache into a latent vector. |
| Approach: | They propose a data-efficient fine-tuning method for transitioning from MHA to MLA using a latent vector cache. |
| Outcome: | The proposed architecture reduces the KV cache size of Llama2-7B by 92.19%, with only 1% drop in LongBench performance. |
MAVEN-FACT: A Large-scale Event Factuality Detection Dataset (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Event factuality detection is under-explored due to the lack of high-quality large-scale data . efd is a subfield of event understanding, which aims to determine the factuity of textual events. |
| Approach: | They propose a large-scale EFD dataset with factuality annotations of 112,276 events . they find that adopting event arguments and relations helps in event factuity detection . |
| Outcome: | The proposed dataset includes factuality annotations of 112,276 events . it is the largest EFD dataset and is challenging for fine-tuned models and large language models . |
Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems (2025.acl-long)
Copied to clipboard
| Challenge: | Existing reward models focus on human preferences, neglecting verifiable correctness signals. |
| Approach: | They propose a reward system that combines human preference rewards with verifiable correctness signals to provide reliable rewards. |
| Outcome: | The proposed reward agent significantly outperforms vanilla reward models on benchmarks and inference-time best-of-n searches on real-world tasks. |