Papers by Bingning Wang
Unveiling the Flaws: Exploring Imperfections in Synthetic Data and Mitigation Strategies for Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | acquiring large amounts of high-quality data can be challenging due to data scarcity, privacy concerns, and high costs. |
| Approach: | They propose a method which reverses instruction-following issues caused by uniform format of synthetic data and proposes unlearning techniques to mitigate these flaws. |
| Outcome: | The proposed method reverses instruction-following issues caused by pattern overfitting without compromising performance on benchmarks at relatively low cost. |
LLaSA: Large Language and Structured Data Assistant (2025.naacl-long)
Copied to clipboard
Yao Xu, Shizhu He, Jiabei Chen, ZengXiangrong ZengXiangrong, Bingning Wang, Guang Liu, Jun Zhao, Kang Liu
| Challenge: | Structured knowledge grounding (SKG) tasks are a key part of many NLP applications. |
| Approach: | They propose a framework for enhancing LLMs' ability to handle structured data . they represent various types of structured data in a unified hypergraph format . |
| Outcome: | The proposed framework outperforms existing methods on SKG tasks using LoRA finetuning. |
LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have extended context windows through scaling positional encodings and lightweight continual pre-training, but performance degradation is still not fully explored. |
| Approach: | They propose a novel approach to reduce short-text performance degradation by minimizing distribution drift in hidden states and attention scores. |
| Outcome: | The proposed approach minimizes the distribution discrepancy between the extended and original models while maintaining or even enhancing the model's long-context abilities. |
Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work relies on training with multi-lingual ability-related data, which may not be available for low-resource languages. |
| Approach: | They propose a multi-lingual ability-enhanced LLM that extracts language-agnostic ability-related weights from LLMs and combine them across different languages by simple addition and subtraction operations without training. |
| Outcome: | The proposed approach extracts language-agnostic ability-related weights from LLMs and combine them across different languages without training. |
Multi-Lingual Question Generation with Language Agnostic Language Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing training data for question generation in English and Chinese is limited . a language-agnostic model is developed to learn the shared representation from several languages in a single architecture. |
| Approach: | They propose a language-agnostic language model which learns the shared representation from several languages in a single architecture. |
| Outcome: | The proposed model improves multi-lingual question generation over five languages. |
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect (2025.findings-acl)
Copied to clipboard
Xin Men, Mingyu Xu, Qingyu Zhang, Qianhao Yuan, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, Weipeng Chen
| Challenge: | Recent studies have identified significant redundancy in large language models . quantization and pruning are two methods that reduce computational resources . |
| Approach: | They propose simple pruning methods that prune redundant layers based on their BI scores. |
| Outcome: | The proposed pruning methods demonstrate superior performance over previous pruning methods. |
MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods face the trilemma of performance, data privacy, and computational costs, which hinders their application to LLMs. |
| Approach: | They propose a model-exclusive task arithmetic method for merging GPT-scale models which is data-agnostic and bypasses the heavy search process. |
| Outcome: | The proposed method achieves state-of-the-art performance on multiple tasks while minimizing the average loss difference between the merged model and each individual task model. |
Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing ABSA test sets cannot be used to distinguish the sentiment of the target aspect from the non-target aspect. |
| Approach: | They propose a simple but effective approach to enrich ABSA test sets by disentangle the confounding sentiments of non-target aspects from the target aspect’s sentiment. |
| Outcome: | The proposed model can distinguish the sentiment of the non-target aspects from the target aspect’s sentiment by using the Aspect Robustness Test Set (ARTS). |