Papers by Hang Ding
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained large-scale language models have been criticized for their limited weight storage and computational speed on hardware platforms. |
| Approach: | They propose an efficient transformer-based large-scale language representation using hardware-friendly block structure pruning. |
| Outcome: | The proposed model achieves 5.0x accuracy on GLUE benchmarks and 1.79x compression rate on DistilBERT. |
TAG: Gradient Attack on Transformer-based Language Models (2021.findings-emnlp)
Copied to clipboard
Jieren Deng, Yijue Wang, Ji Li, Chenghong Wang, Chao Shang, Hang Liu, Sanguthevar Rajasekaran, Caiwen Ding
| Challenge: | Recent studies show that publicly shared gradients in the training process can reveal the private training data to a third-party. |
| Approach: | They propose a gradient attack algorithm to reconstruct the local training data using GLUE benchmarks. |
| Outcome: | The proposed algorithm achieves 1.5x recover rate and 2.5x ROUGE-2 over previous methods without the need of ground truth label. |
ARK: Answer-Centric Retriever Tuning via KG-augmented Curriculum Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) is a powerful framework for knowledge-intensive tasks, but its effectiveness in long-context scenarios is often bottlenecked by the retriever’s inability to distinguish sparse yet crucial evidence. |
| Approach: | They propose a framework that fine-tunes the retriever for Answer Alignment by identifying high-quality positive chunks by evaluating their sufficiency to generate the correct answer. |
| Outcome: | The proposed framework improves 14.5% over the base model and maintains strong efficiency for long-context RAG. |
SciRAG: Adaptive, Citation-Aware, and Outline-Guided Retrieval and Synthesis for Scientific Literature (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing retrieval-augmented generation methods overlook citation graph structure, adapt poorly to complex queries, and yield fragmented, hard-to-verify syntheses. |
| Approach: | They propose a retrieval-augmented generation framework that addresses these gaps by combining adaptive retrieval and symbolic reasoning. |
| Outcome: | Extensive experiments show that SciRAG outperforms prior systems in factual accuracy and synthesis quality. |
Disco-RAG: Discourse-Aware Retrieval-Augmented Generation (2026.acl-long)
Copied to clipboard
Dongqi Liu, Hang Ding, Qiming Feng, Xurong Xie, Zhucun Xue, Chengjie Wang, Jian Li, Jiangning Zhang, Yabiao Wang
| Challenge: | Existing RAG strategies treat retrieved passages in a flat and unstructured way, which prevents the model from capturing structural cues and constrains its ability to synthesize knowledge from dispersed evidence across documents. |
| Approach: | They propose a framework that explicitly injects discourse signals into the generation process. |
| Outcome: | Experiments on question answering and long-document summarization benchmarks show the efficacy of the proposed framework. |
Mathematical Word Problem Generation from Commonsense Knowledge Graph and Equations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for generating mathematical word problems are lacking in educational assessment. |
| Approach: | They propose an end-to-end neural model to generate diverse mathematical word problems from commonsense knowledge graph and equations. |
| Outcome: | The proposed model outperforms the SOTA models in terms of evaluation metrics and topic relevance. |
BFS-Prover: Scalable Best-First Tree Search for LLM-based Automatic Theorem Proving (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to theorem proving in large language models rely on value functions and/or Monte Carlo Tree Search (MCTS), but the potential of simpler methods like Best-First Tree Search remains underexplored. |
| Approach: | They propose a scalable expert iteration framework that implements strategic data filtering at each expert iteration round, excluding problems solvable via beam search node expansion to focus on harder cases. |
| Outcome: | The proposed framework achieves a state-of-the-art score of 72.95 on the MiniF2F test set and challenges the perceived necessity of complex tree search methods. |
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm (2022.acl-long)
Copied to clipboard
Shaoyi Huang, Dongkuan Xu, Ian Yen, Yijue Wang, Sung-En Chang, Bingbing Li, Shiyang Chen, Mimi Xie, Sanguthevar Rajasekaran, Hang Liu, Caiwen Ding
| Challenge: | Conventional wisdom in pruning Transformer-based language models is that it reduces model expressiveness, but new research shows pruning increases risk of overfitting when performed at the fine-tuning phase. |
| Approach: | They propose to reduce pruning risk under pretrain-and-finetune paradigm . they propose to use knowledge distillation to improve pruning performance . |
| Outcome: | The proposed method outperforms the leading competitors on the GLUE benchmark. |
CTAL: Pre-training Cross-modal Transformer for Audio-and-Language Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing audio-language task-specific predictive approaches focus on building complicated late-fusion mechanisms. |
| Approach: | They propose a cross-modal transformer for audio-and-language that learns inter-modal connections between audio and language through two proxy tasks on a large amount of audio- and-language pairs. |
| Outcome: | The proposed model improves on multiple audio-and-language tasks and can be used in fine-tuning phase. |