Papers by Danqing Wang
Scaling LLM Inference Efficiently with Optimized Sample Compute Allocation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to optimize sample allocations for large language models fail to account for the optimal sampling configuration. |
| Approach: | They propose an algorithm that optimizes sample allocation by finding an optimal mix of different inference configurations. |
| Outcome: | The proposed algorithm achieves better accuracy on SWE-Bench with 3x less compute than the default configuration. |
Learning Personalized Alignment for Evaluating Open-ended Text Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Traditional evaluation metrics rely heavily on lexical similarity with human-written references, showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences. |
| Approach: | They propose an interpretable evaluation framework that evaluates alignment with specific human preferences by providing detailed comments and fine-grained scoring. |
| Outcome: | The proposed framework outperforms GPT-4 in Kendall correlation and accuracy with zero-shot reviewers. |
Heterogeneous Graph Neural Networks for Extractive Document Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models capture cross-sentence relations with recurrent neural networks, but they are hard to capture sentence-level long-distance dependency. |
| Approach: | They propose a graph-based neural network for extractive summarization which contains semantic nodes apart from sentences. |
| Outcome: | The proposed graph-based neural network is the first to incorporate different types of nodes into it and perform a qualitative analysis. |
MTG: A Benchmark Suite for Multilingual Text Generation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Using MTG, we train and evaluate multilingual text generation models using human-annotated data. |
| Approach: | They propose a multilingual multiway text generation dataset with 400k human-annotated data that includes four generation tasks across five languages. |
| Outcome: | The proposed dataset includes four generation tasks across five languages (English, German, French, Spanish and Chinese) it provides comprehensive evaluations with diverse generation scenarios. |
Contrastive Aligned Joint Learning for Multilingual Summarization (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing summarization systems for multilingual text summarizing are limited due to the lack of large-scale data in multiple languages. |
| Approach: | They propose a multilingual summarization system that can understand documents in multiple languages and generate summaries in the corresponding language. |
| Outcome: | The proposed model improves over monolingual models in all languages and transferable to other languages. |
CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation methods for text summarization systems are limited to in-domain setting, where supervised pre-trained models are evaluated on the same dataset. |
| Approach: | They propose to use a cross-dataset evaluation approach to evaluate different summarization systems in a multi-domain setting. |
| Outcome: | The proposed model can be used to evaluate text summarization systems on different datasets. |
INSTRUCTSCORE: Towards Explainable Text Generation Evaluation with Automatic Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to evaluate the quality of language generation do not provide explicit explanation of their verdicts. |
| Approach: | They propose a fine-grained explainable evaluation metric for text generation that harnesses human instruction and implicit knowledge of GPT-4 to fine-tune it. |
| Outcome: | The proposed model outperforms all other unsupervised metrics on translation, captioning, data-to-text, and commonsense generation tasks. |
Searching for Effective Neural Extractive Summarization: What Works and What’s Next (P19-1)
Copied to clipboard
| Challenge: | Recent years have seen success in the use of deep neural networks on text summarization, but there is no clear understanding of why they perform so well or how they might be improved. |
| Approach: | They propose to use different types of model architectures to improve extractive summarization systems. |
| Outcome: | The proposed framework achieves state-of-the-art on CNN/DailyMail by a large margin based on observations and analysis. |
Extractive Summarization as Text Matching (2020.acl-main)
Copied to clipboard
| Challenge: | Currently, most of the neural extractive summarization systems score and extract sentences individually and model the relationship between sentences. |
| Approach: | They propose to instantiate a neural extractive summarization task as a semantic text matching problem and use it to match a source document and candidate summaries in a semantic space. |
| Outcome: | The proposed framework is faster and more efficient than existing frameworks. |
A Closer Look at Data Bias in Neural Extractive Summarization Models (D19-54)
Copied to clipboard
| Challenge: | In this paper, we examine the generalization behaviour of summarization models . we propose several properties of datasets that matter for generalization . |
| Approach: | They propose several properties of datasets which matter for generalization of summarization models. |
| Outcome: | The proposed approach improves the state-of-the-art model by rethinking the model design process on a typical dataset. |
Learning from Mistakes via Cooperative Study Assistant for Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated their potential to refine their generation based on their own feedback, but the feedback from LLM itself is often inaccurate, thereby limiting its benefits. |
| Approach: | They propose a framework with an auxiliary agent to assist the main LLM in learning from mistakes through interactive cooperation. |
| Outcome: | The proposed framework can significantly boost large language models by an accuracy margin of up to 6.6 on BBH and 12.6 on BBQ. |