Papers by Michael Du
RWKV: Reinventing RNNs for the Transformer Era (2023.findings-emnlp)
Copied to clipboard
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
| Challenge: | recurrent neural networks struggle to match the performance of Transformers due to limitations in parallelization and scalability. |
| Approach: | They propose a model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs. |
| Outcome: | The proposed model performs on par with similarly sized RNNs, suggesting future work can leverage this architecture to create more efficient models. |
Grounding Open-Domain Instructions to Automate Web Support Tasks (2021.naacl-main)
Copied to clipboard
| Challenge: | RUSS is a task and dataset to ground natural language instructions on the web to perform previously unseen tasks. |
| Approach: | They build a task and dataset to ground AI agents from open-domain, step-by-step instructions on the web. |
| Outcome: | The proposed model outperforms existing models that map instructions to actions without WebLang. |
Self-training Improves Pre-training for Natural Language Understanding (2021.naacl-main)
Copied to clipboard
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Veselin Stoyanov, Alexis Conneau
| Challenge: | Unsupervised pretraining has led to improvements in natural language understanding . a data augmentation method can be used to generate labels for unlabeled examples . |
| Approach: | They propose a semi-supervised method which uses unlabeled data to retrieve sentences from a database of billions of unlabed sentences crawled from the web. |
| Outcome: | The proposed method improves on standard text classification benchmarks by 2.6% and knowledge distillation by few shots. |
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)
Copied to clipboard
Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
Accelerating Code Search with Deep Hashing and Code Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Code search is to search reusable code snippets from source code corpus based on natural languages queries. |
| Approach: | They propose a method to accelerate code search with deep hashing and code classification by using deep hashes and code hash. |
| Outcome: | The proposed method can save 90% of retrieval time while preserving at least 99% of retrievals accuracy. |
AlphaEdit+: Model Editing in the Presence of Conflicting and Inconsistent Knowledge (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge editing struggle with knowledge conflicts and inconsistencies. |
| Approach: | They propose a new method for knowledge editing that relaxes null-space constraints and introduces a weighting scheme to mitigate conflicts between new and historical knowledge. |
| Outcome: | The proposed method outperforms existing methods on challenging datasets and outperformed existing methods. |
FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows (2022.emnlp-main)
Copied to clipboard
| Challenge: | Despite recent progress in dialogue evaluation, how to develop automatic metrics remains an open problem. |
| Approach: | They propose a consensus-based framework for dialog evaluation using segment act flows . they propose to crowdsource a large-scale dataset for it to be evaluated . |
| Outcome: | The proposed framework can reach the best or comparable correlation with human evaluation. |
Language models can learn implicit multi-hop reasoning, but only if they have lots of training data (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies explore the ability of language models to solve multi-hop reasoning tasks without chain of thought. |
| Approach: | They propose to use GPT2-style language models to train k-hop reasoning models . they show that the required training data grows exponentially in k . |
| Outcome: | The proposed models can learn implicit reasoning without chain-of-thoughts, the authors show . their training data grows exponentially in k, and the required number of transformer layers grows linearly in the model. |