Papers by Shuai Fu
GeoDRL: A Self-Learning Framework for Geometry Problem Solving using Reinforcement Learning in Deductive Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for automated geometry problem solving lack labeled data. |
| Approach: | They propose a framework that integrates logic graph deduction and deep reinforcement learning to optimize geometry reasoning as a Markov Decision Process. |
| Outcome: | The proposed framework improves accuracy and interpretability in the Geometry3K dataset while maintaining correctness. |
Code Execution with Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
Chenxiao Liu, Shuai Lu, Weizhu Chen, Daxin Jiang, Alexey Svyatkovskiy, Shengyu Fu, Neel Sundaresan, Nan Duan
| Challenge: | Pre-trained code intelligence models ignore the execution trace and only rely on source code and syntactic structures to understand code execution. |
| Approach: | They develop a mutation-based data augmentation technique to create a Python dataset and task for code execution that challenges existing models. |
| Outcome: | The proposed model outperforms existing models on code execution and shows its potential for zero-shot code-to-code search and text-to code generation. |
Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for parameter-efficient fine-tuning (PEFT) are not effective for weight-poisoning backdoor attacks. |
| Approach: | They propose a parameter-efficient fine-tuning (PEFT) method that updates only a limited set of model parameters and provides a robust defense against weight-poisoning backdoor attacks. |
| Outcome: | The proposed method identifies poisoned samples through confidence and is robust against weight-poisoning backdoor attacks. |
On Orthogonality Constraints for Transformers (2021.acl-short)
Copied to clipboard
Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee
| Challenge: | a dedicated study on orthogonality constraints for transformers has been lacking . plug-and-play constraints increase the BLEU of transformers . |
| Approach: | They propose to use plug-and-play constraints to encourage matrices to be orthogonal for numerical stability. |
| Outcome: | The proposed constraint increases the BLEU on the large-scale WMT’16 EnDe benchmark by a factor of 28.4 to 29.6. |
Learning Retrieval Augmentation for Personalized Dialogue Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Personalized dialogue generation is a popular approach for conversational AI applications . however, persona profiles may not provide comprehensive descriptions of the persona . |
| Approach: | They propose a method that leverages persona profiles and dialogue context to generate personalized dialogues by leveraging personas and persona profile. |
| Outcome: | The proposed method outperforms baselines on the CONVAI2 dataset . it is expected to generate personalized dialogues based on persona profiles and dialogue context . |
Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks (P19-1)
Copied to clipboard
| Challenge: | Existing models for natural language processing are heavily parameterized and memory inefficient. |
| Approach: | They propose a series of lightweight and memory efficient neural architectures for NLP tasks . they propose quaternion algebra and hypercomplex spaces for computation . |
| Outcome: | The proposed models enable expressive inter-component interactions and significantly reduce parameter size without loss of performance. |
Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | ProAttack is a novel and efficient method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger. |
| Approach: | They propose a method for performing clean-label backdoor attacks based on the prompt, which uses the prompt itself as a trigger. |
| Outcome: | The proposed method achieves state-of-the-art performance on several NLP tasks, particularly in few-shot settings. |