Papers with PGD
Rewriter-Evaluator Architecture for Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve neural machine translation models with multiple decoding passes lack proper policies to terminate multi-pass processes. |
| Approach: | They propose a novel architecture of Rewriter-Evaluator to terminate multi-pass decoding . they propose prioritized gradient descent to jointly and efficiently train rewriter and evaluator . |
| Outcome: | The proposed architecture significantly outperforms existing methods on three translation tasks and reduces performance gaps to oracle policies. |
PANDA: Persona Attributes Navigation for Detecting and Alleviating Overuse Problem in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | In persona-grounded dialogue, it is required to respond fluently and ground attributes according to the current conversation topic properly. |
| Approach: | They propose a framework to quantify the persona overuse problem of LLMs by establishing clear standards and verifying various LLM based on them. |
| Outcome: | The proposed framework aims to quantify the persona overuse problem of LLMs by establishing clear standards and verifying various LLM based on them. |
PAC-tuning: Fine-tuning Pre-trained Language Models with PAC-driven Perturbed Gradient Descent (2023.emnlp-main)
Copied to clipboard
| Challenge: | PAC-tuning is a two-stage fine-tune method for pretrained language models . PAC training minimizes the PACBayes generalization bound to learn proper parameter distribution . |
| Approach: | They propose a two-stage fine-tuning method to minimize the PAC-Bayes generalization bound . they use PAC to inject noise with variance learned in the first stage into the model parameters . |
| Outcome: | The proposed method outperforms baseline methods on 5 GLUE benchmark tasks. |