How Helpful is Inverse Reinforcement Learning for Table-to-Text Generation? (2021.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to Table-to-Text generation suffer from issues such as missing information, repetition and repetition. |
| Approach: | They propose to use Inverse Reinforcement Learning (IRL) to solve the Table-to-Text task . they use multiple interpretable unsupervised reward components that are combined linearly to form a composite reward function. |
| Outcome: | The proposed task outperforms strong RL baselines marginally in the Table-to-Text task. |
Similar Papers
Inverse Reinforcement Learning for Text Summarization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that inverse reinforcement learning (RL) training has certain disadvantages such as object mismatch and exposure bias. |
| Approach: | They propose inverse reinforcement learning (IRL) as an effective paradigm for training abstractive summarization models. |
| Outcome: | The proposed model outperforms MLE and RL baselines on ROUGE, coverage, novelty, compression ratio, factuality, and human evaluations. |
Inverse Reinforcement Learning Meets Large Language Model Alignment (2025.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide a comprehensive review of recent advances in LLM alignment . it will highlight the necessity of constructing neural reward models from human data . |
| Approach: | This tutorial will provide a comprehensive review of recent advances in LLM alignment through the lens of inverse reinforcement learning. |
| Outcome: | This tutorial will provide a comprehensive review of recent advances in LLM alignment through the lens of inverse reinforcement learning (IRL). |
Mapping Language to Programs using Multiple Reward Components with Inverse Reinforcement Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches focus on likelihood-based training or using reinforcement learning to fine-tune models based on a single reward. |
| Approach: | They propose an approach to fine-tune programs from natural language instruction . they propose a reward function that linearly combines them and a policy for program generation . |
| Outcome: | The proposed approach achieves better performance than competing methods using Reinforcement Learning. |
Reinforcement Learning with Token-level Feedback for Controllable Text Generation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for controllable text generation are guided by coarse-grained feedback, which may lead to suboptimal performance owing to semantic twists or progressions within sentences. |
| Approach: | They propose a reinforcement learning algorithm which formulates TOken-LEvel rewards for controllable text generation and employs a "first-quantize-then-noise" paradigm to enhance the robustness of the RL algorithm. |
| Outcome: | The proposed algorithm can achieve superior performance on single-attribute and multi-attract control tasks. |
Reward Gaming in Conditional Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work has used reward functions learned from human annotations to align conditional text generation models with desired behaviors. |
| Approach: | They propose to use reinforcement learning to train conditional text generation models with reward functions learned from human annotations to align outputs with desired behaviors. |
| Outcome: | The proposed framework improves the quality of generated summaries by using saliency and faithfulness metrics. |
Text Simplification with Reinforcement Learning Using Supervised Rewards on Grammaticality, Meaning Preservation, and Simplicity (2020.aacl-srw)
Copied to clipboard
| Challenge: | Existing studies in text-to-text generation do not align with human-perspectives for these perspectives. |
| Approach: | They propose to use BERT regressors fine-tuned for grammaticality, meaning preservation, and simplicity as reward estimators to optimize rewards for text simplification. |
| Outcome: | The proposed method achieves text simplification conforming to human-perspectives. |
Navigating Noisy Feedback: Enhancing Reinforcement Learning with Error-Prone Language Models (2024.findings-emnlp)
Copied to clipboard
Muhan Lin, Shuyang Shi, Yue Guo, Behdad Chalaki, Vaishnav Tadiparthi, Ehsan Moradi Pari, Simon Stepputtis, Joseph Campbell, Katia Sycara
| Challenge: | Reward hacking is a problem in reinforcement learning where the ability to specify the desired behavior of a reward function is difficult. |
| Approach: | They propose to use feedback as a potential-based shaping function to solicit and apply feedback from large language models to improve convergence speed and policy returns. |
| Outcome: | The proposed method improves convergence speed and policy returns over baselines even with significant ranking errors and eliminates the need for complex post-processing of reward functions. |
Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning (P19-1)
Copied to clipboard
| Challenge: | Abstract meaning representations (AMRs) are labeled directed acyclic graphs that represent a non intersentential abstraction of natural language with broad-coverage semantic representations. |
| Approach: | They build upon a transition-based AMR parser that uses Stack-LSTMs and augment training with policy learning. |
| Outcome: | The proposed parser performs comparable to the best published parsers. |
Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards (P19-1)
Copied to clipboard
| Challenge: | Existing generative models generate too few keyphrases, but they often generate too many . et al. (2017) propose a reinforcement learning approach for keyphrase generation . |
| Approach: | They propose a reinforcement learning approach that encourages a model to generate sufficient keyphrases with an adaptive reward function. |
| Outcome: | The proposed method improves state-of-the-art generative models with conventional and new evaluation methods on real-world datasets. |
Exploring Supervised and Unsupervised Rewards in Machine Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Autoregressive sequence-to-sequence (seq2sequ) neural architectures have become the de facto approach in Machine Translation (MT). |
| Approach: | They propose to make models less reliant on cross-entropy loss and evaluation metrics . they propose an entropicity-regularised RL method that explores the action space . |
| Outcome: | The proposed method exploits the action space and unsupervised reward function to balance between exploration and exploitation. |