| Challenge: | Existing techniques fine-tune on input-output pairs or with numerical rewards that gauge the output quality are not effective. |
| Approach: | They propose to fine-tune pre-trained language models with binary labels and a Python interpreter to get textual feedback from the inputs. |
| Outcome: | The proposed model outperforms the base model on unseen problems and achieves comparable or better performance on humanEval. |
Similar Papers
Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs (2025.naacl-srw)
Copied to clipboard
| Challenge: | Code-generating Large Language Models (LLMs) have become essential tools in modern software development, enhancing productivity and accelerating development. |
| Approach: | They propose to use Reinforcement Learning and Direct Preference Optimization to fine-tune code-generating Large Language Models (LLMs) by enhancing the training data with symbolic execution techniques. |
| Outcome: | The proposed model improves on the CodeRL benchmark and shows that it is more accurate and objective than the baseline model. |
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated that a large language model (LLM) can generate training data for another LLM, or for creating supplementary training materials, such as rationales. |
| Approach: | They conduct an in-depth investigation to understand why fine-tuning an LLM with responses generated by a LLM often yields better results than using responses generated from humans. |
| Outcome: | The proposed approach can be used to transfer knowledge from a larger model to a smaller one, or for creating supplementary training materials, such as rationales. |
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-fine-tuned large language models (LLMs) under 14B parameters underperform on NLU tasks . we explore a framework to improve the NLU capabilities of LLMs . |
| Approach: | They propose to use Proximal Policy Optimization to improve NLU capabilities . they frame NLU as a reinforcement learning environment and optimize for reward signals . |
| Outcome: | The proposed framework outperforms supervised fine-tuning on GLUE and superGLUE tasks. |
Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to improve robustness require changing the fine-tuning process or large-scale data augmentation, which are infeasible or cost prohibitive for closed-source models. |
| Approach: | They propose to prioritize more complex examples or replace existing training examples with LLM-generated data to improve performance on OOD NLI datasets. |
| Outcome: | The proposed methods improve performance on difficult OOD datasets while training with synthetic data leads to substantial improvements on easier OOD data. |
A Rigorous Evaluation of LLM Data Generation Strategies for Low-Resource Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used to generate synthetic textual data for training smaller specialized models. |
| Approach: | They evaluate the performance of large language models and their generation strategies in 11 different languages using 3 NLP tasks and 4 open-source LLMs. |
| Outcome: | The proposed generation strategies and their combinations yield strong results across 11 languages, including several extremely low-resource ones. |
Training Language Models to Generate Text with Citations via Fine-grained Rewards (2024.acl-long)
Copied to clipboard
| Challenge: | Recent Large Language Models (LLMs) are prone to hallucination and their outputs often contain incorrect or unverifiable claims. |
| Approach: | They propose a training framework using fine-grained rewards to teach LLMs to generate highly supportive and relevant citations while ensuring the correctness of their responses. |
| Outcome: | The proposed training framework outperforms existing methods on QA datasets and surpasses GPT-3.5-turbo on LLaMA-2-7B. |
Less for More: Enhanced Feedback-aligned Mixed LLMs for Molecule Caption Generation and Fine-Grained NLI Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent trends have led to the use of multimodal models to learn molecular and linguistic representations, either in separate but coordinated spaces or in a common space. |
| Approach: | They propose a novel atomic-level evaluation method leveraging off-the-shelf Natural Language Inference (NLI) models for use in the unseen chemical domain. |
| Outcome: | The proposed method surpasses state-of-the-art models in the unseen chemical domain while relying on a granularity-based evaluation method. |
Toward Beginner-Friendly LLMs for Language Learning: Controlling Difficulty in Conversation (2026.findings-eacl)
Copied to clipboard
| Challenge: | Practicing conversations with large language models is a promising alternative to traditional in-person language learning. |
| Approach: | They propose a new token-level evaluation metric, Token Miss Rate, that measures the proportion of incomprehensible tokens per utterance and correlates strongly with human judgments. |
| Outcome: | The proposed methods improve comprehensibility for beginner speakers from 39.4% to 83.3%, compared with prompting alone and a token-level evaluation metric, Token Miss Rate (TMR). |
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing instruction-tuned open-source LLMs have only been instruction- tuned for English and a few popular languages, thus hindering their accessibility to many other languages in the world. |
| Approach: | They propose a framework that uses supervised fine-tuning and reinforcement learning from human feedback to improve the accessibility of large language models. |
| Outcome: | The proposed framework enables the evaluation of generative LLMs in multiple languages. |
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions (2024.eacl-long)
Copied to clipboard
| Challenge: | Large language models with instruction tuning are resource-intensive . a recent study suggests that the performance of LLMs scales proportionally with the size of the model. |
| Approach: | They propose to distill knowledge from instruction-tuned LLMs into much smaller ones . they develop a large set of 2.58M instructions based on existing and newly-generated instructions . |
| Outcome: | The proposed models are comparable to strong baselines while being much smaller in size. |