Eliciting Instruction-tuned Code Language Models’ Capabilities to Utilize Auxiliary Function for Code Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Using auxiliary functions to implement functions is important for instruction-tuned models because it reduces the implementation difficulty of a target function compared to implementing them from scratch. |
| Approach: | They propose several ways to provide auxiliary functions to the models by adding them to the query or providing a response prefix to incorporate the ability to utilize auxiliary function with the instruction following capability. |
| Outcome: | The proposed models outperform the recent powerful language models, gpt-4o, in the code generation task. |
Similar Papers
Exploring Data Augmentation for Code Generation Tasks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have impacted how models are trained for programming language tasks. |
| Approach: | They propose to use augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. |
| Outcome: | The proposed methods improve translation and summarization by 6.9% and 7.5% respectively. |
Advancing Language Models through Instruction Tuning: Recent Progress and Challenges (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | tutorial addresses three critical questions within the field of instruction tuning: (1) What are the current focal points in instruction tuning research? (2) What are best practices in training an instruction-following model? (3) What new challenges have emerged? |
| Approach: | This tutorial presents a systematic overview of recent advances in instruction tuning. |
| Outcome: | The tutorial covers different stages in model training: supervised fine-tuning, preference optimization, and reinforcement learning. |
Fine-Tuning Large Language Models with Sequential Instructions (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing instruction-tuned models struggle to adhere to a query with multiple intentions, which impairs their performance when the completion of several tasks is demanded by a single command. |
| Approach: | They develop an automatic process that turns existing data into diverse and complex task chains and a new benchmark to evaluate a model’s ability to follow all the instructions in a sequence. |
| Outcome: | The proposed model can follow instructions better and deliver higher results in coding, maths, and open-ended generation. |
Unified Pragmatic Models for Generating and Following Instructions (N18-1)
Copied to clipboard
| Challenge: | a new technique for layering explicit pragmatic inference on top of models for sequential tasks is proposed . explicit pragmatic reasoning is used to generate and follow natural language instructions . |
| Approach: | They propose a pragmatic speaker that uses the base listener to simulate the interpretation of candidate descriptions and a listener that reasons counterfactually about alternative descriptions. |
| Outcome: | The proposed model improves state-of-the-art models for interpreting human instructions and speaker models in diverse settings. |
CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation (2025.acl-industry)
Copied to clipboard
| Challenge: | CodeIF assesses the ability of large language models to adhere to task-oriented instructions in code generation tasks. |
| Approach: | They introduce a benchmark designed to assess LLMs' ability to adhere to task-oriented instructions within diverse code generation scenarios. |
| Outcome: | The proposed benchmark assesses LLMs' ability to adhere to task-oriented instructions in code generation tasks across a wide range of complexity levels and programming domains. |
Can You Tell Me How to Get Past Sesame Street? Sentence-Level Pretraining Beyond Language Modeling (P19-1)
Copied to clipboard
Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, Samuel R. Bowman
| Challenge: | State-of-the-art models in natural language processing (NLP) often incorporate sentence encoder functions which generate a sequence of vectors intended to represent the in-context meaning of each word in an input text. |
| Approach: | They conduct the first large-scale systematic study of candidate pretraining tasks, comparing 19 different tasks as alternatives and complements to language modeling. |
| Outcome: | The proposed model can be used to train sentences on language modeling tasks. |
Instruction Induction: From Few Examples to Natural Language Task Descriptions (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models can perform unseen tasks by conditioning on a few input-output demonstrations, but task inference is implicit and the ability of models to explicitly reason about it remains unexplored. |
| Approach: | They propose an instruction induction challenge in which a model is asked to generate a natural language instruction that fits a set of labeled examples. |
| Outcome: | The proposed model achieves 65.7% of human performance while the original model only reaches 9.8% of human performances. |
WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that Code Large Language Models can address a wide range of code-related tasks. |
| Approach: | They propose a method to generate widespread and versatile instruction data from open source code datasets and use it to train code-related models. |
| Outcome: | The proposed model outperforms open-source models in generalization ability across code-related tasks. |
Selective Prefix Tuning for Pre-trained Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models are time-consuming and memory-inefficient. |
| Approach: | They propose a method that inserts learnable vectors into each Transformer layer . they propose SL to encourage diversity in prefix tokens . |
| Outcome: | Extensive experiments validate the effectiveness of Prefix Tuning in sentence and token classification tasks. |
How Many Data Samples is an Additional Instruction Worth? (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent introduced instruction-paradigm empowers non-expert users to leverage NLP resources by defining a new task in natural language. |
| Approach: | They propose to define a task in natural language without creating task-specific datasets or building models. |
| Outcome: | The proposed model outperforms multitask learning models but is far from state-of-the-art task-specific models. |