Diverse In-Context Example Selection After Decomposing Programs and Aligned Utterances Improves Semantic Parsing (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are well suited for seq2seq translation . a lack of pretraining corpora can hinder the use of LLMs for structured interpretation . |
| Approach: | They propose to decompose available ICE trees into fragments and use additional invocations to map them to corresponding utterances. |
| Outcome: | The proposed method shows visible gains on diverse parsing benchmarks on popular languages. |
Similar Papers
Ungrammatical-syntax-based In-context Example Selection for Grammatical Error Correction (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) has shown impressive results on many tasks, but applying LLMs to grammatical error correction (GEC) is still a challenging task. |
| Approach: | They propose an ungrammatical-syntax-based in-context example selection strategy that measures similarity of sentences based on their syntactic structures and identify optimal ICL examples sharing the most similar ill-formed syntax to the test input. |
| Outcome: | The proposed strategy outperforms word-matching and semantics-based methods on a syntax-oriented task like GEC on benchmark English datasets. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that in-context examples for machine translation are beneficial for high-resource languages. |
| Approach: | They propose to use in-context examples for machine translation (MT) they argue that similarity-based selection can improve MT . |
| Outcome: | The proposed approach improves machine translation (MT) and low-resource languages. |
SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning improves performance of large language models (LLMs) performance of ICL highly depends on quality of demonstrations . |
| Approach: | They propose a syntactic-augmented COverage-based In-context example selection strategy that leverages syntastic knowledge beyond word matching to select better examples for machine translation. |
| Outcome: | The proposed strategy obtains the highest average COMET score among learning-free methods. |
What can Large Language Models Capture about Code Functional Equivalence? (2025.findings-naacl)
Copied to clipboard
| Challenge: | SeqCoBench is a benchmark to assess how Code-LLMs can capture code semantics. |
| Approach: | They propose a benchmark to assess how Code-LLMs capture code semantics . they use seqCoBench to evaluate whether they can discern semantically equivalent or different pairs of programs . |
| Outcome: | The proposed benchmarks show that they can capture code semantics better than classical match-based retrieval scores. |
Submodular-based In-context Example Selection for LLMs-based Machine Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Prior studies have focused on the role of well-chosen examples in in-context learning . |
| Approach: | They propose to use multiple translational factors for in-context example selection by using monotone submodular function maximization. |
| Outcome: | The proposed approach outperforms random selection and robust single-factor baselines across various NLP tasks. |
An End-to-End Submodular Framework for Data-Efficient In-Context Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in natural language tasks leverage the emergent In-Context Learning ability of pretrained Large Language Models (LLMs). |
| Approach: | They propose a framework for exemplar selection for in-context learning that uses a pool-based active learning approach to select Diverse and informative exemplars from the target tasks’ unlabeled pool. |
| Outcome: | The proposed framework outperforms existing methods for data annotation and similarity-based methods for test query-specific exemplar retrieval on 7 different NLP datasets and 5 LLMs of varying complexities. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
Effective Self-Mining of In-Context Examples for Unsupervised Machine Translation with LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive performance on a wide range of natural language processing tasks. |
| Approach: | They propose an unsupervised approach to mine in-context examples for machine translation (MT) they use word-level mining to acquire word translations that are then used to perform sentence-level mines . |
| Outcome: | The proposed approach outperforms state-of-the-art methods on 288 directions on 287 languages and is based on word-level mining and sentence-level extraction. |
Evaluating In-Context Learning of Libraries for Code Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work shows that large proprietary LLMs can learn novel library usage in-context from demonstrations. |
| Approach: | They evaluate large proprietary LLMs to understand library usage in-context . they find they are able to generate code based on library specification presented in-constext - a promising area . |
| Outcome: | The proposed models can learn library usage in-context from demonstrations . the results pave the way for more adaptable and dynamic coding environments. |