Large Language Model Is Not a Good Few-shot Information Extractor, but a Good Reranker for Hard Samples! (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models have made remarkable strides in various tasks, but whether they are competitive few-shot solvers remains an open question. |
| Approach: | They propose an adaptive filter-then-rerank paradigm to combine the strengths of LLMs and SLMs. |
| Outcome: | The proposed system achieves promising improvements on various IE tasks with acceptable time and cost investment. |
Similar Papers
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models pre-trained on massive corpora have shown impressive few-shot learning ability on many NLP tasks. |
| Approach: | They propose to recast structured output in the form of code instead of natural language and use generative LLMs of code to perform IE tasks. |
| Outcome: | The proposed method outperforms fine-tuning moderate-size pre-trained models and prompting NL-LLMs under few-shot settings. |
How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a systematic and comprehensive empirical evaluation of state-of-the-art reranking methods is presented. |
| Approach: | They evaluate 22 reranking methods including 40 variants across established benchmarks . primary goal is to determine whether performance disparity exists between LLM-based reranters and lightweight counterparts based on novel queries . |
| Outcome: | The proposed methods perform better on familiar queries than lightweight models, the authors show . |
Large Language Models are few(1)-shot Table Reasoners (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent literature has shown that large language models are excellent few-shot reasoners to solve text reasoning tasks. |
| Approach: | They evaluated LLMs on popular table QA and fact verification datasets like WikiTableQuestion, FetaQA, TabFact, and FEVEROUS and found they are competent at complex reasoning over table structures. |
| Outcome: | The proposed models are more competent at complex reasoning over table structures than tuned T5-large models. |
What do Large Language Models Need for Machine Translation Evaluation? (2024.emnlp-main)
Copied to clipboard
Shenbin Qian, Archchana Sindhujan, Minnie Kabra, Diptesh Kanojia, Constantin Orasan, Tharindu Ranasinghe, Fred Blain
| Challenge: | Existing research shows that large language models can perform better in machine translation tasks. |
| Approach: | They propose to use large language models for machine translation evaluations . authors explore what translation information is needed for LLMs to evaluate MT quality . |
| Outcome: | The proposed model performs comparable to fine-tuned multilingual pre-trained models. |
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers (2023.emnlp-main)
Copied to clipboard
Jon Saad-Falcon, Omar Khattab, Keshav Santhanam, Radu Florian, Martin Franz, Salim Roukos, Avirup Sil, Md Sultan, Christopher Potts
| Challenge: | Existing methods for information retrieval tasks require large labeled datasets for fine-tuning, but they can experience significant drops in accuracy due to distribution shifts from the training to the target domain. |
| Approach: | They propose a method for using large language models to generate large numbers of synthetic queries cheaply using an expensive LLM. |
| Outcome: | The proposed method boosts zero-shot accuracy in long-tail domains and achieves substantially lower latency than standard reranking methods. |
A Dataset for Expert Reviewer Recommendation with Large Language Models as Zero-shot Rankers (2025.coling-main)
Copied to clipboard
Vanja M. Karan, Stephen McQuistin, Ryo Yanagida, Colin Perkins, Gareth Tyson, Ignacio Castro, Patrick G.T. Healey, Matthew Purver
| Challenge: | state of the art reviewer recommendation systems still have relatively high error rates . |
| Approach: | They propose to use a large language model to improve on SotA, but not a cure-all . they first create a new dataset and introduce LLMs with prompting to evaluate their performance. |
| Outcome: | The proposed approach improves on SotA but not cure-all, the authors argue . they show that the proposed approach can be extended to many related tasks . |
A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have enabled advances in the field of natural language processing . however, their application and potential are still underexplored . |
| Approach: | They evaluate four state-of-the-art instruction-tuned Large Language Models on 13 NLP tasks in English. |
| Outcome: | The evaluated models outperform state-of-the-art models on 13 real-world clinical and biomedical NLP tasks in English. |
Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | generative large language models are increasingly used for data augmentation tasks . text samples are mostly selected randomly and a comprehensive overview of other sample selection strategies is lacking. |
| Approach: | They compare random sample selection strategies and random sample sampling strategies to evaluate their effects in a low-resource setting. |
| Outcome: | The proposed model performance improvements are compared with other sample selection strategies. |
SLM-Mod: Small Language Models Surpass LLMs at Content Moderation (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are expensive to query in real-time and do not allow for a community-specific approach to content moderation. |
| Approach: | They propose to use small language models for community-specific content moderation tasks by fine-tuning and evaluating their performance against larger open- and closed-sourced models. |
| Outcome: | The proposed models outperform zero-shot LLMs in content moderation tasks with 11.5% higher accuracy and 25.7% higher recall across all communities. |
ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent Large Language Models (LLMs) have demonstrated remarkable performance in document reranking tasks. |
| Approach: | They propose a two-stage training approach for document reranking using reinforcement learning and fine-grained score learning. |
| Outcome: | The proposed approach outperforms open-source and proprietary reranking models on BEIR benchmark. |