Task-Level Thinking Steps Help Large Language Models for Challenging Classification Task (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results prove the superiority of our proposed method on challenging classification tasks. |
| Approach: | They propose a task-level thinking step that eliminates bias introduced by demonstrations . they propose 'progressive revision framework' which can improve the thinking steps by correcting hard demonstrations. |
| Outcome: | The proposed method achieves best performance on three kinds of classification tasks in zero-shot and few-shot settings. |
Similar Papers
Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has improved performance of large language models, but descriptive instructions are still under-explored. |
| Approach: | They propose an ensemble prompt framework to describe selection criteria of multiple in-context examples. preliminary experiments on machine translation confirm that this framework boosts ICL performance. |
| Outcome: | The proposed framework improves on commonsense, math, logical reasoning and hallucination tasks with three LLMs. |
Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates (2025.emnlp-main)
Copied to clipboard
Hy Dang, Tianyi Liu, Zhuofeng Wu, Jingfeng Yang, Haoming Jiang, Tao Yang, Pei Chen, Zhengyang Wang, Helen Wang, Huasheng Li, Bing Yin, Meng Jiang
| Challenge: | Large language models (LLMs) have strong reasoning and tool-use capabilities, yet fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent. |
| Approach: | They propose a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function calls. |
| Outcome: | The proposed framework reduces tool-use errors and improves interpretability and transparency of tool-using agents. |
Unveiling In-Context Learning: A Coordinate System to Understand Its Working Mechanism (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit remarkable in-context learning (ICL) capabilities, but the underlying working mechanism of ICL remains unclear. |
| Approach: | They propose a Two-Dimensional Coordinate System that unifies both views into a systematic framework that explains the behavior of ICL through two orthogonal variables: whether similar examples are presented in the demonstrations and whether LLMs can recognize the task. |
| Outcome: | The proposed method can interpret ICL for generation tasks effectively. |
Beyond Examples: Towards Automated Thought-level In-Context Reasoning for Large Language Models (2026.acl-long)
Copied to clipboard
Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zhengqi Wen, Chonghua Liao, Ling Yang, Haoran Luo, Zheng Lian, Jianhua Tao
| Challenge: | In-context learning (ICL) struggles with complex reasoning due to superficial, example-level implicit imitation. |
| Approach: | They propose an automated method that shifts from surface-level examples to more guidance-oriented thought patterns. |
| Outcome: | The proposed method achieves 80.6% accuracy on MATH and 62.5% on AMC, surpassing GPT-4o’s 77.2% and 57.5% accuracy. |
Tutor-ICL: Guiding Large Language Models for Improved In-Context Learning Performance (2024.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a dominant paradigm in natural language processing. |
| Approach: | They propose a prompting method for classification tasks using exemplar answers in a *comparative format' they also propose introducing a test instance before the exemplars to improve performance . |
| Outcome: | The proposed method achieves up to 13.76% increase in accuracy on classification tasks across decoder-only and encoder-decoder LLMs. |
Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines (2025.findings-acl)
Copied to clipboard
Do Xuan Long, Duong Ngoc Yen, Do Xuan Trong, Anh Tuan Luu, Kenji Kawaguchi, Shafiq Joty, Min-Yen Kan, Nancy F. Chen
| Challenge: | In-context learning is an important but not fully understood ability of pre-trained large language models. |
| Approach: | They propose a tool that generates two streams of guidelines capturing task language and format distributions and prompts them to define them by prompting. |
| Outcome: | The proposed model improves both strong open- and closed-source LLMs by over 5% in both zero- and few-shot settings. |
What Do Language Models Learn in Context? The Structured Task Hypothesis. (2024.acl-long)
Copied to clipboard
| Challenge: | Pre-trained large language models have exhibited an impressive ability to learn in context across various domains, e.g., code generation, education, medicine and even medicine. |
| Approach: | They taxonomize existing candidate theories into three competing hypotheses that explain LLMs’ ability to learn in context. |
| Outcome: | The proposed model can learn a task from in-context examples presented in a demonstration and generalize it to the prompt. |
How Many Demonstrations Do You Need for In-context Learning? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are capable of complex reasoning when given a few input-output demos. |
| Approach: | They use fewer input-output demos for each test query to study ICL . they do not observe significant degradation when using only one randomly chosen demo . |
| Outcome: | The proposed model outperforms multi-demo models on the tasks in 2022. |
In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is a common method for teaching large language models new tasks . given labeled examples in the input context, the model learns to perform the task without weight updates. |
| Approach: | They examine whether models guided via ICL infer the underlying structure of the task defined by the context or rely on superficial heuristics that only generalize to identically distributed examples. |
| Outcome: | The proposed model generalizes syntactically or linearly on out-of-distribution examples . the proposed model is able to generalize better on pre-trained models . |
In-Context Learning with Iterative Demonstration Selection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing literature has highlighted the importance of selecting examples that are diverse or semantically similar to the test sample . Existing studies have shown that the optimal selection dimension, i.e., diversity or similarity, is task-specific. |
| Approach: | They propose to use zero-shot chain-of-thought reasoning to iteratively select examples that are diverse but still strongly correlated with the test sample as ICL demonstrations. |
| Outcome: | The proposed method outperforms existing demonstration selection methods on reasoning, question answering, and topic classification tasks. |