| Challenge: | Existing methods for structured prediction rely on large labeled datasets. Existing approaches for structured predictions require detailed annotation guidelines about the task, the label set, and the interactions between labels. |
| Approach: | They propose a framework for constructing zero- and few-shot linguistic structure predictors using structural constraints and combinatorial inferences. |
| Outcome: | The proposed framework can be extended to build zero- and few-shot label predictors on two structured prediction tasks and five datasets. |
Similar Papers
Mapping the Course for Prompt-based Structured Prediction (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models have demonstrated strong performance in a wide-range of language tasks without task-specific fine-tuning. |
| Approach: | They combine large language models with combinatorial inference to marry predictive power of LLMs with structural consistency provided by inference methods. |
| Outcome: | The proposed model incorporates symbolic inference to provide consistent and accurate predictions on challenging tasks. |
Prompt Consistency for Zero-Shot Task Generalization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that pre-trained language models can perform zero-shot generalization to new tasks without annotated examples. |
| Approach: | They propose to regularize prompt consistency to encourage consistent predictions over a diverse set of prompts. |
| Outcome: | The proposed approach outperforms the state-of-the-art zero-shot learner, T0, on 9 out of 11 datasets across 4 NLP tasks by 10.6 absolute points in terms of accuracy. |
Prompting Language Models for Linguistic Structure (2023.acl-long)
Copied to clipboard
| Challenge: | Existing prompting methods can test this hypothesis on autoregressive PLMs. |
| Approach: | They propose a structured prompting approach for linguistic structured prediction tasks that performs zero- and few-shot sequence tagging with autoregressive PLMs. |
| Outcome: | The proposed approach shows that the model can perform few-shot sequence tagging on part-of-speech taging, named entity recognition, and sentence chunking tasks. |
Autoregressive Structured Prediction with Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have seen a paradigm shift in NLP towards using pretrained language models for a wide range of tasks. |
| Approach: | They propose to model structures as sequences of actions in autoregressive manner with PLMs . their approach allows in-structure dependencies to be learned without any loss . |
| Outcome: | The proposed approach achieves state-of-the-art on all structured prediction tasks. |
Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent prompt-based approaches allow pretrained language models to achieve strong performances on few-shot finetuning by reformulating downstream task instances as a language modeling problem. |
| Approach: | They propose to reformulate downstream tasks as a language modeling problem and add a regularization that preserves pretraining weights to the model to mitigate the destructive tendency of few-shot finetuning. |
| Outcome: | The proposed model performs better on low data regimes than the standard model on few-shot finetuning. |
Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at straightforward reasoning tasks, but struggle when faced with complex multi-step reasoning. |
| Approach: | They propose a framework that converts unstructured text into a graph and instructs LLMs to navigate this graph using task-specific strategies. |
| Outcome: | The proposed framework improves the multi-step reasoning capabilities of Large Language Models in a zero-shot setting. |
Pre-trained Language Models Can be Fully Zero-Shot Learners (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to pre-trained language models require fine-tuning on labeled datasets or manually constructing proper prompts. |
| Approach: | They propose a nonparametric prompting PLM for fully zero-shot language understanding . they compare it to previous methods for text classification and text entailment . |
| Outcome: | The proposed method outperforms previous methods on diverse tasks. |
Randomized Deep Structured Prediction for Discourse-Level Processing (2021.eacl-main)
Copied to clipboard
| Challenge: | Expressive text encoders have been at the center of recent NLP work . however, some tasks require complex structural dependencies between texts . |
| Approach: | They propose to leverage deep structured prediction and expressive neural encoders for argumentation mining tasks. |
| Outcome: | The proposed framework can be used for argumentation mining tasks without expensive inference tools. |
Word Reordering for Zero-shot Cross-lingual Structured Prediction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current sentence encoders are word order sensitive, resulting in poor performance . Adapting word order from one language to another is key in cross-lingual structured prediction. |
| Approach: | They propose a new module to organize words following the source language order . they build structured prediction models with bag-of-words inputs and introduce a module to do this . |
| Outcome: | The proposed model significantly improves target language performance for languages that are distant from the source language. |
Learning Constraints for Structured Prediction Using Rectifier Networks (2020.acl-main)
Copied to clipboard
| Challenge: | Various natural language processing tasks require domain expertise to design good constraints. |
| Approach: | They propose a framework for learning constraints in a network of linear inequalities over the output variables. |
| Outcome: | The proposed framework can be used to learn constraints from data on natural language processing tasks. |