| Challenge: | Recent studies have shown that pre-trained language models generate similar output embeddings which makes it difficult to discriminate for the prompt-based classifier. |
| Approach: | They propose a calibration method which rotates the embedding feature into a new metric space and adapts the ratio of each dimension to a uniform distribution. |
| Outcome: | The proposed method improves the distinguishability of learning embeddings on three datasets under various settings. |
Similar Papers
Generative Calibration for In-context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning is one of the most exciting features of large language models . performance is sensitive to various configurations of the prompt, such as the choice or order of the training examples. |
| Approach: | They propose to calibrate the in-context predictive distribution by adjusting the label marginal . they find that the proposed method outperforms the ICL and state-of-the-art calibration methods . |
| Outcome: | The proposed method outperforms state-of-the-art methods by 27% absolute in macro-F1. |
A Study on the Calibration of In-context Learning (2024.naacl-long)
Copied to clipboard
Hanlin Zhang, YiFan Zhang, Yaodong Yu, Dhruv Madeka, Dean Foster, Eric Xing, Himabindu Lakkaraju, Sham Kakade
| Challenge: | Prior research has demonstrated improvements in the calibration of language models (LMs) in-context learning is a popular method for adapting static LMs to safety-critical domains. |
| Approach: | They use in-context learning to adapt static language models through tailored prompts to a wide range of tasks and find that miscalibration occurs in low-shot settings. |
| Outcome: | The proposed calibrations show that models exhibit increased miscalibration before achieving better calibration in low-shot settings. |
Token-based Decision Criteria Are Suboptimal in In-context Learning (2025.naacl-long)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) typically utilizes output probabilities of manually selected label tokens, but such calibrations lead to suboptimal decision boundaries. |
| Approach: | They propose a method which renounces token probabilities and uses the nearest centroid classifier on the Language Model’s last hidden states to predict the label of the nearest ctroid. |
| Outcome: | The proposed method outperforms current token-based baselines by about 20%50% and provides a strong state-of-the-art in ICL. |
A Generic Method for Fine-grained Category Discovery in Natural Language Texts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-grained category discovery neglect semantic similarities of fine-grain categories. |
| Approach: | They propose a method that detects fine-grained clusters of semantically similar texts guided by a novel objective function. |
| Outcome: | The proposed method surpasses state-of-the-art methods on three benchmark tasks. |
Coarse-to-Fine Pre-training for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a task of discovering information entities and identifying their corresponding categories. |
| Approach: | They propose a NER-specific framework to inject coarse-to-fine named entity knowledge into pre-trained models by using a remote supervision strategy. |
| Outcome: | The proposed framework achieves significant improvements against several pre-trained base-lines, demonstrating its effectiveness in label-few and low-resource scenarios. |
Debiasing Pre-trained Contextualised Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | a study of contextualised word embeddings shows discriminative biases are encoded in contextualised embeddables. |
| Approach: | They propose a fine-tuning method that can be applied at token- or sentence-levels to debias pre-trained contextualised embeddings. |
| Outcome: | The proposed method can be applied at token- or sentence-levels to debias pre-trained models without requiring retrains. |
Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been widely explored for embedding generation. |
| Approach: | They propose an embedding-based in-context prompt training strategy that leverages in-constext learning to generate high-quality embeddables while reducing computational burden. |
| Outcome: | The proposed method surpasses models trained on publicly available retrieval data and achieves state-of-the-art embedding performance on the MTEB benchmark. |
Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text classification methods focus on a fixed label set, but many real-world applications require extending to new fine-grained classes as the number of samples per label increases. |
| Approach: | They propose a problem called coarse-to-fine grained classification that leverages label surface names as the only human guidance. |
| Outcome: | The proposed method outperforms existing methods on two real-world datasets. |
On the Transformation of Latent Space in Fine-Tuned NLP Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a large body of work analyzed the knowledge learned within representations of pre-trained models. |
| Approach: | They use hierarchical clustering to discover latent concepts in representational space . they compare pre-trained and fine-tuned models and perform a thorough analysis . |
| Outcome: | The results show that the model space evolves towards task-specific concepts whereas the lower layers retain generic concepts acquired in the pre-trained model. |
A Parameter-Efficient and Fine-Grained Prompt Learning for Vision-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Current vision-language models extract semantic information from large-scale cross-modal associations, limiting performance and efficiency. |
| Approach: | They propose a detail-oriented prompt learning method to implement fine-grained multi-modal semantic alignment with merely 0.25M trainable parameters. |
| Outcome: | The proposed method implements fine-grained multi-modal semantic alignment with merely 0.25M trainable parameters. |