Papers with SCD
Speculative Contrastive Decoding (2024.acl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit exceptional performance in language tasks, yet their auto-regressive inference is limited due to high computational requirements and is sub-optimal due to the exposure bias. |
| Approach: | They propose a decoding approach that leverages predictions from smaller language models to achieve both decoding acceleration and quality improvement. |
| Outcome: | The proposed method achieves both decoding acceleration and quality improvement on four diverse language tasks. |
SCD: Self-Contrastive Decorrelation of Sentence Embeddings (2022.acl-short)
Copied to clipboard
| Challenge: | Existing methods for self-supervised learning of representations are based on contrastive learning. |
| Approach: | They propose a self-supervised approach that optimizes a joint decorrelation and self-contrastive objective by leveraging the contrast arising from standard dropout at different rates. |
| Outcome: | The proposed method achieves comparable results with state-of-the-art methods on multiple benchmarks without using contrastive pairs. |
Investigating the Contextualised Word Embedding Dimensions Specified for Contextual and Temporal Semantic Changes (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on the meaning of contextualised word embeddings (SCWEs) have not shown how meaning changes are encoded in the embeddable space. |
| Approach: | They compare pre-trained and fine-tuned contextualised word embeddings on contextual and temporal semantic change detection benchmarks. |
| Outcome: | The pre-trained and fine-tuned versions of (SCWE) and their fine- tuned versions on contextual and temporal semantic change detection benchmarks show that they represent semantic changes across all dimensions when fine--and that they are more efficient than ICA. |
Can Word Sense Distribution Detect Semantic Changes of Words? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect semantic variations of words are not accurate for time-sensitive predictions. |
| Approach: | They propose to use pretrained static sense embeddings to annotate a word's occurrence with a sense id to compare its distributions. |
| Outcome: | The proposed method compares word sense distributions across two corpora to predict meaning change . the results show that pretrained LLMs can detect changes in words over time . |
FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Reranking (2024.findings-emnlp)
Copied to clipboard
Zhuoer Wang, Leonardo Ribeiro, Alexandros Papangelis, Rohan Mukherjee, Tzu-Yen Wang, Xinyan Zhao, Arijit Biswas, James Caverlee, Angeliki Metallinou
| Challenge: | Existing supervised and in-context learning approaches suffer from high training costs, poor data efficiency, and generated API calls that can be unfaithful to the API documentation and the user’s request. |
| Approach: | They propose an output-side optimization approach that uses state-tracked constrained decoding and a lightweight model as the discriminator to rerank the beam-searched candidate generations of the large language model. |
| Outcome: | The proposed approach outperforms existing supervised and in-context learning approaches in the DSTC8 and API Bank datasets in terms of call generation accuracy, inference efficiency, and context efficiency. |
A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing Word-in-Context (WiC) datasets are used to detect temporal semantic changes of words. |
| Approach: | They propose a supervised two-staged SCD method that uses existing Word-in-Context (WiC) datasets to predict temporal semantic changes of words. |
| Outcome: | The proposed method achieves strong performance in multiple languages and significant improvements on WiC benchmarks. |
SCDTour: Embedding Axis Ordering and Merging for Interpretable Semantic Change Detection (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to improve interpretability of SCD often lead to performance degradation . agglomerating axes produces a more refined set of word senses, which improves performance . |
| Approach: | They propose a method that orders and merges interpretable axes to improve SCD performance. |
| Outcome: | The proposed method preserves performance while maintaining high interpretability . it produces a more refined set of word senses, which improves performance . |