T-VEC: A Telecom-Specific Vectorization Model with Enhanced Semantic Understanding via Deep Triplet Loss Fine-Tuning (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Generic embedding models struggle to represent telecom-specific semantics . specialized terminology and ambiguous terms often limit their utility in retrieval and downstream tasks. |
| Approach: | They propose a domain-adapted embedding model fine-tuned from a gte-Qwen2-1.5B-instruct backbone. |
| Outcome: | The proposed model outperforms MPNet, BGE, Jina and E5 on a custom benchmark . it is open source and has a triplet loss objective . |
Similar Papers
Vec2Sent: Probing Sentence Embeddings with Natural Language Generation (2020.coling-main)
Copied to clipboard
| Challenge: | a new unsupervised probing task is able to retrieve black-box sentence embeddings . a variety of problems surround probing tasks, including manual construing . |
| Approach: | They propose a method to generate black-box sentence embeddings by conditionally generating from them . they also illustrate how the language generated from different encoders differs . |
| Outcome: | The proposed probing task improves the performance of black-box sentence embeddings . the proposed task is based on a conditional natural language generation approach . |
G-Tuning: Improving Generalization of Pre-trained Language Models with Generative Adversarial Network (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirical evaluations on the GLUE benchmark demonstrate that fine-tuning can enhance the generalization performance of pre-trained language models (PLMs) in downstream tasks. |
| Approach: | They propose a fine-tuning framework that transforms the latent representation of pre-trained language models from a universal space to a target space and integrates a generative adversarial network into the fine-untun process. |
| Outcome: | Empirical evaluations on the GLUE benchmark and two additional demanding scenarios show that the proposed framework can improve the generalization performance of pre-trained language models (PLMs) in downstream tasks. |
How to Make LMs Strong Node Classifiers? (2026.findings-eacl)
Copied to clipboard
Zhe Xu, Kaveh Hassani, Si Zhang, Hanqing Zeng, Michihiro Yasunaga, Limei Wang, Dongqi Fu, Ning Yao, Bo Long, Hanghang Tong
| Challenge: | Language Models (LMs) are increasingly challenging the dominance of domain-specific models, such as Graph Neural Networks (GNNs) and Graph Transformers (GTs). |
| Approach: | They propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the art (SOTA) GNNs on node classification tasks without requiring any architectural modifications. |
| Outcome: | The proposed approach outperforms existing GNNs on node classification tasks and is open-source upon publication. |
Shallow Domain Adaptive Embeddings for Sentiment Analysis (D19-1)
Copied to clipboard
| Challenge: | Existing domain adaptation algorithms for text classification are limited by lack of training data and exploiting domain idiosyncrasies to improve performance. |
| Approach: | They propose a domain adaptation layer that learns weights to combine a generic and a specific word embedding into a DA embeddable. |
| Outcome: | The proposed approach improves on binary and multi-class classification tasks using popular encoder architectures. |
T-FREE: Subword Tokenizer-Free Generative LLMs via Sparse Representations for Memory-Efficient Embeddings (2024.emnlp-main)
Copied to clipboard
| Challenge: | Tokenizers are crucial for encoding information in Large Language Models, but their development has stagnated. |
| Approach: | They propose a tokenizer that embeds words through sparse activation patterns over character triplets . they show competitive downstream performance with a parameter reduction of more than 85% . |
| Outcome: | The proposed approach achieves competitive downstream performance with a parameter reduction of more than 85% on embedding layers. |
Non-Parametric Domain Adaptation for End-to-End Speech Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | End-to-end speech translation (E2E-ST) systems have received increasing attention due to its less error propagation, lower latency and fewer parameters. |
| Approach: | They propose a non-parametric method that leverages in-domain text translation corpus to achieve domain adaptation for E2E-ST systems. |
| Outcome: | The proposed method outperforms the existing in-domain fine-tuning strategies on the Europarl-ST benchmark. |
Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that powerful pre-trained language models can be fooled by small perturbations or intentional attacks. |
| Approach: | They propose a framework for fine-tuning PLMs using a masked language model and Gaussian noise to augment semantically relevant examples with sufficient diversity. |
| Outcome: | The proposed framework improves the robustness of pre-trained language models and alleviates performance degradation under adversarial attacks. |
Exploring Semantic Properties of Sentence Embeddings (P18-2)
Copied to clipboard
| Challenge: | Neural vector representations are ubiquitous throughout all subfields of natural language processing. |
| Approach: | They propose a framework that generates triplets of sentences to explore how changes in the syntactic structure or semantics of a given sentence affect their similarity. |
| Outcome: | The proposed framework generates triplets of sentences to explore how changes in the syntactic structure or semantics of a given sentence affect the similarities obtained between their embeddings. |
VAEGPT-Sim: Improving Sentence Representation with Limited Corpus Using Gradually-Denoising VAE (2024.findings-acl)
Copied to clipboard
| Challenge: | Text embedding requires a highly efficient method for training domain-specific models on limited corpora. |
| Approach: | They propose a model that combines a denoising variational autoencoder with a target-specific discriminator to generate synonymous sentences that closely resemble human language. |
| Outcome: | The proposed model surpasses ConSERT by 2.8 points in small-dataset training on STS benchmarks. |
Language-Codec: Bridging Discrete Codec Representations and Speech Language Models (2025.acl-long)
Copied to clipboard
Shengpeng Ji, Minghui Fang, Jialong Zuo, Ziyue Jiang, Dingdong Wang, Hanting Wang, Hai Huang, Zhou Zhao
| Challenge: | Existing gaps between discrete acoustic codecs and downstream speech language models . initial channel of codebooks contains excessive information, making it difficult to generate tokens from weakly supervised signals such as text. |
| Approach: | They propose a discrete acoustic codec for generating acustic tokens from weakly supervised signals. |
| Outcome: | The proposed language-codec outperforms competing audio compression algorithms and validates on downstream speech language models. |