Contextualized Semantic Distance between Highly Overlapped Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Conventional semantic metrics are based on word representations and are vulnerable to disturbance of overlapped components with similar representations. |
| Approach: | They propose a mask-and-predict strategy to evaluate the semantic distance between the overlapped sentences using words in the longest common sequence as neighboring words and use masked language modeling to predict their positions. |
| Outcome: | The proposed method outperforms the state-of-the-art in domain adaption by a huge margin. |
Similar Papers
Benchmarking LLMs on Semantic Overlap Summarization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are the most capable text generation models in a variety of tasks and fields. |
| Approach: | They benchmark Large Language Models (LLMs) on SOS and introduce PrivacyPolicyPairs (3P) a dataset of 135 high-quality privacy policy documents is used to evaluate the model. |
| Outcome: | The proposed dataset complements existing resources and broadens domain coverage. |
Quantifying Context Overlap for Training Word Embeddings (D18-1)
Copied to clipboard
| Challenge: | Experimental results show that word embeddings can be improved using word embeds . word embedings are a popular form of natural language processing . |
| Approach: | They propose to estimate second order co-occurrence relations based on context overlap . they use the augmented data to enhance word embeddings learning . |
| Outcome: | The proposed model improves word vectors for word similarity and downstream NLP tasks. |
Matching Varying-Length Texts via Topic-Informed and Decoupled Sentence Embeddings (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to matching text with non-comparable lengths are limited due to truncation issues. |
| Approach: | They propose a model that decouples sentences and embeds them into natural sentences for matching texts of significantly different lengths. |
| Outcome: | The proposed model matches texts of significantly different lengths across three well-studied datasets. |
MATCHA: Matching Text via Contrastive Semantic Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | MATCHA is an automatic metric that rewards semantic agreement with a reference and penalizes contradictions. |
| Approach: | They introduce a metric that jointly rewards semantic agreement with a reference and penalizes contradictions. |
| Outcome: | The proposed metric outperforms popular metrics on eight public benchmarks compared with human annotations on question-answering, image caption generation, natural language inference, summarization, and semantic textual similarity tasks. |
Lost in the Distance: Large Language Models Struggle to Capture Long-Distance Relational Knowledge (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent large language models have demonstrated impressive capabilities in handling long contexts . however, as context length increases, LLMs struggle more with filtering out irrelevant information . |
| Approach: | They propose to use unrelated sentences to capture relational knowledge over long contexts . they find that LLMs can handle edge noise with little impact, but can reason about distant relationships . |
| Outcome: | The proposed model can handle edge noise with little impact, but its ability to reason about distant relationships declines as the noise grows. |
Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on word-level predictions and highlighting semantic differences in natural language documents did not focus on semantic differences as the main target. |
| Approach: | They propose to perform a token-level regression task to highlight semantic differences between two documents . they use word alignment and sentence-level contrastive learning to evaluate the approaches . |
| Outcome: | The proposed approach has a robust correlation to gold labels, but all unsupervised approaches leave a margin of improvement. |
Semantic Overlap Summarization among Multiple Alternative Narratives: An Exploratory Study (2022.coling-1)
Copied to clipboard
| Challenge: | Existing tasks for summarizing multiple alternate narratives with different perspectives are under-explored. |
| Approach: | They propose a task which entails generating a single summary from multiple alternative narratives . they use a web-based dataset and human annotations to evaluate the task . |
| Outcome: | The proposed task is based on a novel dataset and human annotations. |
Bridging Distribution Gap via Semantic Rewriting with LLMs to Enhance OOD Robustness (2024.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning on indistribution data fail to provide robustness against distribution shifts limiting the practical deployment of LLMs in dynamic real-world scenarios. |
| Approach: | They propose a method that leverages the flexibility of LLMs to align both in-distribution (ID) and OOD data with the LLM's distributions. |
| Outcome: | The proposed method outperforms fine-tuning methods on OOD tasks and benchmark datasets. |
SEM-F1: an Automatic Way for Semantic Evaluation of Multi-Narrative Overlap Summaries at Scale (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has introduced an important yet relatively under-explored NLP task called Semantic Overlap Summarization (SOS) that entails generating a summary from multiple alternative narratives which conveys the common information provided by those narratives. |
| Approach: | They propose to use a sentence-level precision-recall style automated evaluation metric to evaluate a new NLP task called Semantic Overlap Summarization (SOS) they propose to employ the popular ROUGE metric and use it to compare the two tasks. |
| Outcome: | The proposed metric yields higher correlation with human judgment and higher inter-rater agreement compared to the existing metric. |
Predicting Numerals in Text Using Nearest Neighbor Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | naive language models treat numerals as string tokens, resulting in difficulty in acquiring commonsense . kNN-LM is an extension of pre-trained neural LMs with the k-nearest neighbor (kNN) search . |
| Approach: | They apply k-nearest neighbor LM to a masked numeral prediction task . they found it is effective for fine-grained predictions of numerals from context . |
| Outcome: | The retrieval-based method is effective for fine-grained numeral prediction from context . it improves accuracy for the OOV numerals, the study shows . |