Unsupervised Concept Representation Learning for Length-Varying Text Similarity (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing document similarity approaches suffer from the information gap caused by context and vocabulary mismatches when comparing varying-length texts. |
| Approach: | They propose an unsupervised concept representation learning approach to address this issue . they propose a concept-based document matching method to leverage recognition of local phrase features . |
| Outcome: | The proposed method achieves a better F1 score than baseline models on real-world data sets. |
Similar Papers
Length is a Curse and a Blessing for Document-level Semantics (2023.emnlp-main)
Copied to clipboard
| Challenge: | In recent years, contrastive learning (CL) has been extensively utilized to recover sentence and document-level encoding capability from pre-trained language models. |
| Approach: | They propose a document-based contrastive learning framework that is length-agnostic self-reference based on document length. |
| Outcome: | The proposed framework achieves state-of-the-art on the standard information retrieval benchmark. |
Self-Supervised Document Similarity Ranking via Contextualized Language Models and Hierarchical Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to document-to-document similarity ranking are limited to relatively short documents or lack similarity labels. |
| Approach: | They propose a self-supervised method for document similarity ranking that can be applied to documents of arbitrary length. |
| Outcome: | The proposed model outperforms existing methods on large documents datasets. |
Self-Discriminative Learning for Unsupervised Document Embedding (N19-1)
Copied to clipboard
| Challenge: | Existing methods for document embedding learning do not consider inter-document relationships. |
| Approach: | They propose to exploit the inter-document information and directly model the relations of documents in embedding space with a discriminative network and a novel objective. |
| Outcome: | The proposed method has errors that are 5 to 13% lower than state-of-the-art models and is even more pronounced in scarce label setting. |
RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank (2023.acl-long)
Copied to clipboard
Jiduan Liu, Jiahao Liu, Qifan Wang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, Kai Chen, Rui Yan
| Challenge: | Unsupervised sentence representation learning is one of the fundamental problems in natural language processing . contrastive learning methods fail to capture fine-grained ranking information among the sentences . |
| Approach: | They propose a novel approach for unsupervised sentence representation learning that integrates ranking consistency and ranking distillation with contrastive learning into a unified framework. |
| Outcome: | The proposed approach performs better over state-of-the-art models on STS and TR tasks. |
Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prior work relies on discrete citation relations to generate contrast samples, but discrete ones enforce a hard cut-off to similarity. |
| Approach: | They propose to use nearest neighbor sampling to learn continuous similarity and to sample hard-to-learn negatives and positives by controlling the sampling margin between them. |
| Outcome: | The proposed method outperforms the state-of-the-art on the SciDocs benchmark and can train (or tune) language models sample-efficiently. |
Representation Learning for Information Extraction from Form-like Documents (2020.acl-main)
Copied to clipboard
| Challenge: | Form-like documents like invoices, purchase orders, tax forms and insurance quotes are common in day-to-day business workflows, but current techniques for processing them largely still employ manual effort or brittle and error-prone heuristics for extraction. |
| Approach: | They propose an extraction system that uses knowledge of the types of the target fields to generate extraction candidates and a neural network architecture that learns a dense representation of each candidate based on neighboring words in the document. |
| Outcome: | The proposed system generates extraction candidates based on neighboring words in the document and is interpretable, as shown using loss cases. |
Matching Varying-Length Texts via Topic-Informed and Decoupled Sentence Embeddings (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to matching text with non-comparable lengths are limited due to truncation issues. |
| Approach: | They propose a model that decouples sentences and embeds them into natural sentences for matching texts of significantly different lengths. |
| Outcome: | The proposed model matches texts of significantly different lengths across three well-studied datasets. |
Fine-grained Contrastive Learning for Definition Generation (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent pre-trained transformer-based definition generation models lack effective representation learning to contain full semantic components of the given word, leading to under-specific definitions. |
| Approach: | They propose a novel contrastive learning method that encourages the model to capture more detailed semantic representations from the definition sequence encoding. |
| Outcome: | The proposed method could generate more specific definitions compared with state-of-the-art models. |
Document Similarity for Texts of Varying Lengths via Hidden Topics (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to measure document similarity are inadequate for document pairs with non-comparable lengths, such as a long document and its summary. |
| Approach: | They propose a document matching approach to bridge the gap between long documents and their abstract information in a common space of hidden topics. |
| Outcome: | The proposed approach outperforms strong baselines on two matching tasks and incorporates domain knowledge to gain further performance improvement. |
Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Word vector representations suffer from a monolingual bias due to the amount of data available across languages. |
| Approach: | They propose a technique for building language-independent vector representations of concepts which places multilinguality at its core while retaining explicit relationships between concepts. |
| Outcome: | The proposed representations outperform the state-of-the-art in multilingual and cross-lingual Semantic Word Similarity and Word Sense Disambiguation on low-resource languages. |