Explainable and Sparse Representations of Academic Articles for Knowledge Exploration (2020.coling-main)
Copied to clipboard
| Challenge: | a system for summarizing academic articles by concept tagging has shown great coverage and high accuracy of concept identification. |
| Approach: | They propose to transform tagged concepts into sparse vectors as representations of academic documents. |
| Outcome: | The proposed system can be applied to a broader class of applications. |
Similar Papers
MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents (2023.acl-short)
Copied to clipboard
| Challenge: | Pretrained language models can learn rich textual representations, but they cannot provide powerful document-level representations for scientific articles. |
| Approach: | They propose a transformer-based method that learns semantically meaningful representations from scientific papers by fine-tuning transformer models to predict the target journal class based on the abstract. |
| Outcome: | The proposed method outperforms six existing models for representation learning on scientific documents across four evaluation standards. |
Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing sparse retrieval methods suffer from a lack of interpretability . we propose a new interpretability framework that decomposes dense embeddings into distinct, interpretable latent concepts. |
| Approach: | They propose a new interpretability framework that leveragesSparse Autoencoders to decompose uninterpretable dense embeddings fromDPR models into distinct, interpretable latent concepts. |
| Outcome: | The proposed interpretability framework achieves high index-space and computational efficiency while maintaining robust performance across vocabulary and semantic mismatches. |
Large Language Models for Scientific Information Extraction: An Empirical Study for Virology (2024.findings-eacl)
Copied to clipboard
| Challenge: | Scholarly communication in the digital age is facing significant challenges due to the overwhelming volume of publications. |
| Approach: | They propose to use Wikipedia infoboxes and structured Amazon product descriptions to create structured scholarly contribution summaries using text generation capabilities of LLMs. |
| Outcome: | The proposed model can be applied to complex IE tasks within terse domains like Science with 1000x fewer parameters than the state-of-the-art GPT-davinci. |
A Survey of AMR Applications (2024.emnlp-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic representation that takes the form of a rooted, directed graph. |
| Approach: | They analyze more than 100 papers which use Abstract Meaning Representation (AMR) they highlight the range of applications for which AMR has been harnessed and techniques for incorporating it . they also highlight broader AMR engineering patterns and outline areas of future work that seem ripe for AMR incorporation. |
| Outcome: | The results highlight the range of applications for which AMR has been harnessed and the techniques for incorporating it into those applications. |
Taxonomy-guided Semantic Indexing for Academic Paper Search (2024.emnlp-main)
Copied to clipboard
| Challenge: | Academic paper search often struggles to match underlying academic concepts between queries and documents. |
| Approach: | They propose a framework that extracts key concepts from papers and organizes them as a semantic index guided by an academic taxonomy. |
| Outcome: | The proposed framework can be flexibly employed to enhance existing retrieval frameworks. |
Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to summarize documents are not extractive and require an abstractive approach. |
| Approach: | They propose a novel abstractive model which is conditioned on the article’s topics and based entirely on convolutional neural networks. |
| Outcome: | The proposed model outperforms an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |
Automatic Generation of Citation Texts in Scholarly Papers: A Pilot Study (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on automatic generation of citation texts in scholarly papers have not investigated this problem. |
| Approach: | They propose to train an implicit citation extraction model based on BERT and a multi-source pointer-generator network with cross attention mechanism for citation text generation. |
| Outcome: | The proposed model can generate short texts to describe cited papers in scholarly papers with training data. |
SPECTER: Document-level Representation Learning using Citation-informed Transformers (2020.acl-main)
Copied to clipboard
| Challenge: | Recent Transformer language models do not leverage information on inter-document relatedness, which limits their document-level representation power. |
| Approach: | They propose a method to generate document-level embeddings using citation graphs. |
| Outcome: | The proposed method outperforms baselines on document-level tasks. |
Fine-grained Information Extraction from Biomedical Literature based on Knowledge-enriched Abstract Meaning Representation (2021.acl-long)
Copied to clipboard
| Challenge: | Compared with general natural language texts, sentences from scientific papers usually possess wider contexts between knowledge elements. |
| Approach: | They propose a novel biomedical Information Extraction model to extract scientific entities and events from English research papers using Abstract Meaning Representation (AMR) they construct a sentence-level knowledge graph from an external knowledge base and encode it to improve the model's understanding of complex scientific concepts. |
| Outcome: | The proposed model can extract scientific entities and events from scientific literature and improve its understanding of complex scientific concepts. |