Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine (D19-62)
Copied to clipboard
| Challenge: | Existing work has trained medical embeddings to rep-resent medical concepts using specific medical data. |
| Approach: | They use intrinsic methods to evaluate pre-trained word embeddings from the various fields of medicine as defined by their ICD-9 systems. |
| Outcome: | The results show that the embeddings perform better in one field of medicine than in other fields. |
Similar Papers
Embedding Strategies for Specialized Domains: Application to Clinical Entity Recognition (P19-2)
Copied to clipboard
| Challenge: | Off-the-shelf word embeddings tend to perform poorly on texts from specialized domains such as clinical reports. |
| Approach: | They combine off-the-shelf contextual embeddings with static word2vec embedders trained on a small in-domain corpus built from task data to reach and sometimes outperform representations learned from a large corpus in the medical domain. |
| Outcome: | The proposed embedding strategies outperform representations learned from a large corpus in the medical domain. |
BioReddit: Word Embeddings for User-Generated Biomedical NLP (D19-62)
Copied to clipboard
| Challenge: | a corpus of medical-themed posts was scrapped from Reddit to train word embeddings on downstream tasks. |
| Approach: | They propose to train word embeddings from a corpus of medical forums from reddit scrapping posts from medical-themed subreddits. |
| Outcome: | The proposed system outperforms embeddings trained on general purpose data or on scientific papers when applied on user-generated content. |
Analyzing Code Embeddings for Coding Clinical Narratives (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work on automated ICD coding learn mappings between low-dimensional representations of clinical text reports and codes. |
| Approach: | They propose novel neural networks for encoding medical codes based on textual, structural and statistical characteristics using a single deep learning baseline model. |
| Outcome: | The proposed methods improve the accuracy of medical codes based on their textual, structural and statistical characteristics. |
What’s in Your Embedding, And How It Predicts Task Performance (C18-1)
Copied to clipboard
| Challenge: | Attempts to find a single technique for general-purpose intrinsic evaluation of word embeddings have so far not been successful. |
| Approach: | They propose a method that quantifies interpretable characteristics of word vector neighborhoods and shows how they correlate with performance on 14 extrinsic and intrinsic task datasets. |
| Outcome: | The proposed approach enables multi-faceted evaluation, parameter search, and generally – a more principled, hypothesis-driven approach to development of distributional semantic representations. |
Just Rank: Rethinking Evaluation with Word and Sentence Similarities (2022.acl-long)
Copied to clipboard
| Challenge: | Word and sentence similarity tasks are the de facto evaluation method for embeddings. |
| Approach: | They propose a new intrinsic evaluation method called EvalRank which shows a much stronger correlation with downstream tasks. |
| Outcome: | The proposed method shows a much stronger correlation with downstream tasks and is released for future benchmarking purposes. |
Benchmarking Meta-embeddings: What Works and What Does Not (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to build meta-embeddings have been evaluated using a variety of methods and datasets, which makes it difficult to draw meaningful conclusions regarding the merits of each approach. |
| Approach: | They propose a unified framework for a fair and objective meta-embedding evaluation using intrinsic and extrinsic tasks. |
| Outcome: | The proposed framework outperforms existing methods on intrinsic and extrinsic evaluation benchmarks and outperformed existing methods. |
A Deeper Look into Dependency-Based Word Embeddings (N18-4)
Copied to clipboard
| Challenge: | Word embeddings trained with dependency contexts excel at different tasks, and enhanced dependencies often improve performance. |
| Approach: | They propose to use dependency-based word embeddings to capture semantic similarity rather than relatedness. |
| Outcome: | The results show that word embeddings trained with Universal and Stanford dependencies excel at different tasks and that enhanced dependencies often improve performance. |
Contextual Embeddings: When Are They Worth It? (2020.acl-main)
Copied to clipboard
| Challenge: | In recent years, rich contextual embeddings have enabled rapid progress on benchmarks like GLUE, but require significant computational resources during pretraining and during downstream task training and inference. |
| Approach: | They empirically compare contextual embeddings with classic pretrained embedders and a random word embeddable with a simple baseline. |
| Outcome: | The proposed models perform within 5 to 10% accuracy on industry-scale data. |
Writing habits and telltale neighbors: analyzing clinical concept usage patterns with sublanguage embeddings (D19-62)
Copied to clipboard
| Challenge: | Existing biomedical concepts may have multiple, often non-compositional surface forms, making them difficult to analyze using lexical occurrence alone. |
| Approach: | They propose a method for characterizing usage patterns of clinical concepts among different document types by embedding concepts on clinical documents of different types and measuring their nearest neighborhood structures. |
| Outcome: | Experiments on the MIMIC-III corpus show that the proposed method captures clinically relevant differences in concept usage while correcting for noise in embedding learning. |
Is Language Modeling Enough? Evaluating Effective Embedding Combinations (2020.lrec-1)
Copied to clipboard
Rudolf Schneider, Tom Oberhauser, Paul Grundmann, Felix Alexander Gers, Alexander Loeser, Steffen Staab
| Challenge: | specialized embeddings are not available for tasks like entity linking or paragraph classification. |
| Approach: | They evaluate whether universal embeddings can be complemented by specialized embeddables. |
| Outcome: | The proposed embeddings outperform state-of-the-art embeddables without any fine-tuning. |