Augmenting word2vec with latent Dirichlet allocation within a clinical application (N19-1)
Copied to clipboard
| Challenge: | Existing models that combine latent Dirichlet allocation and word embedding for distinguishing between speakers with and without Alzheimer’s disease from transcripts of picture descriptions are not suitable for clinical binary text classification tasks. |
| Approach: | They propose three models that combine latent Dirichlet allocation and word embedding for distinguishing between speakers with and without Alzheimer’s disease from transcripts of picture descriptions. |
| Outcome: | The proposed models outperform word2vec and LDA models on a clinical binary text classification task. |
Similar Papers
Embedding Strategies for Specialized Domains: Application to Clinical Entity Recognition (P19-2)
Copied to clipboard
| Challenge: | Off-the-shelf word embeddings tend to perform poorly on texts from specialized domains such as clinical reports. |
| Approach: | They combine off-the-shelf contextual embeddings with static word2vec embedders trained on a small in-domain corpus built from task data to reach and sometimes outperform representations learned from a large corpus in the medical domain. |
| Outcome: | The proposed embedding strategies outperform representations learned from a large corpus in the medical domain. |
How to represent a word and predict it, too: Improving tied architectures for language modelling (D18-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art models use word embeddings as input and output mappings instead of tied models. |
| Approach: | They propose to decouple hidden state from word embedding prediction . they extend their proposed modification to word2vec models . |
| Outcome: | The proposed architectures achieve comparable or better results compared to previous models without tying . the proposed architecture reduces parameters, enabling more compact models and faster learning. |
Multilingual prediction of Alzheimer’s disease through domain adaptation and concept-based language modelling (N19-1)
Copied to clipboard
Kathleen C. Fraser, Nicklas Linz, Bai Li, Kristina Lundholm Fors, Frank Rudzicz, Alexandra König, Jan Alexandersson, Philippe Robert, Dimitrios Kokkinakis
| Challenge: | Existing work on speech and language models has been limited by the size of available datasets. |
| Approach: | They propose to augment a small French dataset with a much larger English dataset to augment the language model to model the order in which information units are produced by dementia patients and controls. |
| Outcome: | The proposed model improves classification performance in English and French separately. |
Text Classification by Contrastive Learning and Cross-lingual Data Augmentation for Alzheimer’s Disease Detection (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for AD detection are too expensive and time-consuming to cover all potential patients. |
| Approach: | They propose a contrastive learning method to obtain effective text representations based on monolingual embeddings of BERT and a cross-lingual data augmentation method by building autoencoders to learn the text representation shared by both languages. |
| Outcome: | The proposed method outperforms other methods on a Mandarin AD corpus and achieves 81.6% detection accuracy. |
Quantifying Compositionality of Classic and State-of-the-Art Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Static word embeddings make strong claims about compositionality, but the SOTA generative models go too far in the other direction. |
| Approach: | a new study evaluates the compositionality of word embeddings by canonical correlation analysis . strong compositional signals are observed in later training stages across data modalities . |
| Outcome: | a new evaluation of compositional models shows that they exploit access meanings when justified . strong compositional signals are observed in later training stages and in deeper layers of the transformer-based model before a decline at the top layer. |
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 6: Tutorial Abstracts) (2023.acl-tutorials)
Copied to clipboard
| Challenge: | ACL 2023 tutorials are organized to give conference attendees a comprehensive introduction by expert researchers to some topics of importance drawn from our rapidly growing and changing research field. |
| Approach: | ACL 2023 tutorials session is organized to give conference attendees a comprehensive introduction by experts in the field. 42 tutorial submissions were received, of which 6 were selected for presentation at ACL. |
| Outcome: | ACL 2023 tutorials are organized to give conference attendees a comprehensive introduction . the review committee includes the EACL tutorial chairs and the ACL tutorial chair . |
Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging (D18-1)
Copied to clipboard
| Challenge: | Low-resource languages lack manual annotated data to learn basic models such as part-of-speech (POS) taggers. |
| Approach: | They propose a cross-lingual neural part-of-speech tagger that learns from disparate sources of distant supervision in a uniform framework. |
| Outcome: | The proposed model scales to hundreds of low-resource languages without access to gold annotated data. |
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 5: Tutorial Abstracts) (2024.acl-tutorials)
Copied to clipboard
| Challenge: | ACL tutorial session is a highlight of the conference . it aims to provide attendees with a thorough introduction to key topics in our fast-evolving research field . |
| Approach: | the ACL tutorial session is a highlight of the conference . it aims to provide attendees with a thorough introduction to key topics in the field . |
| Outcome: | the ACL tutorial session is a highlight of the conference . the review process involved multiple conferences and three reviewers . |
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 5: Tutorial Abstracts) (2025.acl-tutorials)
Copied to clipboard
| Challenge: | ACL 2025 tutorial sessions are a cornerstone event of the conference . 76 tutorial submissions were received this year, many of which were very engaging . |
| Approach: | 76 tutorial submissions were received this year for the tutorial session at ACL 2025 . the tutorials are designed to equip you with the latest insights, tools, and methodologies . |
| Outcome: | the tutorial sessions at ACL 2025 will be held in london on november 8 . the conference received 76 tutorial submissions this year . |
Domain-Specific Word Embeddings with Structure Prediction (2023.tacl-1)
Copied to clipboard
| Challenge: | Current word embedding methods do not provide a way to use or predict information on structure between sub-corpora, time or domain. |
| Approach: | They propose a word embedding method that provides general word representations for the whole corpus, domain-specific representations and embeddable alignment simultaneously. |
| Outcome: | The proposed method provides better performance than baselines on a dataset of science and philosophy articles. |