Papers by Trevor Cohen
APPLS: Evaluating Evaluation Metrics for Plain Language Summarization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for plain language summarization (PLS) lack a dedicated assessment metric and the suitability of text generation evaluation metrics is unclear due to unique transformations. |
| Approach: | They propose a granular meta-evaluation testbed to evaluate PLS metrics . they identify four PLS criteria and define perturbations that sensitive metrics should be able to detect . |
| Outcome: | The proposed testbed assesses performance of 14 existing metrics including scores, features, and prompt-based evaluations. |
GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large numbers of model parameters have shown impressive performance on the task of discriminating between language produced by cognitively healthy individuals and those with Alzheimer’s disease (AD). |
| Approach: | They propose to use a Transformer DL model pre-trained on general English text to combine an artificially degraded version of itself with a model that generalizes well to spontaneous conversations. |
| Outcome: | The proposed method generalizes well to spontaneous conversations and generates text with characteristics associated with AD, demonstrating the induction of dementia-related linguistic anomalies. |
Personalized Jargon Identification for Enhanced Interdisciplinary Communication (2024.naacl-long)
Copied to clipboard
| Challenge: | Identifying and translating scientific jargon for individual researchers could speed up research, but current methods of jaron identification rely on corpus-level familiarity indicators rather than modeling researcher-specific needs. |
| Approach: | They collect over 10K term familiarity annotations from 11 computer science researchers and investigate supervised and prompt-based methods to predict individual jargon familiarity. |
| Outcome: | The proposed method improves jargon familiarity prediction by using domain, subdomain, and individual knowledge. |
Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that the attention mechanism in transformer-based NLMs may present an analogue to the notions of cognitive and brain reserve. |
| Approach: | They propose a bidirectional ablation method that masks attention heads to display degradation of similar magnitude to masking in smaller models. |
| Outcome: | The proposed method exhibits properties attributed to the concepts of cognitive and brain reserve in human brain studies. |
Mitigating Confounding in Speech-Based Dementia Detection through Weight Masking (2025.acl-long)
Copied to clipboard
| Challenge: | Pre-trained neural language models fine-tuned on AD transcripts perform well, but little research has explored the effects of the gender of the speakers represented by these transcripts. |
| Approach: | They propose to use the Extended Confounding Filter and the Dual Filter to isolate and ablate weights associated with gender in dementia datasets. |
| Outcome: | The proposed methods overfit to training data distributions and disrupt gender-related weights, with the trade-off of slightly reduced dementia detection performance. |
Improving Classification of Infrequent Cognitive Distortions: Domain-Specific Model vs. Data Augmentation (2022.naacl-srw)
Copied to clipboard
| Challenge: | Cognitive distortions are one of the targets of cognitive behavioral therapy (CBT). |
| Approach: | They propose to use Easy Data Augmentation, back translation, and mixup techniques to detect distortions in text-based therapy messages. |
| Outcome: | The proposed methods improve performance with optimized parameter settings for rare classes with an augmented model, MentalBERT. |
A Tale of Two Perplexities: Sensitivity of Neural Language Models to Lexical Retrieval Deficits in Dementia of the Alzheimer’s Type (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that cognitive manifestations of future dementia may appear as early as 18 years prior to clinical diagnosis . lack of clear diagnosis and prognosis, possibly for an Alzheimer's type, is a major limitation of current methods for identifying dementia-specific cognitive markers. |
| Approach: | They propose to interrogate neural LMs trained on participants with and without dementia by manipulating lexical frequency. |
| Outcome: | The proposed model improves upon the current state-of-the-art for models trained on transcripts of speech produced by healthy participants and those with dementia. |