Papers by Serguei Pakhomov
GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large numbers of model parameters have shown impressive performance on the task of discriminating between language produced by cognitively healthy individuals and those with Alzheimer’s disease (AD). |
| Approach: | They propose to use a Transformer DL model pre-trained on general English text to combine an artificially degraded version of itself with a model that generalizes well to spontaneous conversations. |
| Outcome: | The proposed method generalizes well to spontaneous conversations and generates text with characteristics associated with AD, demonstrating the induction of dementia-related linguistic anomalies. |
Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that the attention mechanism in transformer-based NLMs may present an analogue to the notions of cognitive and brain reserve. |
| Approach: | They propose a bidirectional ablation method that masks attention heads to display degradation of similar magnitude to masking in smaller models. |
| Outcome: | The proposed method exhibits properties attributed to the concepts of cognitive and brain reserve in human brain studies. |
A Tale of Two Perplexities: Sensitivity of Neural Language Models to Lexical Retrieval Deficits in Dementia of the Alzheimer’s Type (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show that cognitive manifestations of future dementia may appear as early as 18 years prior to clinical diagnosis . lack of clear diagnosis and prognosis, possibly for an Alzheimer's type, is a major limitation of current methods for identifying dementia-specific cognitive markers. |
| Approach: | They propose to interrogate neural LMs trained on participants with and without dementia by manipulating lexical frequency. |
| Outcome: | The proposed model improves upon the current state-of-the-art for models trained on transcripts of speech produced by healthy participants and those with dementia. |
Conversational Agent for Daily Living Assessment Coaching Demo (2021.eacl-demos)
Copied to clipboard
| Challenge: | Conversational Agent for Daily Living Assessment Coaching (CADLAC) is a multi-modal conversational agent system designed to impersonate “individuals” with various levels of ability in activities of daily living. |
| Approach: | They propose to use a multi-modal conversational agent system to impersonate individuals with various levels of ability in activities of daily living to train assessors how to conduct interviews . |
| Outcome: | The system is implemented on the MindMeld platform for conversational AI and features a bidirectional long short-term memory topic tracker that allows the agent to navigate conversations spanning 18 different ADL domains. |