Papers by Changye Li
GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large numbers of model parameters have shown impressive performance on the task of discriminating between language produced by cognitively healthy individuals and those with Alzheimer’s disease (AD). |
| Approach: | They propose to use a Transformer DL model pre-trained on general English text to combine an artificially degraded version of itself with a model that generalizes well to spontaneous conversations. |
| Outcome: | The proposed method generalizes well to spontaneous conversations and generates text with characteristics associated with AD, demonstrating the induction of dementia-related linguistic anomalies. |
Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that the attention mechanism in transformer-based NLMs may present an analogue to the notions of cognitive and brain reserve. |
| Approach: | They propose a bidirectional ablation method that masks attention heads to display degradation of similar magnitude to masking in smaller models. |
| Outcome: | The proposed method exhibits properties attributed to the concepts of cognitive and brain reserve in human brain studies. |
Mitigating Confounding in Speech-Based Dementia Detection through Weight Masking (2025.acl-long)
Copied to clipboard
| Challenge: | Pre-trained neural language models fine-tuned on AD transcripts perform well, but little research has explored the effects of the gender of the speakers represented by these transcripts. |
| Approach: | They propose to use the Extended Confounding Filter and the Dual Filter to isolate and ablate weights associated with gender in dementia datasets. |
| Outcome: | The proposed methods overfit to training data distributions and disrupt gender-related weights, with the trade-off of slightly reduced dementia detection performance. |
"Newspaper Eat" Means "Not Tasty": A Taxonomy and Benchmark for Coded Language in Real-World Chinese Online Reviews (2026.acl-long)
Copied to clipboard
| Challenge: | Current language models handle coded language poorly, with limited real-world datasets and clear taxonomies. |
| Approach: | They propose a taxonomy that captures common encoding strategies including phonetic, orthographic, and cross-lingual substitutions. |
| Outcome: | The proposed model fails to detect or understand coded language in Chinese reviews . negative reviews can expose users to social pressure, retaliation, or reduced visibility . |
Language Models Resist Alignment: Evidence From Data Compression (2025.acl-long)
Copied to clipboard
Jiaming Ji, Kaile Wang, Tianyi Alex Qiu, Boyuan Chen, Jiayi Zhou, Changye Li, Hantao Lou, Josef Dai, Yunhuai Liu, Yaodong Yang
| Challenge: | Large language models (LLMs) may exhibit undesirable behaviors due to the inevitable biases and harmful content present in training. |
| Approach: | They propose to investigate the elasticity of large language models by examining their performance. |
| Outcome: | The proposed model performance declines rapidly before reverting to the pre-training distribution, the authors show . the proposed model weight and code are available at pku-lm-res ist-alignment.github.io. |