Papers by Vivek Khetan
TADPOLE: Task ADapted Pre-Training via AnOmaLy DEtection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to solve domain shifts in NLP tasks require additional pre-training . current approaches focus on the downstream corpus when it is small, but are not effective . |
| Approach: | They propose a task-adapted pre-training framework that can be used when the downstream corpus is too small for additional pre-tuning. |
| Outcome: | The proposed framework outperforms baseline methods on biomedical, computer science, news, and movie reviews tasks. |
Representation Projection Invariance Mitigates Representation Collapse (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Fine-tuning contextualized representations by pre-trained models can lead to representation degradation, which can result in instability, sub-optimal performance, and weak generalization. |
| Approach: | They propose a regularization method to maintain the information content of representations and reduce representation collapse during fine-tuning. |
| Outcome: | The proposed method outperforms baselines on most tasks and improves out-of-distribution performance. |
RedHOT: A Corpus of Annotated Medical Questions, Experiences, and Claims on Social Media (2023.findings-eacl)
Copied to clipboard
| Challenge: | Social media platforms such as Reddit are vulnerable to misinformation and disinformation. |
| Approach: | They propose a method to automatically derive (noisy) supervision for retrieval of trustworthy evidence relevant to a given claim made on social media. |
| Outcome: | The proposed method outperforms baseline models in the retrieval task performed by medical doctors. |
CHARD: Clinical Health-Aware Reasoning Across Dimensions for Text Generation Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies show that pretrained language models can act as knowledge bases and reason like humans. |
| Approach: | They propose to use pretrained language models to generate free-flow textual explanations about 52 health conditions across three clinical dimensions. |
| Outcome: | The proposed model can generate concise and readable text, but can be improved on medical accuracy and QA. |
DEFT-UCS: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection for Text-Editing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language modelling have led to the availability of many pre-trained language models (PLMs); however, how much data is needed to fine-tune PLMs for downstream tasks? |
| Approach: | They propose a data-efficient fine-tuning framework that leverages unsupervised core-set selection to identify a smaller, representative dataset to fine- tune PLMs for text-editing tasks. |
| Outcome: | The proposed framework is as accurate as CoEDIT across eight different datasets consisting of six different editing tasks, while finetuning on 70% less data. |
MIMICause: Representation and automatic extraction of causal relation types from clinical notes (2022.findings-acl)
Copied to clipboard
| Challenge: | Extracted causal information from clinical notes can be combined with structured EHR data such as demographics, diagnoses, and medications. |
| Approach: | They propose to annotate clinical notes and develop an annotated corpus and provide baseline scores to identify types and direction of causal relations between a pair of biomedical concepts. |
| Outcome: | The proposed annotation guidelines achieved a high inter-annotator agreement and a macro F1 score on the clinical text. |