Papers by Timothy Miller
Depth-bounding is effective: Improvements and evaluation of unsupervised PCFG induction (D18-1)
Copied to clipboard
| Challenge: | Recent attempts to improve grammar induction systems by bounding complexity of the model have not been compared against unbounded induction models. |
| Approach: | They propose to use a Bayesian induction inducer to limit the search space of the model and then sample trees with or without bounding. |
| Outcome: | The proposed model produces trees more accurately than or competitively with state-of-the-art constituency grammar induction models. |
Cross-document coreference: An approach to capturing coreference without context (D19-62)
Copied to clipboard
| Challenge: | a cross-document coreference annotation schema was developed to extract timelines in the clinical domain. |
| Approach: | They propose a cross-document coreference annotation schema that is governed by schematic rules to create meaningful and consistent cross- document relations. |
| Outcome: | The proposed approach produces an agreement score of 93.77% for identical relations between the two sets of notes. |
Hierarchical Annotation for Building A Suite of Clinical Natural Language Processing Tasks: Progress Note Understanding (2022.lrec-1)
Copied to clipboard
Yanjun Gao, Dmitriy Dligach, Timothy Miller, Samuel Tesch, Ryan Laffin, Matthew M. Churpek, Majid Afshar
| Challenge: | Existing corpus and annotations focus on textual features and relation prediction, but there are no structured corpus models for clinical diagnostic thinking. |
| Approach: | They propose a hierarchical annotation schema with three stages to address clinical diagnostic thinking. |
| Outcome: | The proposed model is based on a large collection of publicly available daily progress notes. |
Summarizing Patients’ Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models (2022.coling-1)
Copied to clipboard
| Challenge: | Problem list summarization requires a model to understand, abstract, and generate clinical documentation. |
| Approach: | They propose a task that summarises patients' main problems from daily progress notes using input from the provider's progress notes during hospitalization. |
| Outcome: | The proposed model outperforms two state-of-the-art seq2seq transformer architectures in summarizing patients' main problems from daily progress notes in the medical information mart for Intensive Care (MIMIC)-III. |
When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications? (2024.findings-emnlp)
Copied to clipboard
Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy Miller, Danielle Bitterman, Matthew Churpek, Majid Afshar
| Challenge: | Numerical data is pivotal for medical questions and answers, but tabular data is not fully integrated into LLMs. |
| Approach: | They examine the effectiveness of vector representations from last hidden states of LLMs for medical diagnostics and prognostics using electronic health record data. |
| Outcome: | The proposed representations outperform those using raw numerical EHR data in medical diagnostics and prognostics. |
Simplified Neural Unsupervised Domain Adaptation (N19-1)
Copied to clipboard
| Challenge: | Existing unsupervised domain adaptation methods use neural networks to learn representations that are trained to predict the values of subset of important features called “pivot features.” |
| Approach: | They propose to combine the representation learner and task learner to improve on existing neural domain adaptation algorithms by removing heuristically-selected "pivot features" they show competitive performance with a simpler model. |
| Outcome: | The proposed model outperforms existing models by removing heuristically-selected pivot features. |
Two-Stage Fine-Tuning for Improved Bias and Variance for Large Pretrained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work challenges the bias-variance trade-off . large pretrained models can have large variance and overfit domain-specific data . |
| Approach: | They propose a bias-variance trade-off that implies learning methods need to balance complexity with data size to minimize under-fitting and over-fit. |
| Outcome: | The proposed method achieves strong results on SuperGLUE and clinical information extraction tasks. |
Spotting Spurious Data with Neural Networks (N18-1)
Copied to clipboard
| Challenge: | Existing methods to identify spurious instances require either annotations generated by each individual annotator or both task-specific and instance-type annotations. |
| Approach: | They propose an approach that discriminates instances based on their "difficulty to learn" they use queueing theory and psychology of learning to improve annotations . |
| Outcome: | The proposed methods outperform state-of-the-art baselines and have a MAP of 0.85 and 0.22 in identifying spurious instances in synthetic and carefully-crowdsourced real-world datasets respectively. |
Unsupervised Learning of PCFGs with Normalizing Flow (P19-1)
Copied to clipboard
| Challenge: | Existing induction models unable to incorporate semantics and morphology into induction . current models lack a robust model for generating morphologically rich sentences . |
| Approach: | They propose a PCFG inducer which uses context embeddings to generalize over rare, morphologically rich forms. |
| Outcome: | The proposed model produces grammars with state-of-the-art accuracy on a variety of languages. |