Papers by Jeffrey Bigham
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation (2021.findings-acl)
Copied to clipboard
| Challenge: | Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks. |
| Approach: | They propose methods for automatically creating adversarial negative training data . they use mask-and-fill and keyword-guided approaches to generate negative examples . |
| Outcome: | The proposed approaches outperform baseline models in providing informative negative examples for training dialogue systems. |
DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit (2022.lrec-1)
Copied to clipboard
| Challenge: | DialCrowd 2.0 helps requesters obtain higher quality data from human intelligence tasks. |
| Approach: | They propose to use DialCrowd 2.0 to help requesters obtain higher quality data . they aim to improve the way requesters present tasks and facilitate effective communication with workers. |
| Outcome: | The proposed toolkit enables requesters to obtain higher quality data by presenting tasks more clearly and facilitating effective communication with workers. |
Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques (2021.acl-long)
Copied to clipboard
| Challenge: | Creating digital SOAP notes is burdensome and contributes to physician burnout . authors propose a pipeline to generate these notes based on transcripts of clinical conversations . |
| Approach: | They propose a pipeline to leverage deep summarization models based on conversations between physicians and patients . they propose an algorithm that extracts important utterances relevant to each section and generates one summary sentence per cluster . |
| Outcome: | The proposed algorithm outperforms its abstract counterpart by 8 ROUGE-1 points and produces more factual sentences as assessed by human evaluators. |
Does Pretraining for Summarization Require Knowledge Transfer? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing theories claim that pretraining models learn linguistic knowledge from the pretraining corpus, but scientific explanations for these benefits remain unknown. |
| Approach: | They propose to use random character n-grams to test models on real corpora to see if the small residual benefit of using real data could be accounted for by the structure of the pretraining task. |
| Outcome: | The proposed task performs on documents consisting of character n-grams, whereas pretrained models perform on real corpora with no residual benefit. |
Target-Guided Dialogue Response Generation Using Commonsense and Data Augmentation (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for target-guided response generation are inconsistent with human judgement ratings. |
| Approach: | They propose a technique that finds a bridging path between the source and target and uses it to generate transition responses. |
| Outcome: | The proposed technique outperforms baselines on target-guided response generation task. |
Controlling Dialogue Generation with Semantic Exemplars (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to control dialogue generation are manual labelling and manual editing of data. |
| Approach: | They propose a method to control dialogue generation using exemplar responses . they use semantic frames present in exemplars to guide response generation . |
| Outcome: | The proposed model improves coherence while preserving semantic meaning and conversation goals . exemplar responses are handwritten or strategically curated to promote highlevel goals without explicit labels . |
USB: A Unified Summarization Benchmark Across Tasks and Domains (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing summarization benchmarks lack the rich annotations needed to address important problems related to control and reliability. |
| Approach: | They propose a Wikipedia-derived summarization benchmark with crowd-sourced annotations . they find that fine-tuned models outperform larger few-shot prompted language models . |
| Outcome: | The proposed model outperforms many-shot prompted language models on multiple tasks . the proposed model is based on Wikipedia annotations and can be used in other domains . |
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Instruction tuning is emerging in NLP, but has not been explored for dialogue-related tasks. |
| Approach: | They propose an instruction tuning framework for dialogue that leverages natural language instructions with language models to induce zero-shot generalization on unseen tasks. |
| Outcome: | The proposed framework enables good zero-shot performance on unseen datasets and tasks such as dialogue evaluation and intent detection. |
Downstream Datasets Make Surprisingly Good Pretraining Corpora (2023.acl-long)
Copied to clipboard
| Challenge: | a dominant practice is to fine tune large pretrained transformer models using smaller downstream datasets . performance gains are not always attributable to the use of external data in massive amounts . |
| Approach: | They propose to use the same (downstream) training data for pretraining and finetuning to compare models. |
| Outcome: | The proposed model outperforms standard pretraining on the BookWiki corpus on 7 and 5 datasets. |