Papers by Bill Dolan
Investigating Agency of LLMs in Human-AI Collaboration Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | We examine how LLMs can be measured and managed for Agency . a model that manifests high Intentionality, Motivation, Self-Efficacy, and Self-Regulation is more likely to be perceived as strongly agentive. |
| Approach: | They collect a dataset of 83 human-human collaborative interior design conversations containing 908 conversational snippets annotated for Agency features. |
| Outcome: | The proposed models show that they manifest high Intentionality, Motivation, Self-Efficacy, and Self-Regulation, and are more likely to be perceived as agentive. |
Contextualized Perturbation for Textual Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for generating adversarial examples are driven by local heuristic rules that are agnostic to the context, resulting in unnatural and ungrammatical outputs. |
| Approach: | They propose a ContextuaLized AdversaRial Example generation model that generates fluent and grammatical outputs through a mask-then-infill procedure. |
| Outcome: | The proposed model outperforms baseline models in terms of attack success rate, textual similarity, fluency and grammaticality. |
Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading (P19-1)
Copied to clipboard
Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, Jianfeng Gao
| Challenge: | a new approach to contentful neural conversation is proposed . end-to-end models are effective in learning fluent responses, but their responses are often vacuous and uninformative. |
| Approach: | They propose a model that provides the conversation model with relevant text on the fly as a source of external knowledge. |
| Outcome: | The proposed model improves the informativeness and diversity of generated output compared to previous methods. |
Substance over Style: Document-Level Targeted Content Transfer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an entire document to fit a set of constraints. |
| Approach: | They propose a document-level targeted content transfer task that addresses the challenge of rewriting an entire document coherently by generating coherent and diverse rewrites that obey a constraint while remaining close to the original document. |
| Outcome: | The proposed model outperforms existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document. |
Interactive Text Generation (2023.emnlp-main)
Copied to clipboard
Felix Faltings, Michel Galley, Kianté Brantley, Baolin Peng, Weixin Cai, Yizhe Zhang, Jianfeng Gao, Bill Dolan
| Challenge: | Advances in generative modeling have made it possible to automatically generate high-quality texts, code, and images, but they can be unsatisfactory in many respects. |
| Approach: | They propose a task that allows training generation models interactively without the costs of involving real users. |
| Outcome: | The proposed model trains with Imitation Learning without the cost of involving real users and is superior to non-interactive models. |
MixingBoard: a Knowledgeable Stylized Integrated Text Generation Platform (2020.acl-demos)
Copied to clipboard
| Challenge: | Neural text generation algorithms have seen great improvements over the past several years. |
| Approach: | They propose a platform for quickly building demos with a focus on knowledge grounded stylized text generation. |
| Outcome: | The proposed framework unifies existing text generation algorithms in a shared codebase and further adapts earlier algorithms for constrained generation. |
Microsoft Icecaps: An Open-Source Toolkit for Conversation Modeling (P19-3)
Copied to clipboard
Vighnesh Leonardo Shiv, Chris Quirk, Anshuman Suri, Xiang Gao, Khuram Shahid, Nithya Govindarajan, Yizhe Zhang, Jianfeng Gao, Michel Galley, Chris Brockett, Tulasi Menon, Bill Dolan
| Challenge: | upcoming open-source natural language processing repository aims to train conversational agents for multi-turn situations. |
| Approach: | They present the Intelligent Conversation Engine: Code and Pre-trained Systems (ICECAPS) the framework wraps TensorFlow functionality in a modular component-based architecture. |
| Outcome: | The Intelligent Conversation Engine: Code and Pre-trained Systems (ICECAPS) is an open-source natural language processing repository. |
A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | a web-based algorithm can be used to align instructions for different tasks . video instructions can be noisy and contain far more information than textual instructions. |
| Approach: | They propose an algorithm that learns pairwise alignments between different recipes . they then use a graph algorithm to derive a joint alignment between multiple video and text recipes based on the same recipe. |
| Outcome: | The proposed algorithm learns pairwise alignments between different recipes for the same dish. |
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation (2020.acl-demos)
Copied to clipboard
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
| Challenge: | DIALOGPT is a large, tunable neural conversational response generation model . trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Approach: | They present a large, tunable neural conversational response generation model, DIALOGPT . the model is trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Outcome: | The proposed model can generate more relevant, contentful and context-consistent responses than baseline systems. |
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models have enabled open-ended generation frameworks to tackle a variety of tasks beyond data-to-text generation. |
| Approach: | They propose a new task to generate a factual description about an entity given guiding keys and grounding passages using a dataset. |
| Outcome: | The proposed model improves factual correctness and recall significantly compared to previous models. |
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints (D18-1)
Copied to clipboard
| Challenge: | Neural conversation models tend to generate safe, generic responses for most inputs . this is due to the limitations of likelihood-based decoding objectives in generation tasks with diverse outputs, such as conversation. |
| Approach: | They propose a distributional constraint approach that incorporates side information into the generated responses. |
| Outcome: | The proposed approach generates responses that are less generic without sacrificing plausibility. |
Dialogue Response Ranking Training with Large-Scale Human Feedback Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain dialog models can minimize the perplexity of target human responses . however, some human responses are more engaging than others, spawning more followup interactions . |
| Approach: | They train open-domain dialog models to minimize perplexity of target human responses . they use social media feedback data to train models to predict engaging dialog turns . |
| Outcome: | The proposed model outperforms existing models on 133M human feedback pairs . it also outperformed the conventional dialog perplexity baseline model . |
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work on pre-trained generative models often fails to detect non-existent or incorrect content . Existing studies have attempted to detect hallucinations based on oracle references . |
| Approach: | They propose a token-level, reference-free hallucination detection task based on Wikipedia annotations to detect non-existent or incorrect content. |
| Outcome: | The proposed task is token-level, reference-free hallucination detection task and dataset . authors argue that the proposed task can be used in real-time to detect hallucines . |
Automatic Document Sketching: Generating Drafts from Analogous Texts (2021.findings-acl)
Copied to clipboard
| Challenge: | Large pre-trained language models have made it possible to make high-quality predictions on how to add or change a sentence in a document. |
| Approach: | They propose a task to generate entire draft documents for the writer to review and revise. |
| Outcome: | The proposed model can make high-quality predictions on how to add or change a sentence in a document, but it lacks the branching factor to offer useful editing suggestions at a global or document level. |
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models cannot be directly employed to generate text under specified lexical constraints. |
| Approach: | They propose a method for insertion-based text generation that inserts tokens between existing tokens in a parallel manner. |
| Outcome: | The proposed method is intuitive and interpretable on Wikipedia and Yelp datasets. |
Domain Adaptive Text Style Transfer (D19-1)
Copied to clipboard
| Challenge: | Text style transfer without parallel data is a promising method for learning, but in the scenario where less data is available, it may yield poor performance. |
| Approach: | They propose to leverage available data to learn domain-adaptive text style transfer models . they evaluate two style transfer tasks where only limited non-parallel data is available . |
| Outcome: | The proposed models learn from the source domain to: (i) distinguish stylized information and generic content information; (ii) maximally preserve content information and (iv) adaptively transfer the styles in a domain-aware manner. |
Towards More Efficient Insertion Transformer with Fractional Positional Encoding (2023.eacl-main)
Copied to clipboard
| Challenge: | Empirical studies on text generation tasks demonstrate the effectiveness of insertion-based models. |
| Approach: | They propose a reusable positional encoding scheme for insertion transformers that allows reusing representations calculated in previous steps. |
| Outcome: | Empirical studies show that the proposed model reduces the time required to generate a token and improves decoding efficiency. |
Structuring Latent Spaces for Stylized Response Generation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for generating responses in a targeted style are limited by the lack of parallel data. |
| Approach: | They propose a method that bridges conversation modeling and non-parallel style transfer by sharing a structured latent space. |
| Outcome: | The proposed system generates responses of the targeted style and outperforms baselines without sacrificing appropriateness. |
Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs (2024.findings-acl)
Copied to clipboard
| Challenge: | Advancements in large language models (LLMs) are revolutionizing interactive game design, but they may exhibit flaws such as hallucinations, forgetfulness, or misinterpretation of prompts. |
| Approach: | They propose a method for automatically identifying LLM bugs from player game logs . their method surpasses unstructured bug-catching methods and fills the gap . |
| Outcome: | The proposed method surpasses unstructured bug-catching methods and fills the gap in detection of logical and design flaws. |
Text Editing by Command (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has focused on making such models more controllable and factually grounded. |
| Approach: | They propose a novel interactive text generation setting in which the user interacts with the system by issuing commands to edit existing text. |
| Outcome: | The proposed model outperforms baseline models and obtains positive results in automatic and human evaluations. |
Jointly Optimizing Diversity and Relevance in Neural Response Generation (N19-1)
Copied to clipboard
| Challenge: | Recent neural conversation models often generate bland and generic responses . however, the improvement often comes at the cost of decreased relevance . |
| Approach: | They propose a spacefusion model to jointly optimize diversity and relevance that fuses the latent space of a sequence-to-sequence model and that of an autoencoder model by leveraging novel regularization terms. |
| Outcome: | The proposed model improves diversity and relevance compared to baselines in both diversity and diversity. |
Contrastive Multi-document Question Generation (2021.eacl-main)
Copied to clipboard
Woon Sang Cho, Yizhe Zhang, Sudha Rao, Asli Celikyilmaz, Chenyan Xiong, Jianfeng Gao, Mengdi Wang, Bill Dolan
| Challenge: | Multi-document question generation focuses on generating a question that covers the common aspect of multiple documents, but a naive model trained only using the targeted document set may generate too generic questions that cover a larger scope than delineated by the document set. |
| Approach: | They propose a contrastive learning strategy where given ‘positive’ and ‘negative’ sets of documents, generate a question that is closely related to the ‘positive' set but far away from the ‘negative' set. |
| Outcome: | The proposed model significantly outperforms several strong baselines, as measured by automatic metrics and human evaluation. |