Papers by Chris Brockett
Investigating Agency of LLMs in Human-AI Collaboration Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | We examine how LLMs can be measured and managed for Agency . a model that manifests high Intentionality, Motivation, Self-Efficacy, and Self-Regulation is more likely to be perceived as strongly agentive. |
| Approach: | They collect a dataset of 83 human-human collaborative interior design conversations containing 908 conversational snippets annotated for Agency features. |
| Outcome: | The proposed models show that they manifest high Intentionality, Motivation, Self-Efficacy, and Self-Regulation, and are more likely to be perceived as agentive. |
Contextualized Perturbation for Textual Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing techniques for generating adversarial examples are driven by local heuristic rules that are agnostic to the context, resulting in unnatural and ungrammatical outputs. |
| Approach: | They propose a ContextuaLized AdversaRial Example generation model that generates fluent and grammatical outputs through a mask-then-infill procedure. |
| Outcome: | The proposed model outperforms baseline models in terms of attack success rate, textual similarity, fluency and grammaticality. |
Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading (P19-1)
Copied to clipboard
Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, Jianfeng Gao
| Challenge: | a new approach to contentful neural conversation is proposed . end-to-end models are effective in learning fluent responses, but their responses are often vacuous and uninformative. |
| Approach: | They propose a model that provides the conversation model with relevant text on the fly as a source of external knowledge. |
| Outcome: | The proposed model improves the informativeness and diversity of generated output compared to previous methods. |
Microsoft Icecaps: An Open-Source Toolkit for Conversation Modeling (P19-3)
Copied to clipboard
Vighnesh Leonardo Shiv, Chris Quirk, Anshuman Suri, Xiang Gao, Khuram Shahid, Nithya Govindarajan, Yizhe Zhang, Jianfeng Gao, Michel Galley, Chris Brockett, Tulasi Menon, Bill Dolan
| Challenge: | upcoming open-source natural language processing repository aims to train conversational agents for multi-turn situations. |
| Approach: | They present the Intelligent Conversation Engine: Code and Pre-trained Systems (ICECAPS) the framework wraps TensorFlow functionality in a modular component-based architecture. |
| Outcome: | The Intelligent Conversation Engine: Code and Pre-trained Systems (ICECAPS) is an open-source natural language processing repository. |
A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | a web-based algorithm can be used to align instructions for different tasks . video instructions can be noisy and contain far more information than textual instructions. |
| Approach: | They propose an algorithm that learns pairwise alignments between different recipes . they then use a graph algorithm to derive a joint alignment between multiple video and text recipes based on the same recipe. |
| Outcome: | The proposed algorithm learns pairwise alignments between different recipes for the same dish. |
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation (2020.acl-demos)
Copied to clipboard
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
| Challenge: | DIALOGPT is a large, tunable neural conversational response generation model . trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Approach: | They present a large, tunable neural conversational response generation model, DIALOGPT . the model is trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Outcome: | The proposed model can generate more relevant, contentful and context-consistent responses than baseline systems. |
Generating a Common Question from Multiple Documents using Multi-source Encoder-Decoder Models (D19-56)
Copied to clipboard
| Challenge: | Ambiguous user queries can result in multiple topics being retrieved from search engines. |
| Approach: | They propose a task of generating a common question from multiple documents by training an RNN-based single encoder-decoder generator from document pairs and then a model that aggregates these word distributions to generate a question. |
| Outcome: | The proposed model significantly outperforms existing models when evaluated using automated metrics and human judgments on the MS-MARCO-QA dataset. |
Dialogue Response Ranking Training with Large-Scale Human Feedback Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing open-domain dialog models can minimize the perplexity of target human responses . however, some human responses are more engaging than others, spawning more followup interactions . |
| Approach: | They train open-domain dialog models to minimize perplexity of target human responses . they use social media feedback data to train models to predict engaging dialog turns . |
| Outcome: | The proposed model outperforms existing models on 133M human feedback pairs . it also outperformed the conventional dialog perplexity baseline model . |
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work on pre-trained generative models often fails to detect non-existent or incorrect content . Existing studies have attempted to detect hallucinations based on oracle references . |
| Approach: | They propose a token-level, reference-free hallucination detection task based on Wikipedia annotations to detect non-existent or incorrect content. |
| Outcome: | The proposed task is token-level, reference-free hallucination detection task and dataset . authors argue that the proposed task can be used in real-time to detect hallucines . |
Automatic Document Sketching: Generating Drafts from Analogous Texts (2021.findings-acl)
Copied to clipboard
| Challenge: | Large pre-trained language models have made it possible to make high-quality predictions on how to add or change a sentence in a document. |
| Approach: | They propose a task to generate entire draft documents for the writer to review and revise. |
| Outcome: | The proposed model can make high-quality predictions on how to add or change a sentence in a document, but it lacks the branching factor to offer useful editing suggestions at a global or document level. |
POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models cannot be directly employed to generate text under specified lexical constraints. |
| Approach: | They propose a method for insertion-based text generation that inserts tokens between existing tokens in a parallel manner. |
| Outcome: | The proposed method is intuitive and interpretable on Wikipedia and Yelp datasets. |
Domain Adaptive Text Style Transfer (D19-1)
Copied to clipboard
| Challenge: | Text style transfer without parallel data is a promising method for learning, but in the scenario where less data is available, it may yield poor performance. |
| Approach: | They propose to leverage available data to learn domain-adaptive text style transfer models . they evaluate two style transfer tasks where only limited non-parallel data is available . |
| Outcome: | The proposed models learn from the source domain to: (i) distinguish stylized information and generic content information; (ii) maximally preserve content information and (iv) adaptively transfer the styles in a domain-aware manner. |
Structuring Latent Spaces for Stylized Response Generation (D19-1)
Copied to clipboard
| Challenge: | Existing methods for generating responses in a targeted style are limited by the lack of parallel data. |
| Approach: | They propose a method that bridges conversation modeling and non-parallel style transfer by sharing a structured latent space. |
| Outcome: | The proposed system generates responses of the targeted style and outperforms baselines without sacrificing appropriateness. |
Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs (2024.findings-acl)
Copied to clipboard
| Challenge: | Advancements in large language models (LLMs) are revolutionizing interactive game design, but they may exhibit flaws such as hallucinations, forgetfulness, or misinterpretation of prompts. |
| Approach: | They propose a method for automatically identifying LLM bugs from player game logs . their method surpasses unstructured bug-catching methods and fills the gap . |
| Outcome: | The proposed method surpasses unstructured bug-catching methods and fills the gap in detection of logical and design flaws. |
Text Editing by Command (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work has focused on making such models more controllable and factually grounded. |
| Approach: | They propose a novel interactive text generation setting in which the user interacts with the system by issuing commands to edit existing text. |
| Outcome: | The proposed model outperforms baseline models and obtains positive results in automatic and human evaluations. |
Jointly Optimizing Diversity and Relevance in Neural Response Generation (N19-1)
Copied to clipboard
| Challenge: | Recent neural conversation models often generate bland and generic responses . however, the improvement often comes at the cost of decreased relevance . |
| Approach: | They propose a spacefusion model to jointly optimize diversity and relevance that fuses the latent space of a sequence-to-sequence model and that of an autoencoder model by leveraging novel regularization terms. |
| Outcome: | The proposed model improves diversity and relevance compared to baselines in both diversity and diversity. |