Papers by Sudha Rao
Investigating Agency of LLMs in Human-AI Collaboration Tasks (2024.eacl-long)
Copied to clipboard
| Challenge: | We examine how LLMs can be measured and managed for Agency . a model that manifests high Intentionality, Motivation, Self-Efficacy, and Self-Regulation is more likely to be perceived as strongly agentive. |
| Approach: | They collect a dataset of 83 human-human collaborative interior design conversations containing 908 conversational snippets annotated for Agency features. |
| Outcome: | The proposed models show that they manifest high Intentionality, Motivation, Self-Efficacy, and Self-Regulation, and are more likely to be perceived as agentive. |
Substance over Style: Document-Level Targeted Content Transfer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an entire document to fit a set of constraints. |
| Approach: | They propose a document-level targeted content transfer task that addresses the challenge of rewriting an entire document coherently by generating coherent and diverse rewrites that obey a constraint while remaining close to the original document. |
| Outcome: | The proposed model outperforms existing methods by generating coherent and diverse rewrites that obey the constraint while remaining close to the original document. |
A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | a web-based algorithm can be used to align instructions for different tasks . video instructions can be noisy and contain far more information than textual instructions. |
| Approach: | They propose an algorithm that learns pairwise alignments between different recipes . they then use a graph algorithm to derive a joint alignment between multiple video and text recipes based on the same recipe. |
| Outcome: | The proposed algorithm learns pairwise alignments between different recipes for the same dish. |
Multi-Task Neural Models for Translating Between Styles Within and Across Languages (C18-1)
Copied to clipboard
| Challenge: | Generating natural language requires conveying content in an appropriate style. |
| Approach: | They propose to solve two related tasks on generating text of varying formality jointly using multi-task learning. |
| Outcome: | The proposed models achieve state-of-the-art performance for formality transfer and formality-sensitive machine translation without training on style-annotated translation examples. |
Answer-based Adversarial Training for Generating Clarification Questions (N19-1)
Copied to clipboard
| Challenge: | a goal of natural language processing is to develop techniques that enable machines to process naturally occurring language. |
| Approach: | They propose a model where hypothetical answers are latent variables that can guide the model into generating more useful clarification questions. |
| Outcome: | The proposed model outperforms retrieval-based models and ablations that exclude utility model and adversarial training on two datasets. |
Grounded Keys-to-Text Generation: Towards Factual Open-Ended Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Large pre-trained language models have enabled open-ended generation frameworks to tackle a variety of tasks beyond data-to-text generation. |
| Approach: | They propose a new task to generate a factual description about an entity given guiding keys and grounding passages using a dataset. |
| Outcome: | The proposed model improves factual correctness and recall significantly compared to previous models. |
Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer (N18-1)
Copied to clipboard
| Challenge: | a lack of training and evaluation datasets, benchmarks and automatic metrics has blocked progress in this field. |
| Approach: | They propose to use a grammarly's Yahoo Answers Formality corpus to create the largest corpus for a particular style . they also propose to apply machine translation metrics to the task . |
| Outcome: | The proposed model can be used to train and evaluate a text in a particular style . the proposed model is based on the existing model and can be applied to other tasks . |
Generating a Common Question from Multiple Documents using Multi-source Encoder-Decoder Models (D19-56)
Copied to clipboard
| Challenge: | Ambiguous user queries can result in multiple topics being retrieved from search engines. |
| Approach: | They propose a task of generating a common question from multiple documents by training an RNN-based single encoder-decoder generator from document pairs and then a model that aggregates these word distributions to generate a question. |
| Outcome: | The proposed model significantly outperforms existing models when evaluated using automated metrics and human judgments on the MS-MARCO-QA dataset. |
GPT Perdetry Test: Generating new meanings for new words (2021.naacl-main)
Copied to clipboard
| Challenge: | We create a set of nonce words and prompt GPT-3 to generate their dictionary definitions. |
| Approach: | They create a set of nonce words and prompt GPT-3 to generate their dictionary definitions. |
| Outcome: | The proposed model can process new words and make them 'neologisms' . it can also adapt to and extend a changing vocabulary, the authors found . |
Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization (2021.naacl-main)
Copied to clipboard
Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, Jianfeng Gao
| Challenge: | Abstractive summarization is the task of generating a concise summary of input documents . a middle-aged man and a young girl died after they were unable to avoid the plane . |
| Approach: | They propose a model that enriches the original Transformer with a Tensor Product Representation for abstractive summarization. |
| Outcome: | The proposed model outperforms the Transformer and the original TP-Transformer significantly on several datasets. |
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information (P18-1)
Copied to clipboard
| Challenge: | StackExchange users routinely ask clarifying questions to fill information gaps . a principle goal of asking questions is to fill this information gap . |
| Approach: | They build a model to rank candidates by their usefulness to a given post . they use data from StackExchange to evaluate the model against human judgments . |
| Outcome: | The proposed model outperforms baselines on 500 samples of StackExchange's clarification questions. |
Comparing and Developing Tools to Measure the Readability of Domain-Specific Texts (D19-1)
Copied to clipboard
Elissa Redmiles, Lisa Maszkiewicz, Emily Hwang, Dhruv Kuchhal, Everest Liu, Miraida Morales, Denis Peskov, Sudha Rao, Rock Stevens, Kristina Gligorić, Sean Kross, Michelle Mazurek, Hal Daumé III
| Challenge: | Despite this, we lack a thorough understanding of how to validly measure readability at scale, especially for domain-specific texts. |
| Approach: | They present a comparison of the validity of well-known readability measures and introduce a novel approach to measure readability at scale. |
| Outcome: | The proposed approach addresses shortcomings of existing measures. |
Ask what’s missing and what’s useful: Improving Clarification Question Generation using Global Knowledge (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models that generate clarification questions fail to identify useful information in contexts . human ability to generate fluent and relevant questions is important in reducing ambiguity . |
| Approach: | They propose a model that first identifies what is missing and then generates a question about it. |
| Outcome: | The proposed model outperforms baselines as judged by automatic metrics and humans. |
Automatic Bug Detection in LLM-Powered Text-Based Games Using LLMs (2024.findings-acl)
Copied to clipboard
| Challenge: | Advancements in large language models (LLMs) are revolutionizing interactive game design, but they may exhibit flaws such as hallucinations, forgetfulness, or misinterpretation of prompts. |
| Approach: | They propose a method for automatically identifying LLM bugs from player game logs . their method surpasses unstructured bug-catching methods and fills the gap . |
| Outcome: | The proposed method surpasses unstructured bug-catching methods and fills the gap in detection of logical and design flaws. |
Contrastive Multi-document Question Generation (2021.eacl-main)
Copied to clipboard
Woon Sang Cho, Yizhe Zhang, Sudha Rao, Asli Celikyilmaz, Chenyan Xiong, Jianfeng Gao, Mengdi Wang, Bill Dolan
| Challenge: | Multi-document question generation focuses on generating a question that covers the common aspect of multiple documents, but a naive model trained only using the targeted document set may generate too generic questions that cover a larger scope than delineated by the document set. |
| Approach: | They propose a contrastive learning strategy where given ‘positive’ and ‘negative’ sets of documents, generate a question that is closely related to the ‘positive' set but far away from the ‘negative' set. |
| Outcome: | The proposed model significantly outperforms several strong baselines, as measured by automatic metrics and human evaluation. |