Papers by Kathleen McKeown
Event-Guided Denoising for Multilingual Relation Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for general purpose relation extraction use a fixed set of predetermined relations, but research has shifted to the identification of unseen relations in any language. |
| Approach: | They propose a method for collecting high quality relation training data for relation extraction from unlabeled text that achieves a near-recreation of their zero-shot and few-shot results at a fraction of the training cost. |
| Outcome: | The proposed method achieves comparable results to the current state-of-the-art when trained on a smaller multilingual encoder . |
Summarization of Opinionated Political Documents with Varied Perspectives (2025.coling-main)
Copied to clipboard
| Challenge: | Political ideologies can lead people to develop misperceptions of groups with opposing opinions, such as the 2024 US presidential election, French legislative election, or the Brexit referendum. |
| Approach: | They propose a dataset and task for independently summarizing political perspectives in a set of opinionated news articles. |
| Outcome: | The proposed dataset and task evaluates models of varying sizes and architectures on a set of opinionated news articles. |
Layered Insights: Generalizable Analysis of Human Authorial Style by Leveraging All Transformer Layers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to authorship attribution model only learn from the output layer of pre-trained transformers, ignoring representations learned at other layers. |
| Approach: | They propose a model that leverages the various linguistic representations learned at different layers of pre-trained transformer-based models to model the authorship attribution task more effectively. |
| Outcome: | The proposed model performs better on out-of-domain and in-domain scenarios, while ignoring representations learned at other layers. |
Exploring Chain-of-Thought Reasoning for Steerable Pluralistic Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are typically trained to reflect a relatively uniform set of values, which limits their applicability to tasks that require understanding of nuanced human perspectives. |
| Approach: | They propose to use Chain-of-Thought reasoning techniques to build steerable pluralistic models by fine-tuning on human-authored CoT and synthetic explanations. |
| Outcome: | The proposed methods outperform others and demonstrate strong sample efficiency. |
Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Abstractive summarization systems still suffer from faithfulness errors, authors say . prior work has proposed models that improve faithfulness, but it is unclear whether this improvement comes from an increased level of extractiveness of the outputs. |
| Approach: | They propose a faithfulness-abstractiveness trade-off curve that serves as a control . they also learn a selector to identify the most faithful and abstractive summary for a given document . |
| Outcome: | The proposed model achieves higher faithfulness scores while being abstractive than the baseline system on two datasets. |
A Bag of Tricks for Dialogue Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using a pretrained sequence-to-sequence language model, we explore speaker name substitution, negation scope highlighting, multi-task learning with relevant tasks, and pretraining on in-domain data. |
| Approach: | They propose a pretrained sequence-to-sequence language model that can handle different parts of dialogue belonging to multiple speakers and combine them to produce a coherent monologue summary. |
| Outcome: | The proposed techniques outperform baseline models on a dialogue summarization dataset. |
Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events (2020.emnlp-main)
Copied to clipboard
Miguel Ballesteros, Rishita Anubhai, Shuai Wang, Nima Pourdamghani, Yogarshi Vyas, Jie Ma, Parminder Bhatia, Kathleen McKeown, Yaser Al-Onaizan
| Challenge: | Existing models for temporal ordering of events rely on pretrained representations, transfer and multitask learning, and self-training techniques. |
| Approach: | They propose a neural architecture and a set of training methods for ordering events by predicting temporal relations by pre-training models. |
| Outcome: | The proposed models can predict temporal relations between two pairs of events within a span of text and identify temporal relationships between them. |
Seeded Hierarchical Clustering for Expert-Crafted Taxonomies (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Practitioners from many disciplines use expert-crafted taxonomies to make sense of large, unlabeled corpora. |
| Approach: | They propose a weakly supervised algorithm for seeded hierarchical clustering that fits unlabeled data to taxonomies using a small set of labeled examples. |
| Outcome: | The proposed algorithm outperforms baselines on three real-world datasets. |
When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (2023.eacl-main)
Copied to clipboard
Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, Tatsunori Hashimoto
| Challenge: | Existing studies have shown that large language models contain linguistic and societal biases, but it is unclear how these biase amplify to downstream tasks. |
| Approach: | They investigate how name-nationality bias propagates from pre-training to downstream tasks . they show that these biases manifest themselves as hallucinations in summarization . |
| Outcome: | The proposed model can reduce the rate of hallucinations, but does not change the types of biases that do appear. |
iBERT: Interpretable Embeddings via Sense Decomposition (2026.eacl-long)
Copied to clipboard
| Challenge: | iBERT is an encoder that produces inherently interpretable and controllable embeddings without compromising performance. |
| Approach: | They propose an encoder that produces interpretable embeddings that modularize and expose discriminative cues present in language. |
| Outcome: | The proposed model outperforms baselines on style-focused tasks while maintaining competitive performance on authorship verification. |
Cross-language Sentence Selection via Data Augmentation and Rationale Training (2021.acl-long)
Copied to clipboard
| Challenge: | a new approach to cross-language sentence selection is proposed for low-resource contexts . a cross-lingual embedding-based model is proposed that avoids translation entirely . |
| Approach: | They propose a cross-lingual embedding-based query relevance model that uses data augmentation and negative sampling techniques to directly learn a query-sentence pair. |
| Outcome: | The proposed approach performs better than state-of-the-art models on noisy parallel data . consistent improvements are seen across three language pairs over state- of-the art models . |
Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing fact-checking benchmarks require systems to verify claims from everyday text against evidence from scientific journal articles. |
| Approach: | They propose a benchmark system that checks claims from news against scientific journal articles and veracity labels. |
| Outcome: | The new benchmark achieves F1 scores of 76.99 and 69.90 on both a fact-checking specific system and GPT-3.5, respectively. |
Evaluating Defeasible Reasoning in LLMs with DEFREASING (2025.naacl-long)
Copied to clipboard
| Challenge: | Defeasible inferences are highly plausible but can be impacted by new information. |
| Approach: | They construct a dataset to evaluate defeasible reasoning about property inheritance . they use generics to represent the inheritance rules because their semantics include exceptions . |
| Outcome: | The proposed model performs poorly across all pattern types and achieves 0.64 F 1 . the best performing model only achieves F 1 and the model is not well tuned . |
ManiTweet: A New Benchmark for Identifying Manipulation of News on Social Media (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the identification of social media posts that contain misrepresentations of information within associated news articles. |
| Approach: | They propose a data collection schema and curated a dataset called ManiTweet, consisting of 3.6K pairs of tweets and corresponding articles. |
| Outcome: | The proposed model outperforms large language models on the ManiTweet dataset and reveals intriguing connections between manipulation and the domain and factuality of news articles. |
Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for stance detection are topic-specific and cross-target stance. |
| Approach: | They propose a new dataset for zero-shot stance detection that captures a wider range of topics and lexical variation than in previous datasets. |
| Outcome: | The proposed model improves performance on a number of challenging linguistic phenomena. |
Characterizing and Evaluating Working Emotion Vocabularies in Multilingual Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work evaluating emotion and affective understanding in large language models rely on predetermined label sets or focus on a singular evaluation task. |
| Approach: | They examine the ability of multilingual language models to predict any term used by an author to label their own feelings or emotions. |
| Outcome: | The proposed models perform poorly on three different tasks in English and Spanish. |
Is the Top Still Spinning? Evaluating Subjectivity in Narrative Understanding (2025.emnlp-main)
Copied to clipboard
| Challenge: | In many domains, determining faithfulness of a claim to a source document is a binary judgment . but, whether a document is factual or whether it is entailed given some input is highly subjective. |
| Approach: | They propose a task to manage the subjectivity involved with factuality judgments of ambiguous claims. |
| Outcome: | The proposed method improves the annotator agreement on faithfulness of a claim by 21%. |
Learning to Revise References for Faithful Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that noisy reference summaries can be detrimental to model performance. |
| Approach: | They propose to selectively re-write unsupported reference sentences to better reflect source data. |
| Outcome: | The proposed method improves reference quality while retaining all data. |
Event-Centric Natural Language Processing (2021.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide an introduction to various methods for automating the extraction, conceptualization and prediction of events and their relations. |
| Approach: | This tutorial will provide an introduction to various methods for automating events and their relations, and a wide range of NLU and commonsense understanding tasks. |
| Outcome: | This tutorial will provide an introduction to various methods for automating extraction, conceptualization and prediction of events and their relations, and a wide range of NLU and commonsense understanding tasks. |
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization (2024.naacl-long)
Copied to clipboard
Liyan Tang, Igor Shalyminov, Amy Wong, Jon Burnsky, Jake Vincent, Yu’an Yang, Siffi Singh, Song Feng, Hwanjun Song, Hang Su, Lijia Sun, Yi Zhang, Saab Mansour, Kathleen McKeown
| Challenge: | Existing LLMs hallucinate significant amounts of factual errors in the dialogue domain, regardless of the model’s size. |
| Approach: | They propose to evaluate topic-focused dialogue summarization by using large language models (LLMs) they use human annotations to evaluate factual consistency and explain factually inconsistent sentences. |
| Outcome: | The proposed evaluation benchmark on topic-focused dialogue summarization shows that existing LLMs hallucinate significant amounts of factual errors regardless of the model’s size. |
Entity-level Factual Consistency of Abstractive Text Summarization (2021.eacl-main)
Copied to clipboard
Feng Nan, Ramesh Nallapati, Zhiguo Wang, Cicero Nogueira dos Santos, Henghui Zhu, Dejiao Zhang, Kathleen McKeown, Bing Xiang
| Challenge: | Existing models exhibit entity hallucination, generating names of entities that are not present in the source document. |
| Approach: | They propose to use entity-level factual consistency to improve model quality . they propose to filter the training data to reduce entity hallucination problem . |
| Outcome: | The proposed model can reduce the entity hallucination problem by filtering the training data. |
Reranking-based Generation for Unbiased Perspective Summarization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation frameworks rely on traditional metrics for measuring key attributes such as coverage and faithfulness without verifying their applicability. |
| Approach: | They propose to use human annotations to measure perspective summary quality and reranking-based methods yield strong results. |
| Outcome: | The proposed methods show that they perform well with synthetically generated and reranking-labeled data. |
Unsupervised Selective Rationalization with Noise Injection (2023.acl-long)
Copied to clipboard
| Challenge: | Unsupervised selective rationalization produces rationales alongside predictions, but does not ensure that the rationale contains a plausible explanation for the prediction. |
| Approach: | They propose a technique that injects noise between a rationale generator and a predictor to limit generation of implausible rationales. |
| Outcome: | The proposed method achieves significant improvements in plausibility and task accuracy over the state-of-the-art models while maintaining or improving model faithfulness. |
STORYSUMM: Evaluating Faithfulness in Story Summarization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating abstractive summarization are lacking in faithfulness evaluation. |
| Approach: | They propose a dataset that measures faithfulness of LLM summaries with localized errors and faithfulness labels for evaluation methods. |
| Outcome: | The proposed method does not achieve more than 70% accuracy on this task. |
Event-Driven News Stream Clustering using Entity-Aware Contextual Embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | a novel method for online news stream clustering is proposed . a user can scour the many news sources multiple times a day to find news articles . |
| Approach: | They propose a method for online news stream clustering that is a variant of the streaming K-means algorithm. |
| Outcome: | The proposed model achieves state-of-the-art on a standard stream clustering dataset of English documents. |
Penguins Don’t Fly: Reasoning about Generics through Instantiations and Exceptions (2023.eacl-main)
Copied to clipboard
| Challenge: | Generics express generalizations about the world that are not universally true . commonsense knowledge bases encode some generic knowledge but rarely enumerate exceptions . |
| Approach: | They propose a framework informed by linguistic theory to generate exemplars for generics . they generate 19k exemplar cases for 650 generics and show they outperform a strong baseline . |
| Outcome: | The proposed framework outperforms a baseline framework by 12.8 precision points. |
A Unified Feature Representation for Lexical Connotations (2021.eacl-main)
Copied to clipboard
| Challenge: | ideological attitudes and stance are often expressed through subtle meanings of words and phrases. |
| Approach: | They propose a method for lexical representations that capture connotations within the embedding space . they define six new fine-grained connotation aspects for nouns and adjectives . |
| Outcome: | The proposed method improves stance detection when data is limited. |
Margin-aware Unsupervised Domain Adaptation for Cross-lingual Text Labeling (2020.findings-emnlp)
Copied to clipboard
Dejiao Zhang, Ramesh Nallapati, Henghui Zhu, Feng Nan, Cicero Nogueira dos Santos, Kathleen McKeown, Bing Xiang
| Challenge: | Existing approaches to learn a model from labeled data are expensive or prohibitive. |
| Approach: | They propose an unsupervised domain adaptation algorithm that leverages labeled data in a source domain to learn a well-performing model in . they use the Margin Disparity Discrepancy algorithm to optimize the margin loss on the source domain. |
| Outcome: | The proposed approach improves on a recent theoretical work on cross-lingual document classification and NER by a large margin. |
Fair Abstractive Summarization of Diverse Perspectives (2024.naacl-long)
Copied to clipboard
Yusen Zhang, Nan Zhang, Yixin Liu, Alexander Fabbri, Junru Liu, Ryo Kamoi, Xiaoxin Lu, Caiming Xiong, Jieyu Zhao, Dragomir Radev, Kathleen McKeown, Rui Zhang
| Challenge: | Existing work on summarization metrics and large language models has not explored fair abstractive summarizing. |
| Approach: | They propose four reference-free automatic metrics to measure the differences between target and source perspectives. |
| Outcome: | The proposed methods alleviate fair abstractive summarization on user-generated data. |
What Do Users Care About? Detecting Actionable Insights from User Feedback (2022.naacl-industry)
Copied to clipboard
| Challenge: | a large amount of data can be used to extract actionable insights from user feedback . however, the data is unstructured and voluminous, and is underutilized for most users . |
| Approach: | They propose an unsupervised method for finding actionable insights from user feedback . they cluster data into groups containing coherent insights, followed by theme detection . |
| Outcome: | The proposed approach outperforms baselines on two real-world user feedback datasets and one academic dataset. |
Improving Long Dialogue Summarization with Semantic Graph Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing algorithms for abstractive summarization of short dialogues are challenging . however, they can generate high-quality summaries for long dialogues . |
| Approach: | They propose an algorithm that processes complete dialogues into topic-segment-level Abstract Meaning Representation graphs . they propose a pretrained LLM that exploits the text to leverage graph semantics a new text-graph attention . |
| Outcome: | The proposed algorithm outperforms state-of-the-art models on multiple long dialogue summarization datasets . it also generates additional training signals that facilitate graph feature encoding and content selection. |
Segmenting Subtitles for Correcting ASR Segmentation Errors (2021.eacl-main)
Copied to clipboard
David Wan, Chris Kedzie, Faisal Ladhak, Elsbeth Turcan, Petra Galuscakova, Elena Zotkina, Zhengping Jiang, Peter Bell, Kathleen McKeown
| Challenge: | Typical ASR systems segment input audio into utterances using purely acoustic information, which may not resemble sentence-like units expected by conventional machine translation systems for spoken language translation (SLT). |
| Approach: | They propose a model for correcting ASR acoustic segmentation using subtitles as a proxy dataset for creating synthetic aural utterances by modeling common error modes. |
| Outcome: | The proposed model improves performance on MT and audio-document cross-language information retrieval (CLIR) it uses subtitles as a proxy dataset to correct ASR acoustic segmentation . |
Mitigating Covertly Unsafe Text within Natural Language Systems (2022.findings-emnlp)
Copied to clipboard
Alex Mei, Anisha Kabir, Sharon Levy, Melanie Subbiah, Emily Allaway, John Judge, Desmond Patton, Bruce Bimber, Kathleen McKeown, William Yang Wang
| Challenge: | Existing studies on text safety have focused on overtly unsafe, covertly, or indirectly unsafe statements. |
| Approach: | They propose a method to identify physical harm-causing statements as overtly, covertly or indirectly unsafe and a solution to mitigate the generation of such statements. |
| Outcome: | The proposed methods identify the type of unsafe language that can cause physical harm and identify mitigation strategies to inspire future researchers to tackle this challenging problem. |
InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection (2021.acl-long)
Copied to clipboard
Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, Avi Sil
| Challenge: | a novel approach to detect fake news is needed due to training data scarcity . current methods focus on document-level fake news detection using lexical features and semantic embeddings . |
| Approach: | They propose a novel benchmark for fake news detection at the knowledge element level . they propose synthesis method which manipulates knowledge elements to generate noisy training data . |
| Outcome: | The proposed method outperforms the state-of-the-art in detecting misinformation . it yields fine-grained explanations and outperformed the current methods . |
Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning (2023.acl-long)
Copied to clipboard
Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Yang Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang
| Challenge: | Existing methods for data-to-text generation focus on specific types of structured data. |
| Approach: | They propose a method that provides a unified representation that can handle various forms of structured data such as tables, knowledge graph triples, and meaning representations. |
| Outcome: | The proposed method improves zero-shot and few-shot scenarios and can adapt to new structured data. |
Timeline Summarization based on Event Graph Compression via Time-Aware Optimal Transport (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for timeline summarization ignore the events’ intra-structures and inter-structure connections. |
| Approach: | They propose to represent news articles as an event-graph, thus compressing the whole graph to its salient sub-graph. |
| Outcome: | The proposed method significantly improves on the state-of-the-art on three real-world datasets, including two public benchmarks and a Timeline100 dataset. |
Faithfulness-Aware Decoding Strategies for Abstractive Summarization (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on faithfulness of abstractive summarization have focused on decoding strategies. |
| Approach: | They propose two faithfulness-aware generation methods to further improve faithfulness . they propose to use a distillation approach to generate faithful summaries with greedy decoding . |
| Outcome: | The proposed methods improve faithfulness across two datasets as evaluated by automatic faithfulness metrics and human evaluation. |
MASIVE: Open-Ended Affective State Identification in English and Spanish (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that fail to understand cultural and language influences the meaning of emotional terms like "love" a new study shows that smaller finetuned models outperform much larger LLMs on region-specific span prediction tasks. |
| Approach: | They propose to use a reddit reddits dataset to identify a set of affective states . they find that smaller finetuned multilingual models outperform larger LLMs . |
| Outcome: | The proposed model outperforms larger models on span prediction task even on region-specific Spanish affective states. |
Generating EDU Extracts for Plan-Guided Summary Re-Ranking (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to generate summary candidates for re-ranking produce redundant, and often low quality, content. |
| Approach: | They propose a method to generate candidates for re-ranking that addresses these issues by grounding each abstract on its own unique content plan and creating distinct plan-guided abstracts using a model's top beam. |
| Outcome: | The proposed method outperforms baseline decoding methods on CNN, NYT, and Xsum and shows that prompting GPT-3 to follow EDU plans outperformed sampling-based methods by 1.05 points. |
Read Top News First: A Document Reordering Approach for Multi-Document News Summarization (2022.findings-acl)
Copied to clipboard
Chao Zhao, Tenghao Huang, Somnath Basu Roy Chowdhury, Muthu Kumar Chandrasekaran, Kathleen McKeown, Snigdha Chaturvedi
| Challenge: | Existing methods for extracting multi-document news summarization neglect relative importance of documents. |
| Approach: | They propose to concatenate all documents into a single meta-document and then summarize it using an SDS model. |
| Outcome: | The proposed approach outperforms state-of-the-art methods with more complex architectures. |
Exploring Content Selection in Summarization of Novel Chapters (2020.acl-main)
Copied to clipboard
| Challenge: | We focus on extractive summarization, which requires the creation of a gold-standard set of extractive summary summaries. |
| Approach: | They propose a new metric for aligning summary sentences with chapter sentences to create gold extracts. |
| Outcome: | The proposed method improves on previous methods and automatic metrics and a crowd-sourced pyramid analysis. |
ModelingAgent: Bridging LLMs and Mathematical Modeling for Real-World Challenges (2025.findings-emnlp)
Copied to clipboard
Cheng Qian, Hongyi Du, Hongru Wang, Xiusi Chen, Yuji Zhang, Avirup Sil, ChengXiang Zhai, Kathleen McKeown, Heng Ji
| Challenge: | Existing benchmarks for large language models fail to reflect real-world complexity . existing benchmarks often fail to capture real-life problems . |
| Approach: | They propose a benchmark that features real-world-inspired, open-ended problems from competitions . they propose 'ModelingBench' that supports multiple valid solutions . |
| Outcome: | The proposed framework outperforms baselines and produces well-grounded, creative solutions. |
Supporting Clustering with Contrastive Learning (2021.naacl-main)
Copied to clipboard
Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O. Arnold, Bing Xiang
| Challenge: | Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space, but different categories overlap with each other at the beginning of the learning process. |
| Approach: | They propose a framework to leverage contrastive learning to promote better separation between different categories by optimizing a clustering objective defined in the representation space. |
| Outcome: | The proposed framework improves state-of-the-art accuracy and normalized mutual information on short text clustering and combines top-down and bottom-up instance discrimination to achieve better distances. |
Data Caricatures: On the Representation of African American Language in Pretraining Corpora (2025.acl-long)
Copied to clipboard
Nicholas Deas, Blake Vente, Amith Ananthram, Jessica A Grieser, Desmond U. Patton, Shana Kleiner, James R. Shepard Iii, Kathleen McKeown
| Challenge: | Recent work in linguistics and NLP has investigated the quantity and quality of AAL representation in pretraining corpora. |
| Approach: | They examine the quantity and quality of African American Language (AAL) representation in pretraining corpora. |
| Outcome: | The results show that AAL is underrepresented in all evaluated corpora compared to US demographics . they also show that most automated filters are more likely to conserve white Mainstream English (WME) texts over AAL . |
Social Orientation: A New Feature for Dialogue Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on social orientations in dialogues show they improve performance in low-resource settings. |
| Approach: | They propose to use social orientation tags to model dialogue outcomes . they introduce a new set of dialogue utterances machine-labeled with social orientation tag. |
| Outcome: | The proposed model improves on English and Chinese language benchmarks and shows that social orientation tags explain the outcomes of social interactions when used in neural models. |
Content Selection in Deep Learning Models of Summarization (D18-1)
Copied to clipboard
| Challenge: | Using deep learning models, we find that word embedding does not improve performance over simpler models. |
| Approach: | They propose to use sentence embedding to perform content selection across multiple domains . they propose to propose two alternative models that use auto-regressive sentence extraction . |
| Outcome: | The proposed models improve performance across news, personal stories, meetings, and medical articles. |
Emotion-Infused Models for Explainable Psychological Stress Detection (2021.naacl-main)
Copied to clipboard
| Challenge: | a new study examines the use of emotion detection for detecting psychological stress in online posts . traditional multi-task learning and emotion-based language model fine-tuning are used to improve the model . |
| Approach: | They propose to use a semantically related task, emotion detection, for detecting psychological stress in online posts . they propose multi-task learning and emotion-based language model fine-tuning to improve the model . |
| Outcome: | The proposed model is more explainable and human-like than a black-box model . the proposed model mirrors psychological components of stress, the authors show . |
Latent Space Interpretation for Stylistic Analysis and Explainable Authorship Attribution (2025.coling-main)
Copied to clipboard
| Challenge: | Recent authorship attribution methods learn authorship representations of text in a latent, uninterpretable space, which hinders their usability in real-world applications. |
| Approach: | They propose a method for interpreting latent authorship representations by identifying representative points in the latent space and leveraging large language models to generate informative natural language descriptions of the writing style associated with each point. |
| Outcome: | The proposed method outperforms baseline methods on the authorship attribution task by +20% on average when aided with explanations from the method. |
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021.acl-long)
Copied to clipboard
Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Kathleen McKeown, Ramesh Nallapati, Dejiao Zhang, Zhiguo Wang, Andrew O. Arnold, Bing Xiang
| Challenge: | Recent studies show that about 30% of summaries generated by neural text summarization suffer from fact fabrication. |
| Approach: | They propose an automatic evaluation metric to measure factual consistency and a learning algorithm that maximizes the metric during model training. |
| Outcome: | The proposed method improves factual consistency and overall quality of summarization models. |
Learning Interpretable Style Embeddings via Prompting LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has treated the style of a text as separable from the content. |
| Approach: | They use prompting to perform stylometry on a large number of texts to generate a synthetic stylometric dataset. |
| Outcome: | The proposed model trains human-interpretable representations on a large stylometric dataset and a linguistic model for style representation learning. |
SWING: Balancing Coverage and Faithfulness for Dialogue Summarization (2023.findings-eacl)
Copied to clipboard
Kung-Hsiang Huang, Siffi Singh, Xiaofei Ma, Wei Xiao, Feng Nan, Nicholas Dingwall, William Yang Wang, Kathleen McKeown
| Challenge: | Existing approaches to dialogue summarization rely on features of conversation data. |
| Approach: | They propose to use natural language inference models to improve coverage and faithfulness . they use fine-grained training signals to encourage model to generate missing content . |
| Outcome: | The proposed model achieves higher faithfulness and coverage while maintaining conciseness compared to prior methods. |
WikiLingua: A New Benchmark Dataset for Cross-Lingual Abstractive Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of high quality multilingual data for cross-lingual summarization is a costly endeavor since it requires humans to read, comprehend, condense, and paraphrase entire articles. |
| Approach: | They propose to use a large-scale, multilingual dataset to evaluate cross-lingual abstractive summarization systems. |
| Outcome: | The proposed method significantly outperforms baseline approaches while being more cost efficient during inference. |
Constrained Regeneration for Cross-Lingual Query-Focused Extractive Summarization (2022.coling-1)
Copied to clipboard
Elsbeth Turcan, David Wan, Faisal Ladhak, Petra Galuscakova, Sukanta Sen, Svetlana Tchistiakova, Weijia Xu, Marine Carpuat, Kenneth Heafield, Douglas Oard, Kathleen McKeown
| Challenge: | Query-focused summarization of foreign-language documents can help a user understand whether a document is relevant to a query term. |
| Approach: | They propose to use machine translation and post-editing to improve human relevance judgments . they include a query term in a summary when its translation appears in the source document . |
| Outcome: | The proposed approach improves human relevance judgments by including a query term in a summary when its translation appears in the source document. |
Detecting Urgency Status of Crisis Tweets: A Transfer Learning Approach for Low Resource Languages (2020.coling-main)
Copied to clipboard
| Challenge: | We train monolingual and cross-lingual classifiers on the extracted features of tweets . we use a few state-of-the-art contextual embeddings to extract features of the tweets. |
| Approach: | They propose to use tweets to train a dataset of English and two low-resource languages to train zero-shot transfer models. |
| Outcome: | The proposed model performs well in English and in low-resource languages . the proposed model is based on state-of-the-art embeddings and semi-supervised methods . |
Evaluation of African American Language Bias in Natural Language Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that large language generation models disadvantaging African American Language (AAL) can be biased for certain language varieties, but there is little research on the impact of these biases on other languages. |
| Approach: | They evaluate how well LLMs understand African American Language (AAL) in comparison to white Mainstream English (WME) using a dataset of AAL texts from a variety of regions and contexts, they find dialectal bias in six pre-trained LLM. |
| Outcome: | The proposed models understand African American language in comparison to white mainstream English (WME) the proposed models have performance gaps on two tasks that are not matched by the model. |
Controllable Meaning Representation to Text Generation: Linearization and Data Augmentation Strategies (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using task-oriented dialogue generation benchmarks, we compare the effect of four input linearization strategies on controllability and faithfulness. |
| Approach: | They compare the effect of four input linearization strategies on controllability and faithfulness . they also evaluate how a phrase-based data augmentation method can improve performance . |
| Outcome: | The proposed model can generate utterances whose phrases follow the order of the provided plan. |
The Law of Knowledge Overshadowing: Towards Understanding, Predicting and Preventing LLM Hallucination (2025.findings-acl)
Copied to clipboard
Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R. Fung, Kathleen McKeown, ChengXiang Zhai, Manling Li, Heng Ji
| Challenge: | Hallucination is a persistent challenge in large language models where even with rigorous quality control, models often generate distorted facts. |
| Approach: | They propose a new framework to quantify factual hallucinations by modeling knowledge overshadowing. |
| Outcome: | The proposed framework improves model factuality on Overshadow (27.9%), MemoTrap (13.1%) and NQ-Swap (18.3%). |
Parallel Structures in Pre-training Data Yield In-Context Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs) are capable of in-context learning (ICL) however, it is unclear where this ability comes from as there is a stark distribution shift between pre-training text and ICL prompts. |
| Approach: | They find that pre-trained language models are capable of in-context learning (ICL) they detect parallel structures in the pre-training data and conduct ablation experiments to study their effect on ICL. |
| Outcome: | The proposed model can adapt to a task with a few examples given in the prompt without any parameter update. |
Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions (2025.emnlp-main)
Copied to clipboard
| Challenge: | We introduce and study artificial impressions–patterns in LLMs’ internal representations of prompts that resemble human impressions and stereotypes based on language. |
| Approach: | They introduce and study artificial impressions–patterns in LLMs’ internal representations of prompts that resemble human impressions and stereotypes based on language. |
| Outcome: | The proposed models predict impressions and model behavior based on the two-dimensional Stereotype Content Model (SCM). |
Adversarial Learning for Zero-Shot Stance Detection on Social Media (2021.naacl-main)
Copied to clipboard
| Challenge: | a new model for zero-shot stance detection on Twitter uses adversarial learning to generalize across topics . previous work on zero- shot stance detector on English social media focuses on cross-target stances . |
| Approach: | They propose a model that uses adversarial learning to generalize across topics on Twitter . their model achieves state-of-the-art performance on unseen test topics . |
| Outcome: | The proposed model achieves state-of-the-art performance on unseen topics with minimal computational costs. |
StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples (2025.naacl-long)
Copied to clipboard
Ajay Patel, Jiacheng Zhu, Justin Qiu, Zachary Horvitz, Marianna Apidianaki, Kathleen McKeown, Chris Callison-Burch
| Challenge: | Existing methods for embedding text are limited by the imperfect nature of data acquired under such assumptions. |
| Approach: | They propose a new approach to training stronger content-independent style embeddings using a synthetic dataset of near-exact paraphrases with controlled style variations. |
| Outcome: | The proposed model outperforms existing methods in real-world benchmarks and outperformed leading style representations in downstream applications. |
On the Relation between Sensitivity and Accuracy in In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) suffers from oversensitivity to the prompt, making it unreliable in real-world scenarios. |
| Approach: | They propose a few-shot selective prediction method that abstains from sensitive predictions. |
| Outcome: | The proposed method outperforms confidence-based and entropy-based methods on ten classification datasets. |
Faking Fake News for Real Fake News Detection: Propaganda-Loaded Training Data Generation (2023.acl-long)
Copied to clipboard
| Challenge: | despite advances in detecting fake news, there is a sizable gap between machine-generated and human-authored fake news . a nave solution is to collect human-written news articles that contain inaccurate information by crawling untrustworthy news media. |
| Approach: | They propose a framework for generating training examples informed by the styles and strategies of human-authored propaganda. |
| Outcome: | The proposed framework improves detection of human-written disinformation by 3.62–7.69% on two public datasets. |
TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for text style transfer rely on few-shot capabilities of large language models or complex controllable text generation approaches that are inefficient and underperform on fluency metrics. |
| Approach: | They propose a lightweight but effective approach which leverages a small language model and pre-trained authorship embeddings to perform efficient, few-shot text style transfer. |
| Outcome: | The proposed method outperforms strong approaches such as GPT-4 and performs form attribute style transfer with automatic and human evaluations. |
Using Structured Content Plans for Fine-grained Syntactic Control in Pretrained Language Model Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Large pretrained language models can generate powerful text but cannot be controlled at a sub-sentential level. |
| Approach: | They propose to make such fine-grained control possible in pretrained LMs by generating text directly from a semantic representation, Abstract Meaning Representation (BART), which is augmented at the node level with syntactic control tags. |
| Outcome: | The proposed method can generate text from a semantic representation, which is augmented at the node level with syntactic control tags. |
SafeText: A Benchmark for Exploring Physical Safety in Language Models (2022.emnlp-main)
Copied to clipboard
Sharon Levy, Emily Allaway, Melanie Subbiah, Lydia Chilton, Desmond Patton, Kathleen McKeown, William Yang Wang
| Challenge: | Existing models that generate unsafe text are susceptible to the dangers of unsafe text generation and are deemed unsafe. |
| Approach: | They use a dataset to empirically study commonsense physical safety across various models for text generation and reasoning tasks. |
| Outcome: | The proposed model can generate unsafe text and reject it, but the different harms that can occur do not receive equal attention, which may consequently downplay certain harms. |