Papers by Fred Morstatter
Contextualizing Argument Quality Assessment with Relevant Knowledge (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing methods for assessing argument quality in isolation analyze their quality in the absence of context, which affects their accuracy and generalizability. |
| Approach: | They propose a method for scoring argument quality based on contextualization via relevant knowledge that leverages large language models to provide feedback, infer hidden assumptions, supply a similar-quality argument, or give a counter-argument. |
| Outcome: | The proposed method outperforms existing methods across multiple metrics in both in-domain and zero-shot setups. |
Modeling Cross-Cultural Pragmatic Inference with Codenames Duet (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on pragmatic reasoning tests using simple word reference games with unidentified speakers and listeners, but speakers' sociocultural background shapes their pragmatic assumptions. |
| Approach: | They propose a dataset which operationalizes sociocultural pragmatic inference in a word reference game. |
| Outcome: | The proposed model improves clue-giving and guessing tasks by accounting for background characteristics and the game context. |
Pipeline for modeling causal beliefs from natural language (2023.acl-demo)
Copied to clipboard
| Challenge: | Existing methods to analyze language data for psychological causality are difficult to advance as they do not isolate cognitive mechanisms. |
| Approach: | They propose a pipeline that leverages a Large Language Model to identify causal claims made in natural language documents and applies a clustering algorithm to group causal claims based on their semantic topics. |
| Outcome: | The proposed pipeline analyzes the Covid-19 vaccine in tweets and generates a causal claim network. |
Capturing Perspectives of Crowdsourced Annotators in Subjective Learning Tasks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to label aggregation fail to capture subjective annotations and can lead to biases. |
| Approach: | They propose annotator-aware representations for text for subjective classification tasks that involve learning representations of annotators. |
| Outcome: | The proposed model improves on metrics that assess the performance on capturing individual annotators’ perspectives. |
Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Temporal knowledge graphs (TKGs) are used to represent real-world facts in a structured way. |
| Approach: | They propose to use in-context learning with large language models for TKG forecasting . they compare naive LLMs to state-of-the-art (SOTA) supervised models . |
| Outcome: | The proposed approach performs well against pre-trained large language models . the proposed approach is based on simple heuristics and state-of-the-art models compared with pre-trainers . |
Robust Conversational Agents against Imperceptible Toxicity Triggers (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work to generate adversarial attacks is costly and not scalable . despite the abundance of research in this area, little attention has been given to adversarials . |
| Approach: | They propose an adversarial attack mechanism that mitigates toxic language generation . they propose a defense mechanism that is scalable and can be generalized . |
| Outcome: | The proposed defense is effective at avoiding toxic language generation even against imperceptible toxicity triggers while preserving conversational flow. |
Lawyers are Dishonest? Quantifying Representational Harms in Commonsense Knowledge Resources (2021.emnlp-main)
Copied to clipboard
| Challenge: | Commonsense knowledge bases are mostly human-generated and reflect societal biases . a filtering-based approach can reduce the issues in both resources and models but leads to a performance drop . |
| Approach: | They propose a filtering-based approach to mitigating representational harms in ConceptNet and GenericsKB . they propose filtered-based approaches can reduce issues in both resources and models but leads to performance drop . |
| Outcome: | The proposed approach reduces issues in resources and models but leads to performance drop . the paper proposes a filtering-based approach that reduces biases but leaves room for future work . |
AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction (2023.eacl-main)
Copied to clipboard
Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
| Challenge: | Named entity recognition models have shown impressive results in overcoming label scarcity and generalizing to unseen entities by leveraging distant supervision and auxiliary information such as explanations. |
| Approach: | They propose a framework that automatically generates and leverages “entity triggers” which are human-readable cues in the text that help guide the model to make better decisions. |
| Outcome: | The proposed framework outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on three well-studied datasets. |
“Define Your Terms” : Enhancing Efficient Offensive Speech Classification with Definition (2024.eacl-long)
Copied to clipboard
| Challenge: | Multiple studies have proposed various semantically related yet subtle distinct categories of offensive speech. |
| Approach: | They propose a meta-learning architecture that incorporates the input’s label and definition for classification via Prototypical Network. |
| Outcome: | The proposed model achieves 75% of the maximal F1-score while using less than 10% of the available training data across 4 datasets. |
The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used to label data across many domains and for myriad tasks. |
| Approach: | They ask large language models to label data using a series of decisions by practitioners . they find that even the smallest perturbations can change the LLM's answer . |
| Outcome: | The proposed model can be used to quickly get a response for arbitrary tasks. |
ForecastQA: A Question Answering Challenge for Event Forecasting with Temporal Text Data (2021.acl-long)
Copied to clipboard
| Challenge: | Existing automated forecasting studies rely on structured data to predict future events. |
| Approach: | They propose a question-answering task that limits access to unstructured text data . they use a crowdsourced dataset to form a restricted-domain, multiple-choice, question-announcement task . |
| Outcome: | The proposed model achieves 61.0% accuracy on the dataset, which still lags behind human performance by about 19%. |