Papers by Martin Potthast
Target Inference in Argument Conclusion Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches focus on generating single claims, but there are limitations. |
| Approach: | They propose to use a triplet neural network to infer a conclusion's target from premises' targets and a neural network for a new target. |
| Outcome: | The proposed approach outperforms baselines on two domains. |
The Two Paradigms of LLM Detection: Authorship Attribution vs Authorship Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting texts generated by large language models are disputed . authors argue that there are limitations in the current technology . |
| Approach: | They propose to make LLM detectors robust against domain shifts and build benchmarks . they argue that the limitations lie elsewhere, and open the realm of authorship analysis technology . |
| Outcome: | The proposed method systematically analyzes the benchmarks and validates it using state-of-the-art detectors. |
Heuristic Authorship Obfuscation (P19-1)
Copied to clipboard
| Challenge: | Existing methods for authorship verification are insufficient to control the authorial style of a text. |
| Approach: | They propose a novel method that models writing style difference as the Jensen-Shannon distance between character n-gram distributions of texts and manipulates an author’s subconsciously encoded writing style using heuristic search. |
| Outcome: | The proposed approach defeats state-of-the-art verification approaches while keeping text changes at a minimum. |
Counter-Argument Generation by Attacking Weak Premises (2021.findings-acl)
Copied to clipboard
| Challenge: | a recent work explores the generation of counter-arguments by undermining one of its premises . identifying the argument's weak premises is key to effective countering, we hypothesize . |
| Approach: | They propose a pipeline approach that first assesses the argument's weak premises and generates a counter-argument undermining the weakest among them. |
| Outcome: | The proposed approach undermins arguments by attacking weak premises . human annotators favor the proposed approach over state-of-the-art approaches . |
Bias Analysis and Mitigation in the Evaluation of Authorship Verification (P19-1)
Copied to clipboard
| Challenge: | a paper on authorship verification shows that the underlying experiment design cannot guarantee pushing forward the state of the art. |
| Approach: | They propose a "Basic and Fairly Flawed" authorship verifier that is on a par with the best approaches submitted so far . they pinpoint sources of bias that should be eliminated and propose 'refined' authorship corpus as effective countermeasure. |
| Outcome: | The proposed approach is on par with the best approaches submitted so far . the proposed approach shows that sources of bias should be eliminated . |
Paraphrase Acquisition from Image Captions (2023.eacl-main)
Copied to clipboard
| Challenge: | Using image captions, we hypothesize that different captions for the same image naturally form a set of mutual paraphrases. |
| Approach: | They propose to use image captions as a previously underutilized resource for paraphrases . they analyze captions in the English Wikipedia to find common paraphrase similarities . |
| Outcome: | The proposed dataset compares known paraphrase corpora with their syntactic and semantic similarity to the existing dataset. |
Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers (2022.findings-acl)
Copied to clipboard
| Challenge: | a recent study has investigated how transformer-based language models can be combined with active learning. |
| Approach: | They propose to combine transformer-based language models with active learning to reduce labeling costs . transformers are expensive, but they can be fine-tuned using a query strategy . they compare transformers to experiments from previous research to evaluate their performance . |
| Outcome: | The proposed model outperforms the well-known prediction entropy query strategy on five widely used text classification benchmarks. |
Generating Informative Conclusions for Argumentative Texts (2021.findings-acl)
Copied to clipboard
| Challenge: | Argumentative texts often omit explicit conclusions, expecting readers to infer them rather . a corpus of 136,996 arguments is compiled and used to generate informative conclusions . |
| Approach: | They propose to generate informative conclusions from a large-scale corpus of argumentative texts . they propose to use argumentative knowledge to augment the corpus and refine the model . |
| Outcome: | The proposed corpus of argumentative texts and their conclusions is compiled and analyzed . the results show that the proposed model is informative and concise . |
Small-Text: Active Learning for Text Classification in Python (2023.eacl-demo)
Copied to clipboard
| Challenge: | small-text is an easy-to-use active learning library for text classification . it features a variety of pre-implemented state-of-the-art query strategies and stopping criteria . |
| Approach: | They introduce small-text, an easy-to-use active learning library for Python . it offers pool-based active learning for single- and multi-label text classification . they find it matches vanilla transformer fine-tuning in terms of classification accuracy . |
| Outcome: | The proposed library outperforms vanilla transformer fine-tuning in classification accuracy and area under the curve. |
Casting the Same Sentiment Classification Problem (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Identifying the stance of an argument towards a topic is a fundamental problem in computational argumentation. |
| Approach: | They propose a task where text users are asked to determine if they have the same sentiment . they aim to enable a more topic-agnostic sentiment classification by using Yelp data . |
| Outcome: | The proposed task achieves an accuracy above 83% for category subsets across topics and 89% on average. |
Mining Health-related Cause-Effect Statements with High Precision at Large Scale (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for assessing the health relatedness of phrases and sentences are slower and less effective than state-of-the-art medical entity linkers. |
| Approach: | They propose a termhood score that achieves 69% recall at over 90% precision on a web dataset with cause-effect statements. |
| Outcome: | The proposed method achieves 69% recall at over 90% precision on a web dataset with cause-effect statements. |
Beyond Metadata: What Paper Authors Say About Corpora They Use (2021.findings-acl)
Copied to clipboard
| Challenge: | Currently, dataset retrieval relies almost exclusively on metadata provided by the publishers. |
| Approach: | They propose to use metadata to extract review statements from scientific publications . they argue that a crucial piece of information is missing to inform the examination of search results . |
| Outcome: | The proposed analysis is the first of its kind in the field of Natural Language Processing. |
Trigger Warning Assignment as a Multi-Label Document Classification Problem (2023.acl-long)
Copied to clipboard
| Challenge: | a trigger warning is used to warn people about potentially disturbing content . a webis dataset of 1 million fanfiction works contains up to 36 different warnings per document . |
| Approach: | They introduce a multi-label task to assign a trigger warning to fanfiction . they map 41 million free-form tags assigned by authors into a single taxonomy of trigger warnings . |
| Outcome: | The proposed model achieves micro-F1 scores of about 0.5, which reveals the difficulty of the task. |
On Classifying whether Two Texts are on the Same Side of an Argument (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to same side stance classification (S3C) require domain knowledge and semantic inference to solve the task. |
| Approach: | They propose to use same side stance classification to predict whether two arguments argue for the same stance for a given pair of arguments. |
| Outcome: | The proposed model fails to generalize both within and across topics and domains when adjusting the sampling strategy to a more adversarial scenario. |
Summary Explorer: Visualizing the State of the Art in Text Summarization (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Automatic text summarization is the task of generating a summary of a long text by condensing it to its most important parts. |
| Approach: | They propose a tool to visually explore document summarization systems based on three well-known summary quality criteria . |
| Outcome: | The proposed tool compiles outputs of 55 state-of-the-art document summarization approaches and visually explores them during a qualitative assessment. |
Indicative Summarization of Long Discussions (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using large language models, we generate indicative summaries instead of informative summary for long discussions. |
| Approach: | They propose an unsupervised approach to generating indicative summaries using large language models using large-scale language models. |
| Outcome: | The proposed method clusters argument sentences, generates abstractive summaries, and classifies the generated cluster labels into argumentation frames. |
Differential Bias: On the Perceptibility of Stance Imbalance in Argumentation (2022.findings-aacl)
Copied to clipboard
| Challenge: | a theoretical model of bias classification is not feasible because of complexity of interpreting language phenomena. |
| Approach: | They propose to analyze whether a text is biased based on an algorithmic analysis . they propose to use a model to determine whether x is more biased than y . |
| Outcome: | a crowdsourcing study shows that differences in stance bias are perceptible when (light) support is provided through training or visual aids. |
Citance-Contextualized Summarization of Scientific Papers (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Current automatic summarization approaches generate abstracts, but abstracts do not show relationship between paper and references. |
| Approach: | They propose a contextualized summarization approach that generates an informative summary . they extract and model the citances of a paper, retrieve relevant passages from cited papers, and generate abstractive summaries tailored to each citance. |
| Outcome: | The proposed method extracts and models the citances of a paper, retrieves relevant passages from cited papers, and generates abstractive summaries tailored to each citance. |
Clickbait Spoiling via Question Answering and Passage Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Clickbait is a term used to describe posts intended to entice readers to visit a web page . clickbait spoiling is generating a short text that satisfies the curiosity induced by a clickbaiting post . |
| Approach: | They propose to use clickbait spoiling to generate a short text that satisfies curiosity . they classify the type of spoiler needed and generate appropriate spoilers . |
| Outcome: | The proposed method outperforms all other methods in generating spoilers for both types of clickbait posts. |
Revisiting Query Variation Robustness of Transformer Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Despite their proficiency with natural language, transformer-based large language models are not robust to query variations such as typos and paraphrases. |
| Approach: | They extend their findings to include more recent large language models . they find that instruct-LLMs are more robust to query variations . |
| Outcome: | The proposed model can be prompted for robustness by a set of instruction-tuned LLMs. |
A Stylometric Inquiry into Hyperpartisan and Fake News (P18-1)
Copied to clipboard
| Challenge: | a style analysis of hyperpartisan news and fake news can distinguish them from mainstream news . left-wing and right-wing news share significantly more stylistic similarities than mainstream news does . |
| Approach: | a comparative style analysis of hyperpartisan news and fake news is carried out . authors show that left-wing and right-wing news share significantly more stylistic similarities . |
| Outcome: | a style analysis can distinguish hyperpartisan news from mainstream, satire from both . left-wing and right-wing news share significantly more stylistic similarities than mainstream . |
SUMMARY WORKBENCH: Unifying Application and Evaluation of Text Summarization Models (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Summary Workbench is a tool for developing and evaluating text summarization models. |
| Approach: | They propose a tool for developing and evaluating text summarization models that integrates with Docker plugins and provides visual analysis of models’ strengths and weaknesses. |
| Outcome: | The proposed model and evaluation measures can be easily integrated as Docker-based plugins and provide insights into the models’ strengths and weaknesses. |
Celebrity Profiling (P19-1)
Copied to clipboard
| Challenge: | Using a corpus of 71,706 verified accounts, we construct a profile of a wide cross-section of local and global celebrities. |
| Approach: | They propose to use Twitter feeds of 71,706 verified accounts to build a corpus of celebrity profiles using Wikidata crawling. |
| Outcome: | The proposed corpus contains an average of 29,968 words per profile and up to 239 pieces of personal information. |
Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face (2023.emnlp-demo)
Copied to clipboard
Christopher Akiki, Odunayo Ogundepo, Aleksandra Piktus, Xinyu Zhang, Akintunde Oladipo, Jimmy Lin, Martin Potthast
| Challenge: | a toolkit for reproducible information retrieval research is available for free. |
| Approach: | They present a tool that integrates Pyserini and Hugging Face to enable the seamless construction and deployment of interactive search engines. |
| Outcome: | The proposed tool makes state-of-the-art retrieval models more accessible to non-IR practitioners while minimizing deployment effort. |
Topic Ontologies for Arguments (2023.findings-eacl)
Copied to clipboard
| Challenge: | Many computational argumentation tasks, such as stance classification, are topic-dependent. |
| Approach: | They map the argumentation landscape using the World Economic Forum, Wikipedia and Debatepedia as sources for argument topics. |
| Outcome: | The argument ontology is the first comprehensive assessment of argument topics in argument corpora. |
Crawling and Preprocessing Mailing Lists At Scale for Dialog Analysis (2020.acl-main)
Copied to clipboard
| Challenge: | a new neural segmentation model is used to segment 153 million emails . email is perhaps the most reliable and ubiquitous means of digital communication . |
| Approach: | They present a new neural segmentation model that crawls 153 million emails . it achieves 96% accuracy on 15 classes of email segments . |
| Outcome: | The proposed model achieves state-of-the-art performance while being more efficient to train than previous ones. |
Trigger Warnings: Bootstrapping a Violence Detector for Fan Fiction (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing guidelines for proactively alerting readers of potentially disturbing content have been proposed. |
| Approach: | They propose to use a labeled corpus of narrative fiction from a popular fan fiction site to determine whether to assign a trigger warning to an English story. |
| Outcome: | The proposed task achieves F1 scores between 0.8 and 0.9 on three datasets . the authors show that assigning trigger warnings for violence is feasible . |
Generalizing Unmasking for Short Texts (N19-1)
Copied to clipboard
| Challenge: | Authorship verification is the problem of inferring whether two texts were written by the same author. |
| Approach: | They propose a generalized unmasking approach which allows for authorship verification of short texts with high precision at an adjustable recall tradeoff. |
| Outcome: | The proposed approach achieves accuracies of 75–80% while allowing for easy adjustment to forensic scenarios that require higher levels of confidence. |
News Editorials: Towards Summarizing Long Argumentative Texts (2020.coling-main)
Copied to clipboard
| Challenge: | Using news summarization, we aim to target opinionated articles with a well-defined argumentation structure. |
| Approach: | They present a corpus of carefully curated summaries for 266 news editorials. |
| Outcome: | The summarization of opinionated articles with a well-defined argumentation structure is evaluated using a tailored annotation scheme. |
Modeling Appropriate Language in Argumentation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on offensive language has not been systematically addressed in debates . a new taxonomy of 14 dimensions determines inappropriate language in online discussions . |
| Approach: | They propose a taxonomy of 14 dimensions that determine inappropriate language in online discussions . they build on arguments quality corpora and annotate them on a corpus of 2191 arguments . |
| Outcome: | The proposed taxonomy covers the concept of appropriateness comprehensively, showing plausible correlations with argument quality dimensions. |
TL;DR Progress: Multi-faceted Literature Exploration in Text Summarization (2024.eacl-demo)
Copied to clipboard
| Challenge: | TL;DR Progress is a literature explorer designed specifically for the text summarization literature. |
| Approach: | They propose to organize 514 papers based on a comprehensive annotation scheme for text summarization approaches and a fine-grained, faceted search. |
| Outcome: | The proposed tool organizes 514papers based on a comprehensive annotation scheme for text summarization approaches and enables fine-grained, faceted search. |
Crowdsourcing a Large Corpus of Clickbait on Twitter (C18-1)
Copied to clipboard
Martin Potthast, Tim Gollub, Kristof Komlossy, Sebastian Schuster, Matti Wiegmann, Erika Patricia Garces Fernandez, Matthias Hagen, Benno Stein
| Challenge: | Clickbait is a nuisance on social media. |
| Approach: | a corpus of 38,517 annotated Twitter tweets was constructed to detect clickbait . the corpus was annotating tweets on 4-point scale by five annotators at Amazon's Mechanical Turk . |
| Outcome: | The corpus of 38,517 annotated Twitter tweets was used to evaluate 12 clickbait detectors submitted to the Clickbait Challenge 2017 . |
Efficient Pairwise Annotation of Argument Quality (2020.acl-main)
Copied to clipboard
| Challenge: | Especially crowdsourcing suffers from assessors having different reference frames to base their judgments on and task instructions being nondescript and therefore unhelpful in ensuring consistency. |
| Approach: | They propose an efficient annotation framework for argument quality that uses a stochastic transitivity model and an effective sampling strategy to infer high-quality labels. |
| Outcome: | The proposed model significantly outperforms existing annotation procedures and offers statistical insights into argument quality. |
CausalQA: A Benchmark for Causal Question Answering (2022.coling-1)
Copied to clipboard
Alexander Bondarenko, Magdalena Wolska, Stefan Heindorf, Lukas Blübaum, Axel-Cyrille Ngonga Ngomo, Benno Stein, Pavel Braslavski, Matthias Hagen, Martin Potthast
| Challenge: | Existing causal question answering datasets are relatively small and only include one type of causal question. |
| Approach: | They construct a benchmark corpus of 1.1 million causal questions with answers . they use a typology derived from a data-driven, manual analysis of QA datasets . |
| Outcome: | The proposed model achieves a ROUGE-L F1 score of 0.48 on the new QA benchmark. |
GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration (2023.acl-demo)
Copied to clipboard
Aleksandra Piktus, Odunayo Ogundepo, Christopher Akiki, Akintunde Oladipo, Xinyu Zhang, Hailey Schoelkopf, Stella Biderman, Martin Potthast, Jimmy Lin
| Challenge: | Using the mature and well-tested methods from the domain of Information Retrieval (IR) we propose to integrate Pyserini with Hugging Face to provide qualitative analysis tools for NLP researchers. |
| Approach: | They propose to integrate Pyserini with Hugging Face to provide qualitative analysis tools for NLP researchers. |
| Outcome: | The proposed tools can be integrated with the Hugging Face ecosystem of open-source AI libraries and artifacts. |
Task-Oriented Paraphrase Analytics (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on paraphrasing have applied different criteria to the task . authors have previously unmasked related tasks as paraphrases . |
| Approach: | They propose a taxonomy to organize 25 identified paraphrasing tasks . authors propose to use classifiers to identify tasks that a given paraphrased instance fits . |
| Outcome: | The proposed taxonomy identifies 25 paraphrasing tasks that fit the proposed task. |