Papers by Torsten Zesch
Text or Image? What is More Important in Cross-Domain Generalization Capabilities of Hate Meme Detection Models? (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies show that only the textual component of hateful memes enables the multimodal classifier to generalize across domains while the image component proves highly sensitive to a specific training dataset. |
| Approach: | They propose to use only the textual component of hateful memes to generalize across different domains while the image component is highly sensitive to a specific training dataset. |
| Outcome: | The proposed model performs similarly to hate-meme classifiers in a zero-shot setting, while the introduction of meme’s image captions worsens performance by an average F1 of 0.02. |
EVil-Probe - a Composite Benchmark for Extensive Visio-Linguistic Probing (2024.lrec-main)
Copied to clipboard
| Challenge: | Visual question answering, image-text retrieval and retrieving image patches that match an expression are some of the tasks visio-linguistic models show impressive performance on. |
| Approach: | They propose a composite benchmark that processes existing probing datasets into a unified format and reorganizes them based on the linguistic categories they probe. |
| Outcome: | The proposed benchmark is challenging for all models as they are sensitive to linguistic categories and only handles nouns. |
Rainbow - A Benchmark for Systematic Testing of How Sensitive Visio-Linguistic Models are to Color Naming (2024.eacl-long)
Copied to clipboard
| Challenge: | Visio-linguistic models have been gaining popularity for tasks that require a deeper understanding of multimodalities. |
| Approach: | They compile a probing dataset to test multi-modal alignment around color . they show that models have trouble with prepositions and verbs . |
| Outcome: | The proposed model is superior to models that do not rely on pre-extracted image features and is able to perform well with noisy pre-training data. |
Quantifying Qualitative Data for Understanding Controversial Issues (L18-1)
Copied to clipboard
| Challenge: | 'Controversy' is a state of sustained public debate on a topic or issue that evokes conflicting opinions, beliefs, claims, arguments, and points of view. |
| Approach: | They propose a crowdsourced approach to quantifying qualitative information on controversial issues by analyzing crowdsourced assertions in social media. |
| Outcome: | The proposed dataset consists of over 2,000 assertions on 16 controversial issues. |
Every Verb in Its Right Place? A Roadmap for Operationalizing Developmental Stages in the Acquisition of L2 German (2024.lrec-main)
Copied to clipboard
| Challenge: | Developmental stages are a linguistic concept claiming that language learning progresses in an ordered, step-like manner. |
| Approach: | They propose to translate a linguistic specification into a computational procedure that can assign clauses to a developmental stage based on verb placement. |
| Outcome: | The proposed system lacks a coherent linguistic specification of developmental stages . it also lacks the ability to translate the specification into a computational procedure based on verb placement. |
Similarity-Based Content Scoring - A more Classroom-Suitable Alternative to Instance-Based Scoring? (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work suggests that similarity-based content scoring methods can yield comparable results to instance-based supervised learning. |
| Approach: | They propose to use similarity-based scoring to achieve similar results . they compare different instance-based and similarity based methods on multiple data sets . |
| Outcome: | The proposed approach has a lower need for annotated training data and better zero-shot performance, but the results are not consistent with previous studies. |
From legal to technical concept: Towards an automated classification of German political Twitter postings as criminal offenses (N19-1)
Copied to clipboard
| Challenge: | 'Network Enforcement Act' provides for a regulatory framework for 'illegal content' on social network platforms like Twitter or Facebook. |
| Approach: | They propose a data annotation schema to determine whether a particular tweet could constitute a criminal offense and a binary classification schema to help with this. |
| Outcome: | The proposed schema shows that the majority of offensive posts do not constitute a criminal offense and still contribute to public discourse. |
Don’t take “nswvtnvakgxpm” for an answer –The surprising vulnerability of automatic content scoring systems to adversarial input (2020.coling-main)
Copied to clipboard
| Challenge: | Automated content scoring systems can be used on short answer tasks to save human effort, but can invite cheating strategies such as writing irrelevant answers. |
| Approach: | They generate adversarial answers for benchmark content scoring datasets based on different methods of increasing sophistication and examine countermeasures such as adversarials. |
| Outcome: | The proposed methods show that even simple methods can reduce content scoring performance but do not solve the problem. |
Decomposing and Comparing Meaning Relations: Paraphrasing, Textual Entailment, Contradiction, and Specificity (2020.lrec-1)
Copied to clipboard
| Challenge: | SHARel is a new typology for decomposing and comparing multiple meaning relations . it consists of 26 linguistic and 8 reason-based categories and can be applied to all relations with a high inter-annotator agreement. |
| Approach: | They propose a new typology that consists of 26 linguistic and 8 reason-based categories and propose SHARel for decomposing and comparing multiple meaning relations. |
| Outcome: | The proposed method can be applied to all relations with high inter-annotator agreement. |
ESCRITO - An NLP-Enhanced Educational Scoring Toolkit (L18-1)
Copied to clipboard
| Challenge: | Existing implementations are very specific to specific use cases and datasets. |
| Approach: | ESCRITO is a toolkit for scoring student writings using NLP techniques . authors propose teachers and NLP researchers to use APIs for scoring pipelines . |
| Outcome: | ESCRITO is a toolkit for scoring student writings using NLP techniques . it addresses two main user groups: teachers and NLP researchers . |
Chinese Content Scoring: Open-Access Datasets and Features on Different Segmentation Levels (2020.aacl-main)
Copied to clipboard
| Challenge: | Unlike English, which uses spaces as natural separators between words, segmentation of Chinese texts into tokens is challenging. |
| Approach: | They present two data sets for Chinese content scoring that use Chinese short answer questions and a new scoring system that uses Chinese short-answer questions. |
| Outcome: | The proposed system performs better on lower segmentation levels than on token level. |
LeSpell - A Multi-Lingual Benchmark Corpus of Spelling Errors to Develop Spellchecking Methods for Learner Language (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing spellcheckers do not work well with learner data. |
| Approach: | They propose a multi-lingual evaluation data set of spelling mistakes in context that is highly customizable for the DKPro architecture. |
| Outcome: | The proposed spellchecker improves performance in many settings and can be customized to meet learners' needs. |
PictureStories: Predicting the Task Adherence of Language Learner Answers to a Picture Story-Based Writing Task (2026.eacl-long)
Copied to clipboard
| Challenge: | a lack of suitable training and evaluation data limits the evaluation of language learning tasks to language proficiency only. |
| Approach: | They develop a marking rubric that covers task adherence with respect to form and content. |
| Outcome: | The proposed model can predict the adherence of learners to written tasks using picture stories. |
Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding (2024.lrec-main)
Copied to clipboard
Ahmad Idrissi-Yaghir, Amin Dada, Henning Schäfer, Kamyar Arzideh, Giulia Baldini, Jan Trienes, Max Hasin, Jeanette Bewersdorff, Cynthia S. Schmidt, Marie Bauer, Kaleb E. Smith, Jiang Bian, Yonghui Wu, Jörg Schlötterer, Torsten Zesch, Peter A. Horn, Christin Seifert, Felix Nensa, Jens Kleesiek, Christoph M. Friedrich
| Challenge: | Pre-trained language models can struggle in specialized domains such as medicine . existing generalpurpose pre-tried models can be used and refined through further pre-training on domainspecific unlabeled data. |
| Approach: | They pre-trained German medical language models on 2.4B tokens from translated public data and 3B token of German clinical data. |
| Outcome: | The proposed models outperform clinical models on various downstream tasks in germany . the authors show that continuous pre-training can match or exceed clinical models trained from scratch . |
Unraveling the Dynamics of Semi-Supervised Hate Speech Detection: The Impact of Unlabeled Data Characteristics and Pseudo-Labeling Strategies (2024.findings-eacl)
Copied to clipboard
| Challenge: | Semi-supervised learning addresses the need for large amounts of labeled training data for state-of-the-art approaches. |
| Approach: | They propose to leverage unlabeled data to reduce the amount of annotated data required for machine learning based hate speech detection by using a semi-supervised approach. |
| Outcome: | The proposed approach reduces the amount of annotated data required by state-of-the-art models by leveraging unlabeled data. |
DeepTC – An Extension of DKPro Text Classification for Fostering Reproducibility of Deep Learning Experiments (L18-1)
Copied to clipboard
| Challenge: | a current state of DKPro TC does not allow integration of deep learning . we integrate Keras, DyNet, and DeepLearning4J as proof-of-concept . |
| Approach: | They propose a deep learning extension for the multi-purpose text classification framework DKPro Text Classification. |
| Outcome: | The proposed extension improves readability and reduces redundant source code. |