Papers by Bernhard Schoelkopf
Logical Fallacy Detection (2022.findings-emnlp)
Copied to clipboard
Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, Xiaoyu Shen, Yiwen Ding, Zhiheng Lyu, Mrinmaya Sachan, Rada Mihalcea, Bernhard Schoelkopf
| Challenge: | Existing language models perform poorly on logical fallacy detection . fallacious arguments can lead to disagreements, conflicts, endless debates, and a lack of consensus . |
| Approach: | They propose a task of logical fallacy detection and propose LogicClimate to detect fallacies in text. |
| Outcome: | The proposed task outperforms the best language model on Logic and LogicClimate . human reasoning is marred by logical fallacies, and some exacerbate misinformation . |
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP (2021.emnlp-main)
Copied to clipboard
Zhijing Jin, Julius von Kügelgen, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan, Bernhard Schoelkopf
| Challenge: | a meta-analysis of published studies shows that the causal direction of data collection can explain some trends in NLP . semi-supervised learning and domain adaptation performance differ on a number of tasks . |
| Approach: | They argue that the causal direction of the data collection process has nontrivial implications . authors categorize common NLP tasks according to their causal direction . they also empirically assay the validity of the ICM principle for text data . |
| Outcome: | The proposed model can explain differences in semi-supervised learning and domain adaptation performance across settings. |
A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work shows that language models can rely on shallow patterns in problem description when generating a solution. |
| Approach: | They propose a framework which pins down the causal effect of various factors on the output solution. |
| Outcome: | The proposed framework improves robustness and sensitivity to direct interventions on a test bed of math word problems. |
Differentially Private Language Models for Secure Data Sharing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a variety of deanonymization attacks allow the re-identification of individuals from tabular data. |
| Approach: | They propose to train a language model in a differentially private manner and sample data from it . they find that the model generates fluent textual datasets with privacy guarantees . |
| Outcome: | The proposed methods outperform direct classifiers with DP-SGD in the real-world. |
Mining the Cause of Political Decision-Making from Social Media: A Case Study of COVID-19 Policies across the US States (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on political responsiveness focus on long-term policies collected over decades . recent COVID-19 pandemic has given rise to a new political phenomenon, where political leaders make frequent short-term decisions on the same controlled topic. |
| Approach: | They propose to use Twitter data to classify the sentiments toward governors of each state and conduct controlled studies and comparisons. |
| Outcome: | The proposed model focuses on the COVID-19 pandemic, where political leaders make frequent short-term decisions on the same controlled topic. |
Membership Inference Attacks against Language Models via Neighbourhood Comparison (2023.findings-acl)
Copied to clipboard
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schoelkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick
| Challenge: | Existing membership inference attacks aim to predict whether a data sample was present in training data of a machine learning model. |
| Approach: | They propose to compare model scores to neighbour texts to eliminate access to training data by comparing model scores with a given sample. |
| Outcome: | The proposed attacks outperform reference-based attacks with perfect knowledge of the training data distribution and outperformed reference-free attacks with imperfect knowledge. |