‘Am I the Bad One’? Predicting the Moral Judgement of the Crowd Using Pre–trained Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on NLP touch upon moral contexts in text. |
| Approach: | They construct a dataset that can be used for moral judgement tasks on a popular reddit subreddit. |
| Outcome: | The proposed model passes moral judgements on posts from a popular reddit subreddit . it shows that the model can be fine tuned and improves across the datasets . |
Similar Papers
On the Machine Learning of Ethical Judgments from Natural Language (2022.naacl-main)
Copied to clipboard
| Challenge: | a recent study examines the morality of NLP models that can take in arbitrary text and output a moral judgment . a Delphi project is a popular system for moral prediction, but it has received criticism . |
| Approach: | They propose to critique NLP methods for automating ethical decision-making . they examine a nascent task of predicting moral and ethical decisions from text . |
| Outcome: | The proposed model is unsafe at any accuracy, the authors argue . they argue that the proposed model could be useful in NLP, but not in AI. |
Predicting Performance for Natural Language Processing Tasks (2020.acl-main)
Copied to clipboard
| Challenge: | Natural language processing (NLP) is a vast field, with a wide variety of tasks, languages, and domains. |
| Approach: | They build regression models to predict evaluation score of an NLP experiment . they find that their models can produce meaningful predictions over unseen languages . |
| Outcome: | The proposed model outperforms baseline models and human experts on 9 different tasks. |
Constructing a Japanese Verdict Prediction Dataset for Fact-Checking of LLM-Generated Texts (2026.acl-srw)
Copied to clipboard
Miwa Masano, Hirokazu Kiyomaru, Atsushi Keyaki, Kaito Horio, Rei Minamoto, Ribeka Keyaki, Kouta Nakayama, Hideyuki Tachibana, Daisuke Kawahara
| Challenge: | Text generated by Large Language Models (LLMs) may contain plausible but incorrect information known as hallucinations. |
| Approach: | They extend the label set for verdict prediction to capture claim-evidence relationships humans would commonly interpret as supported or refuted. |
| Outcome: | The proposed system improves F1 by 4 percentage points compared to baseline. |
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023.acl-long)
Copied to clipboard
| Challenge: | Hundreds of studies have highlighted ethical issues in NLP models . |
| Approach: | They propose to measure media biases in LMs trained on diverse data sources . they focus on hate speech and misinformation detection . |
| Outcome: | The proposed methods quantify the fairness of downstream NLP models trained on politically biased LMs. |
Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Study (2022.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive models combined with stochastic decodings are the most promising for generating CNs with regard to an unseen target of hate. |
| Approach: | They propose to use pre-trained language models to generate counter-narratives in English by adding an automatic post-editing step to refine generated CNs. |
| Outcome: | The proposed pipeline could be used to generate counter-narratives in English using pre-trained language models and stochastic decoding mechanisms. |
Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies have highlighted a variety of biases in pre-trained language models . however, these studies focus on fine-grained analysis of educational corpora and text that is not English . |
| Approach: | They analyze bias across text and through multiple architectures on a corpus of 9,165 German peer-reviews collected from university students over five years. |
| Outcome: | The proposed dataset shows that pre-trained language models exhibit conceptual, racial, and gender biases. |
A Survey on Modelling Morality for Text Analysis (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work on modelling morality in text has garnered increasing attention due to its complexity and complexity. |
| Approach: | They provide a systematic review of recent work on modelling morality in text . they discuss challenges and research gaps in the area of NLP . |
| Outcome: | The authors present their work on the modelling of morality in text, which has garnered increasing attention in recent years. |
Bias and Fairness in Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | a tutorial will review the history of bias and fairness studies in machine learning and language processing . |
| Approach: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it presents recent community effort to quantify and mitigat bias in natural language processing models . |
| Outcome: | This tutorial reviews the history of bias and fairness studies in machine learning and language processing . it aims to quantify and mitigate bias in natural language processing models for a wide spectrum of tasks . |
Detecting Label Errors by Using Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for label error detection focus on label errors in training data. |
| Approach: | They propose a method for introducing realistic, human-originated label noise into existing crowdsourced datasets such as SNLI and TweetNLP. |
| Outcome: | The proposed method outperforms existing methods for detecting label errors in natural language datasets. |
Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation (D19-1)
Copied to clipboard
| Challenge: | Existing evaluation methods for natural language generation are inadequate . distinguishing machine-generated text is challenging even for human evaluators . |
| Approach: | They compare human-based evaluators with automated evaluation procedures . they find human evaluers do not correlate well with discriminative evalators . |
| Outcome: | The proposed evaluation methods are compared with a dozen state-of-the-art generators for online product reviews. |