Papers by Zeerak Talat
Exploring the Limitations of Detecting Machine-Generated Text (2025.coling-main)
Copied to clipboard
| Challenge: | Recent advances in the quality of the generation of text by large language models have spurred research into identifying machine-generated text. |
| Approach: | They audit classification performance for detecting machine-generated text by evaluating on texts with varying writing styles. |
| Outcome: | The proposed methods are highly sensitive to stylistic changes and complexity, and in some cases degrade entirely to random classifiers. |
The Only Way is Ethics: A Guide to Ethical Research with Large Language Models (2025.coling-main)
Copied to clipboard
Eddie L. Ungless, Nikolas Vitsakis, Zeerak Talat, James Garforth, Bjorn Ross, Arno Onken, Atoosa Kasirzadeh, Alexandra Birch
| Challenge: | Existing literature on the ethical aspects of large language models (LLMs) is lacking a single practical guide on the subject. |
| Approach: | They propose to translate ethics literature into concrete recommendations for computer scientists by presenting an open and living resource for NLP practitioners and those tasked with evaluating the ethical implications of others’ work. |
| Outcome: | The proposed guide is an open and living resource for NLP practitioners and those tasked with evaluating the ethical implications of others’ work. |
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels (2024.naacl-long)
Copied to clipboard
| Challenge: | a line of recent work has illustrated that annotators disagree for many reasons . capturing disagreements can improve model performance and calibration, authors argue . |
| Approach: | They propose a new paradigm shift in data labeling for machine learning that challenges annotator disagreement by treating disagreement as a valuable source of information. |
| Outcome: | The proposed approaches challenge annotator disagreement and provide recommendations for the data labeling pipeline and avenues for future research. |
Classist Tools: Social Class Correlates with Performance in NLP (2024.acl-long)
Copied to clipboard
| Challenge: | despite growing concerns surrounding fairness and bias in NLP, there is a dearth of studies delving into the effects it may have on NLP systems. |
| Approach: | They argue that NLP systems’ performance is affected by speakers’ SES, potentially disadvantaging less-privileged socioeconomic groups. |
| Outcome: | The proposed model shows that NLP systems perform better on tasks with social class, ethnicity and geographical variation than those without social class. |
Mirages. On Anthropomorphism in Dialogue Systems (2023.emnlp-main)
Copied to clipboard
| Challenge: | Automated dialogue systems are anthropomorphised by developers and personified by users. |
| Approach: | They propose to examine linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise thereof. |
| Outcome: | The proposed systems are anthropomorphised and personified by users . linguistic factors can also be used to reinforce gender stereotypes and conceptions of acceptable language. |
Zero-shot Sentiment Analysis in Low-Resource Languages Using a Multilingual Sentiment Lexicon (2024.eacl-long)
Copied to clipboard
| Challenge: | Prior work extended multilingual models to other languages due to the unavailability of labeled and unlabeled training data. |
| Approach: | They use multilingual lexicons to enhance multilingual models capabilities in low-resource languages . they focus on zero-shot sentiment analysis tasks across 34 languages based on a single sentence . |
| Outcome: | The proposed model improves zero-shot performance across 34 languages without using any sentence-level sentiment data. |
Impoverished Language Technology: The Lack of (Social) Class in NLP (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing work on socio-demographic factors has focused on how much a person's socioeconomic status affects their language production and perception. |
| Approach: | They propose to include socio-economic class in future natural language processing (NLP) research aimed at understanding relationships between socio-demographic factors and language production and perception. |
| Outcome: | The proposed definition of class can be operationalised by NLP researchers and argue for including socio-economic class in future language technologies. |
Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP (2024.emnlp-main)
Copied to clipboard
| Challenge: | a measure’s intended use and reliability assessment are often unclear or entirely absent from the literature examining bias measures in natural language processing. |
| Approach: | They propose a set of desiderata to assess the degree to which a measure’s results enable informed action and a review of 146 papers proposing bias measures in NLP. |
| Outcome: | The proposed desiderata are based on 146 papers proposing bias measures in natural language processing (NLP) . they show that key elements of actionability, including a measure’s intended use and reliability assessment, are often unclear or entirely absent. |
On the Machine Learning of Ethical Judgments from Natural Language (2022.naacl-main)
Copied to clipboard
| Challenge: | a recent study examines the morality of NLP models that can take in arbitrary text and output a moral judgment . a Delphi project is a popular system for moral prediction, but it has received criticism . |
| Approach: | They propose to critique NLP methods for automating ethical decision-making . they examine a nascent task of predicting moral and ethical decisions from text . |
| Outcome: | The proposed model is unsafe at any accuracy, the authors argue . they argue that the proposed model could be useful in NLP, but not in AI. |
Thorny Roses: Investigating the Dual Use Dilemma in Natural Language Processing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Dual use is a problem in the context of natural language processing, says aaron eliotta . eelisa et al.: it is important to examine their rightful use and potential misuse . |
| Approach: | They propose a definition and checklist for dual-use in natural language processing based on a survey of NLP researchers and practitioners. |
| Outcome: | The proposed checklist focuses on dual-use in NLP based on a survey of NLP researchers and practitioners. |
Understanding “Democratization” in NLP and ML Research (2024.emnlp-main)
Copied to clipboard
| Challenge: | a large number of NLP and ML papers mention terms related to democracy . authors find that democratization is most frequently used to convey (ease of) access to or use of technologies without meaningfully engaging with theories of democratisation. |
| Approach: | They analyze papers using the term "democra*" to clarify how it is understood in NLP and ML . they find that democratization is most frequently used to convey (ease of) access to or use of technologies . |
| Outcome: | The authors analyze papers using the term "democra*" they find that democratization is most frequently used to convey (ease of) access to or use of technologies without meaningfully engaging with theories of democratisation. |
IYKYK: Using language models to decode extremist cryptolects (2026.eacl-long)
Copied to clipboard
Christine de Kock, Arij Riabi, Zeerak Talat, Michael Sejr Schlichtkrull, Pranava Madhyastha, Eduard Hovy
| Challenge: | Extremist groups develop complex in-group language to exclude or mislead outsiders . general purpose LLMs cannot consistently detect or decode extremist language . |
| Approach: | They evaluate the ability of current language technologies to detect and interpret the cryptolects of two online extremist platforms. |
| Outcome: | The proposed models can detect and interpret extremist language better than current models. |
A Federated Approach to Predicting Emojis in Hindi Tweets (2022.emnlp-main)
Copied to clipboard
| Challenge: | emojis are a visual modality to, often private, textual communication, but their use tends to cluster into the frequently used and the rarely used eojis. |
| Approach: | They propose to use 118k tweets to predict emojis in Hindi and a federated learning algorithm to achieve a balance between model performance and user privacy. |
| Outcome: | The proposed approach achieves comparative scores with more complex centralised models while minimising risks to user privacy. |
Directions for NLP Practices Applied to Online Hate Speech Detection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to address hate speech in online spaces have relied on conventions and practices from NLP. |
| Approach: | They argue that many conventions in NLP are poorly suited for the problem and encourage researchers to develop methods that are more appropriate for the task. |
| Outcome: | The proposed methods are poorly suited for the problem and should be adapted to address the propagation of online harms. |
FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data (2026.acl-long)
Copied to clipboard
| Challenge: | Social media text data is often used to train machine learning models to identify users exhibiting high-risk mental health behaviors. |
| Approach: | They apply federatedlearning and Differentially Private FL to two widely-studied mental health prediction tasks using social media text data. |
| Outcome: | The proposed methods achieve comparable performance to centralized training on depression identification, but have a large performance-privacy trade-off even with low levels of noise. |
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models (2025.naacl-long)
Copied to clipboard
Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, null Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, Zeerak Talat
| Challenge: | Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited. |
| Approach: | They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs. |
| Outcome: | The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide. |
A Federated Approach for Hate Speech Detection (2023.eacl-main)
Copied to clipboard
| Challenge: | Despite the scale of social media content, privacy preservation in hate speech detection has remained understudied. |
| Approach: | They propose to use federated machine learning to address privacy concerns in hate speech detection by obtaining a 6.81% improvement in F1-score. |
| Outcome: | The proposed method improves the F1-score of hate speech detection by 6.81% while maintaining public data privacy. |