Papers by Jad Kabbara
On the Relationship between Truth and Political Bias in Language Models (2024.emnlp-main)
Copied to clipboard
Suyash Fulay, William Brannon, Shrestha Mohanty, Cassandra Overney, Elinor Poole-Dayan, Deb Roy, Jad Kabbara
| Challenge: | Language model alignment research often attempts to ensure that models are helpful and harmless, but can obscure how improving one aspect might impact the other. |
| Approach: | They analyze the relationship between truthfulness and political bias in language models. |
| Outcome: | The results show that optimizing models for truthfulness results in a left-leaning political bias. |
Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive performance in many annotation tasks, including subjective tasks common in content moderation and text analysis in the social sciences. |
| Approach: | They propose to give crowdworkers LLM-generated annotation suggestions to "review" LLMs for subjective tasks can impact model performance and analysis downstream . |
| Outcome: | The proposed approach improves self-reported confidence in annotators and models . it also significantly improves model performance by analyzing human-approved datasets. |
Fora: A corpus and framework for the study of facilitated dialogue (2024.acl-long)
Copied to clipboard
| Challenge: | a new study of facilitated dialogues focuses on the sharing of personal experience . social media is a popular method of civic engagement but lacks the tools to analyze it . |
| Approach: | They compile 262 facilitated conversations hosted with partner organizations . they taxonomize personal sharing behaviors and facilitation strategies in the corpus . |
| Outcome: | The proposed framework can be used to analyze facilitated dialogues and parse spoken conversations . the data can be applied to other fields, including civic use in governance and social science . |
Computational Analysis of Conversation Dynamics through Participant Responsivity (2025.emnlp-main)
Copied to clipboard
| Challenge: | Growing literature explores toxicity and polarization in discourse, with comparatively little work on characterizing what makes dialogue prosocial and constructive. |
| Approach: | They develop and evaluate methods for quantifying responsivity through semantic similarity of speaker turns and large language models to identify the relation between two speaker turns. |
| Outcome: | The proposed method is based on semantic similarity of speaker turns and large language models to identify the relation between two speaker turns. |
Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used in high-stakes areas such as healthcare, law, and education. |
| Approach: | They propose a concept of Confidence-Probability Alignment that connects an LLM’s internal confidence to the confidence conveyed in the model’s response when explicitly asked about its certainty. |
| Outcome: | The proposed model shows the strongest confidence-probability alignment across a wide range of tasks. |
Debiasing should be Good and Bad: Measuring the Consistency of Debiasing Techniques in Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in deep learning have led to the creation of large Transformer-based language models (LMs). |
| Approach: | They propose a protocol which distinguishes methods that yield desirable results . they apply this protocol to a popular debiasing method, Self-Debiase, and compare it to one called Instructive Debiaser. |
| Outcome: | The proposed protocol provides essential insights into the generalizability and interpretability of debiasing methods that may otherwise go overlooked. |
Post-Editing Extractive Summaries by Definiteness Prediction (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Abstract: Extractive summarization has been the mainstay of automatic summarizing for decades, but it still suffers from coreference issues arising from extracting sentences away from their original context. |
| Approach: | They propose a post-editing step that generates linguistic decisions that lead to improved extractive summaries by predicting definiteness of noun phrases. |
| Outcome: | The proposed system generates linguistic decisions that improve the quality of the extractive summaries. |
Common to Whom? Regional Cultural Commonsense and LLM Bias in India (2026.acl-long)
Copied to clipboard
Sangmitra Madhusudan, Trush Shashank More, Steph Buongiorno, Renata Dividino, Jad Kabbara, Ali Emami
| Challenge: | Existing cultural commonsense benchmarks treat nations as monolithic, assuming uniform practices within national boundaries. |
| Approach: | They evaluate eight state-of-the-art LLMs and find two critical gaps . commonsense knowledge is fundamentally long-tailed, with most facts rare in training data . |
| Outcome: | The proposed model achieves only 13.4%–20.9% accuracy on region-specific questions and exhibits geographic bias over-selecting Central and North India as the "default" while under-representing East and West. |
Bridging Context Gaps: Enhancing Comprehension in Long-Form Social Conversations Through Contextualized Excerpts (2025.coling-main)
Copied to clipboard
| Challenge: | a recent rise in polarization has led to a rise in the use of loud and extreme voices in public spaces. |
| Approach: | They propose ways to parse and convey information from small-group recorded conversations . they show that LLMs can provide socially relevant context to improve comprehension . |
| Outcome: | The proposed models improve comprehension, readability, and empathy in small-group conversations . the proposed models struggle with capturing key social aspects, the authors show . |
Investigating the Performance of Transformer-Based NLI Models on Presuppositional Inferences (2022.coling-1)
Copied to clipboard
| Challenge: | Presuppositions are assumptions that are taken for granted by an utterance. |
| Approach: | They propose to use heuristics to create alternative "contrastive" test cases . they also analyze samples from ImpPres datasets to better understand their predictions . |
| Outcome: | The proposed model performs better on the ImpPres dataset than on the other datasets. |
Computational Investigations of Pragmatic Effects in Natural Language (N19-3)
Copied to clipboard
| Challenge: | a recent paper examines the relationship between semantics and pragmatics in language. |
| Approach: | They propose to develop computational models that leverage pragmatic knowledge in language . goal is to build better and more pragmatically-aware natural language generation and understanding systems . |
| Outcome: | The proposed models leverage pragmatic knowledge in language crucial to performing many NLP tasks correctly. |
Investigating the Effect of Pre-finetuning BERT Models on NLI Involving Presuppositions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a study of presupposition, discourse and sarcasm suggests that pre-finetuning can improve models' performance on presimplified cases. |
| Approach: | They propose to leverage the connection between presupposition, discourse and sarcasm to improve models' performance. |
| Outcome: | The proposed model improves on cases involving presupposition by pre-finetuning on additional tasks and datasets. |
Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling (2024.acl-long)
Copied to clipboard
Hang Jiang, Xiajie Zhang, Robert Mahari, Daniel Kessler, Eric Ma, Tal August, Irene Li, Alex Pentland, Yoon Kim, Deb Roy, Jad Kabbara
| Challenge: | a novel application of large language models (LLMs) to legal education helps non-experts learn complex legal concepts . authors find storytelling helps nonexperts understand complex legal terms and concepts compared to definitions . |
| Approach: | They propose a novel application of large language models to legal education . they use LLMs to generate legal stories explaining complex legal concepts . |
| Outcome: | The proposed method improves comprehension and interest among non-native speakers compared to definitions . the novel method also shows that non-experts retain more stories . |
PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have shown that LLMs can generate content that aligns with their assigned personality traits, but there is limited research on whether they consistently reflect specific personality traits. |
| Approach: | They propose to study the behavior of LLM-based agents which they refer to as LLM personas and simulate them to measure their personality traits. |
| Outcome: | The proposed model is based on the Big Five personality model and has been validated by human evaluations and automatic evaluations. |
DART: Mitigating Harm Drift in Difference-Aware LLMs via Distill-Audit-Repair Training (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) tuned for safety often avoid acknowledging demographic differences . current safety alignment forces LLMs to default to identity-blindness even when demographic differences are factually correct or contextually justified. |
| Approach: | They propose a tool to classify whether a correct answer requires recognizing group differences . they use label-conditioned reasoning from a teacher to audit outputs for harm drift cases . |
| Outcome: | The proposed model improves accuracy and safety on eight benchmarks. |
Let’s do it “again”: A First Computational Approach to Detecting Adverbial Presupposition Triggers (P18-1)
Copied to clipboard
| Challenge: | a novel task of predicting adverbial presupposition triggers is useful for natural language generation . a focus is on a new attention mechanism for predicting presuposition trigger . |
| Approach: | They propose a new attention mechanism for predicting adverbial presupposition triggers . they propose to augment a baseline neural network without additional trainable parameters . |
| Outcome: | The proposed model outperforms baseline models in predicting adverbial presupposition triggers. |