Papers by Omar Shaikh
Navigating Rifts in Human-LLM Grounding: Study and Benchmark (2025.acl-long)
Copied to clipboard
| Challenge: | Language models excel at following instructions but struggle with collaborative aspects of conversation that humans naturally employ. |
| Approach: | They analyze logs from WildChat, MultiWOZ, and Bing Chat to examine grounding challenges . they propose a benchmark to determine when LLMs fail to initiate grounding . |
| Outcome: | The proposed model predicts interactions that fail to ground with users . the proposed model is based on human-human interactions with humans . |
Modeling Cross-Cultural Pragmatic Inference with Codenames Duet (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on pragmatic reasoning tests using simple word reference games with unidentified speakers and listeners, but speakers' sociocultural background shapes their pragmatic assumptions. |
| Approach: | They propose a dataset which operationalizes sociocultural pragmatic inference in a word reference game. |
| Outcome: | The proposed model improves clue-giving and guessing tasks by accounting for background characteristics and the game context. |
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | Prior work has focused on logical reasoning tasks; it remains unclear whether improvements hold for more diverse types of reasoning, especially in socially situated contexts. |
| Approach: | They perform a controlled evaluation of zero-shot CoT reasoning in two socially sensitive domains: harmful questions and stereotype benchmarks. |
| Outcome: | The results show that zero-shot CoT reasoning increases model’s likelihood to produce harmful or undesirable output, but decreases with improved instruction following. |
SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent calls for pluralistic alignment of Large Language Models encourage adapting models to diverse user preferences. |
| Approach: | They propose a method to induce synthetic user personas from user interactions for personalized reward modeling. |
| Outcome: | The proposed approach improves LLM-as-a-judge accuracy by 4.4% on Chatbot Arena. |
Examining the Ordering of Rhetorical Strategies in Persuasive Requests (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Numerous studies have been conducted to understand persuasiveness of text, from explorations of rhetoric in presidential campaigns to the impact of a communicator's likability on persuasiveness. |
| Approach: | They use a Variational Autoencoder model to disentangle content and rhetorical strategies in textual requests from a large-scale loan request corpus and visualize interplay between content and strategy through an attentional LSTM that predicts the success of textual request. |
| Outcome: | The proposed model disentangles content and rhetorical strategies in textual requests from a large-scale loan request corpus and visualizes interplay between content and strategy through attentional LSTM that predicts the success rate of textual request. |
Grounding Gaps in Language Model Generations (2024.naacl-long)
Copied to clipboard
| Challenge: | Effective conversation requires common ground, but it does not emerge spontaneously. |
| Approach: | They propose a set of grounding acts and metrics that quantify attempted grounding . they find that large language models generate language with less conversational grounding than humans . |
| Outcome: | The proposed models generate language with less conversational grounding than humans . compared to humans, they generate language that appears to presume common ground . |