Papers by Ahmed Elgohary
Jailbreak Distillation: Renewable Safety Benchmarking (2025.findings-emnlp)
Copied to clipboard
Jingyu Zhang, Ahmed Elgohary, Xiawei Wang, A S M Iftekhar, Ahmed Magooda, Benjamin Van Durme, Daniel Khashabi, Kyle Jackson
| Challenge: | a new framework for safety benchmarking is being developed for large language models . the framework "distills" jailbreak attacks into high-quality and easily-updatable safety benchmarks . |
| Approach: | They propose a framework that "distills" jailbreak attacks into high-quality safety benchmarks. |
| Outcome: | The proposed framework "distills" jailbreak attacks into high-quality safety benchmarks . it requires minimal human effort to rerun the pipeline and produce updated benchmarks. |
NL-EDIT: Correcting Semantic Parse Errors through Natural Language Interaction (2021.naacl-main)
Copied to clipboard
Ahmed Elgohary, Christopher Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, Ahmed Hassan Awadallah
| Challenge: | Existing systems frame semantic parsing as a one-shot translation from a natural language question to the logical form. |
| Approach: | They propose a model that uses natural language feedback to correct parsers . they show that NL-EDIT can boost the accuracy of existing parser by 20% . |
| Outcome: | The proposed model can boost parsers' accuracy by 20% with just one turn of correction. |
It Takes Two to Lie: One to Lie, and One to Listen (2020.acl-main)
Copied to clipboard
Denis Peskov, Benny Cheng, Ahmed Elgohary, Joe Barrow, Cristian Danescu-Niculescu-Mizil, Jordan Boyd-Graber
| Challenge: | Deception is a powerful tool for predicting when a lie occurs in long-lasting relationships . a functioning society is impossible without trust, but deception can be betrayed through false identities, spearphishing attacks and disinformation campaigns. |
| Approach: | They propose a dataset to analyze the use of deception in online negotiation-based game Diplomacy . it captures deceptions in long-lasting relationships where interlocutors combine truth with lies to advance objectives. |
| Outcome: | The proposed model predicts when a lie occurs nearly as well as human players. |
Construction of the Literature Graph in Semantic Scholar (N18-3)
Copied to clipboard
Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
| Challenge: | Fig. 1 summarizes a scalable system for organizing published scientific literature into a heterogeneous graph . authors describe methods used to enable semantic features in www.semanticscholar.org . |
| Approach: | They describe a scalable system for organizing published scientific literature into a heterogeneous graph to facilitate algorithmic manipulation and discovery. |
| Outcome: | The proposed system can be deployed on a scalable platform and report empirical results for each task. |
Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedback (2020.acl-main)
Copied to clipboard
| Challenge: | a natural language interface (NLI) can be used to correct semantic parsing errors . human correction accuracy is 81.5%, but the best model achieves only 25.1% . |
| Approach: | They propose a task where humans can provide free-form natural language feedback to correct a system when it generates an inaccurate interpretation of an initial utterance. |
| Outcome: | The proposed model improves the parsing accuracy while maintaining flexibility of natural language interaction. |
SIRAJ: Diverse and Efficient Red-Teaming for LLM Agents via Distilled Structured Reasoning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing red-teaming frameworks do not cover all the risks associated with arbitrary black-box LLMs. |
| Approach: | They propose a generic red-teaming framework for arbitrary black-box LLM agents that iteratively constructs and refines model-based adversarial attacks based on the execution trajectories of former attempts. |
| Outcome: | The proposed model improves attack success rate by 100%, surpassing the 671B Deepseek-R1 model. |
A dataset and baselines for sequential open-domain question answering (D18-1)
Copied to clipboard
| Challenge: | Existing question-answering systems focus on answering individual questions, assuming they are devoid of context. |
| Approach: | They propose to ask multiple related questions in a dataset that includes human-authored questions. |
| Outcome: | The proposed system can answer human-authored questions better than existing systems. |
Can You Unpack That? Learning to Rewrite Questions-in-Context (D19-1)
Copied to clipboard
| Challenge: | Existing QA datasets lack key NLP problems like coreference and ellipsis resolution. |
| Approach: | They propose a task of question-in-context rewriting to rewrite a context-dependent question into a self-contained question with the same answer. |
| Outcome: | The proposed task is based on a dataset of 40,527 questions based in QuAC . it requires models to link questions together to resolve conversational dependencies . |
Generating Natural Language Adversarial Examples (D18-1)
Copied to clipboard
| Challenge: | Recent research has shown that deep neural networks are vulnerable to adversarial examples, perturbations to correctly classified examples which can cause the model to misclassify. |
| Approach: | They propose to generate adversarial examples that fool well-trained sentiment analysis and textual entailment models by using a black-box population-based optimization algorithm. |
| Outcome: | The proposed model is able to fool well-trained sentiment analysis and textual entailment models with success rates of 97% and 70%, respectively. |
Assessing Composition in Sentence Vector Representations (C18-1)
Copied to clipboard
| Challenge: | opacity of sentence vector representations is a challenge to achieving language understanding . current neural network models are unable to capture meaning information in dense vectors . |
| Approach: | They propose a method that targets compositional meaning information in sentence embeddings with a high degree of precision and control. |
| Outcome: | The proposed method extracts useful information about the different capacities of existing sentences models. |