Papers by Erik Miehling
Language Models in Dialogue: Conversational Maxims for Human-AI Interactions (2024.findings-emnlp)
Copied to clipboard
Erik Miehling, Manish Nagireddy, Prasanna Sattigeri, Elizabeth Daly, David Piorkowski, John Richards
| Challenge: | Modern language models exhibit some inherent shortcomings, particularly in conversational settings. |
| Approach: | They propose a set of maxims for describing effective human-AI conversation that include quantity, quality, relevance, manner, benevolence, and transparency. |
| Outcome: | The proposed maxims are applied to human-AI interactions and are based on extensive research from the social science and AI communities. |
Evaluating the Prompt Steerability of Large Language Models (2025.naacl-long)
Copied to clipboard
Erik Miehling, Michael Desmond, Karthikeyan Natesan Ramamurthy, Elizabeth M. Daly, Kush R. Varshney, Eitan Farchi, Pierre Dognin, Jesus Rios, Djallel Bouneffouf, Miao Liu, Prasanna Sattigeri
| Challenge: | a primary question underlying alignment research is: whose views are we aligning to? |
| Approach: | They propose to evaluate the steerability of model personas as a function of prompting by defining a benchmark and inspecting how these indices change as if steering effort is a factor. |
| Outcome: | The proposed benchmark reveals that the steerability of many current models is limited due to skew in baseline behavior and an asymmetry in their steerability across many persona dimensions. |
Granite Guardian: Comprehensive LLM Safeguarding (2025.naacl-industry)
Copied to clipboard
Inkit Padhi, Manish Nagireddy, Giandomenico Cornacchia, Subhajit Chaudhury, Tejaswini Pedapati, Pierre Dognin, Keerthiram Murugesan, Erik Miehling, Martín Santillán Cooper, Kieran Fraser, Giulio Zizzo, Muhammad Zaid Hameed, Mark Purcell, Michael Desmond, Qian Pan, Inge Vejsbjerg, Elizabeth M. Daly, Michael Hind, Werner Geyer, Ambrish Rawat, Kush R. Varshney, Prasanna Sattigeri
| Challenge: | a suite of advanced models is designed to detect and mitigate risks associated with prompts and responses. |
| Approach: | a team of researchers develop a model family to detect and mitigate risks associated with prompts and responses. the model family is based on the Granite 3.0 language models. |
| Outcome: | a new model family is designed to detect and mitigate risks associated with prompts and responses. |
Synthetic Data for Evaluation: Supporting LLM-as-a-Judge Workflows with EvalAssist (2025.emnlp-demos)
Copied to clipboard
Martín Santillán Cooper, Zahra Ashktorab, Hyo Jin Do, Erik Miehling, Werner Geyer, Jasmina Gajcin, Elizabeth M. Daly, Qian Pan, Michael Desmond
| Challenge: | EvalAssist is a web-based application designed to assist human-centered evaluation of language model outputs. |
| Approach: | They propose a synthetic data generation tool integrated into EvalAssist to assist human-centered evaluation of language model outputs. |
| Outcome: | The proposed tool supports flexible prompting, RAG-based grounding, persona diversity, and iterative generation workflows. |
AI Steerability 360: A Toolkit for Steering Large Language Models (2026.acl-demo)
Copied to clipboard
Erik Miehling, Karthikeyan Natesan Ramamurthy, Praveen Venkateswaran, Ching-Yun Ko, Pierre Dognin, Moninder Singh, Tejaswini Pedapati, Avinash Balakrishnan, Matthew Riemer, Dennis Wei, Inge Vejsbjerg, Elizabeth M. Daly, Kush R. Varshney
| Challenge: | The AI Steerability 360 toolkit is an extensible, open-source Python library for steering LLMs. |
| Approach: | The AI Steerability 360 toolkit is an extensible, open-source Python library for steering LLMs. |
| Outcome: | The toolkit is available under an Apache 2.0 license and is available on https://github.com/IBM/AISteer360. |