Papers by Brihi Joshi
Are Machine Rationales (Not) Useful to Humans? Measuring and Improving Human Utility of Free-text Rationales (2023.acl-long)
Copied to clipboard
Brihi Joshi, Ziyi Liu, Sahana Ramnath, Aaron Chan, Zhewei Tong, Shaoliang Nie, Qifan Wang, Yejin Choi, Xiang Ren
| Challenge: | Existing metrics like task performance of the LM generating the rationales or similarity between generated and gold rationale are not good indicators of their human utility. |
| Approach: | They propose to use a large language model to generate rationales with better human utility by estimating its conciseness and novelty. |
| Outcome: | The proposed model can measure human utility to a better extent by estimating its usefulness in answering similar unseen instances. |
Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. |
| Approach: | They propose to evaluate two complementary qualities of VLM-generated explanations via two quality scoring functions to improve their accuracy. |
| Outcome: | The proposed explanations improve accuracy on the A-OKVQA, VizWiz, and MMMU-Pro tasks by 11.1%, including a 15.4% reduction in falsely believing incorrect predictions. |
Amulet: Putting Complex Multi-Turn Conversations on the Stand with LLM Juries (2025.emnlp-main)
Copied to clipboard
| Challenge: | a typical human-assistant conversation is lengthy and shows significant diversity in topics, intents, and requirements across turns. |
| Approach: | They propose a framework that leverages pertinent linguistic concepts of dialog-acts and maxims to improve the accuracy of LLM-judges on preference data with complex, multi-turn conversational context. |
| Outcome: | The proposed framework improves on 4 challenging datasets showing that humans frequently change their intents from one turn of the conversation to the next. |
ELI-Why: Evaluating the Pedagogical Utility of Language Model Explanations (2025.findings-acl)
Copied to clipboard
Brihi Joshi, Keyu He, Sahana Ramnath, Sadra Sabouri, Kaitlyn Zhou, Souti Chattopadhyay, Swabha Swayamdipta, Xiang Ren
| Challenge: | Language models are widely used in education, yet their ability to tailor responses to learners with varied informational needs and knowledge backgrounds remains under-explored. |
| Approach: | They conduct two extensive human studies to assess the utility of language model-generated explanatory answers (explanations) on a benchmark of 13.4K "Why" questions. |
| Outcome: | The proposed model explanations match learners' educational backgrounds only 50% of the time, compared to 79% for lay explanations. |
PrimeX: A Dataset of Worldview, Opinion, and Explanation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that an individual's worldview -or beliefs about the overall character of the world -can explain persistent behavioral patterns and correlates with personality, well-being, political, religious, and demographic variables. |
| Approach: | They develop a dataset of public opinion survey data from 858 US residents with written explanations from the respondents for why they hold specific opinions and the Primal World Belief survey for assessing respondent worldview. |
| Outcome: | The proposed model can be used to better represent an individual's belief system and improve opinion prediction. |
XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Models (2023.acl-demo)
Copied to clipboard
Dong-Ho Lee, Akshen Kadakia, Brihi Joshi, Aaron Chan, Ziyi Liu, Kiran Narahari, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
| Challenge: | Existing models are susceptible to learning spurious biases that do not reflect the underlying task. |
| Approach: | They propose an open-source framework for explanation-based model debugging that allows users to provide various forms of feedback on model explanations. |
| Outcome: | The proposed framework improves model’s OOD performance on text classification tasks by up to 18%. |
ER-Test: Evaluating Explanation Regularization Methods for Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Explanation regularization (ER) aims to improve NLM generalization by pushing the NLM’s machine rationales to align with human rationale. |
| Approach: | They propose a framework for evaluating ER models’ OOD generalization along three dimensions: unseen datasets, contrast set tests, and functional tests. |
| Outcome: | The proposed framework evaluates ER models’ OOD generalization across unseen datasets, contrast set tests, and functional tests. |
Improving Language Model Personas via Rationalization with Psychological Scaffolds (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to building personas rely on a user’s demographic attributes and/or prior judgments, but not on any underlying reasoning behind a person’s judgments. |
| Approach: | They propose a framework that integrates rationales for why a user could have made a certain judgment into LM personas by incorporating potential rationale. |
| Outcome: | The proposed framework outperforms models conditioned on demographic attributes and/or prior judgments on public opinion and movie preference prediction tasks. |
OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistants (2024.emnlp-main)
Copied to clipboard
Jaspreet Ranjit, Brihi Joshi, Rebecca Dorn, Laura Petry, Olga Koumoundouros, Jayne Bottarini, Peichen Liu, Eric Rice, Swabha Swayamdipta
| Challenge: | a large-scale analysis of millions of tweets on homelessness is challenging to understand at scale. |
| Approach: | They propose a framing typology: Online Attitudes Towards Homelessness (OATH) They use large language models to analyze millions of tweets to find patterns in public attitudes . |
| Outcome: | The proposed model speeds up annotations while incurring a 3 point performance reduction compared to existing classifiers . |
The Devil is in the Details: Evaluating Limitations of Transformer-based Methods for Granular Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Contextual embeddings have shown state-of-the-art performance for various tasks such as question answering, sentiment analysis, and textual similarity. |
| Approach: | They propose to integrate transformer-based neural language models into their models to achieve relative improvements of up to 36% on granular tasks. |
| Outcome: | The proposed model outperforms baselines for more granular tasks while outperforming TF-IDF for more complex tasks. |