Dr ChatGPT tell me what I want to hear: How different prompts impact health answer correctness (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using the TREC Misinformation dataset, we empirically evaluate ChatGPT to show not just its effectiveness but reveal that knowledge passed in the prompt can bias the model to the detriment of answer correctness. |
| Approach: | They empirically evaluate ChatGPT to find out whether a prompt can bias the model to the detriment of answer correctness. |
| Outcome: | The proposed model can be biased to the detriment of answer correctness by using retrieved-then-generate pipelines and how a user phrases their question as well as the question type. |
Similar Papers
The Death and Life of Great Prompts: Analyzing the Evolution of LLM Prompts from the Structural Perspective (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has shown that high-quality prompts are essential for LLMs to produce accurate and relevant responses. |
| Approach: | They analyze 10,538 in-the-wild prompts collected from various platforms and develop a framework that decomposes the prompts into eight key components. |
| Outcome: | The proposed framework decomposes 10,538 in-the-wild prompts into eight components. |
Towards Interpretable Mental Health Analysis with Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on large language models lack adequate evaluations and prompting strategies for explainability. |
| Approach: | They evaluate the mental health analysis and emotional reasoning ability of large language models (LLMs) using 11 datasets across 5 tasks. |
| Outcome: | The proposed model shows strong in-context learning ability but still has a significant gap with advanced task-specific methods. |
Rethinking Prompt-based Debiasing in Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods for debiasing are often superficial and lack a thorough understanding of complex bias concepts. |
| Approach: | They analyze a BBQ and stereoSet benchmarks to examine the assumption that large language models understand biases. |
| Outcome: | The proposed model misclassified 90% of unbiased content as biased despite high accuracy on BBQ dataset . the proposed model may have been flawed in previous attempts to debiase . |
“What’s Up, Doc?”: Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets (2025.findings-emnlp)
Copied to clipboard
Akshay Paruchuri, Maryam Aziz, Rohit Vartak, Ayman Ali, Best Uchehara, Xin Liu, Ishan Chatterjee, Monica Agrawal
| Challenge: | a growing number of people are seeking healthcare information from large language models via chatbots, yet the nature and inherent risks of these interactions remain unexplored. |
| Approach: | They use a curated dataset of 11K real-world conversations composed of 25K user messages to analyze user interactions across 21 health specialties. |
| Outcome: | The proposed dataset consists of 11K real-world conversations composed of 25K user messages. |
Primacy Effect of ChatGPT (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing machine learning models may lead to poor performance in discriminative natural language understanding tasks. |
| Approach: | They propose to use ChatGPT to query large amounts of human-written text to find the answer to a question. |
| Outcome: | The proposed model has a high chance to select labels at earlier positions as the answer. |
On Measuring Social Biases in Prompt-Based Multi-Task Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | a large body of work within prompt engineering attempts to understand the effects of input forms and prompts in achieving superior performance. |
| Approach: | They propose a large-scale text-to-text language model trained using prompts . they consider two different forms of semantically equivalent inputs - question-answer format and premise-hypothesis format . |
| Outcome: | The proposed model can generalize into novel forms of language and handle novel tasks. |
Can LLMs Replace Clinical Doctors? Exploring Bias in Disease Diagnosis by Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new study examines the bias of disease prediction in large language models . the model biases are prevalent across gender, age range and disease judgment behaviors . |
| Approach: | They propose a prompt-based approach to alleviate the bias in disease prediction with LLMs. |
| Outcome: | The proposed model alleviates the observed bias in disease prediction with LLMs. |
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Huang
| Challenge: | Currently, the evaluation of large language models (LLMs) such as ChatGPT in academic datasets is difficult due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. |
| Approach: | They evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in academic datasets. |
| Outcome: | The proposed model performs well on 140 tasks and generates 255K responses in these datasets. |
Do Prompt-Based Models Really Understand the Meaning of Their Prompts? (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that prompts help models to learn faster in the same way that humans learn faster when provided with task instructions expressed in natural language. |
| Approach: | They experiment with 30 prompts manually written for natural language inference (NLI) they find that models can learn just as fast with many irrelevant or pathologically misleading prompts . |
| Outcome: | The proposed model can learn as fast with irrelevant or pathologically misleading prompts as with instructively “good” prompts. |
Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study (2024.eacl-short)
Copied to clipboard
| Challenge: | pharmacovigilance event extraction is a key field of healthcare that involves identifying, evaluating, understanding, and preventing adverse effects. |
| Approach: | They investigate the ability of large language models (LLMs) to extract adverse events from medical text. |
| Outcome: | The proposed model performs reasonably well with demonstration selection strategies, but falls short compared to fully fine-tuned small models. |