Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study (2024.eacl-short)
Copied to clipboard
| Challenge: | pharmacovigilance event extraction is a key field of healthcare that involves identifying, evaluating, understanding, and preventing adverse effects. |
| Approach: | They investigate the ability of large language models (LLMs) to extract adverse events from medical text. |
| Outcome: | The proposed model performs reasonably well with demonstration selection strategies, but falls short compared to fully fine-tuned small models. |
Similar Papers
PHEE: A Dataset for Pharmacovigilance Event Extraction from Text (2022.emnlp-main)
Copied to clipboard
Zhaoyue Sun, Jiazheng Li, Gabriele Pergola, Byron Wallace, Bino John, Nigel Greene, Joseph Kim, Yulan He
| Challenge: | Using NLP methods to discover and extract adverse drug events from unstructured textual data is difficult because it requires time-consuming manual curation. |
| Approach: | They propose to use a hierarchical event schema to extract annotated events from medical case reports and biomedical literature to analyze patient data. |
| Outcome: | The proposed dataset is the largest public dataset to date and contains over 5000 events from medical case reports and biomedical literature. |
A Survey on Detection of LLMs-Generated Content (2024.findings-emnlp)
Copied to clipboard
Xianjun Yang, Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, Wei Cheng
| Challenge: | Recent advances in large language models have led to an increase in synthetic content generation . the ability to detect LLMs-generated content has become of paramount importance . |
| Approach: | They propose to provide a detailed overview of existing detection strategies and benchmarks, scrutinizing their differences and advocating for more adaptable and robust models to enhance detection accuracy. |
| Outcome: | The proposed model will be able to detect human-written content in real time. |
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Huang
| Challenge: | Currently, the evaluation of large language models (LLMs) such as ChatGPT in academic datasets is difficult due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. |
| Approach: | They evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in academic datasets. |
| Outcome: | The proposed model performs well on 140 tasks and generates 255K responses in these datasets. |
On the Generalization of Training-based ChatGPT Detection Methods (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that training-based methods are ineffective to detect LLM generated texts from unseen tasks or topics which are not collected during training. |
| Approach: | They propose to train classification models to distinguish LLMs from human texts by a distribution shift caused by prompts, text lengths, topics, and language tasks. |
| Outcome: | The proposed methods can detect LLMs from black-box models, but they suffer from distribution shifts due to a wide range of factors, including prompts, text lengths, topics, and language tasks. |
Text-to-Text Extraction and Verbalization of Biomedical Event Graphs (2022.coling-1)
Copied to clipboard
| Challenge: | Biomedical events represent complex, graphical, and semantically rich interactions expressed in the scientific literature. |
| Approach: | They propose a framework to solve event extraction and event verbalization with a unified text-to-text approach. |
| Outcome: | The proposed framework achieves greater state-of-the-art performance than single-task competitors and can generate coherent natural language utterances from structured data. |
Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT (2023.emnlp-main)
Copied to clipboard
Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun, Ming Gu
| Challenge: | Large language models (LLMs) are capable of performing tasks but are likely to be misused. |
| Approach: | They propose a zero-shot black-box method to detect LLM-generated texts . they revise the text to be detected using the ChatGPT model . |
| Outcome: | The proposed method can detect LLM-generated texts with a zero-shot black-box model . it is based on intuition that the model will make fewer revisions to LLMs than to human-written texts . |
Ask the experts: sourcing a high-quality nutrition counseling dataset through Human-AI collaboration (2024.findings-emnlp)
Copied to clipboard
Simone Balloccu, Ehud Reiter, Karen Li, Rafael Sargsyan, Vivek Kumar, Diego Reforgiato, Daniele Riboni, Ondrej Dusek
| Challenge: | Large Language Models (LLMs) are being used by end-users for various tasks, including sensitive ones such as health counseling, disregarding potential safety concerns. |
| Approach: | They use ChatGPT to crowd-source dietary struggles and work with nutrition experts to generate supportive text using ChatGPS. |
| Outcome: | The proposed model outperforms other models on dietary struggles and mental health tasks. |
HuatuoGPT, Towards Taming Language Model to Be a Doctor (2023.findings-emnlp)
Copied to clipboard
Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Guiming Chen, Jianquan Li, Xiangbo Wu, Zhang Zhiyi, Qingying Xiao, Xiang Wan, Benyou Wang, Haizhou Li
| Challenge: | Experimental results show that the distilled language model outperforms its teacher model (ChatGPT) in most cases. |
| Approach: | They propose a Large Language Model (LLM) that leverages both distilled data from **ChatGPT** and real-world data from**doctors** in the supervised fine-tuning stage. |
| Outcome: | The proposed model outperforms the teacher model in most cases by using additional real-world data and RLMF to align the language model with the merits of both sources. |
Enhancing Adverse Drug Event Detection with Multimodal Dataset: Corpus Creation and Model Development (2024.findings-acl)
Copied to clipboard
| Challenge: | ADEs are a serious public health concern and cost healthcare systems billions of dollars . despite advancements in healthcare, ADE detection remains a significant challenge . |
| Approach: | They propose a multimodal adverse drug event detection dataset that merges ADE-related textual information with visual aids to enhance patient safety. |
| Outcome: | The proposed dataset integrates ADE-related textual information with visual aids to improve patient safety and healthcare accessibility. |
ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model Robustness (2023.emnlp-main)
Copied to clipboard
| Challenge: | generative large language models (LLMs) are replacing human workers for some tasks . crowdsourcing has several downsides: 1) the workforce is costly, 2) output quality is difficult to achieve, and 3) there are overheads related to the design and organization of the process. |
| Approach: | They investigate whether ChatGPT-created paraphrases are more diverse and robust . they use a crowdsourcing tool to collect training or validation examples . |
| Outcome: | The proposed models are more diverse and robust than the existing models. |