AnnoHID: LLM-Assisted Annotation Framework for Low-Resource Medical Texts (2026.acl-demo)
Copied to clipboard
Annisa Maulida Ningtyas, Guntur Budi Herwanto, Yunita Sari, Rifki Afina Putri, Filip Kovacevic, Alaa El-Ebshihy, Varvara Arzt, Florina Piroi
| Challenge: | Social media platforms are a popular way to communicate with medical experts and improve health literacy. |
| Approach: | They introduce a semi-automated annotation framework for medical texts in low-resource languages . they use large language models for pre-annotation and human validation to support efficient annotation . |
| Outcome: | The proposed framework is applied to medical social media texts in Bahasa Indonesia . it yields higher inter-annotator agreement and human review improves output . future work focuses on mitigating pre-annotation bias and reducing annotation overhead . |
Similar Papers
Exploring LLM Annotation for Adaptation of Clinical Information Extraction Models under Data-sharing Restrictions (2025.findings-acl)
Copied to clipboard
| Challenge: | In-hospital text data often contains valuable clinical information, yet fine-tuned small language models (SLMs) for information extraction remain challenging due to differences in formatting and vocabulary across institutions. |
| Approach: | They leverage large language models to annotate the target domain data for adaptation . they use in-hospital text data to extract clinical information . |
| Outcome: | The proposed model outperforms manual annotation on four clinical information extraction tasks with a larger number of annotated data. |
Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models generate human-like text, making them unreliable for biomedical relation extraction tasks. |
| Approach: | They propose to use Large Language Models as judges to evaluate biomedical relation extraction . they propose structured output formatting for LLM-generated responses that helps LLMs improve their performance by 15%. |
| Outcome: | The proposed method improves LLM-Judges' performance by 15% . it is cheaper and more efficient than human evaluation metrics, the authors say . |
MEGAnno+: A Human-LLM Collaborative Annotation System (2024.eacl-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) can label data faster and cheaper than humans . however, they may fall short in understanding of complex contexts, leading to incorrect labels . |
| Approach: | They propose a collaborative approach where humans and LLMs work together to produce reliable labels. |
| Outcome: | The proposed system can produce reliable and high-quality labels faster and cheaper than humans . compared to traditional models, it can generate labels faster, at a lower cost . |
LLM as Entity Disambiguator for Biomedical Entity-Linking (2025.acl-short)
Copied to clipboard
| Challenge: | Entity linking involves normalizing a mention in medical text to a unique identifier in a knowledge base, such as UMLS or MeSH. |
| Approach: | They propose to use a large language model as an entity disambiguator to enhance the accuracy of alias-matching entity linking methods. |
| Outcome: | The proposed method surpasses existing methods on biomedical datasets by up to 16 points in accuracy. |
Pushing the Limits of Low-Resource NER Using LLM Artificial Data Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is an important task, but it requires a large amount of labeled data to perform well. |
| Approach: | They propose to use open-source Large Language Models to generate NER data with only a few labeled examples, reducing the cost of human annotations. |
| Outcome: | The proposed method significantly improves the baseline on diverse low-resource NER datasets and can be used to augment datasets with class-imbalance problems. |
An LLM-based Framework for Biomedical Terminology Normalization in Social Media via Multi-Agent Collaboration (2025.coling-main)
Copied to clipboard
| Challenge: | Experimental results indicate that our approach exhibits competitive performance. |
| Approach: | They propose a tuning-free approach to normalize non-standard terms using large language models . they use a search engine and a domain knowledge base to expand the short texts into accurate descriptions . |
| Outcome: | The proposed approach is based on the "Recall and Re-rank" framework . it can be used to identify the standard term in a specified termbase for non-standardized mentions . |
Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Framework (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that large language models (LLMs) can perform in named entity recognition tasks. |
| Approach: | They propose a framework for clinical named entity recognition that decomposes the entity recognition task into several retrievals of sub-types and then filters them. |
| Outcome: | The proposed framework improves on the clinical named entity recognition task. |
Can Synthetic Text Help Clinical Named Entity Recognition? A Study of Electronic Health Records in French (2023.eacl-main)
Copied to clipboard
| Challenge: | In sensitive domains, the sharing of corpora is restricted due to confidentiality, copyrights or trade secrets. |
| Approach: | They use auto-regressive neural models to generate a clinical case corpus annotated with clinical entities and evaluate it for a named entity recognition task. |
| Outcome: | The proposed model can produce clinical case corpus annotated with clinical entities while maintaining confidentiality. |
Towards a Versatile Medical-Annotation Guideline Feasible Without Heavy Medical Knowledge: Starting From Critical Lung Diseases (2020.lrec-1)
Copied to clipboard
| Challenge: | Current annotation policies for medical corpora are not standardized across clinical texts of different types. |
| Approach: | They propose to annotate medical records of various types using a named entity recognition (NER) task. |
| Outcome: | The proposed annotation scheme is applicable to large-scale clinical NLP projects. |
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit remarkable adaptability across domains, but they are often not suitable for structured knowledge extraction tasks such as named entity recognition (NER). |
| Approach: | They propose a method that instructs LLMs to self-reflect on the specific domain and generates domain-relevant attributes for creating attribute-rich training data. |
| Outcome: | The proposed method produces NER datasets in domains with domain-relevant attributes and generates entity terms and NER context data around these entities. |