Taming the Real-world Complexities in CPT E/M Coding with Large Language Models (2025.emnlp-industry)
Copied to clipboard
Islam Nassar, Yang Lin, Yuan Jin, Rongxin Zhu, Chang Wei Tan, Zenan Zhai, Nitika Mathur, Thanh Tien Vu, Xu Zhong, Long Duong, Yuan-Fang Li
| Challenge: | Evaluation and Management (E/M) coding is performed by physicians and trained human coders who review clinical encounter notes and electronic health record data to assign appropriate codes. |
| Approach: | They propose a framework that automates evaluation and management coding tasks using the Current Procedural Terminology (CPT) taxonomy. |
| Outcome: | The proposed framework achieves an increase in coding accuracy of more than 36% over a commercial CPT E/M coding system and almost 5% over our strongest single-prompt baseline. |
Similar Papers
Toward Reliable Clinical Coding with Language Models: Verification and Lightweight Adaptation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for clinical code verification fail to account for hierarchical misalignments . standardized coding systems such as ICD-10-CM1 ensure consistency across medical records. |
| Approach: | They propose to use prompt engineering and small-scale fine-tuning to improve accuracy without the computational overhead of search-based methods. |
| Outcome: | The proposed task is a standalone task and a pipeline component to address hierarchical near-miss errors without the computational overhead of search-based methods. |
Navigating the Modern Evaluation Landscape: Considerations in Benchmarks and Frameworks for Large Language Models (LLMs) (2024.lrec-tutorials)
Copied to clipboard
| Challenge: | General-purpose Language Models have changed the world of Natural Language Processing, if not the world itself. |
| Approach: | This tutorial will lay the foundations and explain the basics of evaluation and compare traditional methods to newly developed methods. |
| Outcome: | The tutorial assumes little familiarity with metrics, datasets, prompts and benchmarks . it will compare traditional methods to newly developed methods . |
LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validation (2025.findings-emnlp)
Copied to clipboard
Ming Zhang, Yujiong Shen, Zelin Li, Huayu Sha, Binze Hu, Yuhui Wang, Chenhao Huang, Shichun Liu, Jingqi Tong, Changhao Jiang, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Current medical benchmarks have limitations in question design, data sources and evaluation methods. |
| Approach: | They propose a new benchmark covering five core medical areas . it includes 2,996 questions created from real-world electronic health records . |
| Outcome: | The proposed model covers five core medical areas and includes 2,996 questions created from real-world electronic health records and expert-designed clinical scenarios. |
Measuring What Matters: Evaluating Ensemble LLMs with Label Refinement in Inductive Coding (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are prone to inconsistencies and individual biases, limiting their reliability. |
| Approach: | They propose a framework that combines ensemble methods with code refinement methodology to address these challenges. |
| Outcome: | The proposed framework outperforms large language models and LLMs with a low-rank averaging and a moderator-based mechanism to simulate human consensus. |
From Scores to Steps: Diagnosing and Improving LLM Performance in Evidence-Based Medical Calculations (2025.emnlp-main)
Copied to clipboard
Benlu Wang, Iris Xia, Yifan Zhang, Junda Wang, null Feiyun Ouyang, Shuo Han, Arman Cohan, Hong Yu, Zonghai Yao
| Challenge: | Existing benchmarks assess only the final answer with a wide numerical tolerance, overlooking systematic reasoning failures and potentially causing serious clinical misjudgments. |
| Approach: | They propose a new step-by-step evaluation pipeline that assesses formula selection, entity extraction, and arithmetic computation. |
| Outcome: | The proposed method improves the accuracy of large language models on medical benchmarks from 16.35% to 53.19%. |
CPTCoder: A Reliable LLM System for Medical Procedure Code Prediction (2026.acl-demo)
Copied to clipboard
| Challenge: | Clinical procedure coding is an extreme multi-label classification problem . CPTCoder predicts standardized medical procedure codes from clinical text . |
| Approach: | a new human-in-the-loop system predicts standardized medical procedure codes from clinical text. |
| Outcome: | CPTCoder outperforms baseline system by 12 and 5 points in a clinical procedure classification problem. |
From Tools to Teammates: Evaluating LLMs in Multi-Session Coding Interactions (2025.acl-long)
Copied to clipboard
Nathanaël Carraz Rakotonirina, Mohammed Hamdy, Jon Ander Campos, Lucas Weber, Alberto Testoni, Marzieh Fadaee, Sandro Pezzelle, Marco Del Tredici
| Challenge: | Large Language Models excel at solving individual problems in isolation, but are they able to effectively collaborate over long-term interactions? |
| Approach: | They propose to use a multi-session dataset to test LLMs' ability to track and execute simple coding instructions amid irrelevant information, simulating a realistic setting. |
| Outcome: | The proposed model performs poorly when instructions are spread across sessions, suggesting that they are not able to integrate information over long interactions. |
Text Annotation via Inductive Coding: Comparing Human Experts to LLMs in Qualitative Data Analysis (2025.findings-naacl)
Copied to clipboard
| Challenge: | Qualitative data analysis (QDA) is an important research method across fields such as marketing, media studies, social science, psychology, medical research, and others. |
| Approach: | They evaluate the performance of open-source LLMs by comparing them to human experts. |
| Outcome: | The proposed method is based on inductive coding using large language models. |
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations (2024.emnlp-main)
Copied to clipboard
Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdullah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, Jimmy Huang
| Challenge: | Large Language Models (LLMs) have gained significant attention due to their capabilities in performing diverse tasks across domains. |
| Approach: | They review the primary challenges and limitations causing inconsistencies in evaluations . early models could generate coherent text but limited to simple tasks . |
| Outcome: | The proposed evaluations are reproducible, reliable, and robust. |
CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios (2024.emnlp-main)
Copied to clipboard
| Challenge: | Chinese medical large language models (LLMs) are underperforming on this benchmark, especially where medical reasoning and factual consistency are vital. |
| Approach: | They propose a benchmark with 14 expert-guided clinical scenarios to assess the medical ability of large language models across 7 pivot dimensions. |
| Outcome: | The proposed benchmark has been validated in several ways. |