Papers by Nitika Mathur
Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for judging metrics are sensitive to the translations used for evaluation, leading to falsely confident conclusions about a metric’s efficacy. |
| Approach: | They propose a method for thresholding performance improvement under an automatic metric against human judgements by using a pairwise system ranking method. |
| Outcome: | The proposed method allows quantification of type I versus type II errors incurred, i.e., insignificant human differences in system quality that are accepted, and significant human differences that are rejected. |
Putting Evaluation in Context: Contextual Embeddings Improve Machine Translation Evaluation (P19-1)
Copied to clipboard
| Challenge: | Existing evaluation metrics are limited and can be easily portable to new languages. |
| Approach: | They propose a simple unsupervised metric and additional supervised metrics which rely on contextual word embeddings to encode the translation and reference sentences. |
| Outcome: | The proposed model outperforms existing metrics on the WMT 2017 dataset and is more accurate than existing models. |
Taming the Real-world Complexities in CPT E/M Coding with Large Language Models (2025.emnlp-industry)
Copied to clipboard
Islam Nassar, Yang Lin, Yuan Jin, Rongxin Zhu, Chang Wei Tan, Zenan Zhai, Nitika Mathur, Thanh Tien Vu, Xu Zhong, Long Duong, Yuan-Fang Li
| Challenge: | Evaluation and Management (E/M) coding is performed by physicians and trained human coders who review clinical encounter notes and electronic health record data to assign appropriate codes. |
| Approach: | They propose a framework that automates evaluation and management coding tasks using the Current Procedural Terminology (CPT) taxonomy. |
| Outcome: | The proposed framework achieves an increase in coding accuracy of more than 36% over a commercial CPT E/M coding system and almost 5% over our strongest single-prompt baseline. |