Papers by Simerjot Kaur
Translating Domain-Specific Terminology in Typologically-Diverse Languages: A Study in Tax and Financial Education (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing public terminology datasets for MT research are limited in language coverage or domain specificity, making it difficult to assess or improve MT systems in specialized settings. |
| Approach: | They propose a multilingual terminology resource for tax and financial education covering seven typologically diverse languages: English, Spanish, Russian, Vietnamese, Korean, Chinese (traditional and simplified) and Haitian Creole. |
| Outcome: | The proposed terminology resource covers seven typologically diverse languages: English, Spanish, Russian, Vietnamese, Korean, Chinese (traditional and simplified) and Haitian Creole. |
DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding (2024.acl-long)
Copied to clipboard
Dongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma, Petr Babkin, Simerjot Kaur, Yulong Pei, Armineh Nourbakhsh, Xiaomo Liu
| Challenge: | Documents with rich layouts are a significant portion of enterprise corpora and document AI is still a challenge. |
| Approach: | They propose a lightweight extension to traditional large language models for reasoning over visual documents that takes into account both textual semantics and spatial layout. |
| Outcome: | The proposed model outperforms existing large language models on 14 out of 16 datasets and generalizes well to 4 out of 5 previously unseen datasets. |
How Reliable are Confidence Estimators for Large Reasoning Models? A Systematic Benchmark on High-Stakes Domains (2026.eacl-long)
Copied to clipboard
Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur, Charese Smiley, Ivan Brugere, Kundan S Thind, Mohammad M. Ghassemi
| Challenge: | Large Reasoning Models often struggle with confidence calibration, authors say . authors: accurate confidence scores are essential to build trustworthy systems . |
| Approach: | They propose a Reasoning Model Confidence estimation benchmark to assess LRM confidence . the benchmark is constructed from 347,496 reasoning traces from six popular LRMs . |
| Outcome: | The proposed benchmark compares ten different representation-based methods on a wide range of architectures. |
FinNLI: Novel Dataset for Multi-Genre Financial Natural Language Inference Benchmarking (2025.findings-naacl)
Copied to clipboard
| Challenge: | FinNLI is a benchmark dataset for Financial Natural Language Inference (NLI) across diverse financial texts like SEC Filings, Annual Reports, and Earnings Call transcripts. |
| Approach: | They propose to use FinNLI to evaluate financial natural language inference models across diverse financial texts like SEC Filings, Annual Reports, and Earnings Call transcripts. |
| Outcome: | The proposed dataset is based on a high-quality test set of 3,304 instances annotated by finance experts. |
Calibrating LLM Confidence by Probing Perturbed Representation Stability (2025.emnlp-main)
Copied to clipboard
Reza Khanmohammadi, Erfan Miahi, Mehrsa Mardikoraem, Simerjot Kaur, Ivan Brugere, Charese Smiley, Kundan S Thind, Mohammad M. Ghassemi
| Challenge: | Despite their impressive performance, large language models (LLMs) consistently struggle with confidence calibration. |
| Approach: | They propose a method to analyze internal representational stability in large language models by applying adversarial perturbations to final hidden states and using a lightweight classifier to predict answer correctness. |
| Outcome: | CCPS significantly outperforms existing methods on LLMs from 8B to 32B parameters in multiple-choice and open-ended formats. |
Distill and Align Decomposition for Enhanced Claim Verification (2026.findings-eacl)
Copied to clipboard
Jabez Magomere, Elena Kochkina, Samuel Mensah, Simerjot Kaur, Fernando Acero, Arturo Oncevay, Charese Smiley, Xiaomo Liu, Manuela Veloso
| Challenge: | Existing methods for complex claim verification struggle to align decomposition quality with verification performance. |
| Approach: | They propose a reinforcement learning approach that optimizes decomposition quality and verifier alignment using Group Relative Policy Optimization. |
| Outcome: | The proposed method outperforms prompt-based approaches and existing methods in six evaluation settings. |
Conservative Bias in Large Language Models: Measuring Relation Predictions (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit pronounced conservative bias in relation extraction tasks, often defaulting to no_relation label when an appropriate option is unavailable. |
| Approach: | They systematically evaluate the trade-off between conservative bias and hallucination in relation extraction tasks by using SBERT and LLM prompts to quantify this effect. |
| Outcome: | The proposed model defaults to no_relation label twice as often as hallucination, resulting in significant information loss when reasoning is not explicitly included in the output. |
Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency (2024.lrec-main)
Copied to clipboard
Toyin D. Aguda, Suchetha Siddagangappa, Elena Kochkina, Simerjot Kaur, Dongsheng Wang, Charese Smiley
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance in data annotation tasks on general domain datasets, but their effectiveness on domain specific datasets remains under-explored. |
| Approach: | They compare the annotations produced by three LLMs against expert annotators and crowdworkers. |
| Outcome: | The proposed models outperform expert crowdworkers and crowd-sourced annotators on domain specific datasets. |
A Variational Approach for Mitigating Entity Bias in Relation Extraction (2025.acl-short)
Copied to clipboard
| Challenge: | Relation Extraction (RE) models often rely excessively on entities, resulting in poor generalization. |
| Approach: | They propose a Variational Information Bottleneck (VIB) framework to reduce entity bias in Relation Extraction (RE) . their method extracts relational information from unstructured data to improve generalization . |
| Outcome: | The proposed method achieves state-of-the-art on general and financial domain RE datasets, excelling in in-domain settings and out-of domain. |
Advanced Messaging Platform (AMP): Pipeline for Automated Enterprise Email Processing (2025.acl-industry)
Copied to clipboard
Simerjot Kaur, Charese Smiley, Keshav Ramani, Elena Kochkina, Mathieu Sibue, Samuel Mensah, Pietro Totis, Cecilia Tilli, Toyin Aguda, Daniel Borrajo, Manuela Veloso
| Challenge: | a lack of publicly available datasets for training and benchmarking limits current AI techniques' effectiveness in industry-specific applications. |
| Approach: | They propose an email automation pipeline that automates email response generation at scale in real-world enterprise settings. |
| Outcome: | The proposed pipeline automates email response generation at scale in real-world environments. |