Papers by Charese Smiley
Translating Domain-Specific Terminology in Typologically-Diverse Languages: A Study in Tax and Financial Education (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing public terminology datasets for MT research are limited in language coverage or domain specificity, making it difficult to assess or improve MT systems in specialized settings. |
| Approach: | They propose a multilingual terminology resource for tax and financial education covering seven typologically diverse languages: English, Spanish, Russian, Vietnamese, Korean, Chinese (traditional and simplified) and Haitian Creole. |
| Outcome: | The proposed terminology resource covers seven typologically diverse languages: English, Spanish, Russian, Vietnamese, Korean, Chinese (traditional and simplified) and Haitian Creole. |
AfroCS-xs: Creating a Compact, High-Quality, Human-Validated Code-Switched Dataset for African Languages (2025.acl-long)
Copied to clipboard
Kayode Olaleye, Arturo Oncevay, Mathieu Sibue, Nombuyiselo Zondi, Michelle Terblanche, Sibongile Mapikitla, Richard Lastrucci, Charese Smiley, Vukosi Marivate
| Challenge: | AfroCS-xs is a low-quality dataset for code-switching in multilingual communities . code-witching is prevalent in multicultural societies but lacks high-quality data for model development . |
| Approach: | They propose to use human-validated synthetic code-switched datasets to generate code-witched sentences for four African languages and English within a specific domain—agriculture. |
| Outcome: | The proposed model improves translation accuracy on the high-quality dataset for four African languages and English within a specific domain—agriculture. |
How Reliable are Confidence Estimators for Large Reasoning Models? A Systematic Benchmark on High-Stakes Domains (2026.eacl-long)
Copied to clipboard
Reza Khanmohammadi, Erfan Miahi, Simerjot Kaur, Charese Smiley, Ivan Brugere, Kundan S Thind, Mohammad M. Ghassemi
| Challenge: | Large Reasoning Models often struggle with confidence calibration, authors say . authors: accurate confidence scores are essential to build trustworthy systems . |
| Approach: | They propose a Reasoning Model Confidence estimation benchmark to assess LRM confidence . the benchmark is constructed from 347,496 reasoning traces from six popular LRMs . |
| Outcome: | The proposed benchmark compares ten different representation-based methods on a wide range of architectures. |
FinNLI: Novel Dataset for Multi-Genre Financial Natural Language Inference Benchmarking (2025.findings-naacl)
Copied to clipboard
| Challenge: | FinNLI is a benchmark dataset for Financial Natural Language Inference (NLI) across diverse financial texts like SEC Filings, Annual Reports, and Earnings Call transcripts. |
| Approach: | They propose to use FinNLI to evaluate financial natural language inference models across diverse financial texts like SEC Filings, Annual Reports, and Earnings Call transcripts. |
| Outcome: | The proposed dataset is based on a high-quality test set of 3,304 instances annotated by finance experts. |
When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain (2022.emnlp-main)
Copied to clipboard
Raj Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, Diyi Yang
| Challenge: | Pre-trained language models have shown impressive performance on a variety of tasks and domains. |
| Approach: | They propose a domain specific financial LANGuage model which uses financial keywords and phrases for better masking. |
| Outcome: | The proposed model outperforms existing models on a variety of tasks and domains. |
ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pre-trained language models have brought the NLP field into a new era. |
| Approach: | They propose a large-scale dataset to study the chain of numerical reasoning in conversational question answering. |
| Outcome: | The proposed dataset should push forward the exploration of real-world, complex reasoning tasks as the next research focus. |
Calibrating LLM Confidence by Probing Perturbed Representation Stability (2025.emnlp-main)
Copied to clipboard
Reza Khanmohammadi, Erfan Miahi, Mehrsa Mardikoraem, Simerjot Kaur, Ivan Brugere, Charese Smiley, Kundan S Thind, Mohammad M. Ghassemi
| Challenge: | Despite their impressive performance, large language models (LLMs) consistently struggle with confidence calibration. |
| Approach: | They propose a method to analyze internal representational stability in large language models by applying adversarial perturbations to final hidden states and using a lightweight classifier to predict answer correctness. |
| Outcome: | CCPS significantly outperforms existing methods on LLMs from 8B to 32B parameters in multiple-choice and open-ended formats. |
Distill and Align Decomposition for Enhanced Claim Verification (2026.findings-eacl)
Copied to clipboard
Jabez Magomere, Elena Kochkina, Samuel Mensah, Simerjot Kaur, Fernando Acero, Arturo Oncevay, Charese Smiley, Xiaomo Liu, Manuela Veloso
| Challenge: | Existing methods for complex claim verification struggle to align decomposition quality with verification performance. |
| Approach: | They propose a reinforcement learning approach that optimizes decomposition quality and verifier alignment using Group Relative Policy Optimization. |
| Outcome: | The proposed method outperforms prompt-based approaches and existing methods in six evaluation settings. |
Conservative Bias in Large Language Models: Measuring Relation Predictions (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit pronounced conservative bias in relation extraction tasks, often defaulting to no_relation label when an appropriate option is unavailable. |
| Approach: | They systematically evaluate the trade-off between conservative bias and hallucination in relation extraction tasks by using SBERT and LLM prompts to quantify this effect. |
| Outcome: | The proposed model defaults to no_relation label twice as often as hallucination, resulting in significant information loss when reasoning is not explicitly included in the output. |
Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency (2024.lrec-main)
Copied to clipboard
Toyin D. Aguda, Suchetha Siddagangappa, Elena Kochkina, Simerjot Kaur, Dongsheng Wang, Charese Smiley
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable performance in data annotation tasks on general domain datasets, but their effectiveness on domain specific datasets remains under-explored. |
| Approach: | They compare the annotations produced by three LLMs against expert annotators and crowdworkers. |
| Outcome: | The proposed models outperform expert crowdworkers and crowd-sourced annotators on domain specific datasets. |
The Impact of Domain-Specific Terminology on Machine Translation for Finance in European Languages (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing datasets for evaluating MT systems in this domain are limited. |
| Approach: | They propose to use a multi-parallel corpus from the European Central Bank to analyze the impact of domain-specific terminology on multilingual machine translation for finance. |
| Outcome: | The proposed method compares open-source multilingual MT systems with large language models (LLMs) that possess multilingual capabilities. |
A Variational Approach for Mitigating Entity Bias in Relation Extraction (2025.acl-short)
Copied to clipboard
| Challenge: | Relation Extraction (RE) models often rely excessively on entities, resulting in poor generalization. |
| Approach: | They propose a Variational Information Bottleneck (VIB) framework to reduce entity bias in Relation Extraction (RE) . their method extracts relational information from unstructured data to improve generalization . |
| Outcome: | The proposed method achieves state-of-the-art on general and financial domain RE datasets, excelling in in-domain settings and out-of domain. |
FinQA: A Dataset of Numerical Reasoning over Financial Data (2021.emnlp-main)
Copied to clipboard
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, William Yang Wang
| Challenge: | Popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge and in complex multi-step numerical reasoning on that knowledge. |
| Approach: | They propose a large-scale dataset with Question-Answering pairs over financial reports written by financial experts to facilitate analytical progress. |
| Outcome: | The proposed dataset is the first of its kind and is available on github. |
Advanced Messaging Platform (AMP): Pipeline for Automated Enterprise Email Processing (2025.acl-industry)
Copied to clipboard
Simerjot Kaur, Charese Smiley, Keshav Ramani, Elena Kochkina, Mathieu Sibue, Samuel Mensah, Pietro Totis, Cecilia Tilli, Toyin Aguda, Daniel Borrajo, Manuela Veloso
| Challenge: | a lack of publicly available datasets for training and benchmarking limits current AI techniques' effectiveness in industry-specific applications. |
| Approach: | They propose an email automation pipeline that automates email response generation at scale in real-world enterprise settings. |
| Outcome: | The proposed pipeline automates email response generation at scale in real-world environments. |