When FLUE Meets FLANG: Benchmarks and Large Pretrained Language Model for Financial Domain (2022.emnlp-main)
Copied to clipboard
Raj Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, Diyi Yang
| Challenge: | Pre-trained language models have shown impressive performance on a variety of tasks and domains. |
| Approach: | They propose a domain specific financial LANGuage model which uses financial keywords and phrases for better masking. |
| Outcome: | The proposed model outperforms existing models on a variety of tasks and domains. |
Similar Papers
FinMTEB: Finance Massive Text Embedding Benchmark (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing text embedding benchmarks for financial domains are inadequately addressing the nuanced requirements of specialized domains like finance. |
| Approach: | They propose a finance-adapted embedding model that outperforms general-purpose models . they also introduce a new model, Fin-E5, which is also open-sourced . |
| Outcome: | The proposed framework outperforms general-purpose models on financial embedding tasks. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing financial PLMs are not pretrained on sufficiently diverse financial data, leading to subpar generalization performance. |
| Approach: | They propose to pretrain financial PLMs on financial corpus and train financial models on financial data. |
| Outcome: | The proposed financial language models outperform existing financial PLMs on financial tasks even for unseen corpus groups. |
CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models (2025.naacl-long)
Copied to clipboard
Ying Nie, Binwei Yan, Tianyu Guo, Hao Liu, Haoyu Wang, Wei He, Binfan Zheng, Weihao Wang, Qiang Li, Weijian Sun, Yunhe Wang, Dacheng Tao
| Challenge: | Large language models (LLMs) have achieved remarkable performance on various NLP tasks, yet their potential in more challenging task like finance, has not been fully explored. |
| Approach: | They propose a benchmark to assess the financial knowledge of large language models (LLMs) in China. |
| Outcome: | The proposed benchmark is the most comprehensive evaluation benchmark to date for LLMs in finance. |
Benchmarking Large Language Models on CFLUE - A Chinese Financial Language Understanding Evaluation Dataset (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have revolutionized natural language processing (NLP) there is an urgent need for new benchmarks to keep pace with the development of LLMs. |
| Approach: | They propose a benchmark to assess the capability of large language models (LLMs) they use a dataset to provide both knowledge assessment and application assessment . |
| Outcome: | The proposed benchmark provides datasets tailored for knowledge assessment and application assessment. |
FinMaster: A Holistic Benchmark for Full-Pipeline Financial Management with Large Language Models (2026.findings-acl)
Copied to clipboard
Junzhe Jiang, Chang Yang, Aixin Cui, Sihan Jin, Yujing Zhang, Yilin Xiao, Ruiyu Wang, Bo Li, Xiao Huang, Danny Dongning Sun, Xinrun Wang
| Challenge: | Existing benchmarks lack domain-specific data, realistic workflow-level task design, and standardized workflow- level evaluation. |
| Approach: | a new benchmark evaluates large language models on financial management workflows . the global financial services market is projected to grow to $37 trillion by 2027 . |
| Outcome: | a new benchmark for large language models on financial management workflows reveals critical capability gaps . accuracy drops from 90% on basic tasks to 40% on complex scenarios requiring multi-step reasoning . the global financial services market reached $25.8 trillion in 2022 and is projected to grow to $37 trillion by 2027 . |
FINKRX: Establishing Best Practices for Korean Financial NLP (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing tools to evaluate large language models in the financial domain are limited by the inherently closed nature of the financial industry. |
| Approach: | They present the first open leaderboard for evaluating Korean large language models focused on finance. |
| Outcome: | The proposed model is FINKRX, a fully open and transparent LLM built using these best practices. |
Financial Language Model Evaluation (FLaME) (2025.findings-acl)
Copied to clipboard
| Challenge: | Language Models (LMs) have demonstrated impressive capabilities with core NLP tasks in finance, but their effectiveness is difficult to assess due to gaps in evaluation methodologies. |
| Approach: | They propose to use a framework to evaluate language models against ‘reasoning-reinforced’ LMs to measure their performance on finance NLP tasks. |
| Outcome: | The proposed frameworks are open-source and provide data and data for the study. |
FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models (2025.naacl-long)
Copied to clipboard
Xin Guo, Haotian Xia, Zhaowei Liu, Hanyang Cao, Zhi Yang, Zhiqiang Liu, Sizhe Wang, Jinyi Niu, Chuqi Wang, Yanhui Wang, Xiaolong Liang, Xiaoming Huang, Bing Zhu, Zhongyu Wei, Yun Chen, Weining Shen, Liwen Zhang
| Challenge: | Large language models have demonstrated outstanding performance in various natural language processing tasks, but their security capabilities in the financial domain have not been explored. |
| Approach: | They propose to use a benchmark to evaluate large language models' financial domain knowledge and practical abilities. |
| Outcome: | The proposed benchmark evaluates large language models' financial domain knowledge and practical abilities. |
Is ChatGPT a Financial Expert? Evaluating Language Models on Financial Natural Language Processing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized general natural language preprocessing tasks, but their performance in financial domains is not evaluated comprehensively. |
| Approach: | They propose a framework to evaluate financial language models on financial tasks . they compare performance of auto-encoding language models and ChatGPT . |
| Outcome: | The proposed framework compares the performance of auto-encoding language models and the LLM ChatGPT on financial tasks. |