ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pre-trained language models have brought the NLP field into a new era. |
| Approach: | They propose a large-scale dataset to study the chain of numerical reasoning in conversational question answering. |
| Outcome: | The proposed dataset should push forward the exploration of real-world, complex reasoning tasks as the next research focus. |
Similar Papers
FinQA: A Dataset of Numerical Reasoning over Financial Data (2021.emnlp-main)
Copied to clipboard
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, William Yang Wang
| Challenge: | Popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge and in complex multi-step numerical reasoning on that knowledge. |
| Approach: | They propose a large-scale dataset with Question-Answering pairs over financial reports written by financial experts to facilitate analytical progress. |
| Outcome: | The proposed dataset is the first of its kind and is available on github. |
Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Numerical reasoning remains a challenging area for large language models (LLMs). |
| Approach: | They propose a two-step framework to enhance LLM's capabilities in financial numerical reasoning by using a generative retriever and context-aware program of thought prompting. |
| Outcome: | The proposed model surpasses previous benchmarks with execution accuracy improvements of 5.98% and 4.05%, respectively. |
FIND: Toward Multimodal Financial Reasoning and Question Answering for Indic Languages (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for numerical reasoning in multilingual Indic languages are inadequate . e.g., FinVQA is a framework for evaluating financial numerical reasoning . |
| Approach: | They propose a framework that combines supervised fine-tuning with constraint-aware decoding to promote faithful numerical reasoning. |
| Outcome: | The proposed framework spans English, Hindi, Bengali, Marathi, Gujarati, and Tamil . it combines supervised fine-tuning with constraint-aware decoding to promote faithful numerical reasoning . |
DRIVINGVQA: A Dataset for Interleaved Visual Chain-of-Thought in Real-World Driving Scenarios (2026.findings-eacl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting is a prompting strategy that improves reasoning in large language models, but its effectiveness in vision-language models remains limited due to over-reliance on textual cues and memorized knowledge. |
| Approach: | They propose a visual question-answering dataset derived from driving theory exams that incorporates textual explanations with visual tokens extracted from entities relevant to the reasoning process. |
| Outcome: | The proposed approach outperforms chain-of-thought prompting in large language models and vision-language models in real-world scenarios. |
Facilitating Long Context Understanding via Supervised Chain-of-Thought Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have enabled them to process increasingly longer sequences, ranging from 2K to 2M tokens and even beyond. |
| Approach: | They propose a synthetic dataset in the financial domain that integrates Chain-of-Thought reasoning into LLMs in a supervised manner to facilitate effective long-context understanding. |
| Outcome: | The proposed model outperforms standard GPT-4o-mini on the Loong benchmark and fine tunes LLaMA-3.1-8B-Instruct on the model, achieving a 28.0% gain on the financial subset. |
PACIFIC: Towards Proactive Conversational Question Answering over Tabular and Textual Data in Finance (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on financial question answering systems focus on passively responding to user queries. |
| Approach: | They propose a new dataset to facilitate conversational question answering over hybrid contexts in finance . they propose PACIFIC to combine clarification question generation and CQA . |
| Outcome: | The proposed method performs multi-task learning over all sub-tasks in PACIFIC . it incorporates a simple ensemble strategy to alleviate error propagation issue . |
MarkQA: A large scale KBQA dataset with numerical reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing KBQA datasets are insufficient for numerical reasoning . existing KBqa datasets lack multi-hop reasoning and numerical reasoning. |
| Approach: | They propose a task that necessitates the ability to perform multi-hop reasoning and numerical reasoning. |
| Outcome: | The proposed task necessitates the ability to perform multi-hop reasoning and numerical reasoning. |
Evaluating LLMs’ Mathematical Reasoning in Financial Document Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel in natural language understanding, but their capability for complex mathematical reasoning with a hybrid of structured tables and unstructured text remain uncertain. |
| Approach: | They propose a prompting technique tailored to semi-structured documents that matches or outperforms baselines performance while providing a nuanced understanding of LLMs' abilities. |
| Outcome: | The proposed prompting technique outperforms baseline prompting techniques while providing a nuanced understanding of LLMs' abilities. |
PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations (2023.findings-acl)
Copied to clipboard
| Challenge: | Mars? - PragmatiCQA |
| Approach: | Mars? - The Paper . |
| Outcome: | The proposed dataset features 6873 QA pairs that explores pragmatic reasoning in conversations over a diverse set of topics. |
FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning (2026.acl-long)
Copied to clipboard
Zhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan, Hachem Madmoun, Fan Zhang, Debopriyo Banerjee, Georgi Nenkov Georgiev, Xueqing Peng, Lingfei Qian, Jimin Huang, Jinyan Su, Aaryamonvikram Singh, Rui Xing, Rania Elbadry, Chen Xu, Haonan Li, Fajri Koto, Ivan Koychev, Tanmoy Chakraborty, Yuxia Wang, Salem Lahlou, Veselin Stoyanov, Sophia Ananiadou, Preslav Nakov
| Challenge: | Existing benchmarks emphasize final numerical answers while neglecting intermediate reasoning steps. |
| Approach: | They propose a symbolic benchmark for verifiable Chain-of-Thought evaluation in finance . FINCHAIN spans 58 topics across 12 financial domains and three difficulty levels . |
| Outcome: | The proposed benchmark aims to bridge symbolic reasoning and factual verification. |