Synthesizing question answering data from financial documents: An End-to-End Multi-Agent Approach (2026.eacl-industry)
Copied to clipboard
| Challenge: | Large language models excel at financial reasoning but their deployment for enterprise use cases remains costly and often constrained by latency, privacy, and regulatory requirements. |
| Approach: | They propose a pipeline that extracts and selects relevant content from unstructured financial documents and generates QA pairs from the selected content for SLM fine-tuning. |
| Outcome: | The proposed model outperforms models trained on previous manual models and achieves competitive in-distribution performance. |
Similar Papers
A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering (2022.emnlp-demos)
Copied to clipboard
| Challenge: | Question Answering (QA) is a growing area of research . state-of-the-art QA models struggle on out-of domain documents without fine-tuning . |
| Approach: | They propose a pipeline for validating and training QA data and an interface for human annotation. |
| Outcome: | The proposed pipeline improves QA performance on domain-specific datasets while preserving the accuracy of the model. |
Fine-tuning Smaller Language Models for Question Answering over Financial Documents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research has shown that smaller language models can acquire substantial reasoning abilities when fine-tuned with reasoning exemplars crafted by a significantly larger teacher model. |
| Approach: | They propose to fine-tune several smaller model to generate programs that encode the required financial reasoning and calculations. |
| Outcome: | The proposed model outperforms the teacher model in the financial domain by adjusting the entity extraction for the specific data format. |
End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems (2020.emnlp-main)
Copied to clipboard
Siamak Shakeri, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang
| Challenge: | Existing approaches for synthetic QA data generation have limited or no success in improving the downstream Reading Comprehension task. |
| Approach: | They propose an end-to-end approach for synthetic QA data generation using a transformer-based encoder-decoder network that is trained end- to-end to generate both answers and questions. |
| Outcome: | The proposed model outperforms current state-of-the-art methods in the domain adaptation of QA models. |
David vs. Goliath: Cost-Efficient Financial QA via Cascaded Multi-Agent Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have demonstrated remarkable reasoning capabilities, but performance in FQA remains limited. |
| Approach: | They propose a low-cost yet effective framework that enables small LLMs to perform complex reasoning tasks without expensive models. |
| Outcome: | The proposed framework outperforms the best open-source model on BizBench by 10.46% and achieves competitive performance to GPT-3.5 using significantly fewer parameters. |
LLMs Cannot (Yet) Match the Specificity and Simplicity of Online Communities in Long Form Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have positioned Large Language Models (LLMs) as powerful question answering (QA) tools, shifting users away from interacting in communities towards discourse with AI-driven conversational interfaces. |
| Approach: | They propose to use a QA preference dataset to fine-tune and align Large Language Models (LLMs) from more than 7.4 million submissions and 82 million comments from 2008 to 2022 in Reddit’s 15 largest finance communities. |
| Outcome: | The proposed framework improves on the social quality of the data, and the proposed framework is more accurate and more specific. |
Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA systems do not strictly enforce cross-document synthesis or exploit the explicit inter-paper structure that links sources. |
| Approach: | They propose a pipeline methodology for constructing a multi-document academic QA dataset . they detect communities based on citation networks and leverage Large Language Models . |
| Outcome: | The proposed method generates QA pairs related to multi-document content automatically and forms coherent communities based on citation networks and large language models. |
What Factors Affect LLMs and RLLMs in Financial Question Answering? (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on large language models and reasoning large language model (RLLMs) however, there are few studies that explore what methods can fully unlock the performance of LLMs and RLLM in the financial domain. |
| Approach: | They examine the effects of prompting methods, agentic frameworks, and multilingual alignment methods on financial question-answering tasks. |
| Outcome: | The results show that prompting methods and agent frameworks improve LLMs' performance . the authors suggest that these frameworks can be used to enhance LLM performance if they are implemented in financial domains. |
SAGE: Steerable Agentic Data Generation for Deep Search with Execution Feedback (2026.findings-eacl)
Copied to clipboard
Fangyuan Xu, Rujun Han, Yanfei Chen, Zifeng Wang, I-Hung Hsu, Jun Yan, Vishy Tirumalashetty, Eunsol Choi, Tomas Pfister, Chen-Yu Lee
| Challenge: | High-quality, complex question-answer pairs are pivotal for training and evaluating capable deep search agents. |
| Approach: | They propose a pipeline that generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. |
| Outcome: | The proposed pipeline generates high-quality, difficulty-controlled deep search question-answer pairs for a given corpus and a target difficulty level. |
On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering (2024.findings-naacl)
Copied to clipboard
| Challenge: | a new study evaluates how Large Language Models interact with a SQL interpreter . the model is limited in context and is stochastic, making it less suited for tasks requiring high precision and extensive computations. |
| Approach: | They propose and evaluate two interaction strategies to evaluate how LLMs interact with a SQL interpreter. |
| Outcome: | The proposed framework improves the accuracy and reliability of the evaluations. |
FinQA: A Dataset of Numerical Reasoning over Financial Data (2021.emnlp-main)
Copied to clipboard
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, William Yang Wang
| Challenge: | Popular, large, pre-trained models fall far short of expert humans in acquiring finance knowledge and in complex multi-step numerical reasoning on that knowledge. |
| Approach: | They propose a large-scale dataset with Question-Answering pairs over financial reports written by financial experts to facilitate analytical progress. |
| Outcome: | The proposed dataset is the first of its kind and is available on github. |