Generating Complex Question Decompositions in the Face of Distribution Shifts (2025.naacl-long)
Copied to clipboard
| Challenge: | Question decomposition has been found to improve large language models’ (LLMs) performance on complex question answering (QA) however, performance on the task remains dominated by supervised approaches, suggesting room for making LLMs better decomposers. |
| Approach: | They propose to generate synthetic decomposition data with only five annotated examples by extending recent advances in using LLM-as-judge and for reranking in novel ways. |
| Outcome: | The proposed approach generates synthetic decomposition data with only five examples over two benchmark datasets. |
Similar Papers
Is a Question Decomposition Unit All We Need? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LMs) have achieved state-of-the-art performance on many NLP benchmarks. |
| Approach: | They propose to decompose a hard question into simpler questions that are easier for models to answer. |
| Outcome: | The proposed approach significantly improves model performance (24% for GPT3 and 29% for RoBERTa-SQuAD along with a symbolic calculator) by decomposing a hard question into simpler questions that are easier for models to answer. |
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) provide a data-centric solution to alleviate limitations of real-world data with synthetic data generation. |
| Approach: | They propose a generic workflow for LLM-driven synthetic data generation. |
| Outcome: | The proposed workflows highlight gaps in existing research and outline avenues for future studies. |
Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to end-to-end questionanswering assume that pre-trained language can decompose complex tasks into more straightforward sub-tasks. |
| Approach: | They propose to use distant supervision to train decomposition-based transformers for large-scale parallel news. |
| Outcome: | The proposed model improves on semantic parsing and on hotpotQA and strategyQA datasets by 20% to 30%. |
Better as Generators Than Classifiers: Leveraging LLMs and Synthetic Data for Low-Resource Multilingual Classification (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable multilingual capabilities, making them promising tools in both high- and low-resource languages. |
| Approach: | They use a multilingual LLM to generate synthetic datasets covering 11 languages and 4 classification tasks and use them to train smaller models. |
| Outcome: | The proposed model outperforms the large generator in low-resource languages and tasks. |
Unsupervised Question Decomposition for Question Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing QA systems struggle to answer complex questions because information is scattered in different places. |
| Approach: | They propose an unsupervised algorithm that decomposes hard questions into simpler sub-questions . they propose an algorithm that can be used to generate a final answer from millions of questions . |
| Outcome: | The proposed algorithm decomposes hard questions into simpler sub-questions that existing QA systems can answer. |
When Do Decompositions Help for Machine Reading? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on decompositions of complex questions has focused on multi-step reasoning . but, in machine reading, it is unclear when decomposing is helpful . |
| Approach: | They conduct experiments on decompositions in machine reading to unify recent work . they find that decomposing complex questions can be helpful in zero or limited-data settings . |
| Outcome: | The proposed model can learn decompositions implicitly even with limited data, the study shows . the results are consistent with previous work on decomposing complex questions . |
Chain-of-Question: A Progressive Question Decomposition Approach for Complex Knowledge Base Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to answer complex questions rely on decomposition of complex questions into sub-questions . Existing approaches to decompose complex questions are limited by the original question . |
| Approach: | They propose a question decomposition approach to decompose semantically clear questions . they use the decomposed sub-questions to select relevant patterns as auxiliary information . |
| Outcome: | The proposed method achieves state-of-the-art performance on multiple datasets. |
Small Language Models Fine-tuned to Coordinate Larger Language Models improve Complex Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent attempts at prompt decomposition toward solving complex, multi-step reasoning problems depend on the ability of the LLM to simultaneously decompose and solve the problem. |
| Approach: | They propose a decomposition generator that decomposes complex problems into subproblems that require fewer reasoning steps. |
| Outcome: | The proposed method can produce competitive or even better performance compared to its larger successor, GPT-4. |
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have great potential for synthetic data generation. |
| Approach: | They show that large language models can generate useful data even for complex tasks . they use a symmetric task difficulty asymmetry to prompt an LLM to generate plausible input text for a target output structure. |
| Outcome: | The proposed approach outperforms existing models by a substantial margin on closed information extraction tasks with 1.8M data points and 770M parameters. |
Question Decomposition for Retrieval-Augmented Generation (2025.acl-srw)
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is effective for question answering tasks . multi-hop questions, such as "Which company among NVIDIA, Apple, and Google made the biggest profit in 2023?" challenge RAG because relevant facts are often distributed across multiple documents . |
| Approach: | They propose a pipeline that incorporates question decomposition to ground large language models in verifiable external sources. |
| Outcome: | The proposed approach improves retrieval and answer accuracy over standard RAG . multi-hop questions often require multiple documents to support the model . |