Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Progress in the task of Critical Questions Generation has been hindered by the lack of suitable datasets and automatic evaluation standards. |
| Approach: | They propose a comprehensive approach to support the development and benchmarking of systems for this task. |
| Outcome: | The proposed approach supports the development and benchmarking of systems for this task. |
Similar Papers
Automatic Multiple-Choice Question Generation and Evaluation Systems Based on LLM: A Study Case With University Resolutions (2025.coling-main)
Copied to clipboard
Sérgio Silva Mucciaccia, Thiago Meireles Paixão, Filipe Wall Mutz, Claudine Santos Badue, Alberto Ferreira de Souza, Thiago Oliveira-Santos
| Challenge: | Multiple choice questions (MCQs) are often used in employee selection and training, but their creation is resource-intensive and requires significant effort and investment. |
| Approach: | They propose to use large language models and prompt engineering techniques to automate the generation and validation of MCQs. |
| Outcome: | The proposed system reduces the burden on human resources and enables scalable, cost-effective MCQ generation. |
KHANQ: A Dataset for Generating Deep Questions in Education (2022.coling-1)
Copied to clipboard
| Challenge: | Existing QG datasets are not suitable for educational question generation because the questions are not real questions asked by humans during learning. |
| Approach: | They propose a dataset for question generation that contains 1,034 high-quality learner-generated questions seeking an in-depth understanding of the taught online courses in Khan Academy. |
| Outcome: | The proposed dataset contains 1,034 high-quality learner-generated questions seeking an in-depth understanding of the taught online courses in Khan Academy. |
Socratic Question Generation: A Novel Dataset, Models, and Evaluation (2023.eacl-main)
Copied to clipboard
| Challenge: | Socratic questioning is a form of reflective inquiry often employed in education to encourage critical thinking in students. |
| Approach: | They present a first large dataset of 110K questions, context pairs for Socratic Question Generation. |
| Outcome: | The proposed model produces realistic, type-sensitive, human-like Socratic questions . authors show that the model can be used in counseling and coaching . |
Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting (2021.acl-long)
Copied to clipboard
| Challenge: | Existing QG systems perform substantially worse in answering multi-hop questions than single-hop ones. |
| Approach: | They propose a framework that progressively increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
| Outcome: | The proposed framework increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
Meta-CQG: A Meta-Learning Framework for Complex Question Generation over Knowledge Bases (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods train one encoder-decoder-based model to fit all questions . however, such a one-size-fits-all strategy may not perform well for complex questions involving multiple KB relations or functional constraints. |
| Approach: | They propose a meta-learning framework for complex question generation over knowledge bases . they propose he meta-trained generator can acquire universal meta-knowledge . |
| Outcome: | The proposed framework can acquire universal and transferable meta-knowledge and quickly adapt to long-tailed samples under different dimensions. |
Ask To The Point: Open-Domain Entity-Centric Question Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | *entity-centric question generation (ECQG) is a task motivated by real-world applications such as topic-specific learning, assisted reading, and fact-checking. |
| Approach: | They propose a PLM-based framework GenCONE with two modules: content focusing and question verification. |
| Outcome: | The proposed framework outperforms baselines and is effective and complementary in generating high-quality questions. |
Benchmarking Large Language Model Capabilities for Conditional Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Autoregressive and pre-trained large language models have shifted the field from application-specific to generation-based approaches. |
| Approach: | They propose to adapt existing application-specific generation benchmarks to pre-trained large language models to better suit different tasks. |
| Outcome: | The proposed models differ in their applicability to different data regimes and their generalization to multiple languages. |
Agenda-Driven Question Generation: A Case Study in the Courtroom Domain (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing automated question generation methods focus on unstructured text and lack agenda and background documents as context. |
| Approach: | They propose to leverage large language models for CourtQG by fine-tuning them on two auxiliary tasks, agenda explanation and question type prediction. |
| Outcome: | The proposed method generates better questions according to standard metrics when compared to several baselines. |
ParaQG: A System for Generating Questions and Answers from Paragraphs (D19-3)
Copied to clipboard
| Challenge: | Automated question generation systems generate questions from sentences and paragraphs . manual generation of questions is labour-intensive as it requires reading, parsing and understanding of long passages of text. |
| Approach: | They propose a web-based system for generating questions from sentences and paragraphs . paraQG provides an interactive interface for a user to select answers with visual insights . |
| Outcome: | The proposed system generates questions from sentences and paragraphs on a web-based platform. |
LLM DEBATE OPPONENT : Counter-argument Generation focusing on Implicit and Critical Premises (2025.naacl-srw)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) show promise in automating counter-argument generation. |
| Approach: | They compare multi-step and one-step generation methods for counter-arguments across 100 debate topics. |
| Outcome: | The proposed model outperforms multi-step and one-step pipelines for counter-arguments across 100 debate topics. |