| Challenge: | Existing methods for generating high-quality distractors lack adaptability and control over difficulty levels. |
| Approach: | They propose a two-way distractor generation process to generate plausible distractors using an ensemble QA system and a multitask learning strategy to train a difficulty-controllable generation model. |
| Outcome: | The proposed method significantly outperforms GPT-4o in aligning distractor difficulty with human perception. |
Similar Papers
Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Objective questions such as fill-in-the-blank and multiple-choice require examinees to select one valid answer from a set of invalid options. |
| Approach: | They examine distractor generation tasks, datasets, methods, and evaluation metrics for English objective questions. |
| Outcome: | The proposed task is based on fill-in-the-blank and multiple choice questions and is widely utilized in educational settings across various domains and subjects. |
Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models (2024.findings-naacl)
Copied to clipboard
Wanyong Feng, Jaewook Lee, Hunter McNichols, Alexander Scarlatos, Digory Smith, Simon Woodhead, Nancy Ornelas, Andrew Lan
| Challenge: | Multiple-choice questions (MCQs) are easy to administer and grade . but crafting high-quality distractors remains labor-intensive and limited scalability . |
| Approach: | They propose to automate the generation of distractors in math MCQs by using large language models to generate distractors. |
| Outcome: | The proposed methods can generate valid distractors, but they are less adept at anticipating common errors or misconceptions among real students. |
Automatic Distractor Generation for Multiple Choice Questions in Standard Tests (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to generate distractors for multiple choice questions are expensive and time-consuming. |
| Approach: | They propose a question and answer guided distractor generation framework to automate distractors generated by domain experts. |
| Outcome: | The proposed model outperforms existing models and achieves state-of-the-art on a large-scale dataset. |
CDGP: Automatic Cloze Distractor Generation based on Pre-trained Language Model (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to generate cloze distractors with carefully-designed distractors are limited due to wrong option selection. |
| Approach: | They propose to employ pre-trained language models as an alternative to cloze distractor generation by using pre-designed distractors. |
| Outcome: | The proposed model improves the state-of-the-art cloze test score from 14.94 to 34.17 (NDCG@10) The proposed framework improves clozing distractors by incorporating pre-trained language models. |
CrossQG: Improving Difficulty-Controllable Question Generation through Consistency Enhancement (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models can generate questions with controlled difficulty, but they often fail to align with the given target difficulty. |
| Approach: | They propose a question generation method that requires no tuning of generator parameters yet significantly improves difficulty consistency. |
| Outcome: | The proposed method outperforms several mainstream methods on high-quality question answering datasets and achieves superior consistency with target difficulty. |
Distractor Generation Using Generative and Discriminative Capabilities of Transformer-based Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Multiple Choice Questions (MCQs) are used to test language learners' comprehension and knowledge. |
| Approach: | They propose an automatic distractor generation approach which generates correct and incorrect answer options and then discriminates potential correct options from distractors. |
| Outcome: | The proposed approach outperforms previous models on multiple choice questions and reading comprehension questions. |
Distractor Generation based on Text2Text Language Models with Pseudo Kullback-Leibler Divergence Regulation (2023.findings-acl)
Copied to clipboard
Hui-Juan Wang, Kai-Yu Hsieh, Han-Cheng Yu, Jui-Ching Tsou, Yu An Shih, Chen-Hua Huang, Yao-Chung Fan
| Challenge: | Existing methods for cloze-style multiple choice questions (MCQs) distractor generation are based on knowledge bases and pre-trained language models. |
| Approach: | They propose to formulate cloze distractor generation task as Text2Text task and propose a pseudo Kullback-Leibler divergence for regulating the generation to consider item discrimination index in education evaluation. |
| Outcome: | The proposed model improves state-of-the-art performance from 10.81 to 22.00 (p@1 score) |
Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting (2021.acl-long)
Copied to clipboard
| Challenge: | Existing QG systems perform substantially worse in answering multi-hop questions than single-hop ones. |
| Approach: | They propose a framework that progressively increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
| Outcome: | The proposed framework increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain. |
Fine-Tuning Encoder-Decoder Models with Contrastive Learning for In-Context Distractor Generation (2025.findings-emnlp)
Copied to clipboard
Elaf Alhazmi, Quan Z. Sheng, Wei Emma Zhang, Mohammed I. Thanoon, Haojie Zhuang, Behnaz Soltani, Munazza Zaib
| Challenge: | Distractors are used to generate plausible but incorrect options for fill-in-the-blank questions . research studies focus on fine-tuning pre-trained models with data augmentation techniques to generate distractors . |
| Approach: | They propose a model that trains the model to recognize essential semantic features necessary to generate distractors. |
| Outcome: | The proposed model outperforms existing models on two public datasets. |
DisGeM: Distractor Generation for Multiple Choice Questions with Span Masking (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Multiple-choice cloze tests are a prevalent form of assessment that evaluates students' comprehension and inference abilities. |
| Approach: | They propose a framework for distractor generation using readily available pre-trained language models . human evaluations confirm that their approach produces more effective distractors . |
| Outcome: | The proposed framework outperforms existing methods without training or fine-tuning human evaluations confirm it. |