Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integration (2024.findings-acl)
Copied to clipboard
Han Cheng Yu, Yu An Shih, Kin Man Law, KaiYu Hsieh, Yu Chen Cheng, Hsin Chih Ho, Zih An Lin, Wen-Chuan Hsu, Yao-Chung Fan
| Challenge: | Existing LMs undergo task-agnostic pertaining, but task-specific pretraining has gained prominence. |
| Approach: | They propose retrieval augmented pretraining and task-specific pretraining for DG . they propose to refine language model pretraining to align it more closely with downstream task . |
| Outcome: | The proposed method improves the performance of multiple-choice questions by integrating knowledge graphs and language models. |
Similar Papers
Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Objective questions such as fill-in-the-blank and multiple-choice require examinees to select one valid answer from a set of invalid options. |
| Approach: | They examine distractor generation tasks, datasets, methods, and evaluation metrics for English objective questions. |
| Outcome: | The proposed task is based on fill-in-the-blank and multiple choice questions and is widely utilized in educational settings across various domains and subjects. |
Automatic Distractor Generation for Multiple Choice Questions in Standard Tests (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to generate distractors for multiple choice questions are expensive and time-consuming. |
| Approach: | They propose a question and answer guided distractor generation framework to automate distractors generated by domain experts. |
| Outcome: | The proposed model outperforms existing models and achieves state-of-the-art on a large-scale dataset. |
DisGeM: Distractor Generation for Multiple Choice Questions with Span Masking (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Multiple-choice cloze tests are a prevalent form of assessment that evaluates students' comprehension and inference abilities. |
| Approach: | They propose a framework for distractor generation using readily available pre-trained language models . human evaluations confirm that their approach produces more effective distractors . |
| Outcome: | The proposed framework outperforms existing methods without training or fine-tuning human evaluations confirm it. |
Can We Learn Question, Answer, and Distractors All from an Image? A New Task for Multiple-choice Visual Question Answering (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies focus on generating QADs from image and question, but a novel task is needed to generate meaningful questions, correct answers, and challenging distractors. |
| Approach: | They propose a task to generate QADs from images and encode images together . they use contrastive learning to ensure consistency of QAD generated and tested . |
| Outcome: | Empirical evaluations on the benchmark dataset validate the performance of the proposed task. |
Fine-Tuning Encoder-Decoder Models with Contrastive Learning for In-Context Distractor Generation (2025.findings-emnlp)
Copied to clipboard
Elaf Alhazmi, Quan Z. Sheng, Wei Emma Zhang, Mohammed I. Thanoon, Haojie Zhuang, Behnaz Soltani, Munazza Zaib
| Challenge: | Distractors are used to generate plausible but incorrect options for fill-in-the-blank questions . research studies focus on fine-tuning pre-trained models with data augmentation techniques to generate distractors . |
| Approach: | They propose a model that trains the model to recognize essential semantic features necessary to generate distractors. |
| Outcome: | The proposed model outperforms existing models on two public datasets. |
Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models (2024.findings-naacl)
Copied to clipboard
Wanyong Feng, Jaewook Lee, Hunter McNichols, Alexander Scarlatos, Digory Smith, Simon Woodhead, Nancy Ornelas, Andrew Lan
| Challenge: | Multiple-choice questions (MCQs) are easy to administer and grade . but crafting high-quality distractors remains labor-intensive and limited scalability . |
| Approach: | They propose to automate the generation of distractors in math MCQs by using large language models to generate distractors. |
| Outcome: | The proposed methods can generate valid distractors, but they are less adept at anticipating common errors or misconceptions among real students. |
Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Retrieval-augmented large language models excel in various NLP tasks but are not always helpful when the knowledge required is absent in the model. |
| Approach: | They propose to determine whether the model is knowledgeable on a query via inspecting the (contextualized) pre-trained token embeddings of LLMs. |
| Outcome: | Experiments show that the proposed approach performs better than previous approaches on various benchmarks. |
Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction (2025.acl-long)
Copied to clipboard
| Challenge: | Multiple-choice questions (MCQs) are critical for identifying misconceptions and gaps in knowledge and accurately assessing students' understanding. |
| Approach: | They propose to train a model to generate distractors that are more likely to be selected by students by a pairwise ranker and a distractor generator via Direct Preference Optimization. |
| Outcome: | The proposed model outperforms baseline models and performs comparable to humans in various metrics including pairwise rank accuracy and distractor plausibility. |
Distractor Generation Using Generative and Discriminative Capabilities of Transformer-based Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Multiple Choice Questions (MCQs) are used to test language learners' comprehension and knowledge. |
| Approach: | They propose an automatic distractor generation approach which generates correct and incorrect answer options and then discriminates potential correct options from distractors. |
| Outcome: | The proposed approach outperforms previous models on multiple choice questions and reading comprehension questions. |
RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing graph-based or hybrid systems lack the ability to integrate supplementary evidence as reasoning unfolds. |
| Approach: | They propose a framework that integrates non-parametric knowledge into Large Language Models . they use a RL-based framework to optimize the entire generation process via RL . |
| Outcome: | The proposed framework outperforms existing RAG frameworks in five question answering benchmarks. |