| Challenge: | Existing question answering datasets are imperfect tests that do not expose model limitations. |
| Approach: | They develop an adversarial writing setting where humans interact with trained models and try to break them. |
| Outcome: | The proposed model-driven annotation process systematically stumps automated question answering systems. |
Similar Papers
What Question Answering can Learn from Trivia Nerds (2020.acl-main)
Copied to clipboard
| Challenge: | a question answering dataset is a competition that has a leaderboard that determines the best answers. |
| Approach: | They propose to apply the best practices of trivia tournaments to question answering datasets . they outline key lessons that can transfer to QA research . |
| Outcome: | The proposed model is based on the best practices of trivia tournaments . the model is used to identify the best question answering teams . |
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that adversarial data collection (ADC) models perform better on other adversarially collected data but are liable under plausible domain shifts. |
| Approach: | They conduct a large-scale controlled study on question answering by assigning workers at random to compose questions either adversarially (with a model in the loop) or in the standard fashion (without a modeling). |
| Outcome: | The proposed model performs better on other adversarial datasets but worse on diverse collection of out-of-domain evaluation sets. |
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new approach to generate adversarial data is needed to improve question answering models . crowdworkers can fool a model only 8.8% of the time, compared to 17.6% for a trained model without synthetic data. |
| Approach: | They develop a pipeline that generates questions and then filters or labels them to improve quality. |
| Outcome: | The proposed approach improves state-of-the-art on a human-written adversarial dataset by 3.7F1 and improves model generalisation on nine of the twelve MRQA datasets. |
Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension (2020.tacl-1)
Copied to clipboard
| Challenge: | Innovations in annotation methodologies have been a catalyst for Reading Comprehension (RC) datasets and models. |
| Approach: | They propose to use a model-in-the-annotation-loop approach to train adversarial models in three different settings to explore reproducibility of the adversarial effect, transfer from data collected with varying model- in-the loop strengths, and generalization to data collected without a modeling model. |
| Outcome: | The proposed approach achieves 39.9F1 on questions it cannot answer when trained on SQUAD, but lower than when trained using RoBERTa itself (41.0F1). |
Bend but Don’t Break? Multi-Challenge Stress Test for QA Models (D19-58)
Copied to clipboard
| Challenge: | a gap remains in reasoning ability compared to a human, and performance tends to degrade when models are exposed to less-constrained tasks. |
| Approach: | They conduct extensive qualitative and quantitative analyses on the results of four models across four datasets . they relate common errors to model capabilities and discuss a way forward . |
| Outcome: | The proposed model performance is based on the results of four models across four datasets. |
HellaSwag: Can a Machine Really Finish Your Sentence? (P19-1)
Copied to clipboard
| Challenge: | Existing commonsense models struggle to perform inferences that are trivial for humans, but are often misclassified by state-of-the-art models. |
| Approach: | They propose a dataset that is adversarial to state-of-the-art commonsense reasoning and use it to build a model that is surprisingly robust. |
| Outcome: | The proposed dataset is compared with existing models and scaled up towards a critical 'Goldilocks zone' wherein generated text is ridiculous to humans, yet often misclassified by state-of-the-art models. |
Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attack (2024.lrec-main)
Copied to clipboard
| Challenge: | Despite the development of large language models, there are still significant challenges in detecting whether text is generated by a machine. |
| Approach: | They propose a framework for a broader class of adversarial attacks to perform minor perturbations in machine-generated content to evade detection. |
| Outcome: | The proposed framework can be compromised in as little as 10 seconds, and improves over iterative adversarial learning. |
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering (2021.eacl-main)
Copied to clipboard
| Challenge: | Question-Answering (QA) systems are deployed in the real world . a lack of research attention has been devoted to studying the issues that arise when people use QA systems. |
| Approach: | They show that component components that precede an answering engine can introduce varied and considerable sources of error. |
| Outcome: | The proposed evaluations highlight the need for QA evaluation to expand to consider real-world use. |
RobustQA: A Framework for Adversarial Text Generation Analysis on Question Answering Systems (2023.emnlp-demo)
Copied to clipboard
Yasaman Boreshban, Seyed Morteza Mirbostani, Seyedeh Fatemeh Ahmadi, Gita Shojaee, Fatemeh Kamani, Gholamreza Ghassem-Sani, Seyed Abolghasem Mirroshandel
| Challenge: | Question answering (QA) systems have reached human-level accuracy, but they are not robust enough and vulnerable to adversarial examples. |
| Approach: | They modified the attack algorithms widely used in text classification to fit them for QA systems. |
| Outcome: | The proposed framework is the first open-source toolkit for investigating textual adversarial attacks in QA systems. |
Don’t take “nswvtnvakgxpm” for an answer –The surprising vulnerability of automatic content scoring systems to adversarial input (2020.coling-main)
Copied to clipboard
| Challenge: | Automated content scoring systems can be used on short answer tasks to save human effort, but can invite cheating strategies such as writing irrelevant answers. |
| Approach: | They generate adversarial answers for benchmark content scoring datasets based on different methods of increasing sophistication and examine countermeasures such as adversarials. |
| Outcome: | The proposed methods show that even simple methods can reduce content scoring performance but do not solve the problem. |