CAM 2.0: End-to-End Open Domain Comparative Question Answering System (2024.lrec-main)
Copied to clipboard
Ahmad Shallouf, Hanna Herasimchyk, Mikhail Salnikov, Rudy Alexandro Garrido Veliz, Natia Mestvirishvili, Alexander Panchenko, Chris Biemann, Irina Nikishina
| Challenge: | Comparative Question Answering is a Natural Language Processing task that combines Question Answers and Argument Mining. |
| Approach: | They propose a system for answering comparative questions called CAM 2.0 and a public leaderboard called CompUGE that unifies existing datasets under a single easy-to-use evaluation suite. |
| Outcome: | The proposed system is compared with previous web-form-based systems . it features question identification, object and aspect labeling, stance classification, summarization . the proposed system has a user-friendly interface and is available for free on the web . |
Similar Papers
CompUGE-Bench: Comparative Understanding and Generation Evaluation Benchmark for Comparative Question Answering (2025.coling-demos)
Copied to clipboard
| Challenge: | Comparative Question Answering systems help users make informed decisions by generating comparative information. |
| Approach: | They propose a comprehensive benchmark designed to evaluate Comparative Question Answering systems. |
| Outcome: | The proposed benchmark is available on HuggingFace Spaces . it unifies multiple datasets and provides a robust evaluation platform . |
Open-Domain Question Answering (2020.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering (QA) |
| Approach: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering . focus will shift to cutting- edge models proposed for open- domain QA . |
| Outcome: | The tutorial will cover cutting-edge research in open-domain question answering (QA) it will cover two-stage retriever-reader approaches, dense retriever and end-to-end training, and retriever free methods . |
End-to-End Open-Domain Question Answering with BERTserini (N19-4)
Copied to clipboard
| Challenge: | a new open-domain question answering system integrates best practices from IR with a BERT-based reader to identify answers from a large corpus of Wikipedia articles. |
| Approach: | They propose an end-to-end question answering system that integrates BERT with an IR reader. |
| Outcome: | The proposed system improves on a standard benchmark test collection. |
How to Compare Things Properly? A Study of Argument Relevance in Comparative Question Answering (2025.acl-long)
Copied to clipboard
Irina Nikishina, Saba Anwar, Nikolay Dolgov, Maria Manina, Daria Ignatenko, Artem Shelmanov, Chris Biemann
| Challenge: | Comparative Question Answering (CQA) is a task that involves processing information and diverse viewpoints. |
| Approach: | They construct a dataset of arguments annotated with their relevance and use it to answer comparative questions. |
| Outcome: | The proposed dataset contains arguments annotated with their relevance and enables precise traceability and faithfulness. |
DebateQA: Evaluating Question Answering on Debatable Knowledge (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing QA benchmarks that provide fixed answers to debatable questions are inadequate for evaluating their performance. |
| Approach: | They propose to use a dataset of 2,941 debatable questions to assess their ability to provide comprehensive answers to inherently debatably asked questions. |
| Outcome: | The proposed model performs well on 2,941 debatable questions accompanied by human-annotated partial answers that capture a variety of perspectives. |
ReQA: An Evaluation for End-to-End Answer Retrieval Models (D19-58)
Copied to clipboard
| Challenge: | Popular QA benchmarks like SQuAD have driven progress on identifying answer spans within a specific passage . retrieving relevant answers from a huge corpus of documents is still a challenging problem . |
| Approach: | They propose a benchmark for evaluating large-scale sentence-level answer retrieval models . they establish baselines using both neural encoding models and classical retrieval techniques . |
| Outcome: | The proposed model outperforms human models on identifying answer spans within a specific passage . the proposed model is scalable and can bypass the typical document retrieval step . |
UKP-SQUARE: An Online Platform for Question Answering Research (2022.acl-demo)
Copied to clipboard
Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva, Gregor Geigle, Max Eichler, Clifton Poth, Hannah Sterz, Haritz Puerto, Leonardo F. R. Ribeiro, Jonas Pfeiffer, Nils Reimers, Gözde Şahin, Iryna Gurevych
| Challenge: | Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats and require different model architectures and setups. |
| Approach: | They propose an extensible online QA platform that allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
| Outcome: | The proposed tool allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
Elvis vs. M. Jackson: Who has More Albums? Classification and Identification of Elements in Comparative Questions (2022.lrec-1)
Copied to clipboard
| Challenge: | Comparative Question Answering (cQA) is the task of providing accurate answers to questions . most question answering systems focus on answering factoid questions, but they fail at answering comparative questions in an efficient argumentative manner. |
| Approach: | They propose two new open-domain datasets for identifying and labeling comparative questions . they use a binary classification task and an unsupervised sequence labeling task . |
| Outcome: | The proposed datasets reach close-to-human results on a binary classification task with a neural model using ALBERT embeddings. |
NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets (2020.emnlp-demos)
Copied to clipboard
| Challenge: | Existing tools for Question Answering (QA) have challenges that limit their use in practice. |
| Approach: | They propose a library that integrates with existing infrastructure and offers helpful defaults for QA subtasks. |
| Outcome: | NeuralQA integrates well with existing infrastructure and offers helpful defaults for QA subtasks. |
RG-VQA: Leveraging Retriever-Generator Pipelines for Knowledge Intensive Visual Question Answering (2025.findings-emnlp)
Copied to clipboard
Settaluri Lakshmi Sravanthi, Pulkit Agarwal, Debjyoti Mondal, Rituraj Singh, Subhadarshi Panda, Ankit Mishra, Kiran Pradeep, Srihari K B, Godawari Sudhakar Rao, Pushpak Bhattacharyya
| Challenge: | Existing methods to improve the reasoning capabilities of VQA systems are limited due to complexity of graph neural networks and end-to-end training. |
| Approach: | They propose a method to integrate Dense Passage Retrievers with Vision Language Models to boost the reasoning capabilities of VQA systems. |
| Outcome: | The proposed method outperforms human accuracy and GPT-4 in the ScienceQA dataset. |