Papers by Svitlana Vakulenko
Open-Domain Question Answering Goes Conversational via Question Rewriting (2021.naacl-main)
Copied to clipboard
Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, Srinivas Chappidi
| Challenge: | Existing large-scale benchmarks for conversational QA limit the topic of conversation to the content of a single document. |
| Approach: | They propose a dataset for Question Rewriting in Conversational Context (QReCC) the dataset contains 14K conversations with 80K question-answer pairs. |
| Outcome: | The proposed approach shows that the first baseline for the QReCC dataset is 19.10, compared to the human upper bound of 75.45, indicating the difficulty of the setup and a large room for improvement. |
Neural Ranking with Weak Supervision for Open-Domain Question Answering : A Survey (2023.findings-eacl)
Copied to clipboard
| Challenge: | Neural ranking models require substantial amounts of relevance annotations, which is costly to scale. |
| Approach: | They propose to train a NR model with weak supervision instead of annotations . they use a structured overview of standard WS signals used for training a model . |
| Outcome: | The proposed approach reduces the cost of annotations by using weak supervision instead of a parametric model. |
Robustness Evaluation of Entity Disambiguation Using Prior Probes: the Case of Entity Overshadowing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Entity disambiguation (ED) is the last step of entity linking when candidate entities are reranked according to the context they appear in. |
| Approach: | They propose a dataset that includes 16K short text snippets annotated with entity mentions to evaluate EL models. |
| Outcome: | The proposed dataset shows that the performance of EL systems is overestimated . the results show that the EL system performance is significantly better on the ShadowLink benchmark . |
Retrieving Contextual Information for Long-Form Question Answering using Weak Supervision (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrievers for long-form question answering are optimized for information that directly targets the question, missing out on contextual information. |
| Approach: | They propose to use weak supervision techniques to optimize retrieval for contextual information. |
| Outcome: | The proposed techniques improve the end-to-end QA performance on a conversational QA dataset. |
SCAI-QReCC Shared Task on Conversational Question Answering (2022.lrec-1)
Copied to clipboard
| Challenge: | evaluating systems for conversational QA remains an open research problem in its own right . evaluating (conversational) QA systems remains an important challenge for developing conversational information retrieval (conversional search) systems. |
| Approach: | They propose to use a conversational question answering task to extend the original conversational QA dataset with alternative correct answers produced by participant systems. |
| Outcome: | The proposed task was based on the SCAI-QReCC 2021 shared task on conversational question answering. |