Papers by Rachneet Sachdeva
Are Emergent Abilities in Large Language Models just In-Context Learning? (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models have been claimed to acquire certain capabilities without having been specifically trained on them. |
| Approach: | They propose a theory that explains emergent abilities by taking into account their potential confounding factors and rigorously substantiate this theory through over 1000 experiments. |
| Outcome: | The proposed theory proves that emergent abilities are not truly emergental, but result from a combination of in-context learning, model memory, and linguistic knowledge. |
UKP-SQuARE v2: Explainability and Adversarial Attacks for Trustworthy QA (2022.aacl-demo)
Copied to clipboard
Rachneet Sachdeva, Haritz Puerto, Tim Baumgärtner, Sewin Tariverdian, Hao Zhang, Kexin Wang, Hossain Shaikh Saadi, Leonardo F. R. Ribeiro, Iryna Gurevych
| Challenge: | Question Answering (QA) systems rely on deep neural networks, which are difficult to interpret by humans. |
| Approach: | They propose an interpretable model that provides an explanation infrastructure for comparing models based on saliency maps and graph-based explanations. |
| Outcome: | The proposed methods can be used to compare models based on saliency maps and graph-based explanations. |
CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibration (2024.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable generalization capabilities, performing well on various tasks such as question answering (QA), complex reasoning, and code generation. |
| Approach: | They propose to augment training data of smaller language models with automatically generated counterfactuals (CF) instances to improve out-of-domain (OOD) performance of SLMs in extractive question answering setup. |
| Outcome: | The proposed approach improves out-of-domain (OOD) performance of small language models in extractive question answering setup. |
UKP-SQUARE: An Online Platform for Question Answering Research (2022.acl-demo)
Copied to clipboard
Tim Baumgärtner, Kexin Wang, Rachneet Sachdeva, Gregor Geigle, Max Eichler, Clifton Poth, Hannah Sterz, Haritz Puerto, Leonardo F. R. Ribeiro, Jonas Pfeiffer, Nils Reimers, Gözde Şahin, Iryna Gurevych
| Challenge: | Recent advances in NLP and information retrieval have given rise to a diverse set of question answering tasks that are of different formats and require different model architectures and setups. |
| Approach: | They propose an extensible online QA platform that allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
| Outcome: | The proposed tool allows users to query and analyze a large collection of modern Skills via a user-friendly web interface and integrated behavioural tests. |
UKP-SQuARE v3: A Platform for Multi-Agent QA Research (2023.acl-demo)
Copied to clipboard
Haritz Puerto, Tim Baumgärtner, Rachneet Sachdeva, Haishuo Fang, Hao Zhang, Sewin Tariverdian, Kexin Wang, Iryna Gurevych
| Challenge: | Current approaches to QA models are multi-dataset models, but combining expert agents can yield large performance gains over multi-agent models. |
| Approach: | They extend an online platform for QA research to support three families of multi-agent systems: agent selection, early-fusion of agents, and late-fusion. |
| Outcome: | The proposed model can be compared with multi-dataset models and achieve high inference speed and performance. |