Papers by Aleksandra Piktus
Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements (2022.emnlp-demos)
Copied to clipboard
Leandro Von Werra, Lewis Tunstall, Abhishek Thakur, Sasha Luccioni, Tristan Thrush, Aleksandra Piktus, Felix Marty, Nazneen Rajani, Victor Mustar, Helen Ngo
| Challenge: | Evaluation is a key part of machine learning, yet there is neo-tooling to support it . auxiliary techniques such as testing for significance, measuring statistical power, and auxiliary methods are not available in ML. |
| Approach: | They propose a set of tools to facilitate the evaluation of models and datasets in machine learning . they propose 'evaluation on the Hub' platform that enables large-scale evaluation of over 75,000 models . |
| Outcome: | The proposed tools can be used to evaluate models and datasets on the Hugging Face Hub. |
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021.tacl-1)
Copied to clipboard
Patrick Lewis, Yuxiang Wu, Linqing Liu, Pasquale Minervini, Heinrich Küttler, Aleksandra Piktus, Pontus Stenetorp, Sebastian Riedel
| Challenge: | Open-domain Question Answering models that directly leverage question-answer (QA) pairs show promise in terms of speed and memory compared with conventional models which retrieve and read from text corpora. |
| Approach: | They propose a question-answer (QA)-pair retriever to facilitate improved QA-patch models by introducing Probably Asked Questions (PAQ) they propose QA pair retriever, RePAQ, which preempts and caches test questions, enabling it to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. |
| Outcome: | The proposed model outperforms baseline models by 5% but trails RePAQ by 15% . it can be configured for size (under 500MB) or speed (over 1K questions per second) while retaining high accuracy. |
The ROOTS Search Tool: Data Transparency for LLMs (2023.acl-demo)
Copied to clipboard
Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurençon, Gérard Dupont, Sasha Luccioni, Yacine Jernite, Anna Rogers
| Challenge: | a 1.6TB multilingual text corpus is currently the largest language model . large language models are ubiquitous in modern NLP, used directly to generate text and as building blocks in downstream applications. |
| Approach: | They propose a search engine for the 1.6TB multilingual ROOTS corpus offering both fuzzy and exact search capabilities. |
| Outcome: | The ROOTS Search Tool is an open-source search engine for the 1.6TB multilingual ROOTs corpus. |
How Decoding Strategies Affect the Verifiability of Generated Text (2020.findings-emnlp)
Copied to clipboard
Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktäschel, Vassilis Plachouras, Fabrizio Silvestri, Sebastian Riedel
| Challenge: | Recent advances in pre-trained language models have generated text of an increasingly high quality. |
| Approach: | They propose a decoding strategy that produces less repetitive and more verifiable text. |
| Outcome: | The proposed method produces less repetitive and more verifiable text than previously used decoding strategies. |
Generating Fact Checking Briefs (2020.emnlp-main)
Copied to clipboard
Angela Fan, Aleksandra Piktus, Fabio Petroni, Guillaume Wenzek, Marzieh Saeidi, Andreas Vlachos, Antoine Bordes, Sebastian Riedel
| Challenge: | Existing work has framed fact checking as classification, often supported by a claim as input. |
| Approach: | They propose to use natural language briefs to increase the accuracy of fact checking . they show that QABriefer increases the accuracy by 10% while QABries reduce time . |
| Outcome: | The proposed model increases the accuracy of crowdworkers by 10% while reducing the time required by 20%. |
Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Face (2023.emnlp-demo)
Copied to clipboard
Christopher Akiki, Odunayo Ogundepo, Aleksandra Piktus, Xinyu Zhang, Akintunde Oladipo, Jimmy Lin, Martin Potthast
| Challenge: | a toolkit for reproducible information retrieval research is available for free. |
| Approach: | They present a tool that integrates Pyserini and Hugging Face to enable the seamless construction and deployment of interactive search engines. |
| Outcome: | The proposed tool makes state-of-the-art retrieval models more accessible to non-IR practitioners while minimizing deployment effort. |
KILT: a Benchmark for Knowledge Intensive Language Tasks (2021.naacl-main)
Copied to clipboard
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, Sebastian Riedel
| Challenge: | Existing models for knowledge-intensive language tasks require access to large, external knowledge sources. |
| Approach: | They propose a benchmark for knowledge-intensive language tasks (KILT) they test a shared dense vector index coupled with a seq2seq model to generate disambiguated text. |
| Outcome: | The proposed model outperforms tailor-made approaches on fact checking, open-domain question answering and dialog by generating disambiguated text. |
Domain-matched Pre-training Tasks for Dense Retrieval (2022.findings-naacl)
Copied to clipboard
Barlas Oguz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus, Xilun Chen, Sebastian Riedel, Scott Yih, Sonal Gupta, Yashar Mehdad
| Challenge: | Existing approaches to improve performance of pre-training tasks are needed. |
| Approach: | They propose to pre-train large bi-encoder models on a recently released set of 65 millionsynthetically generated questions and 200 million post-comment pairs from a preexisting reddit conversation dataset. |
| Outcome: | The proposed model can be pre-trained on a set of 65 millionsynthetically generated questions and 200 million post-comment pairs from a preexisting dataset of Reddit conversations. |
FinGPT: Large Generative Models for a Small Language (2023.emnlp-main)
Copied to clipboard
Risto Luukkonen, Ville Komulainen, Jouni Luoma, Anni Eskelinen, Jenna Kanerva, Hanna-Mari Kupari, Filip Ginter, Veronika Laippala, Niklas Muennighoff, Aleksandra Piktus, Thomas Wang, Nouamane Tazi, Teven Scao, Thomas Wolf, Osma Suominen, Samuli Sairanen, Mikko Merioksa, Jyrki Heinonen, Aija Vahtola, Samuel Antao, Sampo Pyysalo
| Challenge: | Neural language models excel in many tasks in NLP but are limited to smaller languages. |
| Approach: | They propose two approaches to pretrain large language models for Finnish . they train seven monolingual models from scratch and use Finnish as pretraining data . |
| Outcome: | The proposed model is based on a dataset of Finnish web crawls, news, social media and eBooks. |
Misspelling Oblivious Word Embeddings (N19-1)
Copied to clipboard
Aleksandra Piktus, Necati Bora Edizel, Piotr Bojanowski, Edouard Grave, Rui Ferreira, Fabrizio Silvestri
| Challenge: | Existing word embeddings have limited applicability to malformed texts . misspellings are frequent and embeddable for words that have not been observed at training time . |
| Approach: | They propose a method to learn word embeddings that are resilient to misspellings . they use FastText with subwords to train embeddables on a new dataset . |
| Outcome: | The proposed method is tested on a publicly available dataset. |
GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration (2023.acl-demo)
Copied to clipboard
Aleksandra Piktus, Odunayo Ogundepo, Christopher Akiki, Akintunde Oladipo, Xinyu Zhang, Hailey Schoelkopf, Stella Biderman, Martin Potthast, Jimmy Lin
| Challenge: | Using the mature and well-tested methods from the domain of Information Retrieval (IR) we propose to integrate Pyserini with Hugging Face to provide qualitative analysis tools for NLP researchers. |
| Approach: | They propose to integrate Pyserini with Hugging Face to provide qualitative analysis tools for NLP researchers. |
| Outcome: | The proposed tools can be integrated with the Hugging Face ecosystem of open-source AI libraries and artifacts. |