Papers by Nadezhda Chirkova
Bayesian Compression for Natural Language Processing (D18-1)
Copied to clipboard
| Challenge: | In natural language processing, recurrent neural networks have a huge number of parameters. |
| Approach: | They propose a Bayesian sparsification technique which allows compressing RNNs dozens or hundreds of times without time-consuming hyperparameters tuning. |
| Outcome: | The proposed technique compresses the RNN dozens or hundreds of times without time-consuming hyperparameters tuning. |
Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies have focused on zero-shot cross-lingual transfer . mBERT, mBART and mT5 provide high-quality representations for texts in various languages . |
| Approach: | They propose to use mBART and NLLB-200 to finetune a multilingual pretrained language model on input-output pairs in one language and use it to make task predictions for inputs in other languages. |
| Outcome: | The proposed approach significantly reduces generation in the wrong language with full finetuning and can be competitive in some cases. |
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation (2024.findings-emnlp)
Copied to clipboard
David Rau, Hervé Déjean, Nadezhda Chirkova, Thibault Formal, Shuai Wang, Stéphane Clinchant, Vassilina Nikoulina
| Challenge: | Retrieval-Augmented Generation allows to enhance Large Language Models with external knowledge. |
| Approach: | They propose a library that allows to benchmark and standardize RAG experiments. |
| Outcome: | The proposed library is an end-to-end library for reproducible research standardizing RAG experiments. |
Should you marginalize over possible tokenizations? (2023.acl-short)
Copied to clipboard
| Challenge: | Autoregressive language models map token sequences to probabilities. |
| Approach: | They propose to marginalize over all tokenizations to compute the probability of a string . they find that the gap in log-likelihood is no larger than 0.5% in most cases . |
| Outcome: | The proposed algorithm shows that the gap in log-likelihood is no larger than 0.5% in most cases . but it becomes more pronounced for data with long complex words, the authors show . |
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to handle out-of-vocabulary identifiers are not suitable for source code processing. |
| Approach: | They propose a method to handle out-of-vocabulary identifiers by identifies anonymization . they show that the method significantly improves the performance of the Transformer . |
| Outcome: | The proposed method significantly improves the performance of the Transformer in two code processing tasks. |
On the Embeddings of Variables in Recurrent Neural Networks for Source Code (2021.naacl-main)
Copied to clipboard
| Challenge: | Source code processing heavily relies on the methods widely used in natural language processing (NLP) but requires specifics that need to be taken into account to achieve higher quality. |
| Approach: | They propose a recurrent mechanism that adjusts the learned semantics of a variable when it obtains more information about the variable’s role in the program. |
| Outcome: | The proposed method significantly improves the performance of the recurrent neural network, in code completion and bug fixing tasks. |