Papers by Dzmitry Bahdanau
MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a knowledge cutoff and are costly to finetune repeatedly. |
| Approach: | They introduce a language evaluation suite that incorporates diverse tokens and prompt settings to simulate real-world complexity. |
| Outcome: | The proposed evaluation suite incorporates diverse tokens and prompt settings to simulate real-world complexity. |
PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work often tackles the problem of text classification when there is a limited amount of training data. |
| Approach: | They propose a method to generate more helpful augmented data by utilizing the LLM's ability to follow instructions and perform few-shot classifications. |
| Outcome: | The proposed method generates more helpful examples near class boundaries, but generating borderline examples increases the risk of false positives in the dataset. |
LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Semantic parsers struggle to generalize to examples with unseen combinations of seen rules from the training set. |
| Approach: | They propose a general framework to produce semantic parses by predicting node labels for a complete multi-layer input-aligned graph. |
| Outcome: | The proposed framework produces better generalizations than the baseline framework . it produces representations directly as a graph and not as sequences . |
DuoRAT: Towards Simpler Text-to-SQL Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent text-to-SQL models can translate natural language questions to corresponding SQL queries on unseen databases. |
| Approach: | They propose a re-implementation of the RAT-SQL model that uses only relation-aware or vanilla transformers as the building blocks. |
| Outcome: | The proposed model is based on the spider dataset and shows it can be used on large databases without human intervention. |
Compositional Generalization in Dependency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Compositionality is the ability to combine familiar units like words into novel phrases and sentences. |
| Approach: | They introduce a set of dependency parses for Compositional Freebase Queries (CFQ) they analyze the behaviour of a state-of-the-art dependency parser on the CFQ dataset . |
| Outcome: | The proposed dependency parser performs lower on the most challenging splits with the highest compound divergence. |
Understanding by Understanding Not: Modeling Negation in Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | Negation is a core construction in natural language, but state-of-the-art pre-trained language models often handle it incorrectly. |
| Approach: | They propose to augment language modeling objective with unlikelihood objective based on negated generic sentences from a raw text corpus. |
| Outcome: | The proposed approach reduces the top 1 error rate to 4% on negated LAMA dataset and improves on negating NLI benchmarks. |
PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large pre-trained language models for textual data have an unconstrained output space . when fine-tuned to target constrained formal languages like SQL, these models often generate invalid code, rendering it unusable. |
| Approach: | They propose a method for constraining auto-regressive decoders of language models through incremental parsing. |
| Outcome: | The proposed method can find valid output sequences by rejecting inadmissible tokens . it can be used on Spider and CoSQL text-to-SQl translation tasks . |
Evaluating In-Context Learning of Libraries for Code Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent work shows that large proprietary LLMs can learn novel library usage in-context from demonstrations. |
| Approach: | They evaluate large proprietary LLMs to understand library usage in-context . they find they are able to generate code based on library specification presented in-constext - a promising area . |
| Outcome: | The proposed models can learn library usage in-context from demonstrations . the results pave the way for more adaptable and dynamic coding environments. |