Papers by Lucas Bandarkar
SNaRe: Domain-aware Data Generation for Low-Resource Event Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for ED struggle with label noise and domain drift when applied to specialized domains. |
| Approach: | They propose a domain-aware synthetic data generation framework composed of three components: Scout, Narrator, and Refiner. |
| Outcome: | The proposed framework outperforms baseline approaches on three diverse domain ED datasets and achieves average F1 gains of 3-7% in the zero-shot/few-shot settings and 4-20% improvement for multilingual generation. |
News Headline Grouping as a Challenging NLU Task (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in Natural Language Understanding (NLU) have seen models outperform human performance on many standard tasks. |
| Approach: | They propose a task of HeadLine Grouping and a dataset consisting of 20,056 pairs of news headlines, each labeled with a binary judgement as to whether the pair belongs within the same group. |
| Outcome: | The proposed model outperforms human models on a task consisting of 20,056 pairs of headlines on HLGD and a dataset with a binary judgement. |
Can Transformer Models Measure Coherence In Text: Re-Thinking the Shuffle Test (2021.acl-short)
Copied to clipboard
| Challenge: | Recent work shows that modern NLP models can detect shuffled text without supervision. |
| Approach: | They propose to use Shuffle Test to evaluate whether NLP models can measure coherence in text . they argue that this is unlikely to lead to a good model of text coherency . |
| Outcome: | The Shuffle Test is the most common task to evaluate whether NLP models can measure coherence in text. |
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants (2024.acl-long)
Copied to clipboard
Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, Madian Khabsa
| Challenge: | Existing benchmarks for text comprehension only cover 30 languages, but lack of labeled data is a major obstacle to building functional systems in most languages. |
| Approach: | They present a multiple-choice machine reading comprehension dataset spanning 122 languages . they use it to evaluate the capabilities of multilingual masked language models and large language models . |
| Outcome: | The proposed dataset enables the evaluation of text models in high-, medium- and low-resource languages. |