Papers with RussianSuperGLUE
Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian (2020.coling-main)
Copied to clipboard
| Challenge: | MRC in other languages, including Russian, has not been well-addressed due to the lack of high-quality and large-scale datasets. |
| Approach: | They propose two Russian machine reading comprehension datasets that require reasoning over multiple sentences and commonsense knowledge to infer the answer. |
| Outcome: | The proposed datasets are more complex than the original ones for Russian . the results show that the proposed models are challenging for advanced models . |
Vygotsky Distance: Measure for Benchmark Task Similarity (2024.lrec-main)
Copied to clipboard
| Challenge: | GLUE, SuperGLUE and RussianSuperGLUE benchmarks are arbitrary sets of tasks that are not generalized. |
| Approach: | They propose a theoretical instrument and an algorithm to calculate similarity between benchmark tasks . they use relative performance of the "students" on a given task to determine similarity . |
| Outcome: | The proposed model reduces the number of evaluation tasks while maintaining high validation quality. |