Papers by Anastasia Shimorina
PoSum-Bench: Benchmarking Position Bias in LLM-based Conversational Summarization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit positional bias, a problem that can undermine the completeness of conversation summarizations. |
| Approach: | They propose a semantic similarity-based sentence-level metric to quantify positional bias in conversational summaries. |
| Outcome: | The proposed benchmark provides the first systematic evaluation of positional bias in conversational summarization across languages and contexts. |
An Error Analysis Framework for Shallow Surface Realization (2021.tacl-1)
Copied to clipboard
| Challenge: | BLEU and METEOR metrics fail to provide information on which linguistic factors impact performance of natural language generation models. |
| Approach: | They propose a framework for error analysis which permits identifying which features of the input affect the models’ results. |
| Outcome: | The proposed framework improves the performance of 174 system runs submitted to the Multilingual SR shared tasks. |
Knowledge Extraction From Texts Based on Wikidata (2022.naacl-industry)
Copied to clipboard
| Challenge: | Existing knowledge extraction pipelines for English are not suitable for enterprise use. |
| Approach: | They propose a knowledge extraction pipeline for English which can be further used for building an entreprise-specific knowledge base. |
| Outcome: | The proposed pipeline can be used to build an entreprise-specific knowledge base. |
LORIA / Lorraine University at Multilingual Surface Realisation 2019 (D19-63)
Copied to clipboard
| Challenge: | WO component is easily transferrable between languages, but needs attention for each language separately. |
| Approach: | They present a LORIA / Lorraine University submission to the shared task . they evaluate it on 11 languages covered by the shared challenge . |
| Outcome: | The proposed approach is evaluated on 11 languages covered by the shared task . the main takeaways are that the WO component is easily transferrable between languages . |
Surface Realisation Using Full Delexicalisation (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to surface realisation model word ordering, morphological inflection and contraction generation are evaluated on 10 languages covered by the SR'18 shared task. |
| Approach: | They propose a modular approach which models each of these components separately and an analysis of the differences in word ordering performance across languages. |
| Outcome: | The proposed model is compared with existing models on 10 languages covered by the SR'18 shared task. |
A Systematic Review of Reproducibility Research in Natural Language Processing (2021.eacl-main)
Copied to clipboard
| Challenge: | Despite the recent progress in reproducibility, the field is far from reaching a consensus on how reproducibility should be defined, measured and addressed. |
| Approach: | They propose to provide a wide-angle snapshot of current work on reproducibility in NLP. |
| Outcome: | The proposed work will provide a wide-angle snapshot of current work on reproducibility in NLP. |