Papers by Serge Gladkoff
HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing automated evaluation metrics for machine translation are expensive and lack inter-rater reliability. |
| Approach: | They propose a task-oriented and human-centric evaluation framework for machine translation output based on professional post-e diting annotations. |
| Outcome: | The proposed framework improves translation quality and system performance and transparency . it is cost-effective, easy to use and faster to implement . |
Measuring Uncertainty in Translation Quality Evaluation (TQE) (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing automated tools are not good enough to evaluate translation quality . existing tools are often accused of having low reliability and agreement . |
| Approach: | They propose to use a method to accurately estimate the confidence intervals depending on the sample size of the translated text. |
| Outcome: | The proposed method aims to estimate the confidence intervals (CITATION) depending on the sample size of the translated text, e.g. the amount of words or sentences, that needs to be processed on TQE workflow step for confident and reliable evaluation of overall translation quality. |