Papers by Juri Opitz
Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Bias (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that embedding models exhibit systematic positional and language biases when documents are longer and consist of multiple segments. |
| Approach: | They propose a permutation-based evaluation framework to quantify embedding biases . they propose an inference-time attention calibration method that redistributes attention more evenly across document positions . |
| Outcome: | The proposed framework reduces the positional and language biases in embedding models . the proposed framework improves the discoverability of later segments . |
Similar, but why? A Toolkit for Explaining Text Similarity (2026.eacl-demo)
Copied to clipboard
| Challenge: | XPLAINSIM is a Python package that explains textual similarity in an easy-to-use way. |
| Approach: | They propose a Python package that unifies three approaches to explain text similarity . they demonstrate the value of the package through intuitive examples and empirical research . |
| Outcome: | XPLAINSIM is a Python package that unifies three approaches to explain text similarity . the authors show that the package is useful for explaining text similarities in a simple way . |
PARAPHRASUS: A Comprehensive Benchmark for Evaluating Paraphrase Detection Models (2025.coling-main)
Copied to clipboard
| Challenge: | prevailing notion of paraphrase is simplistic, offering only limited view of vast spectrum of paraphrasing phenomena. |
| Approach: | They propose a benchmarking tool for paraphrase detection that provides a fine-grained evaluation lens. |
| Outcome: | The proposed benchmark enables rapid calibration of models to specific strictness levels. |
ConLoan: A Contrastive Multilingual Dataset for Evaluating Loanwords (2025.acl-long)
Copied to clipboard
Sina Ahmadi, Micha David Hess, Elena Álvarez-Mellado, Alessia Battisti, Cui Ding, Anne Göhring, Yingqiang Gao, Zifan Jiang, Andrianos Michail, Peshmerge Morad, Joel Niklaus, Maria Christina Panagiotopoulou, Stefano Perrella, Juri Opitz, Anastassia Shaitarova, Rico Sennrich
| Challenge: | Lexical borrowing is a ubiquitous linguistic phenomenon influenced by geopolitical, societal, and technological factors. |
| Approach: | They propose a novel contrastive dataset comprising sentences with and without loanwords across 10 languages to examine how machine translation and language models process loanword . |
| Outcome: | The proposed dataset shows that state-of-the-art models prefer loanwords over native terms and exhibit varying performance across languages. |
Weisfeiler-Leman in the Bamboo: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing metrics for assessing the similarity of abstract meaning representations (AMRs) have complementary strengths and weaknesses, and are expensive to implement. |
| Approach: | They propose a Benchmark for AMR Metrics based on overt objectives that can be used to assess graph-based similarity metrics. |
| Outcome: | The proposed metrics match contextualized substructures and induce n:m alignments between their nodes. |
SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features (2022.aacl-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (S3BERT) embeddings are composed of explainable sub-embeddings that emphasize various sentence meaning features. |
| Approach: | They propose to induce Semantically Structured Sentence BERT embeddings (S3BERT) that emphasize various sentence meaning features. |
| Outcome: | The proposed model shows high correlation to human similarity ratings, but lacks interpretability. |
Interpretable Text Embeddings and Text Similarity Explanation: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Text embeddings are a fundamental component in many NLP tasks, but their interpretation and explanation remain challenging. |
| Approach: | They propose a framework for interpretable text embeddings and text similarity explanation . they characterize the main ideas, approaches, and trade-offs and discuss lessons learned . |
| Outcome: | The proposed methods are compared with existing models and compare them with existing ones. |
On the Role of Summary Content Units in Text Summarization Evaluation (2024.naacl-short)
Copied to clipboard
Marcel Nawrath, Agnieszka Nowak, Tristan Ratz, Danilo Walenta, Juri Opitz, Leonardo Ribeiro, João Sedoc, Daniel Deutsch, Simon Mille, Yixin Liu, Sebastian Gehrmann, Lining Zhang, Saad Mahamood, Miruna Clinciu, Khyathi Chandu, Yufang Hou
| Challenge: | a human written summary content unit (SCU) is used to judge the quality of a summary . a pyramid evaluation method is based on SCUs that decompose a reference summary into concise sentences . |
| Approach: | They propose to use automated SCUs to evaluate the quality of a candidate summary . they propose to generate SCU approximations from AMR meaning representations and large language models . |
| Outcome: | The proposed method can be fully automated, but lacks the human effort to validate it. |
A Survey of AMR Applications (2024.emnlp-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic representation that takes the form of a rooted, directed graph. |
| Approach: | They analyze more than 100 papers which use Abstract Meaning Representation (AMR) they highlight the range of applications for which AMR has been harnessed and techniques for incorporating it . they also highlight broader AMR engineering patterns and outline areas of future work that seem ripe for AMR incorporation. |
| Outcome: | The results highlight the range of applications for which AMR has been harnessed and the techniques for incorporating it into those applications. |
Schroedinger’s Threshold: When the AUC Doesn’t Predict Accuracy (2024.lrec-main)
Copied to clipboard
| Challenge: | AUC is a useful tool for evaluating diverse models without calibration. |
| Approach: | They show that the AUC yields an academic notion of accuracy that can misalign with actual accuracy observed in application. |
| Outcome: | The AUC yields an academic and optimistic notion of accuracy that can misalign with actual accuracy observed in application. |
With a Little Push, NLI Models can Robustly and Efficiently Predict Faithfulness (2023.acl-short)
Copied to clipboard
| Challenge: | Conditional language models generate unfaithful output that is not supported by their input . this jeopardizes trust in real-world applications, raising a need for automatic faithfulness metrics. |
| Approach: | They propose to augment conditional language models with robust inference procedures to improve faithfulness. |
| Outcome: | The proposed approach outperforms existing models on the TRUE benchmark. |
Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR (2021.eacl-main)
Copied to clipboard
| Challenge: | Abstract meaning representations are typically evaluated using surface matching metrics . however, there are problems with these metrics, since they allow multiple surface realizations . |
| Approach: | They propose a decomposable metric that measures the distance between the original and reconstructed AMRs. |
| Outcome: | The proposed metric reduces the complexity of the evaluation process by allowing for multiple surface realizations. |
AMR Similarity Metrics from Principles (2020.tacl-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) graphs are rooted, acyclic, directed, and edge-labeled. |
| Approach: | They propose a canonical Smatch metric that aligns variables of two graphs and assesses triple matches. |
| Outcome: | The proposed metric is more benevolent to only very slight meaning deviations and targets the fulfilment of all established criteria. |
AMR Quality Rating with a Lightweight CNN (2020.aacl-main)
Copied to clipboard
| Challenge: | Structured semantic sentence representations are useful in various NLP tasks, but their quality can vary greatly and jeopardize their usefulness. |
| Approach: | They propose to transfer the AMR graph to the domain of images and create a convolutional neural network that imitates a human judge tasked with rating graph quality. |
| Outcome: | The proposed model can rate quality more accurately than strong baselines, in several quality dimensions, and reduces energy consumption. |
Cheap Character Noise for OCR-Robust Multilingual Embeddings (2025.findings-acl)
Copied to clipboard
| Challenge: | Optical character recognition (OCR) is a key component of the digitization of historical documents. |
| Approach: | They propose a method that fine-tunes existing multilingual models using noisy texts and a contrastive loss. |
| Outcome: | The proposed model improves on the training data of existing models using noisy texts and a contrastive loss. |
A Survey of Meaning Representations – From Theory to Practical Utility (2024.naacl-long)
Copied to clipboard
| Challenge: | Symbolic meaning representations of natural language text have been studied since at least the 1960s . with the availability of large annotated corpora, the field has recently seen several new developments . |
| Approach: | They propose a framework for expressing meaning in natural language text using annotated corpora and a set of tools for machine learning. |
| Outcome: | The frameworks are based on a set of theoretical and practical problems and their applications. |
Similarity-weighted Construction of Contextualized Commonsense Knowledge Graphs for Knowledge-intense Argumentation Tasks (2023.acl-long)
Copied to clipboard
| Challenge: | Arguments often do not make explicit how a conclusion follows from its premises . we present a method for constructing Contextualized Commonsense Knowledge Graphs (CCKGs) that is efficient and high-quality . |
| Approach: | They propose an unsupervised method for constructing Contextualized Commonsense Knowledge Graphs (CCKGs) they use triplet similarities to extract contextually relevant knowledge paths . |
| Outcome: | The proposed method outperforms baselines and a GPT-3 based system in a knowledge-intense argumentation task. |
A Dynamic, Interpreted CheckList for Meaning-oriented NLG Metric Evaluation – through the Lens of Semantic Similarity Rating (2022.starsem-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a popular NLG evaluation metric . however, it is difficult to judge where exactly such a metric fails . |
| Approach: | They propose a checklist for NLG evaluation metrics that focus on meaning by organizing them around meaning-relevant linguistic phenomena. |
| Outcome: | The proposed metric GraCo computes lexical cohesion graphs over AMR concepts. |
Sentence Smith: Controllable Edits for Evaluating Text Embeddings (2025.emnlp-main)
Copied to clipboard
| Challenge: | Controllable and transparent text generation has been a long-standing goal in NLP . but previous approaches were hindered by parsing and generation insufficiencies . |
| Approach: | They propose a framework for English that has three steps: 1. Parsing a sentence into a semantic graph. 2. Applying human-designed semantic manipulation rules. 3. Generating text from the manipulated graph. |
| Outcome: | The proposed framework for English is based on a neural network and parsers. |
SMATCH++: Standardized and Extended Evaluation of Semantic Graphs (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing graph-alignment metrics that measure graph distances are not reliable, we show . metric is spread out and does not provide upper bounds for extended tasks. |
| Approach: | They propose a metric to measure a distance between graphs by aligning nodes and counting matching graph triples. |
| Outcome: | The proposed method reduces search space and improves scoring by reducing the number of errors. |