Papers by Julia Rozanova
Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Models (2024.lrec-main)
Copied to clipboard
| Challenge: | a number of studies have reported high accuracies in NLP tasks due to simple heuristics and dataset artifacts. |
| Approach: | They use a case where two words/terms occur in a shared context to construct a causal diagram . they also investigate the robustness to irrelevant changes and sensitivity to impactful changes of Transformers . |
| Outcome: | The proposed method bolsters the fact that similar benchmark accuracy scores may be observed for models that exhibit very different behaviour. |
Montague semantics and modifier consistency measurement in neural language models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on distributional language models have been focused on linguistics and their relationship with semantic formalisms for decades. |
| Approach: | They propose a method for measuring compositional behavior in contemporary language embedding models by introducing three new tests inspired by Montague semantics. |
| Outcome: | The proposed method measures compositional behavior in language embedding models on adjectival modifier phenomena in adjective-noun phrases. |
Systematicity, Compositionality and Transitivity of Deep NLP Models: a Metamorphic Testing Perspective (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on robustness-like metamorphic relations, which limit the scope of linguistic properties they can test. |
| Approach: | They propose three new classes of metamorphic relations which address the properties of systematicity, compositionality and transitivity. |
| Outcome: | The proposed methods show that metamorphic models do not always behave according to expected linguistic properties. |
Does My Representation Capture X? Probe-Ably (2021.acl-demo)
Copied to clipboard
| Challenge: | Probing (or diagnostic classification) has become a popular strategy for investigating whether a given set of intermediate features is present in the representations of neural models. |
| Approach: | They propose to use an extendable probing framework to automate the application of probing methods to the user’s inputs. |
| Outcome: | The proposed framework automates the application of probing methods to the user’s inputs. |
Diff-Explainer: Differentiable Convex Optimization for Explainable Multi-hop Inference (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing explainable multi-hop inference models are regarded as black-boxes due to their ability to transfer linguistic and semantic information to downstream tasks, posing concerns about interpretability and transparency of their predictions. |
| Approach: | They propose a hybrid framework that integrates explicit constraints with neural architectures through differentiable convex optimization to answer and explain multi-hop questions in natural language. |
| Outcome: | The proposed framework improves performance on scientific and commonsense QA tasks while still providing structured explanations in support of its predictions. |
To be or not to be an Integer? Encoding Variables for Mathematical Text (2022.findings-acl)
Copied to clipboard
| Challenge: | a number of natural language inference models are limited in interpreting mathematical knowledge written in Natural Language . a variable's meaning is determined exclusively by its defining type, i.e., its context . |
| Approach: | They propose a method that can create context-based representations for variables . they propose 'variable slot' approach which can be used to model variables based on their meaning . |
| Outcome: | The proposed model can be used to represent variables in natural language . it can be applied to a task of variable typing and create context-based representations for variables . |
Interventional Probing in High Dimensions: An NLI Case Study (2023.findings-eacl)
Copied to clipboard
| Challenge: | Probing strategies have been shown to detect the presence of various linguistic features inlarge language models; in particular, semantic features intermediate to the “natural logic”fragment of the NLI. |
| Approach: | They propose to use amnesic probing and mnestic probing to investigate the effect of these semantic fea-tures on NLI classification by examining the effects of a mnemonic probing variation on the model. |
| Outcome: | The proposed methods have been shown to detect features intermediate to the “natural logic”fragment of the Natural Language Inferencetask (NLI). |