Papers by Alexey Tikhonov
Transformers in the loop: Polarity in neural models of language (2022.acl-long)
Copied to clipboard
| Challenge: | Recent Transformer-based language representation models (LRMs) show impressive results on practical text analysis tasks, but do they have access to complex linguistic notions? |
| Approach: | They propose to use polarity as a case study to compare metrics derived from language models to human judgments obtained in psycholinguistic experiments. |
| Outcome: | The proposed model is more accurate than linguistic theory predictions for polarity, and allows us to use language models to discover new insights into natural language grammar beyond existing linguistic theories. |
It’s All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | gilbert et al.: commonsense reasoning is a key problem in natural language processing but its capabilities are still unstudied. gilland eetal.: a new approach to commonsensible reasoning is needed to solve the problem. |
| Approach: | They propose a method which trains a linear classifier with weights of multi-head attention as features and a multilingual Winograd Schema corpus to measure cross-lingual generalization ability. |
| Outcome: | The proposed approach performs competitively with recent approaches even when applied to other languages in a zero-shot manner. |
BERT in Plutarch’s Shadows (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to estimate authorship of texts rely on statistical patterns like word or sentence length to distinguish between different authors. |
| Approach: | They propose to use a BERT language model to identify authorship in ancient Greek texts . they find that the Placita Philosophorum and other works are similar to those written by Alexandrian authors . |
| Outcome: | The proposed model can shed new light on the authorship question in ancient Greek and shows similarities with the works written by authors from Alexandrian contexts. |
HeadlineCause: A Dataset of News Headlines for Detecting Causalities (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets focus on commonsense causal reasoning or explicit causal relations . authors present dataset for detecting implicit causal relations between news headlines . |
| Approach: | They present a dataset for detecting implicit causal relations between news headlines . they use 5000 headline pairs from English news and 9000 from Russian news . |
| Outcome: | The proposed dataset shows that it is valid and can be used to predict implicit causal relations between headline pairs. |
Leverage Points in Modality Shifts: Comparing Language-only and Multimodal Word Representations (2023.starsem-1)
Copied to clipboard
| Challenge: | a recent study of the effect of visual grounding on language representations has given a new life to the debate around extractability and quality of semantic information in representations trained solely on textual input. |
| Approach: | They compare word embeddings from vision-and-language models to text-only models . they identify meaning properties and relations that characterize words whose embeddements are most affected by visual grounding . |
| Outcome: | The proposed model differs from text-only models on semantic representations of language . the study is the first large-scale study of the effect of visual grounding on language representations . |
EENLP: Cross-lingual Eastern European NLP Index (2022.lrec-1)
Copied to clipboard
Alexey Tikhonov, Alex Malkhasov, Andrey Manoshin, George-Andrei Dima, Réka Cserháti, Md.Sadek Hossain Asif, Matt Sárdi
| Challenge: | Existing NLP resources for Eastern European languages are sparse. |
| Approach: | They propose to use existing Eastern European language resources to build cross-lingual datasets for five different semantic tasks to support commonsense reasoning. |
| Outcome: | The proposed model trains on 104 languages and shows impressive results on text analysis tasks. |
Smotrom tvoja på ander drogoj verden! Resurrecting Dead Pidgin with Generative Models: Russenorsk Case Study (2025.findings-acl)
Copied to clipboard
| Challenge: | Russenorsk is a pidgin language used in trade interactions between Russian and Norwegian speakers. |
| Approach: | They propose to analyze Russenorsk's lexicon using modern large language models . they construct a structured dictionary of the language, grouped by synonyms and word origins . |
| Outcome: | The proposed model can generate linguistic hypotheses about Russenorsk, the authors show . their model can be used to generate renderings of contemporary Russian and Norwegian texts . |
Connecting degree and polarity: An artificial language learning study (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that degree modifiers are related to sentence polarity, but they are not related to the grammatical number of an expression. |
| Approach: | They propose to generalize degree modifiers to their polarity sensitivity in pre-trained language models by applying the Artificial Language Learning experimental paradigm from psycholinguistics to a neural language model. |
| Outcome: | The proposed generalisations are consistent with existing linguistic observations that relate de-gree semantics to polarity sensitivity, including the main one: low degree semantics is associated with preference towards positive polarities. |
Decomposing Textual Information For Style Transfer (D19-56)
Copied to clipboard
| Challenge: | Using a framework of style transfer for texts, we propose several empirical methods to assess information decomposition quality. |
| Approach: | They propose to use latent representations to effectively decompose different aspects of textual information using a framework of style transfer for texts. |
| Outcome: | The proposed methods show that higher quality representations correlate with higher performance in bilingual evaluation understudy (BLEU) between output and human-written reformulations. |
Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites (D19-1)
Copied to clipboard
| Challenge: | Currently, standard methods for style transfer have several significant problems. |
| Approach: | They propose to take BLEU between input and human-written reformulations into consideration for benchmarks. |
| Outcome: | The proposed architectures outperform state-of-the-art in style transfer metric on human-written reformulations and take BLEU between input and output into consideration for benchmarks. |