Papers by Adam Tsakalidis
Sequential Modelling of the Evolution of Word Representations for Semantic Change Detection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that detect semantically shifted words do not account for its evolution through time. |
| Approach: | They propose three variants of sequential models for detecting semantically shifted words . they demonstrate that temporal modelling of word representations yields a clear-cut advantage . |
| Outcome: | The proposed models account for the changes in word representations over time. |
Template-based Abstractive Microblog Opinion Summarization (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing work on Twitter uses extractive summarization to filter through information, but this approach often includes incomplete or redundant information. |
| Approach: | They propose to use Twitter data to generate 3100 gold-standard opinion summaries. |
| Outcome: | The proposed method outperforms previous work on extractive summarization models and fine-tunes to improve performance. |
Exciting Mood Changes: A Time-aware Hierarchical Transformer for Change Detection Modelling (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on temporally sensitive tasks focuses on predicting mood changes . however, there is little attention given to the importance of longitudinal language modelling . |
| Approach: | They propose a Hawkes process-inspired transformation layer to model the influence of time on users’ posts, capturing both their immediate and historical dynamics. |
| Outcome: | The proposed model outperforms existing models on two existing datasets and shows clear performance gains. |
TempoFormer: A Transformer for Temporally-aware Representations in Change Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to model context and time dynamics are slow and prone to overfitting. |
| Approach: | They propose a transformer-based and temporally-aware model for dynamic representation learning that is task-agnostic and trained on inter and intra context dynamics. |
| Outcome: | The proposed model is task-agnostic and can be used as the temporal representation foundation of other models or applied to different transformer-based architectures. |
Identifying Moments of Change from Longitudinal User Text (2022.acl-long)
Copied to clipboard
| Challenge: | Identifying changes in individuals’ behaviour and mood via shared content is gaining importance given the global increase in mental health disorders and the limited access to support services. |
| Approach: | They propose a task of identifying moments of change in individuals on the basis of their shared content online. |
| Outcome: | The proposed task is based on 500 manually annotated user timelines and shows that it performs best through context aware sequential modelling. |
Unsupervised Opinion Summarisation in the Wasserstein Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on opinion summarisation of social media posts has focused on reviews . however, it is important to capture user opinions in online discussions over specific topics . |
| Approach: | They propose an unsupervised opinion summarisation model which uses the Wasserstein distance to generate a single summary from a group of documents. |
| Outcome: | The proposed model outperforms the state-of-the-art on ROUGE metrics and produces the best summaries with respect to meaning preservation according to human evaluations. |
A Digital Language Coherence Marker for Monitoring Dementia (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that dementia is associated with thought disorders relating to inability to produce coherent communication. |
| Approach: | They propose to capture language coherence as a human-interpretable digital marker for monitoring cognitive changes in people with dementia. |
| Outcome: | The proposed model shows a significant difference between people with mild cognitive impairment, those with Alzheimer’s Disease and healthy controls and high association with clinical bio-markers. |
Sequential Path Signature Networks for Personalised Longitudinal Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Current work on low-dimensional static user representations or more importantly on dynamic user representation is limited. |
| Approach: | They propose to integrate path signatures from rough path theory into neural sequential models by integrating contextual neural representations and recursive neural networks. |
| Outcome: | The proposed model outperforms state-of-the-art models on macro-average F1 score on two available datasets and outperformed previous models which only have access to historical posts. |
Combining Hierachical VAEs with LLMs for clinically meaningful timeline summarisation in social media (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that social media users' posts can help identify depression, bipolar disorder or self-harm. |
| Approach: | They propose a hybrid abstractive summarisation approach combining hierarchical VAEs with LLMs to produce clinically meaningful summaries from social media timelines. |
| Outcome: | The proposed approach produces clinically meaningful summaries from social media user timelines, suitable for mental health monitoring. |
Creation and evaluation of timelines for longitudinal user posts (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for segmenting user posts into timelines improve quality and cost of manual annotation. |
| Approach: | They propose a set of methods for segmenting longitudinal user posts into timelines likely to contain interesting moments of change in a user’s behaviour based on their online posting activity. |
| Outcome: | The proposed framework is able to evaluate two different social media datasets and compares with existing models. |
Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling (2024.eacl-demo)
Copied to clipboard
Talia Tseriotou, Ryan Chan, Adam Tsakalidis, Iman Munire Bilal, Elena Kochkina, Terry Lyons, Maria Liakata
| Challenge: | Existing work on temporal and longitudinal language modelling has focused on taskoriented models. |
| Approach: | They propose an open-source, pip installable toolkit that incorporates Signature-based Neural Network models into various longitudinal language modelling tasks. |
| Outcome: | The proposed model outperforms Transformer-based models in three NLP tasks and provides guidance for future projects. |
Evaluation of Thematic Coherence in Microblogs (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work on grouping together views about tweets expressing opinions about the same entities has been criticized for their lack of thematic coherence. |
| Approach: | They propose to use a corpus of microblogs representing opinions about the same topics within the same time window to evaluate thematic coherence. |
| Outcome: | The proposed method outperforms surface level metrics, topic model coherence and text generation metrics (TGMs) but is not as reliable as TGMs due to being less sensitive to time windows. |