Exciting Mood Changes: A Time-aware Hierarchical Transformer for Change Detection Modelling (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on temporally sensitive tasks focuses on predicting mood changes . however, there is little attention given to the importance of longitudinal language modelling . |
| Approach: | They propose a Hawkes process-inspired transformation layer to model the influence of time on users’ posts, capturing both their immediate and historical dynamics. |
| Outcome: | The proposed model outperforms existing models on two existing datasets and shows clear performance gains. |
Similar Papers
TempoFormer: A Transformer for Temporally-aware Representations in Change Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to model context and time dynamics are slow and prone to overfitting. |
| Approach: | They propose a transformer-based and temporally-aware model for dynamic representation learning that is task-agnostic and trained on inter and intra context dynamics. |
| Outcome: | The proposed model is task-agnostic and can be used as the temporal representation foundation of other models or applied to different transformer-based architectures. |
Identifying Moments of Change from Longitudinal User Text (2022.acl-long)
Copied to clipboard
| Challenge: | Identifying changes in individuals’ behaviour and mood via shared content is gaining importance given the global increase in mental health disorders and the limited access to support services. |
| Approach: | They propose a task of identifying moments of change in individuals on the basis of their shared content online. |
| Outcome: | The proposed task is based on 500 manually annotated user timelines and shows that it performs best through context aware sequential modelling. |
Temporal Attention for Language Models (2022.findings-naacl)
Copied to clipboard
| Challenge: | Pretrained language models are trained on corpora derived from the web, but ignore this information. |
| Approach: | They propose a time-aware self-attention mechanism that captures time-specific contextualized word representations and allows the transformer to capture this information. |
| Outcome: | The proposed model achieves state-of-the-art on three datasets in different languages (English, German, and Latin) that vary in time, size, and genre. |
Sequential Path Signature Networks for Personalised Longitudinal Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Current work on low-dimensional static user representations or more importantly on dynamic user representation is limited. |
| Approach: | They propose to integrate path signatures from rough path theory into neural sequential models by integrating contextual neural representations and recursive neural networks. |
| Outcome: | The proposed model outperforms state-of-the-art models on macro-average F1 score on two available datasets and outperformed previous models which only have access to historical posts. |
Autoregressive Affective Language Forecasting: A Self-Supervised Task (2020.coling-main)
Copied to clipboard
| Challenge: | Using natural language, we can model emotional language in time based on past changes of language. |
| Approach: | They propose a task of affective language forecasting to model emotional language in time based on past changes of language. |
| Outcome: | The proposed model outperforms existing models on a Twitter dataset of 1,900 users and scores for 6 emotions and 2 additional linguistic attributes. |
Measuring and Modeling Language Change (N19-5)
Copied to clipboard
| Challenge: | This tutorial will help researchers answer questions fundamental to the social sciences and humanities . |
| Approach: | This tutorial is designed to help researchers answer questions in the social sciences and humanities . it synthesizes recent computational techniques for handling and modeling temporal data . |
| Outcome: | The tutorial will synthesize recent techniques for handling and modeling temporal data, such as dynamic word embeddings, and identify useful tools for social scientists and digital humanities scholars. |
MTP: A Dataset for Multi-Modal Turning Points in Casual Conversations (2024.acl-short)
Copied to clipboard
| Challenge: | a new problem setting is designed to detect critical moments in conversations . a human-annotated multi-modal dataset is used to classify and detect turning points . |
| Approach: | They propose a problem setting focusing on turning points in conversations as TPs . they propose MTPC, MTPD, & MTPR tasks to classify and detect turning points . |
| Outcome: | The proposed model achieves an F1-score of 0.88 in classification and 0.61 in detection . it uses state-of-the-art vision-language models to construct a narrative from the videos . |
Transformer-Based Temporal Information Extraction and Application: A Review (2025.emnlp-main)
Copied to clipboard
| Challenge: | Temporal information extraction (IE) aims to extract structured temporal information from unstructured text, thereby uncovering the implicit timelines within. |
| Approach: | They summarize and analyze the work using Transformers to highlight potential future directions. |
| Outcome: | The proposed method is applied across healthcare, newswire, and intelligence analysis domains. |
A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Suicide ideation is often linked to a history of mental depression. |
| Approach: | They propose a time-aware transformer based model for preliminary screening of suicidal risk on social media that augments linguistic models with historical context. |
| Outcome: | The proposed model outperforms competing models and shows that it is time-aware and contextually useful for suicide risk assessment. |
Diachronic Sense Modeling with Deep Contextualized Word Embeddings: An Ecological View (P19-1)
Copied to clipboard
| Challenge: | Existing word embeddings only assign one vector to a word for a time period, thus they face the meaning conflation deficiency. |
| Approach: | They propose a sense representation and tracking framework based on deep contextualized embeddings that can be used to answer what and when the word meaning changes. |
| Outcome: | The proposed framework is effective in representing fine-grained word senses, and brings a significant improvement in word change detection task. |