Sequential Path Signature Networks for Personalised Longitudinal Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Current work on low-dimensional static user representations or more importantly on dynamic user representation is limited. |
| Approach: | They propose to integrate path signatures from rough path theory into neural sequential models by integrating contextual neural representations and recursive neural networks. |
| Outcome: | The proposed model outperforms state-of-the-art models on macro-average F1 score on two available datasets and outperformed previous models which only have access to historical posts. |
Similar Papers
From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP (2026.acl-long)
Copied to clipboard
Adithya V Ganesan, Vasudha Varadarajan, Oscar Kjell, Whitney Ringwald, Scott M. Feltman, Benjamin J. Luft, Roman Kotov, Ryan L. Boyd, H. Andrew Schwartz
| Challenge: | a longitudinal model for NLP relies on document-level evaluation to map isolated instances of language to an outcome. |
| Approach: | They propose a longitudinal model that aligns evaluation splits to generalization over people and time . they propose integrating a sequence inputs to incorporate history by default . |
| Outcome: | The proposed model improves on a dataset of 17k daily diary transcripts paired with PTSD symptom severity from 238 participants. |
Creation and evaluation of timelines for longitudinal user posts (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for segmenting user posts into timelines improve quality and cost of manual annotation. |
| Approach: | They propose a set of methods for segmenting longitudinal user posts into timelines likely to contain interesting moments of change in a user’s behaviour based on their online posting activity. |
| Outcome: | The proposed framework is able to evaluate two different social media datasets and compares with existing models. |
Sig-Networks Toolkit: Signature Networks for Longitudinal Language Modelling (2024.eacl-demo)
Copied to clipboard
Talia Tseriotou, Ryan Chan, Adam Tsakalidis, Iman Munire Bilal, Elena Kochkina, Terry Lyons, Maria Liakata
| Challenge: | Existing work on temporal and longitudinal language modelling has focused on taskoriented models. |
| Approach: | They propose an open-source, pip installable toolkit that incorporates Signature-based Neural Network models into various longitudinal language modelling tasks. |
| Outcome: | The proposed model outperforms Transformer-based models in three NLP tasks and provides guidance for future projects. |
Identifying Moments of Change from Longitudinal User Text (2022.acl-long)
Copied to clipboard
| Challenge: | Identifying changes in individuals’ behaviour and mood via shared content is gaining importance given the global increase in mental health disorders and the limited access to support services. |
| Approach: | They propose a task of identifying moments of change in individuals on the basis of their shared content online. |
| Outcome: | The proposed task is based on 500 manually annotated user timelines and shows that it performs best through context aware sequential modelling. |
Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings (P18-1)
Copied to clipboard
| Challenge: | recurrent neural networks have produced significant advances in part-of-speech tagging accuracy . a common feature of these models is the presence of rich initial word encodings . however, word or sub-word information interacts only through subsequent recursive layers . |
| Approach: | They propose to use recurrent neural networks with sentence-level context for initial character and word-based representations. |
| Outcome: | The proposed model has the highest accuracy of all participating systems in the CoNLL 2017 task. |
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning (2023.acl-long)
Copied to clipboard
Kyuyong Shin, Hanock Kwak, Wonjae Kim, Jisu Jeong, Seungjae Jung, Kyungmin Kim, Jung-Woo Ha, Sang-Woo Lee
| Challenge: | Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. |
| Approach: | They propose to use user behavior sequences as plain text to represent rich information in any domain or system without losing generality. |
| Outcome: | The proposed frameworks achieve excellent results on diverse recommendation tasks and can be used on unseen domains and services. |
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that deep contextualized language models (LMs) can extract temporal relations between events and time expressions. |
| Approach: | They propose a temporal relation extraction technique which extracts temporal relations between events and time expressions. |
| Outcome: | The proposed method significantly improves temporal dependency parsing, the authors show . their work compares the proposed method to other methods and shows where they may fail . |
Exciting Mood Changes: A Time-aware Hierarchical Transformer for Change Detection Modelling (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work on temporally sensitive tasks focuses on predicting mood changes . however, there is little attention given to the importance of longitudinal language modelling . |
| Approach: | They propose a Hawkes process-inspired transformation layer to model the influence of time on users’ posts, capturing both their immediate and historical dynamics. |
| Outcome: | The proposed model outperforms existing models on two existing datasets and shows clear performance gains. |
Neural Language Modeling for Contextualized Temporal Graph Generation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for temporal reasoning have been used for a number of applications, but their potential for tempor reasoning over event graphs has not been explored. |
| Approach: | They propose to use large-scale pre-trained language models to generate an event-level temporal graph from a document using existing IE/NLP tools. |
| Outcome: | The proposed method outperforms the closest existing method on several metrics on a hand-labeled, out-of-domain corpus. |
Pretrained Language Models for Sequential Sentence Classification (D19-1)
Copied to clipboard
| Challenge: | Recent successful models for document-level understanding have used hierarchical encoding and CRFs to capture dependencies between subsequent labels. |
| Approach: | They propose a pretrained language model that captures contextual dependencies without hierarchical encoding nor a CRF. |
| Outcome: | The proposed model captures contextual dependencies without hierarchical encoding nor a CRF on four datasets, including a new dataset of structured scientific abstracts. |