Papers by Janet Pierrehumbert
Dynamic Contextualized Word Embeddings (2021.acl-long)
Copied to clipboard
| Challenge: | Static word embeddings that represent words by a single vector cannot capture word meaning in different linguistic and extralinguistic contexts. |
| Approach: | They propose dynamic contextualized word embeddings that represent words as a function of linguistic and extralinguistic contexts. |
| Outcome: | The proposed model models time and social space jointly, making them attractive for NLP tasks involving semantic variability. |
Modeling Ideological Salience and Framing in Polarized Online Groups with Graph Neural Networks and Structured Sparsity (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to detect ideological divides in social media rely on knowing in advance the political orientation of text . fascist and mainstream are among the most polarized concepts in reddit in 2019 . |
| Approach: | They propose a minimally supervised method that leverages the network structure of online discussion forums to detect polarized concepts. |
| Outcome: | The proposed framework captures temporal ideological dynamics such as right-wing and left-wing radicalization using graph neural networks and sparsity learning. |
Predicting the Growth of Morphological Families from Social and Linguistic Factors (2020.acl-main)
Copied to clipboard
| Challenge: | a burst in token frequency of the word "trump" in social media before the 2016 presidential election is a prime indicator of topical dynamics. |
| Approach: | They propose a task of Morphological Family Expansion Prediction to predict the size of a morphological family by analyzing a reddit corpus. |
| Outcome: | The proposed task predicts the increase in the size of a morphological family on a reddit corpus. |
HateCheck: Functional Tests for Hate Speech Detection Models (2021.acl-long)
Copied to clipboard
| Challenge: | Hate speech detection models are evaluated by measuring their performance on held-out test data using metrics such as accuracy and F1 score. |
| Approach: | They propose a suite of functional tests for hate speech detection models that measure model performance on held-out test data and then craft test cases to validate their quality. |
| Outcome: | The proposed tests show that the proposed models perform poorly on a small set of widely-used hate speech datasets. |
Forecasting COVID-19 Caseloads Using Unsupervised Embedding Clusters of Social Media Posts (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies have shown that social media can help predict rises in infectious disease caseloads. |
| Approach: | They propose to use transformer-based language models to integrate infectious disease modelling into reddit embedding features in reddits in specific US states. |
| Outcome: | The proposed model outperforms other features at predicting upward trend signals in areas where epidemiological data is unreliable. |
Time Machine GPT (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large language models are often trained on extensive, temporally indiscriminate text corpora . conventional methods for creating temporal adapted models depend on pre-training static models on time-specific data. |
| Approach: | They propose a series of point-in-time LLMs called TimeMachineGPT to be nonprognosticative . time-series forecasting and event prediction aim to infer a future state from past data . authors propose linguistically-based models that can be used to predict future events . |
| Outcome: | The proposed model is nonprognosticative and ensures it remains uninformed about future factual information and linguistic changes. |
Superbizarre Is Not Superb: Derivational Morphology Improves BERT’s Interpretation of Complex Words (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) are based on fixed-size vocabularies of words and subwords that are generated by compression algorithms such as bytepair encoding. |
| Approach: | They propose to use BERT as an example PLM to study its semantic representations of English derivatives to test their hypothesis. |
| Outcome: | The proposed model outperforms BERT on a series of semantic probing tasks. |
An Embarrassingly Simple Method to Mitigate Undesirable Properties of Pretrained Language Model Tokenizers (2022.acl-short)
Copied to clipboard
| Challenge: | a standard tokenizer does not cover all characters of a word but preserves key aspects of its morphological structure . a novel method to improve tokenization of pretrained language models is proposed . |
| Approach: | They propose a method to improve the tokenization of pretrained language models . they use the vocabulary of a standard tokenizer but preserves morphological structure . |
| Outcome: | The proposed method improves tokenization of pretrained language models on morphological gold segmentations and text classification tasks. |
Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks (2022.naacl-main)
Copied to clipboard
| Challenge: | Labelled data is the foundation of most natural language processing tasks, but there are valid beliefs about what the correct data labels should be. |
| Approach: | They propose two contrasting paradigms for data annotation that encourage annotator subjectivity . they propose a descriptive paradigm that allows for the surveying and modelling of different beliefs . |
| Outcome: | The proposed paradigms encourage annotator subjectivity, while the prescriptive paradigm discourages it. |
A Graph Auto-encoder Model of Derivational Morphology (2020.acl-main)
Copied to clipboard
| Challenge: | Existing words that conform to morphological patterns of a language differ in how likely they are to be actually created by speakers. |
| Approach: | They propose to model the morphological well-formedness of derivatives by combining syntactic and semantic information with associative information from the mental lexicon. |
| Outcome: | The proposed model models the morphological well-formedness of derivatives in English . |
DagoBERT: Generating Derivational Morphology with a Pretrained Language Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) generate derivationally complex words, but it is unclear what they learn about other aspects of language. |
| Approach: | They propose to use BERT to examine its derivational capabilities in different settings, from unmodified pretrained models to full finetuning. |
| Outcome: | The proposed model outperforms the state-of-the-art in derivation generation. |
Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Language use differs between domains and even within a domain, language use changes over time. |
| Approach: | They propose to use social media comments to study temporal adaptations in pre-trained language models. |
| Outcome: | The proposed model performs better on past than on future test sets, whereas adapting to domain does not improve performance on the downstream task. |
STEntConv: Predicting Disagreement between Reddit Users with Stance Detection and a Signed Graph Convolutional Network (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to detect disagreements on social media platforms have focused on supplementing textual information with user network information, such as Twitter's following system, retweets and hashtags. |
| Approach: | They propose a method which builds a graph of users and named entities and trains a Signed Graph Convolutional Network to detect disagreement between comment and reply posts. |
| Outcome: | The proposed model builds a graph of users and named entities weighted by stance and trains a Signed Graph Convolutional Network (SGCN) to detect disagreement between comment and reply posts. |