Papers by Matthew Matero
Human Language Modeling (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing language modeling models treat text sequences as if they were created independently. |
| Approach: | They propose a hierarchical extension to the language modeling problem whereby a human-level exists to connect sequences of documents and capture the notion that human language is moderated by changing human states. |
| Outcome: | The proposed model outperforms the current state-of-the-art in terms of language modeling and fine-tuning for 4 downstream tasks spanning document- and user-levels. |
SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Tasks (2024.eacl-short)
Copied to clipboard
Gourab Dey, Adithya V Ganesan, Yash Kumar Lal, Manal Shah, Shreyashee Sinha, Matthew Matero, Salvatore Giorgi, Vivek Kulkarni, H. Schwartz
| Challenge: | Social science NLP tasks require large data to capture semantics and implicit pragmatics. |
| Approach: | They propose an open-source instruction tuning tool for social science NLP tasks that captures implicit pragmatic cues from text. |
| Outcome: | The proposed model matches or improves on a state-of-the-art, multi-task finetuned model on 80% of social tasks. |
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality (2021.naacl-main)
Copied to clipboard
| Challenge: | In human-level NLP tasks, the number of observations is often smaller than the standard 768+ hidden state sizes of each layer within transformer-based language models. |
| Approach: | They propose to use dimension reduction methods to fine-tune large models with limited data and to use pre-trained dimension reduction regimes to improve model performance. |
| Outcome: | The proposed model outperforms other models in human-level NLP tasks with a pre-trained dimension reduction regime. |
MeLT: Message-Level Transformer with Masked Document Representations as Pre-Training for Stance Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Much of natural language processing is focused on leveraging large capacity language models, typically trained over single messages with a task of predicting one or more tokens. |
| Approach: | They propose a hierarchical message-encoder pre-trained over Twitter for stance prediction task. |
| Outcome: | The proposed model achieves 67% performance on stance prediction task using a pre-trained message-encoder over Twitter. |
Autoregressive Affective Language Forecasting: A Self-Supervised Task (2020.coling-main)
Copied to clipboard
| Challenge: | Using natural language, we can model emotional language in time based on past changes of language. |
| Approach: | They propose a task of affective language forecasting to model emotional language in time based on past changes of language. |
| Outcome: | The proposed model outperforms existing models on a Twitter dataset of 1,900 users and scores for 6 emotions and 2 additional linguistic attributes. |
Discourse-Level Representations can Improve Prediction of Degree of Anxiety (2023.acl-short)
Copied to clipboard
Swanie Juhng, Matthew Matero, Vasudha Varadarajan, Johannes Eichstaedt, Adithya V Ganesan, H. Andrew Schwartz
| Challenge: | Anxiety disorders are the most common of mental illnesses, but little is known about how to detect them from language. |
| Approach: | They propose to use discourse-level information in addition to lexical-level large language model embeddings to evaluate the utility of a lexico-discourse model. |
| Outcome: | The proposed model outperforms models based on state-of-the-art contextual embeddings and uses discourse patterns of causal explanations significantly more than models derived from Sentence-BERT and DiscRE, and is comparable to psychological models. |