Papers by Francesco Barbieri
Multimodal Emoji Prediction (N18-2)
Copied to clipboard
| Challenge: | Emojis are small images that are commonly included in social media text messages. |
| Approach: | They propose a multimodal approach that is able to predict emojis in Instagram posts by using both text and image. |
| Outcome: | The proposed model incorporates both text and image to improve accuracy . |
Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shifts (2022.aacl-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a longstanding NLP task that consists of identifying an entity in a sentence or document. |
| Approach: | They construct a dataset of seven entity types annotated over 11,382 tweets . they provide a set of language model baselines and analyze the performance of the model . |
| Outcome: | The proposed dataset contains seven entity types annotated over 11,382 tweets . the authors focus on short-term degradation of NER models over time and strategies to fine-tune a language model over different periods . |
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond (2022.lrec-1)
Copied to clipboard
| Challenge: | Language models are ubiquitous in NLP, but current analyses focus on (multilingual variants of) standard benchmarks and task-specific corpora as multilingual signals. |
| Approach: | They propose a model to train and evaluate multilingual language models in Twitter using a set of Twitter datasets in eight different languages and a XLM-T model. |
| Outcome: | The proposed model trains and evaluates multilingual models on Twitter. |
Interpretable Emoji Prediction via Label-Wise Attention LSTMs (D18-1)
Copied to clipboard
| Challenge: | Emojis are the evolution of characterbased emoticons and are used to express ideas about a myriad of topics. |
| Approach: | They propose a label-wise attention mechanism to better understand emoji prediction . they propose to model e-mails with eojis and then label them based on their meaning . |
| Outcome: | The proposed model improves over baselines and does particularly well when predicting infrequent emojis. |
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Modern NLP systems are typically ill-equipped when applied to noisy user-generated text. |
| Approach: | They propose a new evaluation framework consisting of seven Twitter-specific classification tasks. |
| Outcome: | The proposed framework is based on seven heterogeneous Twitter-specific classification tasks. |
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning (2021.naacl-main)
Copied to clipboard
| Challenge: | PTLMs can exhibit biases against protected groups in a host of modeling tasks . but, fine-tuned LMs may propagate bias to downstream classifiers . |
| Approach: | They propose to use upstream bias mitigation techniques to reduce bias on downstream tasks by fine-tuning an upstream model and applying it to a downstream model. |
| Outcome: | The proposed model reduces bias on hate speech detection, toxicity detection and coreference resolution tasks over bias factors. |
TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Media (2022.coling-1)
Copied to clipboard
Daniel Loureiro, Aminette D’Souza, Areej Nasser Muhajab, Isabella A. White, Gabriel Wong, Luis Espinosa-Anke, Leonardo Neves, Francesco Barbieri, Jose Camacho-Collados
| Challenge: | Language models are often clean and time-invariant, and do little to no account of social media usage. |
| Approach: | They propose a benchmark to accelerate research in social media-based meaning shift. |
| Outcome: | The proposed benchmark is aimed at accelerating research in social media-based meaning shift. |
SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research (2023.findings-emnlp)
Copied to clipboard
Dimosthenis Antypas, Asahi Ushio, Francesco Barbieri, Leonardo Neves, Kiamehr Rezaee, Luis Espinosa-Anke, Jiaxin Pei, Jose Camacho-Collados
| Challenge: | specialised language models (LMs) have shown to exhibit lower perplexity and higher downstream performance across the board. |
| Approach: | They propose a benchmark for NLP evaluation in social media, SuperTweetEval. |
| Outcome: | The proposed benchmark shows that social media models perform better when compared to general-purpose models, metrics and benchmarks. |
TimeLMs: Diachronic Language Models from Twitter (2022.acl-demo)
Copied to clipboard
| Challenge: | Neural language models (LMs) are a key enabler in NLP, but lack of diachronic specialization affects both the ability to generalize to future data and the reliability of experimental results. |
| Approach: | They propose to use Twitter data to develop time-specific language models that are specialized on the time variable. |
| Outcome: | The proposed models cope with trends and peaks in activity involving specific named entities or concept drift. |
Twitter Topic Classification (2022.coling-1)
Copied to clipboard
Dimosthenis Antypas, Asahi Ushio, Jose Camacho-Collados, Vitor Silva, Leonardo Neves, Francesco Barbieri
| Challenge: | Existing methods to identify topics from posts are difficult to interpret and can differ from corpus to corpus. |
| Approach: | They propose a task based on tweet topic classification and release two datasets that can be used to train and test models. |
| Outcome: | The proposed task is based on two datasets from recent time periods and provides training and testing data. |
PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Instruction tuning has advanced large language models (LLMs) but its application in lower-resource languages faces challenges due to the imbalanced foundational abilities of LLMs across different languages. |
| Approach: | They propose a pivot language guided generation approach that utilizes a high-resource language as the pivot to enhance instruction tuning in lower-resourced languages. |
| Outcome: | The proposed approach improves instruction-following abilities of LLMs by 29% on average compared to directly responding in the target language alone. |
Evaluating Very Long-Term Conversational Memory of LLM Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on long-term open-domain dialogues focus on evaluating responses within contexts spanning no more than five chat sessions. |
| Approach: | They propose a machine-human pipeline to generate very long-term dialogues by leveraging LLMs and retrieval augmented generation techniques. |
| Outcome: | The proposed pipeline generates very long-term dialogues using LLMs and RAGs . the generated conversations are verified and edited by human annotators for long-range consistency and grounding to the event graphs. |
Explainability and Hate Speech: Structured Explanations Make Social Media Moderators Faster (2024.acl-short)
Copied to clipboard
Agostina Calabrese, Leonardo Neves, Neil Shah, Maarten Bos, Björn Ross, Mirella Lapata, Francesco Barbieri
| Challenge: | Existing studies have shown that explanations can support content moderators to make faster decisions, but the benefits of such models have not been studied. |
| Approach: | They propose to use structured explanations to support content moderators to make faster decisions by 7.4%. |
| Outcome: | The proposed models lower the speed of real-world moderators by 7.4% compared to generic explanations and are often ignored . previous studies have shown that explanations can support moderator's decision making by detecting violations of policies but the benefits have not been studied . |
Multilingual Topic Classification in X: Dataset and Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | Social media platforms such as X (Twitter), Snapchat and Instagram provide an environment for content creation and information sharing. |
| Approach: | They propose a multilingual dataset featuring tweet topic classification in four languages . they leverage X-Topic to perform cross-linguistic and multilingual analysis . |
| Outcome: | The proposed dataset includes topics in four languages and is useful for cross-linguistic analysis and the development of robust multilingual models. |
The Devil is in the Details: Evaluating Limitations of Transformer-based Methods for Granular Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Contextual embeddings have shown state-of-the-art performance for various tasks such as question answering, sentiment analysis, and textual similarity. |
| Approach: | They propose to integrate transformer-based neural language models into their models to achieve relative improvements of up to 36% on granular tasks. |
| Outcome: | The proposed model outperforms baselines for more granular tasks while outperforming TF-IDF for more complex tasks. |