Papers by Giuseppe Carenini
Explicit Bayesian Inference to Uncover the Latent Themes of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive generative capabilities, yet their inner mechanisms remain largely opaque. |
| Approach: | They propose a variational autoencoder-based neural topic model to interpret LLMs generation process through an explicit Bayesian framework by inferring latent topic variables via variational inference. |
| Outcome: | The proposed model outperforms state-of-the-art topic models on intrinsic measures of coherence and diversity on multiple datasets and shows significant gains on classification and summarization tasks. |
Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning (2021.acl-short)
Copied to clipboard
| Challenge: | In news articles the lead bias dominates the learning signals for neural extractive summarizations, severely limiting their performance on data with different or even no bias. |
| Approach: | They propose a method to demote the lead bias in news and make the model focus more on the content semantics. |
| Outcome: | The proposed method can demote the model’s learned lead bias and improve its generality on out-of-distribution data with little to no performance loss on in-difference data. |
Comparing the Intrinsic Performance of Clinical Concept Embeddings by Their Field of Medicine (D19-62)
Copied to clipboard
| Challenge: | Existing work has trained medical embeddings to rep-resent medical concepts using specific medical data. |
| Approach: | They use intrinsic methods to evaluate pre-trained word embeddings from the various fields of medicine as defined by their ICD-9 systems. |
| Outcome: | The results show that the embeddings perform better in one field of medicine than in other fields. |
Improving Context Modeling in Neural Topic Segmentation (2020.aacl-main)
Copied to clipboard
| Challenge: | Recent work favors highly effective neural supervised approaches for topic segmentation but current neural solutions are limited in how they model context. |
| Approach: | They propose to enhance a hierarchical attention biLSTM network-based topic segmenter to better model context by adding a coherence-related auxiliary task and restricted self-attention. |
| Outcome: | The proposed model outperforms SOTA approaches on three datasets and on four real-world benchmarks. |
MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing discourse treebanks are limited in the application of data-driven approaches to discourse parsing. |
| Approach: | They propose a method to automatically generate discourse treebanks using distant supervision from sentiment annotated datasets by heuristic beam-search strategy extended with a stochastic component. |
| Outcome: | The proposed method generates discourse trees incorporating structure and nuclearity for documents of arbitrary length using an efficient beam-search strategy, extended with a stochastic component. |
Towards Domain-Independent Text Structuring Trainable on Large Discourse Treebanks (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Text structuring is a fundamental step in natural language generation, especially when generating multi-sentential text. |
| Approach: | They propose a novel task that combines neural dependency tree induction with pointer networks to train on large discourse treebanks. |
| Outcome: | The proposed method outperforms existing content ordering metrics and outperformed existing ones. |
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In recent years, pre-trained language models have become the de facto instrument for the field of natural language processing (NLP). |
| Approach: | They propose a method that injects linguistic structures into pre-trained language models in the parameter-efficient fine-tuning setting. |
| Outcome: | The proposed approach outperforms state-of-the-art methods with a comparable number of parameters. |
From Sentiment Annotations to Sentiment Prediction through Discourse Augmentation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing sentiment analysis models lack temporal information to capture semantics of long texts. |
| Approach: | They propose a framework to exploit task-related discourse structures for sentiment analysis. |
| Outcome: | The proposed framework improves the performance even beyond existing approaches based on human annotated data. |
Diversity-Aware Coherence Loss for Improving Neural Topic Models (2023.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that our method significantly improves the performance of neural topic models without requiring any pretraining or additional parameters. |
| Approach: | They propose a variational autoencoder framework that minimizes the posterior and prior divergence and a diversity-aware coherence loss that encourages the model to learn corpus-level coherency scores while maintaining high diversity between topics. |
| Outcome: | The proposed approach significantly improves the performance of neural topic models without pretraining or additional parameters. |
Delta-KNN: Improving Demonstration Selection in In-Context Learning for Alzheimer’s Disease Detection (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for in-context learning (ICL) perform poorly for AD diagnosis due to inherent complexity of task. |
| Approach: | They propose a demonstration selection strategy that leverages a delta score to assess the relative gains of each training example and a KNN-based retriever that dynamically selects optimal “representatives” for a given input. |
| Outcome: | The proposed model outperforms existing methods on two AD detection datasets and surpasses even supervised classifiers. |
MIMIC: Multi-party Dialogue Augmentation via Speaker Stylistic Transfer (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing discourse annotations are limited and annotated data scarcity has hindered progress in discourse parsing. |
| Approach: | They propose a framework for augmenting discourse-annotated corpora via speaker stylistic transfer using Large Language Models (LLMs). |
| Outcome: | The proposed framework outperforms parsers trained on STAC and Molweni corpora on a multi-party dialogue with consistent gains for underrepresented discourse patterns and in low-resource scenarios. |
Predicting Discourse Structure using Distant Supervision from Sentiment (D19-1)
Copied to clipboard
| Challenge: | Discourse parsing is a fundamental NLP task known to enhance key downstream tasks, such as sentiment analysis, text classification and summarization. |
| Approach: | They propose a method that uses document supervision to generate abundant data for RST-style discourse structure prediction by using an optimal CKY-style tree generation algorithm. |
| Outcome: | The proposed approach performs well on the more difficult task of inter-domain discourse structure prediction, but it does not match the performance of a parser trained and tested on the same dataset. |
Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues (2023.findings-eacl)
Copied to clipboard
| Challenge: | Discourse processing suffers from data sparsity, especially for dialogues . a variety of discourse frameworks have been proposed to extract discourse information from dialogues. |
| Approach: | They propose unsupervised and semi-supervised methods to infer latent discourse structures for dialogues based on attention matrices from Pre-trained Language Models. |
| Outcome: | The proposed methods achieve encouraging results on the STAC corpus, with F1 scores of 57.2 and 59.3 for the unsupervised and semi-supervised methods, respectively. |
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing pretrained models require domain-specific additional information to be effective. |
| Approach: | They propose a pre-trained model for multi-document representation with a focus on summarization that uses efficient encoder-decoder transformers to simplify the processing of concatenated input documents. |
| Outcome: | PRIMERA outperforms current state-of-the-art models on most datasets with large margins . PRImerA uses efficient encoder-decoder transformers to simplify processing of concatenated input documents. |
Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale Pretraining (2020.coling-main)
Copied to clipboard
| Challenge: | Discourse parsing is an important upstream task within the area of Natural Language Processing (NLP) . |
| Approach: | They propose a discourse parser that incorporates recent contextual language models to improve the performance of RST-based discourse parses. |
| Outcome: | The proposed parser outperforms existing models on two key RST datasets and on large-scale "silver-standard" discourse treebank MEGA-DT. |
FM2DS: Few-Shot Multimodal Multihop Data Synthesis with Knowledge Distillation for Question Answering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focus on single-hop, single-modality, or short texts, limiting real-world applications . despite advances in visual question answering, this multihop setting remains underexplored due to a lack of quality datasets. |
| Approach: | They propose a framework for creating a high-quality dataset for multimodal multihop question answering . they use a 5-stage pipeline to acquire relevant multimodal documents from Wikipedia . |
| Outcome: | The proposed framework outperforms existing methods on multimodal multihop question answering datasets. |
T3-Vis: visual analytic for Training and fine-Tuning Transformers in NLP (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Existing visual analytics tools have been shown to support the analysis and interpretation of deep learning models due to the inherent black-box nature of the models. |
| Approach: | They propose to use visual analytic framework to help researchers understand the model's intrinsic properties and behaviours through interactive visualization. |
| Outcome: | The proposed framework provides valuable insights about the model’s intrinsic properties and behaviours through interactive visualization and a suite of built-in algorithms. |
Predicting Discourse Trees from Transformer-based Neural Summarizers (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing extractive summarization tasks use only neural approaches to learn discourse information, but recent work has shown that it is beneficial for summarizing discourse information. |
| Approach: | They propose to generate document-level discourse trees from pre-trained neural summarizers that encode dependency- and constituency-style discourse information. |
| Outcome: | The proposed model learns both, dependency- and constituency-style discourse information, consistent with pre-neural results. |
Neural Multimodal Topic Modeling: A Comprehensive Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Neural topic models can find coherent and diverse topics in textual data, but they are limited in dealing with multimodal datasets. |
| Approach: | They propose two new topic modeling solutions and two new evaluation metrics for document multimodality. |
| Outcome: | The proposed models generate coherent and diverse topics on a rich dataset. |
ChartGaze: Enhancing Chart Understanding in LVLMs with Eye-Tracking Guided Attention Refinement (2025.emnlp-main)
Copied to clipboard
Ali Salamatian, Amirhossein Abaskohi, Wan-Cyuan Fan, Mir Rayat Imtiaz Hossain, Leonid Sigal, Giuseppe Carenini
| Challenge: | Chart question answering (CQA) is a key research challenge for large vision-language models . recent efforts focus on leveraging LVLMs directly on chart images . |
| Approach: | They propose a gaze-guided attention refinement that aligns image-text attention with human fixations to improve chart reasoning quality and interpretability. |
| Outcome: | The proposed approach improves answer accuracy and attention alignment yielding gains of up to 2.56 percentage points across multiple models. |
Evaluating Topic Quality with Posterior Variability (D19-1)
Copied to clipboard
| Challenge: | Probabilistic topic models such as latent Dirichlet allocation (LDA) are widely used for NLP tasks which require the extraction of latent themes. |
| Approach: | They propose to measure topic quality using the variability of posterior distributions of probabilistic topic models. |
| Outcome: | The proposed metric achieves state-of-the-art correlations with human judgments of topic quality in experiments on three corpora. |
Neural RST-based Evaluation of Discourse Coherence (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing discourse parsers cannot predict coherent texts without using silver-standard features. |
| Approach: | They propose a tree-recursive neural model which takes advantage of the text’s RST features produced by a state of the art RST parser and compares it to the current state of art. |
| Outcome: | The proposed model achieves state-of-the-art accuracy on the Grammarly Corpus for Discourse Coherence (GCDC) and has 62% fewer parameters than existing models. |
W-RST: Towards a Weighted RST-style Discourse Framework (2021.acl-long)
Copied to clipboard
| Challenge: | We show that weighted discourse trees from auxiliary tasks can benefit downstream applications . linguistic theories play a less and less critical role in the field of discourse . |
| Approach: | They propose a weighted-RST framework that assigns a binary assessment of importance between text segments by a relation attribute. |
| Outcome: | The proposed framework can be replaced by real-valued scores, the authors show . they show that weighted discourse trees can benefit key NLP downstream applications . |
Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have investigated the multi-head self-attention mechanism of transformers. |
| Approach: | They propose to use a human-in-the-loop pipeline to discover task-specific attention patterns and inject them into transformer models to improve their accuracy. |
| Outcome: | The proposed methods improve the performance of transformer models by incorporating predefined patterns into their attention matrices. |
Discourse Analysis and Its Applications (P19-4)
Copied to clipboard
| Challenge: | Discourse processing is a suite of NLP tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications. |
| Approach: | They present a set of tasks to uncover linguistic structures from texts at several levels, which can support many downstream applications. |
| Outcome: | The tutorial covers the basic concepts of discourse analysis and linguistic structures in monologue vs. conversation, synchronous v. asynchronous conversation, and key linguistic structure in discourse analysis. |
NLP for Conversations: Sentiment, Summarization, and Group Dynamics (C18-3)
Copied to clipboard
| Challenge: | a tutorial focuses on computational models for conversational structure, summarization and sentiment detection, and group dynamics. |
| Approach: | a tutorial will provide examples of specific NLP tasks for conversational structure, summarization and sentiment detection, and group dynamics. |
| Outcome: | The tutorial focuses on the three areas of conversational structure, summarization and sentiment detection, and group dynamics. |
Personalized Abstractive Summarization by Tri-agent Generation Pipeline (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing research shows that large language models do not consistently satisfy users' preferences or expectations. |
| Approach: | They propose a tri-agent generation pipeline that includes a generator, an instructor, and an editor to enhance output personalization. |
| Outcome: | The proposed pipeline generates outputs that better meet user expectations on two abstractive summarization datasets. |
Systematically Exploring Redundancy Reduction in Summarizing Long Documents (2020.aacl-main)
Copied to clipboard
| Challenge: | Summarization tasks are often based on importance and diversity, but there is a trade-off between importance and non-redundancy. |
| Approach: | They propose to organize existing methods into categories based on when and how redundancy is considered and propose three additional methods balancing non-redundancy and importance in a general and flexible way. |
| Outcome: | The proposed methods achieve state-of-the-art on two scientific paper datasets, Pubmed and arXiv, while reducing redundancy significantly. |
DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning (2025.coling-main)
Copied to clipboard
Raymond Li, Yuxi Feng, Zhenan Fan, Giuseppe Carenini, Weiwei Zhang, Mohammadreza Pourreza, Yong Zhang
| Challenge: | In-context Learning (ICL) has proven to be effective in a variety of complex tasks, but the selection of the most beneficial demonstration examples remains an open research problem. |
| Approach: | They propose a demonstration retrieval framework that learns a weighted combination of LLM hidden states where rich semantic information is encoded. |
| Outcome: | Experiments on two popular NL2SQL benchmarks show that the proposed method outperforms state-of-the-art models. |
Infusing Theory of Mind into Socially Intelligent LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Theory of Mind (ToM) is a key aspect of human social intelligence, yet chatbots and LLMs do not typically integrate it. |
| Approach: | They propose a method that integrates Theory of Mind (ToM) into chatbots and dialogue agents to generate mental states between dialogue turns. |
| Outcome: | The proposed method improves dialogue and social interaction by integrating ToM with dialogue lookahead. |
Topic-Guided Reinforcement Learning with LLMs for Enhancing Multi-Document Summarization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive results in single-document summarization, but their performance on MDS still leaves room for improvement. |
| Approach: | They propose a topic-guided reinforcement learning approach to improve content selection in MDS . explicit prompting models with topic labels enhances the informativeness, they show . |
| Outcome: | The proposed method outperforms baselines on multi-News and multi-XScience datasets. |
Extractive Summarization of Long Documents by Combining Global and Local Context (D19-1)
Copied to clipboard
| Challenge: | Existing methods for extractive and abstractive summarization are far from human performance. |
| Approach: | They propose a neural single-document extractive summarization model for long documents that incorporates both the global context of the whole document and the local context. |
| Outcome: | The proposed model outperforms previous models on ROUGE-1, ROUGEE-2 and METEOR scores on two datasets of scientific papers. |
Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to pre-training/fine-tuning are focusing on the alignment of pre-trained and fine-tuned PLMs with large-scale discourse structures. |
| Approach: | They propose a novel approach to infer discourse information for arbitrarily long documents using supervised, distantly supervised and simple baselines. |
| Outcome: | The proposed approach shows that the captured discourse information is local and general, even across fine-tuning tasks. |
BeDiscovER: The Benchmark of Discourse Understanding in the Era of Reasoning Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | BeDiscovER evaluates the discourse-level knowledge of modern LLMs . state-of-the-art models exhibit strong performance in arithmetic aspect of temporal reasoning, but struggle with long-dependency reasoning and some subtle semantic and discourse phenomena, such as rhetorical relation classification. |
| Approach: | They evaluate open-source LLMs Qwen3 series, DeepSeek-R1, and frontier reasoning model GPT-5-mini on BeDiscovER . they find that models exhibit strong performance in arithmetic aspect of temporal reasoning, but struggle with long-dependency reasoning and some subtle semantic and discourse phenomena . |
| Outcome: | The proposed framework evaluates open-source LLMs Qwen3 series, DeepSeek-R1, and frontier reasoning model GPT-5-mini. |
CEMTM: Contextual Embedding-based Multimodal Topic Modeling (2025.emnlp-main)
Copied to clipboard
| Challenge: | CEMTM is a context-enhanced multimodal topic model that can infer coherent topic structures from documents . traditional multimodal topics failed to capture deeper cross-modal interactions . large vision language models (LLMs) and LVLMs have shown remarkable capacity to encode rich semantic knowledge from vast corpora. |
| Approach: | They propose a context-enhanced multimodal topic model that uses tokens to weight contributions to topic inference. |
| Outcome: | The proposed model outperforms unimodal and multimodal benchmarks on six multimodal domains and captures semantics in scientific articles. |