Effectiveness of French Language Models on Abstractive Dialogue Summarization Task (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained language models have established the state-of-the-art on various natural language processing tasks, including dialogue summarization. |
| Approach: | They propose to use several language specific pre-trained models to summarize spontaneous oral dialogues in French using several language-specific pre-trainers: BARThez, BelGPT-2, mBARThes, and mT5. |
| Outcome: | The proposed models outperform the existing models on the DECODA (Call Center) dialogue corpus and show that they are far superior to the current models. |
Similar Papers
Towards Comprehensive Language Analysis for Clinically Enriched Spontaneous Dialogue (2024.lrec-main)
Copied to clipboard
| Challenge: | Contemporary NLP has progressed from feature-based classification to fine-tuning and prompt-based techniques . many of these techniques remain understudied in the context of real-world, clinically enriched spontaneous dialogue. |
| Approach: | They investigate the efficacy and overall performance of a range of NLP techniques on transcribed speech from patients with schizophrenia and other disorders. |
| Outcome: | The proposed methods are effective in analyzing transcribed speech from patients with schizophrenia and healthy controls taking a clinically-validated language test. |
BARThez: a Skilled Pretrained French Sequence-to-Sequence Model (2021.emnlp-main)
Copied to clipboard
| Challenge: | Inductive transfer learning has taken the entire NLU field by storm, with models such as BERT and BART setting new state-of-the-art on countless tasks. |
| Approach: | They introduce a large-scale pretrained seq2seq model for French that is very competitive with state-of-the-art BERT-based French language models such as CamemBERT and FlauBERT. |
| Outcome: | The proposed model outperforms existing models on discriminative and generative tasks on a French summarization dataset. |
Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization (2021.acl-long)
Copied to clipboard
| Challenge: | Existing dialogue summarization systems encode text with a number of general semantic features, but these are often not available in open-domain tools. |
| Approach: | They propose to use DialoGPT to label three types of features on two datasets . they propose to employ pre-trained and non-pre-tried models as dialogue annotators . |
| Outcome: | The proposed method improves on two dialogue summarization datasets and achieves state-of-the-art performance. |
STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension (2022.emnlp-main)
Copied to clipboard
Borui Wang, Chengcheng Feng, Arjun Nair, Madelyn Mao, Jai Desai, Asli Celikyilmaz, Haoran Li, Yashar Mehdad, Dragomir Radev
| Challenge: | Abstractive dialogue summarization is an important standalone task in natural language processing, but no previous work has explored whether it can be used to boost an NLP system's performance on other important dialogue comprehension tasks. |
| Approach: | They propose a novel type of dialogue summarization task that decomposes and imitates the hierarchical, systematic and structured mental process that human beings usually go through when understanding and analyzing dialogues. |
| Outcome: | The proposed model improves the performance of transformer encoder language models on two important dialogue comprehension tasks. |
Hello, It’s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems (D19-56)
Copied to clipboard
| Challenge: | Statistical conversational systems are complex, timeintensive, expensive, and not easily transferable due to data scarcity. |
| Approach: | They propose a task-oriented dialogue model that operates on text input . they validate it on multi-domain task-orientated dialogues from a multi-word dataset . |
| Outcome: | The proposed model bypasses explicit policy and language generation modules on multi-domain task-oriented dialogues from the MultiWOZ dataset. |
FREDSum: A Dialogue Summarization Corpus for French Political Debates (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in deep learning have improved the performance of abstractive summarization systems. |
| Approach: | They present a dataset of french political debates to enhance resources for multi-lingual dialogue summarization. |
| Outcome: | The proposed dataset will be made publicly available for use by the research community. |
Multilingual Generation in Abstractive Summarization: A Comparative Study (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing models for multilingual generation lack thorough analysis due to extensive linguistic diversity. |
| Approach: | They propose to classify multilingual generation methodologies into three categories based on their underlying modeling principles . they introduce an automatic metric to mitigate spurious correlations associated with language mixing . |
| Outcome: | The proposed model improves in high-resource, low-resourced, and zero-shot scenarios. |
Post-Training Dialogue Summarization using Pseudo-Paraphrasing (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to dialogue summarization use dialogue-specific features that require additional knowledge to recognize or make the models harder to tune. |
| Approach: | They propose to post-train pretrained language models to rephrase from dialogue to narratives and fine-tune them as usual. |
| Outcome: | The proposed approach outperforms existing models by summary quality and implementation costs. |
Pretrained Language Models for Dialogue Generation with Multiple Input Sources (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks. |
| Approach: | They propose to fuse attention information from multiple input sources to achieve better relevance with dialogue history than simple fusion baselines. |
| Outcome: | The proposed models deliver higher relevance with dialogue history than baselines. |
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for summarizing dialogues lack in taking into account the structure of dialogues and rely heavily on labeled data. |
| Approach: | They propose a pre-trained encoder-decoder model for summarizing dialogues in any new domain. |
| Outcome: | The proposed model outperforms existing methods on six datasets and shows ROUGE scores in zero-shot and few-shot settings. |