DISRPT: A Multilingual, Multi-domain, Cross-framework Benchmark for Discourse Processing (2024.lrec-main)
Copied to clipboard
Chloé Braud, Amir Zeldes, Laura Rivière, Yang Janet Liu, Philippe Muller, Damien Sileo, Tatsuya Aoyama
| Challenge: | DISRPT is a multilingual, multi-domain, and cross-framework benchmark dataset for discourse processing. |
| Approach: | They present a multilingual, multi-domain, and cross-framework benchmark dataset for discourse processing that includes 13 languages and 24 corpora covering about 4 millions tokens and around 250,000 discourse relation instances from 4 discourse frameworks. |
| Outcome: | The DISRPT dataset includes data from 24 corpora covering about 4 millions tokens and around 250,000 discourse relation instances from 4 discourse frameworks. |
Similar Papers
CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse paRsing in conversations (2025.findings-acl)
Copied to clipboard
| Challenge: | Discourse parsing datasets based on conversations are restricted to a single domain . a lack of discourse structures in audio-based conversations is a challenge . |
| Approach: | They introduce CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse parsing in conversations. |
| Outcome: | The proposed corpus is code-mixed in Hindi and English and annotated with nine discourse relations. |
Joint Learning of Syntactic Features Helps Discourse Segmentation (2020.lrec-1)
Copied to clipboard
| Challenge: | Discourse segmentation is a task of fragmenting text into minimal disjoint chunks of text called Elementary Discourse Units (EDUs). |
| Approach: | They propose a framework for multi-lingual discourse segmentation with BERT . they cast the problem as a token classification problem and jointly learn syntactic features like part-of-speech tags and dependency relations. |
| Outcome: | Experiments in English, Dutch, German, Portuguese Brazilian and Basque show that the proposed model performs better across languages. |
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks focus primarily on language understanding tasks. |
| Approach: | They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages. |
| Outcome: | Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve. |
GlobalBench: A Benchmark for Global Progress in Natural Language Processing (2023.emnlp-main)
Copied to clipboard
Yueqi Song, Simran Khanuja, Pengfei Liu, Fahim Faisal, Alissa Ostapenko, Genta Winata, Alham Aji, Samuel Cahyawijaya, Yulia Tsvetkov, Antonios Anastasopoulos, Graham Neubig
| Challenge: | despite advances in NLP, significant disparities in performance across languages still exist . prior benchmarks focused on a limited number of tasks and languages, but now GlobalBench tracks progress on all languages. |
| Approach: | They propose to use global benchmarks to track progress on all NLP datasets in all languages. |
| Outcome: | a new tool tracks progress on all NLP datasets in all languages and tracks per-speaker utility and equity . globalbench is designed to identify the most under-served languages and reward research efforts . a globalbech is available at https://github.com/neulab/globalbench. |
GDTB: Genre Diverse Data for English Shallow Discourse Parsing across Modalities, Text Types, and Domains (2024.emnlp-main)
Copied to clipboard
Yang Janet Liu, Tatsuya Aoyama, Wesley Scivetti, Yilun Zhu, Shabnam Behzad, Lauren Levine, Jessica Lin, Devika Tiwari, Amir Zeldes
| Challenge: | Existing shallow discourse parsing systems focus on the Wall Street Journal corpus, but the data is limited to the news domain and is 35 years old. |
| Approach: | They propose to use the Wall Street Journal corpus as a benchmark for PDTB-style shallow discourse parsing. |
| Outcome: | The proposed dataset is compatible with PDTB, but suffers from degradation out-of-domain. |
ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to building cross-lingual summarization systems on dialogue documents are limited. |
| Approach: | They propose a benchmark dataset for building cross-lingual summarization systems on dialogue documents. |
| Outcome: | The proposed model outperforms pipeline models on ClidSum and mDialBART. |
DraDDP: A Multimodal Multi-Party Dialogue Discourse Parsing Dataset (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on multi-party dialogue discourse parsing focus on textual modality and two-party dialog . et al., 2016) focused on text-based discourse parses, ignoring the complexity and richness of multimodal interactions in real-world scenarios. |
| Approach: | They construct the first publicly available English multimodal dataset for multi-party dialogue discourse parsing based on American TV dramas. |
| Outcome: | The proposed dataset contains 495 dialogue segments with 6,374 utterances and 9.1 hours of parallel video content, covering rich multi-party interaction scenarios. |
CodeSwitch-Reddit: Exploration of Written Multilingual Discourse in Online Discussion Forums (D19-55)
Copied to clipboard
| Challenge: | a dataset of written multilingual productions is released to explore the sociolinguistic underpinnings of written code-switching . |
| Approach: | They use a reddit discussion platform to collect written code-switched productions . they examine whether oral code-witching findings are carried over to written code . |
| Outcome: | The proposed dataset can facilitate a range of research and practical activities. |
CodeSwitch-Reddit: Exploration of Written Multilingual Discourse in Online Discussion Forums (D19-1)
Copied to clipboard
| Challenge: | a dataset of written code-switched productions is curated from topical threads of multiple bilingual communities on the Reddit discussion platform. |
| Approach: | They analyze a dataset of written code-switched productions curated from multiple bilingual communities on the reddit discussion platform and examine whether findings are carried over to written codeswitching in discussion forums. |
| Outcome: | The proposed dataset can facilitate a range of research and practical activities. |
UniteD-SRL: A Unified Dataset for Span- and Dependency-Based Multilingual and Cross-Lingual Semantic Role Labeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual and cross-lingual Semantic Role Labeling (SRL) has attracted increasing attention as multilingual text representation techniques have become more effective and widely available. |
| Approach: | They propose a benchmark for multilingual and cross-lingual, span- and dependency-based SRL that provides expert-curated parallel annotations using a common predicate-argument structure inventory. |
| Outcome: | The proposed benchmark provides expert-curated parallel annotations using a common predicate-argument structure inventory, allowing direct comparisons across languages and encouraging studies on cross-lingual transfer in SRL. |