Building and curating conversational corpora for diversity-aware language science and technology (2022.lrec-1)
Copied to clipboard
| Challenge: | Language resources that capture language use in its natural habitat of social interaction are rare despite the obvious merits of studying the very environment where we all learn and use it everyday. |
| Approach: | They propose to build an analysis pipeline and best practice guidelines for building and curating corpora of everyday conversation in diverse languages. |
| Outcome: | The proposed pipeline can be used to collect and curate conversational corpora in 67 languages and varieties from 28 phyla. |
Similar Papers
From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology (2022.acl-long)
Copied to clipboard
| Challenge: | Informal social interaction is the primordial home of human language. |
| Approach: | They show that linguistically diverse conversational corpora can provide empirical foundations for flexible, localizable language technologies of the future. |
| Outcome: | The results suggest that even relatively small corpora can support robust generalizations about key aspects of interactional infrastructure. |
A Brief Survey of Textual Dialogue Corpora (2022.lrec-1)
Copied to clipboard
| Challenge: | Several dialogue corpora are available for research purposes, but they do not cover all the necessities of real-world applications. |
| Approach: | They analyze available dialogue corpora and propose possible approaches to create new ones. |
| Outcome: | The proposed corpus of human-human dialogues is based on a list of available dialogue corpora . it covers speakers, size, languages, collection, annotations, and domains . some trends are identified and possible approaches are also discussed . |
An Empirical Evaluation of Annotation Practices in Corpora from Language Documentation (2020.lrec-1)
Copied to clipboard
| Challenge: | Language documentation projects have produced substantial amounts of primary data from a wide variety of endangered languages. |
| Approach: | They propose to use common annotation conventions in existing corpora to facilitate their future processing. |
| Outcome: | The proposed formats are based on the common ELAN and Toolbox formats and are used to facilitate their future processing. |
DELTA: A Toolkit for Measuring Linguistic Diversity in Dependency-Parsed Corpora (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing tools for measuring diversity of specific linguistic phenomena are limited . we present an open-source framework for measuring linguistic diversity . |
| Approach: | They propose an open-source framework that integrates dependency tree querying with diversity computation. |
| Outcome: | The proposed framework can measure diversity across multiple linguistic levels and dimensions. |
Beyond Metadata: What Paper Authors Say About Corpora They Use (2021.findings-acl)
Copied to clipboard
| Challenge: | Currently, dataset retrieval relies almost exclusively on metadata provided by the publishers. |
| Approach: | They propose to use metadata to extract review statements from scientific publications . they argue that a crucial piece of information is missing to inform the examination of search results . |
| Outcome: | The proposed analysis is the first of its kind in the field of Natural Language Processing. |
From Text to Context: Contextualizing Language with Humans, Groups, and Communities for Socially Aware NLP (2024.naacl-tutorials)
Copied to clipboard
Adithya V Ganesan, Siddharth Mangalik, Vasudha Varadarajan, Nikita Soni, Swanie Juhng, João Sedoc, H. Andrew Schwartz, Salvatore Giorgi, Ryan L Boyd
| Challenge: | This tutorial will cover the latest techniques and libraries for doing so at each level of analysis. |
| Approach: | This tutorial will cover the latest techniques and libraries for doing so at each level of analysis. |
| Outcome: | The tutorial covers human-centered techniques that provide benefit to traditional document- or word-level NLP tasks. |
The DReaM Corpus: A Multilingual Annotated Corpus of Grammars for the World’s Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Until recently, language descriptions were available in paper form only, with indexes as the only search aid. |
| Approach: | They propose to digitize a multilingual corpus of language descriptions and annotate it with various meta, word, and text attributes to make searching and analysis easier and more useful. |
| Outcome: | The proposed corpus is searchable through a couple of well-established corpus infrastructures. |
Disentangling Codemixing in Chats: The NUS ABC Codemixed Corpus (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on code-mixing have not been able to model human interactions in context. |
| Approach: | They propose to use a general-purpose code-mixing corpus to model human interactions and relationships in context while maintaining ethical standards. |
| Outcome: | The proposed corpus includes over 355,641 messages spanning various code-mixing patterns, with a primary focus on English, Mandarin, and other languages. |
Processing and Understanding Mixed Language Data (D19-2)
Copied to clipboard
| Challenge: | Multilingual communities exhibit code-mixing, mixing of two or more languages in a single conversation . social media and other informal interactive platforms are allowing code-switching in user-generated text . |
| Approach: | a tutorial aims to provide a foundation for researchers to study code-mixing in multilingual communities. |
| Outcome: | a tutorial aims to provide new researchers with a foundation in linguistics and computational aspects of code-mixing. |
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI (2024.findings-eacl)
Copied to clipboard
Jianguo Zhang, Kun Qian, Zhiwei Liu, Shelby Heinecke, Rui Meng, Ye Liu, Zhou Yu, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | DialogStudio is the largest and most diverse collection of dialogue datasets . existing datasets lack diversity and comprehensiveness, authors say . |
| Approach: | They introduce DialogStudio: the largest and most diverse collection of dialogue datasets . DialogStuio aggregates more than 80 diverse dialogue dataset . |
| Outcome: | a new dataset is created to improve the quality and diversity of dialogue datasets . DialogStudio is the largest and most diverse collection of dialogue data . |