| Challenge: | Existing methods to detect low quality work do not address the correctness of the data. |
| Approach: | They propose an unsupervised method for measuring speaker metadata plausibility of a collection, i.e., evaluating the match (or lack thereof) between contributors and speakers. |
| Outcome: | The proposed method shows high precision in automatically classifying contributor alignment (>0.94). |
Similar Papers
Samrómur: Crowd-sourcing large amounts of data (2022.lrec-1)
Copied to clipboard
| Challenge: | Samrómur is the largest prompted speech collection effort for Icelandic so far and verification is as monumental as the collection itself. |
| Approach: | They propose to collect large and diverse corpus for automatic speech recognition and similar tools using crowd-sourced donations. |
| Outcome: | The collected utterances are based on the Mozilla Common Voice platform and are available for free on the Samrómur collection platform. |
Voices in a Crowd: Searching for clusters of unique perspectives (2024.emnlp-main)
Copied to clipboard
| Challenge: | Proposed solutions aim to capture minority perspectives by either modelling annotator disagreements or grouping annotators based on shared metadata. |
| Approach: | They propose a framework that trains models without encoding annotator metadata and creates clusters of similar opinions, that are called voices. |
| Outcome: | The proposed framework captures minority perspectives based on demographic factors in two distinct datasets while also capturing majority perspectives. |
Speaker Verification in Agent-generated Conversations (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have increased the capabilities of conversational AI to solve challenging dialogue problems. |
| Approach: | They propose a task to verify whether two sets of utterances originate from the same speaker. |
| Outcome: | The proposed task aims to verify whether two sets of utterances originate from the same speaker. |
The Million Authors Corpus: A Cross-Lingual and Cross-Domain Wikipedia Dataset for Authorship Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Authorship verification (AV) is a crucial task for identity verification, accountlinking, historical linguistics, and AI-generated text identification. |
| Approach: | They propose to use Wikipedia's Million Authors Corpus to examine authorship verification models on a broad scale. |
| Outcome: | The proposed dataset includes 60.08M textual chunks, contributed by 1.29M Wikipedia authors. |
Multi-Level Attention Aggregation for Language-Agnostic Speaker Replication (2024.eacl-short)
Copied to clipboard
| Challenge: | Recent advances in speech synthesis research have enabled the generation of natural-sounding speech, which has prompted a notable shift in TTS research towards the synthesis of speech in the voices of both seen and unseen speakers. |
| Approach: | They propose a multi-level attention aggregation approach that probes and amplifies various speaker-specific attributes in a hierarchical manner. |
| Outcome: | The proposed model achieves substantial speaker similarity and generalizes to out-of-domain (OOD) cases. |
SpeakerSleuth: Can Large Audio-Language Models Judge Speaker Consistency across Multi-turn Dialogues? (2026.acl-long)
Copied to clipboard
| Challenge: | Large Audio-Language Models (LALMs) are a popular approach for evaluating speech quality, yet their ability to assess speaker consistency across multi-turn dialogues remains unexplored. |
| Approach: | They construct 1,818 human-verified evaluation instances across four datasets spanning synthetic and real speech, with controlled acoustic difficulty. |
| Outcome: | The proposed model performs better in comparing and ranking acoustic variants, demonstrating inherent acustic discrimination capabilities. |
Far-Field Speaker Recognition Benchmark Derived From The DiPCo Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Using a publicly-available corpus, we propose a far-field speaker verification benchmark. |
| Approach: | They propose a far-field speaker verification benchmark derived from the publicly available DiPCo corpus. |
| Outcome: | The proposed tasks are very challenging and hope to inspire the speech community to develop new methods and systems for this challenging domain. |
SVeritas: Benchmark for Robust Speaker Verification under Diverse Conditions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks only evaluate a subset of potential conditions, missing others entirely. |
| Approach: | a new benchmark suite evaluates speaker verification models under a variety of stressors . a san francisco-based team evaluates models under natural and background conditions . |
| Outcome: | a new benchmark suite evaluates speaker verification models under stressors under a variety of conditions . the results show that some models perform better under stress conditions than others . |
ALLIES: A Speech Corpus for Segmentation, Speaker Diarization, Speech Recognition and Speaker Change Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | a meta corpus of audio files is used to gather, annotate and transcribe speech . a large number of speech databases are needed to perform multi-speaker tasks such as speaker diarization and speaker change detection. |
| Approach: | They propose to use human feedback to homogenize and correct speaker labels among the audio files by integrating human feedback within a speaker verification system. |
| Outcome: | The proposed protocol evaluates speech segmentation, speaker diarization, speech transcription and speaker change detection using human feedback. |
Don’t paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsing (D19-1)
Copied to clipboard
| Challenge: | a major hurdle on the road to conversational interfaces is the difficulty in collecting data that maps language utterances to logical forms . crowdsourcing and crowdsourcing have been used to generate pseudo-language paired with logical form . however, this data collection method often leads to low performance on real data . |
| Approach: | They propose a method that uses crowdsourcing to map language utterances to logical forms . they quantify the effects of mismatches between the true and induced distributions . |
| Outcome: | The proposed method leads to 70.6 accuracy on the true distribution, compared to 51.3 in paraphrase-based data collection. |