Papers by Siyao Peng
VariErr NLI: Separating Annotation Error from Human Label Variation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing work on label variation and annotation errors has focused on them in isolation. |
| Approach: | They propose a 2-round annotation procedure to separate human label variation from annotation errors by pairing valid explanations with annotators' validations. |
| Outcome: | The proposed procedure is based on the NLI task in English and contains 7,732 valid judgements on 1,933 explanations for 500 re-annotated items. |
“Seeing the Big through the Small”: Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Human label variation arises when multiple human annotators provide different labels for valid reasons. |
| Approach: | They propose to use crowd workers to represent human judgment distributions or expert linguists to provide detailed explanations for their chosen labels. |
| Outcome: | The proposed model can approximate human judgment distributions using a small number of expert labels and explanations. |
A Rose by Any Other Name: LLM-Generated Explanations Are Good Proxies for Human Explanations to Collect Label Distributions on NLI (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent research has shown that explanations provide valuable information for understanding human label variation (HLV) Large language models (LLMs) can approximate HJD from a few human-provided label-explanation pairs, but collecting explanations for every label is still time-consuming. |
| Approach: | They propose to use Large Language Models (LLMs) as annotators to generate model explanations for a few given human labels. |
| Outcome: | The proposed models can generate human-provided explanations from human labels, but they are still time-consuming. |
SPLICE: A Singleton-Enhanced PipeLIne for Coreference REsolution (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing attempts to integrate singleton mention detection into end-to-end coreference resolution for English have been hampered by the lack of singletont mention spans in the OntoNotes benchmark. |
| Approach: | They propose a two-step neural mention and coreference resolution system that integrates singleton mentions with OntoNotes syntax trees to achieve a near approximation of the Ontonotes dataset with all singletont mentions. |
| Outcome: | The proposed system achieves 94% recall on a sample of gold singletons. |
Sebastian, Basti, Wastl?! Recognizing Named Entities in Bavarian Dialectal Data (2024.lrec-main)
Copied to clipboard
Siyao Peng, Zihang Sun, Huangyan Shan, Marie Kolm, Verena Blaschke, Ekaterina Artemova, Barbara Plank
| Challenge: | Named Entity Recognition (NER) is a fundamental task to extract key information from texts, but annotated resources are scarce for dialects. |
| Approach: | They present the first dialectal NER dataset for German, BarNER, with 161K tokens annotated on Bavarian Wikipedia articles and tweets. |
| Outcome: | The proposed dataset improves on bar-wiki and moderately on bartweet with training first on Bavarian . |
GCDT: A Chinese RST Treebank for Multigenre and Multilingual Discourse Parsing (2022.aacl-short)
Copied to clipboard
| Challenge: | GCDT is the largest hierarchical discourse treebank for Mandarin Chinese in the framework of Rhetorical Structure Theory (RST). |
| Approach: | They propose to use a Chinese hierarchical discourse treebank to parse Mandarin Chinese using relation inventory and a multilingual training program. |
| Outcome: | The proposed dataset includes state-of-the-art scores for Chinese RST parsing and RST Parsing on the English GUM dataset, using cross-lingual training in Chinese and English with multilingual embeddings. |
What Media Frames Reveal About Stance: A Dataset and Study about Memes in Climate Change Discourse (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Media framing is a method of shaping public perceptions of issues, but the interaction between stance and media frame remains unexplored. |
| Approach: | They propose to use a dataset of climate-change memes annotated with stance and media frames to conceptualize and computationally explore this interaction. |
| Outcome: | The proposed dataset includes 1,184 climate-change memes sourced from 47 subreddits and enables analysis of frame prominence over time and communities. |
EVADE: LLM-Based Explanation Generation and Validation for Error Detection in NLI (2026.findings-acl)
Copied to clipboard
| Challenge: | Human label variation (HLV) arises when multiple labels are valid for the same instance. |
| Approach: | They propose a framework for generating and validating explanations to detect errors using large language models (LLMs) EVADE framework provides broader explanation coverage and requires less human intervention . |
| Outcome: | The proposed framework provides broader explanation coverage, requires less human intervention, and delivers better downstream performance in predicting label distributions. |
Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations (2026.findings-acl)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets often exhibit label variation. |
| Approach: | They extend LiTEx taxonomy to two NLI datasets and jointly analyze label variation and label variation. |
| Outcome: | The proposed model combines explanations as a lens to analyze variation in NLI annotations and examine individual differences in reasoning. |
MaiBaam: A Multi-Dialectal Bavarian Universal Dependency Treebank (2024.lrec-main)
Copied to clipboard
| Challenge: | Despite the success of the Universal Dependencies (UD) project, there is still a lack of diversity within high-resource languages and their closely related non-standard languages and dialects. |
| Approach: | They propose to annotate Bavarian with part-of-speech and syntactic dependency information manually in UD and to highlight morphosyntactical differences between the closely related languages. |
| Outcome: | The proposed treebank covers multiple genres including wiki, fiction, grammar examples, social, non-fiction and Bavarian. |
A Corpus of Adpositional Supersenses for Mandarin Chinese (2020.lrec-1)
Copied to clipboard
| Challenge: | Adpositions are frequent markers of semantic relations, but they are highly ambiguous and vary significantly from language to language. |
| Approach: | They propose to annotate Chinese adpositions in a corpus with all aforementioned supersenses . they adapt a framework that defined a set of supersens according to ostensibly language-independent criteria . |
| Outcome: | The proposed corpus is the first to be broadly annotated with adposition semantics in Chinese . it shows that the supersense categories are well-suited to Chinese adepositions despite syntactic differences from English . |
LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evidence of human label variation in Natural Language Inference (NLI) however, within-label variation is an additional challenge. |
| Approach: | They propose a linguistically-informed taxonomy for categorizing free-text explanations in English that captures different reasoning strategies behind NLI explanations with a particular focus on within-label variation. |
| Outcome: | The proposed taxonomy can be used to classify explanations in English using a linguistically-informed taxonomies. |
AMALGUM – A Free, Balanced, Multilayer English Web Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | a corpus of 4M tokens is available online with a large number of high-quality annotation layers. |
| Approach: | They propose to use a genre-balanced English web corpus with multiple annotation layers . they harness knowledge from multiple annotation layer to achieve a "better than NLP" benchmark . |
| Outcome: | The proposed corpus is genre-balanced and features high-quality automatic annotation layers. |