| Challenge: | Existing methods for argument quality assessment do not consider multi-perspective evaluation due to subjective nature of arguments. |
| Approach: | They propose a multi-persona framework for argument quality assessment that simulates diverse evaluator perspectives through large language models. |
| Outcome: | The proposed framework outperforms baselines while providing comprehensive multi-perspective rationales on IBM-Rank-30k and IBM-ArgQ-5.3kArgs datasets. |
Similar Papers
Towards a Perspectivist Turn in Argument Quality Assessment (2025.naacl-long)
Copied to clipboard
| Challenge: | Argument quality is a key aspect of computational argumentation (CA), but it still exhibits a high degree of subjectivity in perception. |
| Approach: | They propose to use a multi-layered classification to target two aspects of argument quality in a systematic review of NLP datasets. |
| Outcome: | The proposed model improves the quality of annotators and their ability to be used in perspectivist research. |
Automatic Argument Quality Assessment - New Datasets and Methods (D19-1)
Copied to clipboard
Assaf Toledo, Shai Gretz, Edo Cohen-Karlik, Roni Friedman, Elad Venezian, Dan Lahav, Michal Jacovi, Ranit Aharonov, Noam Slonim
| Challenge: | 6.3k arguments were collected from contributors of various levels, and are released as part of this work. |
| Approach: | They propose to use a language model to annotate arguments for argument ranking and argument-pair classification. |
| Outcome: | The proposed methods outperform state-of-the-art methods in the argument ranking task and argument-pair classification task. |
Efficient Pairwise Annotation of Argument Quality (2020.acl-main)
Copied to clipboard
| Challenge: | Especially crowdsourcing suffers from assessors having different reference frames to base their judgments on and task instructions being nondescript and therefore unhelpful in ensuring consistency. |
| Approach: | They propose an efficient annotation framework for argument quality that uses a stochastic transitivity model and an effective sampling strategy to infer high-quality labels. |
| Outcome: | The proposed model significantly outperforms existing annotation procedures and offers statistical insights into argument quality. |
Argument Quality Assessment in the Age of Instruction-Following Large Language Models (2024.lrec-main)
Copied to clipboard
Henning Wachsmuth, Gabriella Lapesa, Elena Cabrio, Anne Lauscher, Joonsuk Park, Eva Maria Vecchi, Serena Villata, Timon Ziegenbein
| Challenge: | Argument quality assessment is critical for opinion formation, decision making, writing education, and the like. |
| Approach: | They propose to use large language models to leverage knowledge across contexts to enable a much more reliable assessment. |
| Outcome: | The proposed approach improves the quality of argumentation and the ability to leverage knowledge across contexts. |
Bridging Argument Quality and Deliberative Quality Annotations with Adapters (2023.findings-eacl)
Copied to clipboard
| Challenge: | Assessing the quality of an argument is a complex, highly subjective task . argument quality dimensions are complex and dependent on the context in which it is assessed . |
| Approach: | They propose a multi-task learning framework that incorporates knowledge about related dimensions into the learning process. |
| Outcome: | The proposed framework improves quality prediction in an extrinsic, out-of-domain task. |
Towards Comprehensive Argument Analysis in Education: Dataset, Tasks, and Method (2025.acl-long)
Copied to clipboard
| Challenge: | Existing research on argument mining has proposed various argument annotation schemes and tasks. |
| Approach: | They propose a framework comprising 14 fine-grained relation types to capture the interplay between argument components for a thorough understanding of argument structure. |
| Outcome: | The proposed framework captures the interplay between argument components for a thorough understanding of argument structure. |
Let’s discuss! Quality Dimensions and Annotated Datasets for Computational Argument Quality Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Argumentation is a key competence and an important cultural technique in democratic societies. |
| Approach: | They propose to create domain-specific datasets and methods to assess argument quality. |
| Outcome: | The proposed methods address gaps in the literature and aid future research in the domain. |
Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It’s Best to Relate Perspectives! (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to subjectivity in natural language processing are subjective . authors argue that disagreement should not be regarded as a problem . |
| Approach: | They propose to account for subjective perspectives of individuals and objective concepts that build a common ground between annotators. |
| Outcome: | The proposed architectures increase the averaged annotator-individual F1-scores up to 43% over a majority-label model. |
ArgAnalysis35K : A large-scale dataset for Argument Quality Analysis (2023.acl-long)
Copied to clipboard
| Challenge: | Existing datasets in argument quality detection lack quality, quantity and diversity of topics and arguments. |
| Approach: | They propose a dataset that adds a detailed explanation of why the argument made is true, applicable or impactful. |
| Outcome: | The proposed dataset covers 34,890 high-quality argument-analysis pairs and is the largest of its kind to our knowledge. |
Graph Embeddings for Argumentation Quality Assessment (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Argumentation is the process by which arguments are constructed, compared, evaluated in several respects and judged in order to establish whether any of them is warranted. |
| Approach: | They propose to annotate 1908 arguments tagged with quality facets from a resource of 402 persuasive essays and to use them to create a neural architecture that takes into account the support and attack relations holding among the arguments. |
| Outcome: | The proposed neural architecture outperforms state-of-the-art and standard arguments on the persuasive essays dataset. |