Identification of Multiple Logical Interpretations in Counter-Arguments (2025.emnlp-main)
Copied to clipboard
Wenzhi Wang, Paul Reisert, Shoichi Naito, Naoya Inoue, Machi Shimmei, Surawat Pothong, Jungmin Choi, Kentaro Inui
| Challenge: | Counter-arguments (CAs) are a good way to improve learners' critical thinking skills . however, it is difficult to provide every learner tailored feedback due to limited human resources and heavy workloads. |
| Approach: | They propose to annotate a dataset of 134 CAs annotated with 13 logical predicate questions and train a model with Reinforcement Learning with Verifiable Rewards to identify multiple logical interpretations. |
| Outcome: | The proposed model performs on par with larger proprietary models. |
Similar Papers
Designing Logic Pattern Templates for Counter-Argument Logical Structure Analysis (2024.findings-emnlp)
Copied to clipboard
Shoichi Naito, Wenzhi Wang, Paul Reisert, Naoya Inoue, Camélia Guerraoui, Kenshi Yamaguchi, Jungmin Choi, Irfan Robbani, Surawat Pothong, Kentaro Inui
| Challenge: | Despite their effectiveness, the logical attack structure of counterarguments remains unexplored due to its complexity. |
| Approach: | They propose a task to analyze logical attack structure of counterarguments in relation to their corresponding opponent argument using 10 new CA logic patterns. |
| Outcome: | The proposed task achieves high annotator agreement and coverage and high coverage on a dataset of 778 CAs. |
Learning to Explain: Datasets and Models for Identifying Valid Reasoning Chains in Multihop Question-Answering (2020.emnlp-main)
Copied to clipboard
| Challenge: | despite rapid progress in multihop question-answering, models still have trouble explaining why an answer is correct. |
| Approach: | They propose three explanation datasets in which explanations from corpus facts are annotated . they first annotate multiple candidate explanations for each answer, then use crowd-sourcing perturbations to test generalization . |
| Outcome: | The proposed datasets improve explanation quality but still behind the upper bound . the proposed dataset can be used to improve explanations using a BERT-based classifier . |
Limited Generalizability in Argument Mining: State-Of-The-Art Models Learn Datasets, Not Arguments (2025.acl-long)
Copied to clipboard
| Challenge: | Identifying arguments is a prerequisite for various tasks in automated discourse analysis. |
| Approach: | They evaluate four BERT-like transformers on 17 English sentence-level datasets . they find that they tend to rely on lexical shortcuts tied to content words . |
| Outcome: | The proposed models perform best on 17 English sentence-level datasets on common tasks, but their performance drops when applied to unseen datasets. |
Learning First-Order Logic Rules for Argumentation Mining (2025.acl-long)
Copied to clipboard
Yang Sun, Guanrong Chen, Hamid Alinejad-Rokny, Jianzhu Bao, Yuqi Huang, Bin Liang, Kam-Fai Wong, Min Yang, Ruifeng Xu
| Challenge: | Argumentation Mining (AM) aims to extract argumentative structures from texts by identifying argumentation components (ACs) and their argumentative relations (ARs). |
| Approach: | They propose a First- Order Logic reasoning framework for AM to capture logical reasoning paths within argumentative texts. |
| Outcome: | The proposed framework outperforms strong baselines while significantly improving explainability. |
CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures (2024.emnlp-main)
Copied to clipboard
Ekaterina Sviridova, Anar Yeginbergen, Ainara Estarrona, Elena Cabrio, Serena Villata, Rodrigo Agerri
| Challenge: | Existing tools to aid residents in teaching medical doctors to explain decisions are a key objective of AI in education. |
| Approach: | They present a multilingual dataset for Medical Question Answering where doctors can annotate correct and incorrect diagnoses with argument components and argument relations. |
| Outcome: | The proposed dataset consists of 558 clinical cases with explanations in English, Spanish, French, Italian and annotated with argument components and argument relations. |
Beyond Recognising Entailment: Formalising Natural Language Inference from an Argumentative Perspective (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for recognizing textual entailment lack a standardized definition of inference, making it difficult to compare methods trained on different datasets. |
| Approach: | They propose a rigorous approach to align entailment recognition with argumentation theory by using a tool to assist humans in annotating arguments according to the PTA. |
| Outcome: | The proposed model is based on a human-trained dataset and provides insights into non-expert annotator training. |
Decompositional Argument Mining: A General Purpose Approach for Argument Graph Construction (P19-1)
Copied to clipboard
| Challenge: | Argument mining is the process of identifying argumentative structure contained within a text. |
| Approach: | They propose to decompose propositions into four functional components and identify the patterns linking those components to determine argument structure. |
| Outcome: | The proposed method is generic in that it is not tuned for a specific corpus and achieved an F score of 0.79, 0.77 and 0.64 respectively. |
Determining Relative Argument Specificity and Stance for Complex Argumentative Structures (P19-1)
Copied to clipboard
| Challenge: | Existing work on claim specificity and stance has been limited to shallow arguments . a system that can determine the stance of claims employed in argumentation is not sufficient . |
| Approach: | They propose to use a dataset of manually curated argument trees to study claim specificity and stance in argumentation. |
| Outcome: | The proposed dataset consists of manually curated argument trees for 741 controversial topics covering 95,312 unique claims. |
Leveraging pre-trained language models for linguistic analysis: A case of argument structure constructions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Argument structure constructions (ASCs) are lexicogrammatical patterns at the clausal level. |
| Approach: | They evaluate the effectiveness of pre-trained language models in identifying argument structure constructions . they use supervised training with RoBERTa and prompt-guided annotation with GPT-4 . |
| Outcome: | The proposed model outperforms the gold-standard model on three methods . the results show that the model performs better on gold-standardized data . |
CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | CRASS data set and benchmark provide novel test scheme to evaluate large language models . authors present and explain the CRAS data set, a novel basis to test reasoning and natural language understanding of LLMs . |
| Approach: | They introduce a new test scheme utilizing questionized counterfactual conditionals to evaluate large language models. |
| Outcome: | The proposed model sets out to be the most powerful and valid tool to evaluate large language models. |