ParsFEVER: a Dataset for Farsi Fact Extraction and Verification (2021.starsem-1)
Copied to clipboard
Majid Zarharan, Mahsa Ghaderan, Amin Pourdabiri, Zahra Sayedi, Behrouz Minaei-Bidgoli, Sauleh Eetemadi, Mohammad Taher Pilehvar
| Challenge: | Existing methods for fact-checking and verification require large amounts of annotated data, but this is limited to low-resource languages. |
| Approach: | They present a first publicly available Farsi dataset for fact extraction and verification . they use the construction procedure of the standard English dataset for the task . |
| Outcome: | The proposed dataset improves on the standard English dataset and is available on github. |
Similar Papers
Exploring Factual Entailment with NLI: A News Media Study (2024.starsem-1)
Copied to clipboard
| Challenge: | Recent studies have focused on the relationship between factuality and Natural Language Inference (NLI). |
| Approach: | They propose a novel annotation scheme that models factual rather than textual entailment and use it to annotate a dataset of naturally occurring sentences from news articles. |
| Outcome: | The proposed annotation scheme can be used to model factual relationships on a dataset of naturally occurring sentences from news articles. |
When Truth Matters - Addressing Pragmatic Categories in Natural Language Inference (NLI) by Large Language Models (LLMs) (2023.starsem-1)
Copied to clipboard
| Challenge: | In this paper, we examine the ability of large language models (LLMs) to accommodate different pragmatic sentence types, such as questions, commands, and sentence fragments for natural language inference (NLI). |
| Approach: | They propose to fine-tune large language models to accommodate different sentence types for natural language inference (NLI) they also explore ChatGPT's concept of entailment by using a symbolic semantic parser. |
| Outcome: | The proposed models can accommodate different sentence types without losing too much accuracy on MNLI-matched models. |
A Closer Look at Claim Decomposition (2024.starsem-1)
Copied to clipboard
| Challenge: | Recent work uses claim decomposition to determine how well supported a claim is for applications in factual precision of generated text, entailment of human generated text and claim verification. |
| Approach: | They propose an LLM-based approach to generating decompositions inspired by Bertrand Russell’s theory of logical atomism and neo-Davidsonian semantics and demonstrate its improved decomposing quality over previous methods. |
| Outcome: | The proposed method improves on the FActScore and a Bertrand Russell-inspired approach to generating decompositions inspired by neo-Davidsonian semantics and improves decomposability quality. |
A Benchmark Suite of Japanese Natural Questions (2024.starsem-1)
Copied to clipboard
| Challenge: | Existing studies to solve QA tasks in an integrated manner are not available in other languages because of the lack of QA datasets. |
| Approach: | They build a Japanese version of Natural Questions using natural questions from query logs of a search engine and crowdsource it using crowdsourcing. |
| Outcome: | The proposed datasets are based on natural questions from Japanese search engines and crowdsourced. |
MASSIVE Multilingual Abstract Meaning Representation: A Dataset and Baselines for Hallucination Detection (2024.starsem-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic formalism that captures the core meaning of an utterance. |
| Approach: | They propose to use AMR to map meanings of 1,685 utterances to 50+ languages to build a dataset 20 times larger than existing resources. |
| Outcome: | The proposed dataset covers more languages, has more utterances, and has localized or translated entities for each language. |
Event Semantic Knowledge in Procedural Text Understanding (2023.starsem-1)
Copied to clipboard
| Challenge: | Annotators’ reliance on commonsense knowledge to annotate implicit state information is a challenge for entity state tracking. |
| Approach: | They propose a method for entity state tracking that incorporates commonsense entity-centric knowledge from ConceptNet into a BERT-based neural-symbolic architecture. |
| Outcome: | The proposed model outperforms existing models on the ProPara dataset and is domain-agnostic. |
Multilingual Neural Semantic Parsing for Low-Resourced Languages (2021.starsem-1)
Copied to clipboard
| Challenge: | a large amount of training data is needed to understand multilingual semantic parsing models. |
| Approach: | They propose to use machine translation to bootstrap multilingual training data from English data. |
| Outcome: | The proposed model outperforms existing models on human-written sentences and the state-of-the-art models on the public NLMaps dataset. |
Learning as Abduction: Trainable Natural Logic Theorem Prover for Natural Language Inference (2020.starsem-1)
Copied to clipboard
| Challenge: | a logic-based approach to Natural Language Inference is becoming less and less common . a new method uses semantic relations to abduct sentences from data . |
| Approach: | They propose a method to reverse a theorem-proving procedure to abduct semantic relations from data. |
| Outcome: | The proposed method improves the performance of the theorem prover on the SICK dataset by 1.4% while maintaining high precision (>94%) |
DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Models (2022.starsem-1)
Copied to clipboard
| Challenge: | ArgumentAnalyst is a multi-dimensional, modular framework for performing deep argument analysis using existing pre-trained language models (PTLMs). |
| Approach: | They propose a multi-dimensional, modular framework for performing deep argument analysis using current pre-trained language models (PTLMs) ArgumentAnalyst reconstructs argumentative texts as valid arguments by inserting missing premises and conclusions, formalizing inferences, and coherently linking the reconstruction to the source text. |
| Outcome: | The proposed framework can replicate established heuristics, cope with plurality of correct solutions, and exploit higher-order evidence. |
Teach the Rules, Provide the Facts: Targeted Relational-knowledge Enhancement for Textual Inference (2021.starsem-1)
Copied to clipboard
| Challenge: | InferBERT is a method to enhance transformer-based inference models with relevant relational knowledge. |
| Approach: | They propose to enhance transformer-based inference models with relevant relational knowledge by injecting relevant facts at test time into the model. |
| Outcome: | The proposed method outperforms existing models on the challenge datasets while outperforming existing models. |