Automatic Assessment of Conceptual Text Complexity Using Knowledge Graphs (C18-1)
Copied to clipboard
| Challenge: | Existing methods to assess text complexity only at lexical and syntactic levels have not been attempted. |
| Approach: | They propose to automatically estimate conceptual complexity using graph-based measures on a large knowledge base. |
| Outcome: | The proposed measures achieve high discriminative power even in a default setup. |
Similar Papers
CoCo: A Tool for Automatically Assessing Conceptual Complexity of Texts (2020.lrec-1)
Copied to clipboard
| Challenge: | Traditional text complexity assessment only takes into account lexical and lexiconal complexity. |
| Approach: | They propose a tool for automatic assessment of conceptual text complexity based on the current state-of-the-art unsupervised approach . they compare the current implementation with the state of the art and discuss the influence of the choice of entity linker on the performance of the tool. |
| Outcome: | The proposed tool can be personalized and adapted to the needs of struggling readers. |
When Shallow is Good Enough: Automatic Assessment of Conceptual Text Complexity using Shallow Semantic Features (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to automatic assessment of text complexity focus on syntactic and lexical complexity. |
| Approach: | They propose to use graph-based deep semantic features to automatically assess conceptual text complexity by using DBpedia as a proxy to human knowledge. |
| Outcome: | The proposed features outperform the state-of-the-art features on pairwise comparison of two versions of the same text and five-level classification task. |
Estimating Lexical Complexity from Document-Level Distributions (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for complexity estimation are limited to entire documents . health assessment tools are too short for existing methods to apply . |
| Approach: | They propose a two-step approach for estimating lexical complexity that does not rely on pre-annotated data. |
| Outcome: | The proposed method is tested on the Norwegian language and compares with other assessment tools. |
A Spreading Activation Framework for Tracking Conceptual Complexity of Texts (P19-1)
Copied to clipboard
| Challenge: | Existing models for assessing conceptual complexity of texts are lacking . conceptual complexity accounts for background knowledge necessary to understand mentioned concepts . |
| Approach: | They propose an unsupervised approach for assessing conceptual complexity of texts based on spreading activation using DBpedia knowledge graph as a proxy to long-term memory. |
| Outcome: | The proposed model outperforms current state of the art in assessing conceptual complexity of texts. |
ARTS: Assessing Readability & Text Simplicity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text simplification evaluation measures do not include simplicity labels on text level, as they are mainly based on a relational concept. |
| Approach: | They propose a method for language-independent construction of datasets for simplicity assessment using pairwise comparisons of texts in conjunction with an Elo algorithm to produce a simplicity ranking and simplicity scores. |
| Outcome: | The proposed method produces a ranking and simplicity scores for human-labeled and three GPT-labelled datasets. |
Inspecting the concept knowledge graph encoded by modern language models (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models are used to solve tasks such as summarization and information retrieval. |
| Approach: | They propose to use word embeddings, text generators, context encoders to extract underlying knowledge graphs of nine influential language models. |
| Outcome: | The proposed model is able to encode word embeddings, text generators, and context encoders, but suffers from several inaccuracies. |
No Simple Answer to Data Complexity: An Examination of Instance-Level Complexity Metrics for Classification Tasks (2025.naacl-long)
Copied to clipboard
| Challenge: | Understanding data complexity at the instance level has become increasingly important in Natural Language Processing (NLP) and machine learning (ML). |
| Approach: | They empirically examine the relationship between instance-level complexity scores and metric selection for classification tasks. |
| Outcome: | The results show that storing training loss provides similar complexity rankings to other methods, but not demographic fairness, even in downstream predictions. |
Editing Conceptual Knowledge for Large Language Models (2024.findings-emnlp)
Copied to clipboard
Xiaohan Wang, Shengyu Mao, Shumin Deng, Yunzhi Yao, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen, Ningyu Zhang
| Challenge: | Existing knowledge editing methods can modify concept-level definitions, but they can distort instantial knowledge in LLMs, leading to poor performance. |
| Approach: | They construct a benchmark dataset ConceptEdit and establish new metrics for evaluation to investigate the editing capability of LLMs. |
| Outcome: | The proposed methods can modify concept definitions but can distort instantial knowledge in LLMs, leading to poor performance. |
Evaluating LLMs for Targeted Concept Simplification for Domain-Specific Texts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Simplifying the entire text makes it understandable but sometimes removes important details. |
| Approach: | They propose a simplification task for rewriting text to help readers comprehend text containing unfamiliar concepts and introduce a dataset of 22k definitions from 13 academic domains paired with a difficult concept within each definition. |
| Outcome: | The proposed model outperforms open-source and commercial models on the task and human judges prefer explanations over simplifications of the difficult concept. |
A Decade of Knowledge Graphs in Natural Language Processing: A Survey (2022.aacl-main)
Copied to clipboard
| Challenge: | Knowledge graphs (KGs) are a representation of semantic relations between entities . despite their popularity, there is still no general understanding of what exactly a KG is or for what tasks it is applicable. |
| Approach: | They analyze 507 papers on knowledge graphs in natural language processing (NLP) they provide a taxonomy of tasks and review the maturity of individual research streams . |
| Outcome: | The findings summarize the literature and highlight directions for future work. |