When Shallow is Good Enough: Automatic Assessment of Conceptual Text Complexity using Shallow Semantic Features (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing approaches to automatic assessment of text complexity focus on syntactic and lexical complexity. |
| Approach: | They propose to use graph-based deep semantic features to automatically assess conceptual text complexity by using DBpedia as a proxy to human knowledge. |
| Outcome: | The proposed features outperform the state-of-the-art features on pairwise comparison of two versions of the same text and five-level classification task. |
Similar Papers
Automatic Assessment of Conceptual Text Complexity Using Knowledge Graphs (C18-1)
Copied to clipboard
| Challenge: | Existing methods to assess text complexity only at lexical and syntactic levels have not been attempted. |
| Approach: | They propose to automatically estimate conceptual complexity using graph-based measures on a large knowledge base. |
| Outcome: | The proposed measures achieve high discriminative power even in a default setup. |
CoCo: A Tool for Automatically Assessing Conceptual Complexity of Texts (2020.lrec-1)
Copied to clipboard
| Challenge: | Traditional text complexity assessment only takes into account lexical and lexiconal complexity. |
| Approach: | They propose a tool for automatic assessment of conceptual text complexity based on the current state-of-the-art unsupervised approach . they compare the current implementation with the state of the art and discuss the influence of the choice of entity linker on the performance of the tool. |
| Outcome: | The proposed tool can be personalized and adapted to the needs of struggling readers. |
An Instance Level Approach for Shallow Semantic Parsing in Scientific Procedural Text (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to parse scientific text using grammatically similar labeled sentences are limited and expensive to create. |
| Approach: | They propose a method where semantic labels from structurally similar sentences are copied to test sentences. |
| Outcome: | The proposed approach outperforms baseline and prior methods by 0.75 to 3 F1 absolute in the wet lab protocol corpus and 1 F1 absolut in the materials science procedural text corpus. |
Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers? (2025.coling-main)
Copied to clipboard
Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, Yongfeng Zhang
| Challenge: | Large language models have shown remarkable performances across a wide range of tasks, but mechanisms by which they encode tasks of varying complexity remain poorly understood. |
| Approach: | They propose to explore the possibility that LLMs process concepts in different layers . they propose to categorize concepts based on their level of abstraction . |
| Outcome: | The proposed model can process complex concepts in shallow layers, the authors show . the proposed model could be used to prob complex tasks in shallow ones . |
Exploring Semantic Capacity of Terms (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that measure semantic capacity of terms are not all considered equal . a good command of semantic capacity will give us more insight into the granularity of terms . |
| Approach: | They propose a model that evaluates semantic capacity of terms if text corpus can provide enough co-occurrence information of terms. |
| Outcome: | The proposed model can evaluate semantic capacity of terms if the corpus can provide enough co-occurrence information of terms. |
A Spreading Activation Framework for Tracking Conceptual Complexity of Texts (P19-1)
Copied to clipboard
| Challenge: | Existing models for assessing conceptual complexity of texts are lacking . conceptual complexity accounts for background knowledge necessary to understand mentioned concepts . |
| Approach: | They propose an unsupervised approach for assessing conceptual complexity of texts based on spreading activation using DBpedia knowledge graph as a proxy to long-term memory. |
| Outcome: | The proposed model outperforms current state of the art in assessing conceptual complexity of texts. |
ARTS: Assessing Readability & Text Simplicity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text simplification evaluation measures do not include simplicity labels on text level, as they are mainly based on a relational concept. |
| Approach: | They propose a method for language-independent construction of datasets for simplicity assessment using pairwise comparisons of texts in conjunction with an Elo algorithm to produce a simplicity ranking and simplicity scores. |
| Outcome: | The proposed method produces a ranking and simplicity scores for human-labeled and three GPT-labelled datasets. |
No Simple Answer to Data Complexity: An Examination of Instance-Level Complexity Metrics for Classification Tasks (2025.naacl-long)
Copied to clipboard
| Challenge: | Understanding data complexity at the instance level has become increasingly important in Natural Language Processing (NLP) and machine learning (ML). |
| Approach: | They empirically examine the relationship between instance-level complexity scores and metric selection for classification tasks. |
| Outcome: | The results show that storing training loss provides similar complexity rankings to other methods, but not demographic fairness, even in downstream predictions. |
Drivel-ology: Challenging LLMs with Interpreting Nonsense with Depth (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite excelling at many natural language processing tasks, large language models fail to grasp the layered semantics of Drivelological text. |
| Approach: | They construct a benchmark dataset of over 1,200+ carefully curated and diverse examples across English, Mandarin, Spanish, French, Japanese, and Korean to examine their Drivelological characteristics. |
| Outcome: | The proposed models lack conceptual understanding and lack conceptual and semantic accuracy. |
Evaluating LLMs for Targeted Concept Simplification for Domain-Specific Texts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Simplifying the entire text makes it understandable but sometimes removes important details. |
| Approach: | They propose a simplification task for rewriting text to help readers comprehend text containing unfamiliar concepts and introduce a dataset of 22k definitions from 13 academic domains paired with a difficult concept within each definition. |
| Outcome: | The proposed model outperforms open-source and commercial models on the task and human judges prefer explanations over simplifications of the difficult concept. |