LLM-Guided Semantic Bootstrapping for Interpretable Text Classification with Tsetlin Machines (2026.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) provide strong semantic representations but are costly and opaque. |
| Approach: | They propose a framework that transfers pretrained language models into symbolic form and integrates them into symbolic models. |
| Outcome: | The proposed framework improves interpretability and accuracy across multiple text classification tasks while remaining fully symbolic and efficient. |
Similar Papers
Infusing Finetuning with Semantic Dependencies (2021.tacl-1)
Copied to clipboard
| Challenge: | Several diagnostics help to localize the benefits of our approach. |
| Approach: | They apply convolutional graph encoders to integrate semantic parses into task-specific finetuning. |
| Outcome: | The proposed approach yields benefits to natural language understanding (NLU) tasks in the GLUE benchmark. |
Guiding Neural Machine Translation with Semantic Kernels (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical studies show that our approach gains approximately an improvement of 1 BLEU score on most benchmarks over the Transformer baseline. |
| Approach: | They propose to extract several semantic kernels from a source sentence to capture global semantic information. |
| Outcome: | Empirical results show that the proposed approach improves 1 BLEU score on benchmarks . it is also 1.7 times faster than previous works on average at inference time . |
Pre-Trained Language-Meaning Models for Multilingual Parsing and Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have been used for tasks in computational semantics but meaning representations are not included in PLMs. |
| Approach: | They propose to include meaning representations besides natural language texts in the same model . they propose to use DRSs to improve performance of non-English tasks . |
| Outcome: | The proposed approach achieves the best performance on multilingual parsing and DRS-to-text generation tasks. |
Proto-lm: A Prototypical Network-Based Framework for Built-in Interpretability in Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for interpreting LLMs are post hoc and focus on low-level features and lack of explainability at higher-level text units. |
| Approach: | They propose a prototypical network-based white-box framework that allows LLMs to learn immediately interpretable embeddings during the fine-tuning stage while maintaining competitive performance. |
| Outcome: | The proposed framework can learn interpretable embeddings during the fine-tuning stage while maintaining competitive performance. |
Graph Pre-training for AMR Parsing and Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Abstract meaning representation (AMR) highlights the core semantic information of text in a graph structure. |
| Approach: | They propose two graph auto-encoding strategies for graph-to-graph pre-training and four tasks to integrate text and graph information during pre-tuning to improve structure awareness. |
| Outcome: | The proposed model is superior to pre-trained language models on AMR parsing and AMR-to-text generation tasks. |
Neuro-Symbolic Natural Language Processing (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have limitations in terms of safe and controlled reasoning, interpretability and adaptability . this tutorial aims to bridge the gap between the practical performance of LLMs and the principled modelling of language and inference of formal methods. |
| Approach: | This tutorial aims to bridge the gap between the practical performance of Large Language Models and the principled modelling of language and inference of formal methods. |
| Outcome: | This tutorial aims to bridge the gap between the performance of LLMs and the principled modelling of language and inference of formal methods. |
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit remarkable adaptability across domains, but they are often not suitable for structured knowledge extraction tasks such as named entity recognition (NER). |
| Approach: | They propose a method that instructs LLMs to self-reflect on the specific domain and generates domain-relevant attributes for creating attribute-rich training data. |
| Outcome: | The proposed method produces NER datasets in domains with domain-relevant attributes and generates entity terms and NER context data around these entities. |
Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models of language understanding are based on explicit representations of hierarchical structure, but there are good reasons to doubt that they can be said to understand language in any meaningful way. |
| Approach: | They examine whether syntactic and semantic graph representations can complement and improve neural language modeling. |
| Outcome: | The proposed model outperforms pretrained models on English WSJ in perplexity and other metrics. |
Masked Latent Semantic Modeling: an Efficient Pre-training Alternative to Masked Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study suggests that masked language models are a useful pre-training technique for natural language processing . a study using mlms pre-trained by a team of researchers has improved performance . |
| Approach: | They propose an alternative to the classic masked language modeling paradigm . they use an unsupervised technique which uses sparse coding to make the prediction possible . |
| Outcome: | The proposed technique improves on pre-trained models compared to vanilla MLM . the proposed model returns distributions over their vocabulary peaking at plausible substitutes . |
Integrating Task Specific Information into Pretrained Language Models for Low Resource Fine Tuning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pretrained language models are agnostic to downstream information and can overfit when fine-tuned with low resource datasets. |
| Approach: | They integrate label information as a task-specific prior into the self-attention component of pretrained BERT models. |
| Outcome: | Experiments on benchmarks and real-word datasets show that the proposed approach can improve the performance of pretrained models when fine-tuned with small datasets. |