Image Manipulation via Multi-Hop Instructions - A New Dataset and Weakly-Supervised Neuro-Symbolic Approach (2023.emnlp-main)
Copied to clipboard
Harman Singh, Poorva Garg, Mohit Gupta, Kevin Shah, Ashish Goswami, Satyam Modi, Arnab Mondal, Dinesh Khandelwal, Dinesh Garg, Parag Singla
| Challenge: | Recent studies have shown that neuro-symbolic models lack interpretability and are not robust to noise. |
| Approach: | They propose to extend Neuro Symbolic Concept Learning (NSCL) which has been quite effective for the task of Visual Question Answering (VQA) they create a new dataset for the image manipulation task and demonstrate that NeuroSIM is highly competitive with or beats SOTA baselines that make use of supervised data for manipulation. |
| Outcome: | The proposed system performs complex multi-hop reasoning over multi-object scenes and only requires weak supervision in the form of annotated data for VQA. |
Similar Papers
Neuron Empirical Gradient: Discovering and Quantifying Neurons’ Global Linear Controllability (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that feed-forward neurons in pre-trained language models (PLMs) can encode factual knowledge, but current methods are costly and lack the link between activations and outputs. |
| Approach: | They propose to compute a global linear relationship between neuron activations and outputs using a knowledge probing dataset. |
| Outcome: | The proposed method exploits the neural empirical gradient to capture changes in neuron activations and model outputs. |
CoRRPUS: Code-based Structured Prompting for Neurosymbolic Story Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Story generation and understanding has seen a surge in neurosymbolic work . symbolic methods are expensive and require a lot of time and expertise . |
| Approach: | They use Code-LLMs to bootstrap the use of symbolic methods for story understanding . they show that they can beat current LLM techniques on pre-existing stories with minimal hand engineering . |
| Outcome: | The proposed system beats state-of-the-art structured LLM techniques on pre-existing story understanding tasks with minimal hand engineering. |
Neuro-Symbolic Natural Language Processing (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have limitations in terms of safe and controlled reasoning, interpretability and adaptability . this tutorial aims to bridge the gap between the practical performance of LLMs and the principled modelling of language and inference of formal methods. |
| Approach: | This tutorial aims to bridge the gap between the practical performance of Large Language Models and the principled modelling of language and inference of formal methods. |
| Outcome: | This tutorial aims to bridge the gap between the performance of LLMs and the principled modelling of language and inference of formal methods. |
Learning to Follow Object-Centric Image Editing Instructions Faithfully (2023.findings-emnlp)
Copied to clipboard
| Challenge: | avrahami et al., 2022b,a): natural language instructions are often underspecified, requiring models to uncover their implicit meaning. |
| Approach: | They propose to use paired data to model the implicit meaning of instructions . they also propose to ground the model to localize where the edit has to be performed . |
| Outcome: | The proposed model performs better than state-of-the-art baselines on paired data, showing improvements in quality and faithfulness. |
Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods to fine-tune Large Language Models without human annotations are lacking in the field of natural language training. |
| Approach: | They propose an environment-guided neural-symbolic self-training framework to overcome two main challenges: the scarcity of symbolic data and the limited proficiency of LLMs in processing symbolic language. |
| Outcome: | The proposed framework overcomes two main challenges: the scarcity of symbolic data, and the limited proficiency of LLMs in processing symbolic language. |
Neuro-symbolic Training for Reasoning over Spatial Language (2025.findings-naacl)
Copied to clipboard
| Challenge: | Spatial reasoning is essential for everyday human tasks and is crucial for robots to interact with their environment in a human-like manner. |
| Approach: | They propose to train language models to adhere to spatial reasoning rules as constraints . this allows them to capture the necessary level of abstraction for spatial reasoning . |
| Outcome: | The proposed technique improves language models in multi-hop spatial reasoning over text . it achieves higher accuracy than other competitive Spatial Question-answering benchmarks . |
Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning (D19-1)
Copied to clipboard
| Challenge: | Existing models for multi-hop question answering require multiple pieces of evidence scattered in a given context. |
| Approach: | They propose an interpretable, controller-based self-assembling Neural Modular Network for multi-hop reasoning . their model can softly decompose a multi-step question into multiple single-hop sub-questions . |
| Outcome: | The proposed model improves on the static, single-hop model on regular and adversarial evaluations. |
From Local Perspective to Global Reasoning: A Neuro-Symbolic Framework for Zero-Shot Relation Extraction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for zero-shot relationship extraction do not distinguish between unseen, semantically similar relations. |
| Approach: | They propose a framework to enable global reasoning across a set of predictions. |
| Outcome: | The proposed framework outperforms existing methods and establishes new state-of-the-art results on widely used datasets. |
Soft Syntactic Reinforcement for Neural Event Extraction (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent event extraction methods rely on pre-trained language models but still suffer from errors due to a lack of syntactic knowledge. |
| Approach: | They propose a method to incorporate syntactic information into PLM-based models for event extraction (EE) this method uses a standard dependency corpus to select syntax-related dimensions of the model's representation. |
| Outcome: | The proposed method outperforms baseline models and existing syntactic reinforcement methods on sentence-level and document-level EE benchmark datasets. |
Complex Reasoning in Natural Language (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Recent research shows that pretrained language models are often brittle for complex reasoning tasks. |
| Approach: | They propose to use pre-trained language models to teach machines to reason over texts . they will review recent promising approaches to tackling complex reasoning tasks . |
| Outcome: | This tutorial reviews promising approaches to complex reasoning tasks . it reviews the methods that can be used to augment models with robustness . |