Implicit Reasoning in Transformers is Reasoning through Shortcuts (2025.findings-acl)
Copied to clipboard
| Challenge: | Language models can perform step-by-step reasoning and achieve high accuracy in both in-domain and out-of-domain tests via implicit reasoning. |
| Approach: | They train GPT-2 from scratch on a curated multi-step mathematical reasoning dataset and conduct analytical experiments to investigate how language models perform implicit reasoning in multi- step tasks. |
| Outcome: | The proposed model performs better on multi-step tasks than the explicit reasoning model. |
Similar Papers
Can Transformers Reason in Fragments of Natural Language? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on natural language inference has identified two strands of research . |
| Approach: | They investigate whether neural networks have acquired logical principles from natural language . they use transformer-based models to detect valid inferences in controlled fragments of natural language. |
| Outcome: | The proposed model overfits to superficial patterns in the data rather than acquiring the logical principles governing reasoning in natural language fragments. |
Training in Step-by-Step Formal Reasoning Improves Pronominal Reasoning in Language Models (2026.eacl-short)
Copied to clipboard
| Challenge: | Large reasoning models are limited to formal reasoning, i.e., math, code, and logic. |
| Approach: | They evaluate a set of large reasoning models on a dataset for pronoun resolution and fidelity. |
| Outcome: | The results show that distilling step-by-step formal reasoning improves pronoun resolution and fidelity. |
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023.findings-acl)
Copied to clipboard
| Challenge: | Large pretrained language models have shown surprising in-context learning ability . despite the great success in performance, its working mechanism remains unclear . |
| Approach: | They explain language models as meta-optimizers and understand in-context learning as implicit finetuning . they find that Transformer attention has a dual form of gradient descent . |
| Outcome: | The proposed model can predict labels for unseen inputs without parameter updates . the proposed model outperforms smaller models with a single parameter update . |
Identifying the limits of transformers when performing model-checking with natural language (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies have focused on transformer models’ ability to perform reasoning on text, but the above question has not been adequately answered. |
| Approach: | They investigated the problem of model-checking with natural language to determine whether transformers can comprehend logical semantics in natural language. |
| Outcome: | The proposed model-checking problem is suited to address this issue but is untouched in natural language inference research. |
In-Context Learning Creates Task Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a powerful new learning paradigm for Large Language Models (LLMs). |
| Approach: | They propose to use a model with a prompt and a query to learn a mapping based on two examples to produce the output. |
| Outcome: | The proposed model can learn functions from a simple structure based on a training set and a single task vector calculated from the training set. |
Do Vision-and-Language Transformers Learn Grounded Predicate-Noun Dependencies? (2022.emnlp-main)
Copied to clipboard
| Challenge: | a recent study examines whether vision-and-language models learn syntactic dependencies . a controlled evaluation of the models is crucial for a precise and rigorous test of their knowledge . |
| Approach: | They propose a task to evaluate understanding of predicate-noun dependencies in a controlled setup. |
| Outcome: | This study compares state-of-the-art models with a case study on predicate-noun dependencies. |
ALERT: Adapt Language Models to Reasoning Tasks (2023.acl-long)
Copied to clipboard
Ping Yu, Tianlu Wang, Olga Golovneva, Badr AlKhamissi, Siddharth Verma, Zhijing Jin, Gargi Ghosh, Mona Diab, Asli Celikyilmaz
| Challenge: | Large language models have shown increasing in-context learning capabilities with scaling up the model and data sizes. |
| Approach: | They propose a benchmark and suite of analyses to evaluate reasoning skills of large language models. |
| Outcome: | The proposed model compares pre-trained and fine-tuned models on tasks that require reasoning skills to solve. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Fine-tuning and in-context learning are two prevalent methods in imbuing large language models with task-specific knowledge. |
| Approach: | They propose to use a circuit shift theory to explain why in-context learning is superior to fine-tuning for tasks with implicit patterns. |
| Outcome: | The proposed method can grasp deep patterns and significantly improve accuracy on implicit patterns, compared with fine-tuning and in-context learning. |
Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that large reasoning models arrive at the correct answer before completing textual reasoning steps, indicating the presence of latent reasoning. |
| Approach: | They conduct a systematic investigation of multilingual latent reasoning in large reasoning models across 11 languages. |
| Outcome: | The proposed model arrive at the correct answer before completing the reasoning steps, indicating the presence of latent reasoning. |