NumeroLogic: Number Encoding for Enhanced LLMs’ Numerical Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language models struggle with numerical and arithmetical tasks, such as multiplying 3-digit numbers. |
| Approach: | They propose a method to include the count of digits before each number instead of “42”. |
| Outcome: | The proposed format improves the reasoning process before generating the actual number. |
Similar Papers
1,729 vs. 1729: The Effect of Scripts and Formats on LLM Numeracy (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive proficiency in basic arithmetic, but little attention has been given to how they perform when numerical expressions deviate from the prevailing conventions present in their training corpora. |
| Approach: | They investigate numerical reasoning across a wide range of numeral scripts and formats . they show that LLM accuracy drops substantially when numerical inputs are rendered in underrepresented scripts or formats despite the underlying mathematical reasoning being identical . |
| Outcome: | The proposed methods can narrow the gap between LLMs and human models when they deviate from prevailing numerical conventions. |
Language Models Encode Numbers Using Digit Representations in Base 10 (2025.naacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) often make errors when handling simple numerical tasks . a natural hypothesis is that these errors stem from how LLMs represent numbers . |
| Approach: | They propose to examine how LLMs represent numbers with circular representations per digit . they propose to use digit-wise representations to shed light on errors on numerical tasks . |
| Outcome: | The proposed model is internally represented with individual circular representations per-digit in base 10 . the proposed model could be used to analyze numerical mechanisms in large language models . |
NUMCoT: Numerals and Units of Measurement in Chain-of-Thought Reasoning using Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing LLMs are not able to handle numerals and units of measurement, but they can be improved by introducing perturbations. |
| Approach: | They propose to analyze existing LLMs on processing numerals and units of measurement by perturbing their datasets. |
| Outcome: | The proposed model improves on ancient Chinese arithmetic problems and can handle numeral and measurement conversions. |
Language Models Encode the Value of Numbers Linearly (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies show that large language models encode the value of numbers linearly. |
| Approach: | They construct a large language model and use linear probes to read out input numbers from hidden states. |
| Outcome: | The proposed model encodes the value of numbers linearly, and can store the outputs via simple vector additions. |
Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles (2025.emnlp-main)
Copied to clipboard
| Challenge: | Across languages, numeral systems vary widely in how they construct and combine numbers. |
| Approach: | They conduct experiments to examine the linguistic and mathematical aspects of numbers in language. |
| Outcome: | The models can't solve linguistic-mathematical puzzles involving cross-linguistic numeral systems, the authors found . they lack the ability to flexibly infer compositional rules from implicit patterns in human-scale data. |
How to Leverage Digit Embeddings to Represent Numbers? (2025.coling-main)
Copied to clipboard
| Challenge: | Existing numerical reasoning models struggle to understand numbers, despite simple generalisations. |
| Approach: | They propose to use mathematical priors to compute digit embeddings and explicitly incorporate them into transformer models by adding a special token to the input embedded digits or introducing an additional loss function to enhance correct predictions. |
| Outcome: | The proposed method is compatible with any pretrained model and easy to implement. |
Language Models Learn Universal Representations of Numbers and Here’s Why You Should Care (2026.acl-long)
Copied to clipboard
Michal Štefánik, Timothee Mickus, Marek Kadlčík, Bertram Højer, Michal Spiegel, Raúl Vázquez, Aman Sinha, Josef Kuchař, Philipp Mondorf, Pontus Stenetorp
| Challenge: | Prior work has shown that large language models (LLMs) often converge to accurate input embedding for numbers, based on sinusoidal representations. |
| Approach: | They show that large language models often converge to accurate input embedding for numbers, based on sinusoidal representations. |
| Outcome: | The proposed representations are strikingly systematic, and are interchangeable in a large swathe of experimental setups. |
Representing Numbers in NLP: a Survey and a Vision (2021.naacl-main)
Copied to clipboard
| Challenge: | Numeracy is an essential skill for language understanding since numbers are often interspersed in text. |
| Approach: | They propose a comprehensive taxonomy of tasks and methods to represent numbers in text . they synthesize best practices for representing numbers in texts and articulate a vision for holistic numeracy . |
| Outcome: | The proposed model synthesizes best practices for representing numbers in text . it argues that the model is more effective than other approaches . |
LLMs Know More About Numbers than They Can Say (2026.eacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly used in mathematical, scientific, financial and engineering domains. |
| Approach: | They probe the hidden states of several smaller open-source LLMs to find out how big they are . |
| Outcome: | The proposed model improves verbalized accuracy by 3.22% over base models. |
Multiplication in Multimodal LLMs: Computation with Text, Image, and Audio Inputs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks lack systematically paired instances across modalities, making it difficult to compare genuine arithmetic limits . a model that computes 4736 may fail on a nearby instance like 8967, despite a well-tuned internal router. |
| Approach: | They propose a controlled multimodal multiplication benchmark that factorially varies digit length, digit sparsity, representation, and modality with paired instances from a reproducible generator. |
| Outcome: | The proposed model can perceive numerical content across modalities but fails to perform exact multi-digit multiplication when presented as numerals, number words, images, or in audio form. |