Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are struggling with performing numeric operations accurately. |
| Approach: | They propose to use different numeral systems to scale different numerates in transformer-based large language models. |
| Outcome: | The proposed model is more data-efficient than base 10 and base 10 3 . the model is also more efficient on addition and multiplication . |
Similar Papers
NUMCoT: Numerals and Units of Measurement in Chain-of-Thought Reasoning using Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing LLMs are not able to handle numerals and units of measurement, but they can be improved by introducing perturbations. |
| Approach: | They propose to analyze existing LLMs on processing numerals and units of measurement by perturbing their datasets. |
| Outcome: | The proposed model improves on ancient Chinese arithmetic problems and can handle numeral and measurement conversions. |
How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs (2025.findings-acl)
Copied to clipboard
Guhao Feng, Kai Yang, Yuntian Gu, Xinyue Ai, Shengjie Luo, Jiacheng Sun, Di He, Zhenguo Li, Liwei Wang
| Challenge: | Despite the success of transformer-based large language models, understanding and enhancing their mathematical capabilities remains a significant challenge. |
| Approach: | They propose to use numerical precision as a key factor that influences LLMs' effectiveness in arithmetical tasks to determine their effectiveness. |
| Outcome: | The proposed models perform better in arithmetic tasks than transformer-based models with standard numerical precision. |
Language Models Learn Universal Representations of Numbers and Here’s Why You Should Care (2026.acl-long)
Copied to clipboard
Michal Štefánik, Timothee Mickus, Marek Kadlčík, Bertram Højer, Michal Spiegel, Raúl Vázquez, Aman Sinha, Josef Kuchař, Philipp Mondorf, Pontus Stenetorp
| Challenge: | Prior work has shown that large language models (LLMs) often converge to accurate input embedding for numbers, based on sinusoidal representations. |
| Approach: | They show that large language models often converge to accurate input embedding for numbers, based on sinusoidal representations. |
| Outcome: | The proposed representations are strikingly systematic, and are interchangeable in a large swathe of experimental setups. |
1,729 vs. 1729: The Effect of Scripts and Formats on LLM Numeracy (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive proficiency in basic arithmetic, but little attention has been given to how they perform when numerical expressions deviate from the prevailing conventions present in their training corpora. |
| Approach: | They investigate numerical reasoning across a wide range of numeral scripts and formats . they show that LLM accuracy drops substantially when numerical inputs are rendered in underrepresented scripts or formats despite the underlying mathematical reasoning being identical . |
| Outcome: | The proposed methods can narrow the gap between LLMs and human models when they deviate from prevailing numerical conventions. |
Language Models Encode Numbers Using Digit Representations in Base 10 (2025.naacl-short)
Copied to clipboard
| Challenge: | Large language models (LLMs) often make errors when handling simple numerical tasks . a natural hypothesis is that these errors stem from how LLMs represent numbers . |
| Approach: | They propose to examine how LLMs represent numbers with circular representations per digit . they propose to use digit-wise representations to shed light on errors on numerical tasks . |
| Outcome: | The proposed model is internally represented with individual circular representations per-digit in base 10 . the proposed model could be used to analyze numerical mechanisms in large language models . |
On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Cultures (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) should be able to provide accurate information irrespective of the measurement system at hand . |
| Approach: | They use newly compiled datasets to test if this is true for seven open-source LLMs. |
| Outcome: | The proposed model can provide accurate information regardless of the measurement system at hand. |
Numeric Magnitude Comparison Effects in Large Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior research on the representational capabilities of LLMs evaluates whether they show human-level performance. |
| Approach: | They ask how well popular LLMs capture the magnitudes of numbers from a behavioral lens. |
| Outcome: | The proposed model captures the magnitudes of numbers from a behavioral lens. |
Hints on the data for language modeling of synthetic languages with transformers (2023.acl-long)
Copied to clipboard
| Challenge: | Language Models (LMs) are becoming more useful for providing representations for NLP applications. |
| Approach: | They evaluated whether the critical amount of data varies for different morphological typologies . they found that the size of the vocabulary due to morphology is directly correlated with LM perplexity . |
| Outcome: | The proposed method reduces perplexity by more than a half for a polysynthetic language like Quechua . |
From A and B to A+B: Can Large Language Models Solve Compositional Math Problems? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies that create problem variants by adding perturbations to a single problem focus on the interaction between problems. |
| Approach: | They propose a pipeline with 98.2% accuracy to combine two original problems with a logical connection and to evaluate LLMs' generalization ability on the compositional problems. |
| Outcome: | The proposed pipeline can combine two original problems with a logical connection to get a new math problem and evaluate its compositional generalization on the compositional problems. |
Do Large Language Models Truly Grasp Addition? A Rule-Focused Diagnostic Using Two-Integer Arithmetic (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel at complex math but fail on basic addition, raising the question of whether they grasp rules or are merely reproducing patterns. |
| Approach: | They systematically probe LLMs’ understanding of two-integer addition by testing three crucial properties: commutativity (A+B=B+A), representation invariance via symbolic remapping and consistent accuracy scaling with operand length. |
| Outcome: | The proposed models achieve high numeric accuracy but fail basic addition tasks. |