Papers with puzzle-solving
LatEval: An Interactive LLMs Evaluation Benchmark with Incomplete Information from Lateral Thinking Puzzles (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks, such as MMLU, C-Eval, and GSM8K, evaluate models by posing a variety of problems, including problems about mathematics, science, law, and general knowledge. |
| Approach: | They propose a benchmark which assesses the model’s lateral thinking within an interactive framework. |
| Outcome: | The proposed evaluation benchmark assesses the model’s lateral thinking within an interactive framework. |
Can LLMs Generate and Solve Linguistic Olympiad Puzzles? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for solving linguistic puzzles are outdated and require expert linguists to ensure validity. |
| Approach: | They propose a novel task that uses Large Language Models to generate and solve linguistic puzzles for high school students. |
| Outcome: | The proposed model outperforms humans on most puzzle types except for those centered on writing systems and for the understudied languages. |
AG-GRPO: Answer-Guided GRPO for Masked Diffusion Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work on large language models (LLMs) has emphasized not only final-answer accuracy but also reliability of reasoning on challenging tasks. |
| Approach: | They propose an answer-guided group-relative policy optimization for masked diffusion language models which generates text through iterative mangled token restoration. |
| Outcome: | The proposed approach improves over pretrained dLLMs and prior RL methods across mathematics, puzzle-solving, and code-generation benchmarks. |