Papers with SimpleQA
Purging the Gray Zone: Latent-Geometric Denoising for Precise Knowledge Boundary Awareness (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing abstention fine-tuning methods cause models to suffer from label noise near the decision boundaries. |
| Approach: | They propose a latent space representation perspective for abstention fine-tuning . they propose 'geometric denoising' framework that constructs a truth hyperplane . |
| Outcome: | The proposed framework significantly enhances model truthfulness and demonstrates strong generalization in out-of-distribution scenarios. |
WebCoT: Enhancing Web Agent Reasoning by Reconstructing Chain-of-Thought in Reflection, Branching, and Rollback (2025.findings-emnlp)
Copied to clipboard
Minda Hu, Tianqing Fang, Jianshu Zhang, Jun-Yu Ma, Zhisong Zhang, Jingyan Zhou, Hongming Zhang, Haitao Mi, Dong Yu, Irwin King
| Challenge: | Web agents powered by Large Language Models lack the ability to perform in uncertain web environments. |
| Approach: | They propose to reconstruct web agents' reasoning skills into chain-of-thought rationales by fine-tuning their LLM backbone into a web-based model. |
| Outcome: | The proposed approach significantly improves the agent self-improving benchmark OpenWebVoyager, demonstrating that it can be used to improve the agent's reasoning skills. |
Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models (2025.acl-long)
Copied to clipboard
Yancheng He, Shilong Li, Jiaheng Liu, Yingshui Tan, Weixun Wang, Hui Huang, Xingyuan Bu, Hangyu Guo, Chengwei Hu, Boren Zheng, Zhuoran Lin, Dekai Sun, Zhicheng Zheng, Wenbo Su, Bo Zheng
| Challenge: | Current frontier models sometimes generate false outputs or answers that are not substantiated by evidence. |
| Approach: | They propose Chinese SimpleQA, a Chinese benchmark to evaluate LLMs' factuality . they focus on Chinese language over 6 major topics with 99 diverse subtopics . |
| Outcome: | The Chinese SimpleQA benchmark evaluates the factuality ability of LLMs . the questions and answers are short and easy-to-evaluate . |
KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality (2026.acl-long)
Copied to clipboard
| Challenge: | Existing Reinforcement Learning approaches rely on outcome-oriented rewards to reinforce fabricated reasoning paths when the final answer is correct. |
| Approach: | They propose a framework that integrates factual supervision directly into reasoning . they propose to decompose chain of thought into atomic facts and verify them against ground-truth knowledge . |
| Outcome: | The proposed framework reduces the Incorrect Rate on SimpleQA by 20.3% while maintaining strong performance on complex reasoning benchmarks. |