Papers by Shiyu Ni
Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perception (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit impressive performance across diverse tasks but struggle to accurately gauge their knowledge boundaries. |
| Approach: | They propose Consistency-based Confidence Calibration (C3) which assesses confidence consistency through question reformulation to improve LLMs’ ability to recognize their knowledge gaps. |
| Outcome: | The proposed method improves the unknown perception rate by 5.6% on NQ and 4.9% on HotpotQA. |
Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) demonstrate strong visual question answering (VQA) capabilities but are shown to hallucinate. |
| Approach: | They propose three confidence-based methods to enhance LVLMs' perception . they propose probabilistic and consistency-based signals are more reliable indicators . |
| Outcome: | Experiments on three LVLMs across three VQA datasets show that LVLs possess a reasonable perception level but there is room for improvement. |
STReasoner: Empowering LLMs for Spatio-Temporal Reasoning in Time Series via Spatial-Aware Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models focus on predictive accuracy over reasoning, a gap exists . time series data are ubiquitous in real-world systems and exhibit complex spatio-temporal structures. |
| Approach: | They propose a time series reasoning model that integrates time series, graph structure, and text for explicit reasoning. |
| Outcome: | The proposed model achieves average accuracy gains between 17% and 135% at 0.004x the cost of proprietary models and generalizes robustly to real-world data. |
How Long Reasoning Chains Influence LLMs’ Judgment of Answer Factuality (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly adopted as scalable judges for open-ended generation, yet how they form judgments remains insufficiently understood. |
| Approach: | They show that exposing reasoning influences LLM-based judgment . they also show that reasoning fluency and factuality critically shape judgment outcomes . |
| Outcome: | Empirical results show that the presence of reasoning significantly alters judgment behavior . stronger judges exhibit more selective behavior and achieve higher judgment accuracy . |
When Do LLMs Need Retrieval Augmentation? Mitigating LLMs’ Overconfidence Helps Retrieval Augmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have difficulty knowing they do not possess certain knowledge and tend to provide specious answers in such cases. |
| Approach: | They propose to use Retrieval Augmentation to enhance LLMs' ability to perceive their knowledge boundaries to reduce overconfidence. |
| Outcome: | The proposed methods reduce overconfidence and improve accuracy in large language models with fewer retrieval calls. |