Papers by Helen Jin
Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to addressing factual inaccuracies require high-quality human factuality annotations to mitigate these hallucinations. |
| Approach: | They propose to leverage the self-evaluation capability of an LLM to provide training signals that steer the model towards factuality. |
| Outcome: | The proposed approach significantly improves factual accuracy over LLMs across three key knowledge-intensive tasks on TruthfulQA and BioGEN. |
Generic Temporal Reasoning with Differential Analysis and Explanation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing temporal reasoning models drop to random guessing on TODAY, suggesting that they heavily rely on spurious information rather than proper reasoning for temporal predictions. |
| Approach: | They propose a task called TODAY that evaluates whether systems can correctly understand the effect of incremental changes in temporal relation distributions. |
| Outcome: | The proposed task outperforms existing models, including GPT-3.5, on in-domain benchmarks while allowing for more appropriate annotations. |
Probabilistic Soundness Guarantees in LLM Reasoning Chains (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting propagated errors in reasoning chains are inadequate . author et al. (2017) show that initial errors propagate and undermine reliability of final conclusion . |
| Approach: | They propose a framework that evaluates each reasoning step based solely on previously-verified premises and provides certified statistical guarantees of its soundness. |
| Outcome: | ARES achieves state-of-the-art performance across four benchmarks and demonstrates superior robustness on very long synthetic reasoning chains. |
Adaptively profiling models with task elicitation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Language model evaluations fail to characterize consequential failure modes, forcing experts to inspect outputs and build new benchmarks. |
| Approach: | They propose a method that automatically builds new evaluations to profile model behavior. |
| Outcome: | The proposed method finds that language models fail in hundreds of tasks . it also finds that o3-mini is prone to hallucination when fabrications are repeated . |