Papers by Chiwei Zhu
Automated Creativity Evaluation for Large Language Models: A Reference-Based Approach (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for evaluating creativity of machine-generated texts rely on costly manual annotations or fail to align closely with human assessments. |
| Approach: | They propose an automated method based on the Torrance Test of Creative Writing (TTCW) . |
| Outcome: | The proposed method improves the alignment between LLM evaluations and human assessments. |
WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for Graph-based Retrieval-Augmented Generation (GraphRAG) rely on short, curated passages as external knowledge, failing to adequately evaluate systems in realistic settings involving long contexts and large-scale heterogeneous documents. |
| Approach: | They propose a benchmark to assess GraphRAG performance in the wild using Wikipedia's unique structure where cohesive narratives are grounded in long and heterogeneous external reference documents. |
| Outcome: | Experiments with articles across 12 top-level topics show that GraphRAG performs better in the wild than existing methods. |
From Real to Synthetic: Synthesizing Millions of Diversified and Complicated User Instructions with Attributed Grounding (2025.acl-long)
Copied to clipboard
| Challenge: | a pursuit of diverse, complex, and large-scale instruction data is crucial for automatically aligning large language models . authors: methods that generate synthetic instructions at scale suffer from limited grounding sources . attributed grounding is a technique that can be used to align language models with human . |
| Approach: | They synthesize 1 million instructions using attributed grounding and a bottom-up synthesis process that leverages web documents to generate a situation, then a meaningful instruction. |
| Outcome: | The proposed framework achieves leading performance on benchmarks and scales with more web corpora. |
FS-Researcher: Test-Time Scaling for Long-Horizon Research Tasks with File-System-Based Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Long trajectories in deep research often exceed model context limits, compressing token budgets for both evidence collection and report writing. |
| Approach: | They propose a file-system-based framework that scales deep research beyond context window . a Context Builder agent acts as a librarian and a Report Writer agent composes the final report . |
| Outcome: | Experiments on two open-ended benchmarks show that FS-Researcher achieves state-of-the-art report quality across different backbone models. |
KNN-Instruct: Automatic Instruction Construction with K Nearest Neighbor Deduction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating synthetic instructions for large language models suffer from stale distribution and scalability. |
| Approach: | They propose a method which incorporates KNN deduction to produce meaningful new instructions by summarizing and learning from existing ones. |
| Outcome: | The proposed method outperforms all 7B models on the LMSYS leaderboard. |
Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that training language models with rationales augmentation is beneficial, but this view does not hold consistently. |
| Approach: | They conduct comprehensive investigations to thoroughly inspect the impact of rationales on model performance and a novel perspective of model reliability. |
| Outcome: | The proposed method outperforms untrained models in several areas and provides informative regulations on the broad utilization of rationales. |
On the Calibration of Large Language Models and Alignment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models are becoming more popular and are proving to be reliable . however, their reliability is often understudied due to their uncertainty and complex structure . |
| Approach: | They conduct a systematic examination of the calibration of aligned language models throughout the entire construction process including pretraining and alignment training. |
| Outcome: | The results shed light on whether popular large language models are well-calibrated and how the training process influences model calibration. |
Grammatical Error Correction via Mixed-Grained Weighted Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical evaluation shows that MainGEC achieves consistent and significant performance improvements on two benchmark datasets. |
| Approach: | They propose to use mixed-grained weighted training to improve the training effect for GEC by analyzing the inherent discrepancies in annotated training data. |
| Outcome: | Empirical results show that the proposed method achieves significant performance improvements on two benchmark datasets. |