Papers by Ruixuan Tu
Is Semantic Chunking Worth the Computational Cost? (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in Retrieval-Augmented Generation (RAG) systems have popularized semantic chunking. |
| Approach: | They evaluate the effectiveness of semantic chunking using three common retrieval tasks . they find that the computational costs associated with semantic chunks are not justified by consistent performance gains. |
| Outcome: | The proposed semantic chunking approach is not able to deliver consistent performance gains in three retrieval-related tasks. |
DocAsRef: An Empirical Study on Repurposing Reference-based Summary Quality Metrics as Reference-free Metrics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing reference-based metrics are limited by their reliance on human input. |
| Approach: | They propose to adapt some reference-based metrics to assess system summary against human-written references. |
| Outcome: | The proposed model outperforms reference-based metrics on two datasets and is comparable to reference-free metrics. |
FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs (2025.naacl-short)
Copied to clipboard
Forrest Sheng Bao, Miaoran Li, Renyi Qu, Ge Luo, Erana Wan, Yujia Tang, Weisi Fan, Manveer Singh Tamber, Suleman Kazi, Vivek Sourabh, Mike Qi, Ruixuan Tu, Chenyu Xu, Matthew Gonzales, Ofer Mendelevitch, Amin Ahmad
| Challenge: | Existing evaluations of hallucinations in large language models suffer from a lack of diversity and recency in the LLM and LLM families considered. |
| Approach: | They propose a summarization hallucination benchmark that challenges models to disagree on hallucines . they use models to generate answers or summaries from textual input . |
| Outcome: | The proposed model combines the best of 10 modern LLMs with ground truth annotations. |