Papers with human-written
RORA: Robust Free-Text Rationale Evaluation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing metrics rely on degree to which rationale supports a label, but they fail to evaluate rationales that inadvertently leak the label. |
| Approach: | They propose a RObust free-text RAtionale evaluation against label leakage that quantifies the new information supplied by a rationale to justify the label. |
| Outcome: | The proposed evaluation outperforms existing methods in evaluating human-written, synthetic, or model-generated rationales, particularly demonstrating robustness against label leakage. |
Localizing and Mitigating Errors in Long-form Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Long-form question answering (LFQA) answers are prone to hallucinations and factual inconsistencies, challenging their faithful evaluation. |
| Approach: | They propose a dataset with localized error annotations for human-written and model-generated LFQA answers. |
| Outcome: | The proposed approach reduces errors and improves quality of the answers across multiple models. |
Linguistic and Embedding-Based Profiling of Texts Generated by Humans and Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on using LLMs to classify text as either human-written or machine-generated . |
| Approach: | They characterize human-written and machine-generated texts using a set of linguistic features across different linguistic levels such as morphology, syntax, and semantics. |
| Outcome: | The proposed model reveals that human-written texts exhibit simpler syntactic structures and more diverse semantic content. |