Papers with ROC
Detecting Overflow in Compressed Token Representations for Retrieval-Augmented Generation (2026.eacl-srw)
Copied to clipboard
| Challenge: | Efficient long-context processing remains a challenge for large language models (LLMs) however, the limits of compressibility remain underexplored. |
| Approach: | They propose a method to characterize and detect token overflow in xRAG soft-compression by mapping long contexts into dense vectors that can be directly consumed by the model. |
| Outcome: | The proposed method identifies token overflow with query-agnostic saturation statistics but lacks the capability to detect it. |
Unsupervised Fact Checking by Counter-Weighted Positive and Negative Evidential Paths in A Knowledge Graph (2020.coling-main)
Copied to clipboard
| Challenge: | Misinformation spreads across media, community, and knowledge graphs in the Web by human agents and information extraction algorithms. |
| Approach: | They propose a rule-based approach that finds positive and negative evidential paths in a knowledge graph for a given factual statement and calculates a truth score for the given statement by unsupervised ensemble. |
| Outcome: | The proposed approach outperforms the state-of-the-art unsupervised approaches by up to 0.12 AUC-ROC and even outperfies the supervised approach by up 0.05 AUC. |
Have a Cake and Eat it Too: Assessing Discriminating Performance of an Intelligibility Index Obtained from a Reduced Sample Size (2020.lrec-1)
Copied to clipboard
| Challenge: | Random vs. phonetically motivated reduction of linguistic material in speech disordered populations has a higher ranking ability than when the sample size is reduced based on a phonetic criterium. |
| Approach: | They investigate random vs. phonetically motivated reduction of linguistic material in speech disordered populations and their impact on the discrimination classifier. |
| Outcome: | The proposed method is reliable and efficient, and it is of significant interest to clinicians and patients. |
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models (2020.emnlp-main)
Copied to clipboard
Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
| Challenge: | Existing pre-trained large language models have shown unparalleled generative capabilities, but they are not controllable. |
| Approach: | They propose a framework that uses large-scale language models and adds control to text generation by incorporating an external knowledge base. |
| Outcome: | The proposed model generates more fluent, consistent, and coherent stories with less repetition and higher diversity compared to previous work on the ROC story dataset. |
PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have recognized hallucination as a notable concern in large autoregressive language models (LLMs). |
| Approach: | They propose a polygraph for large language models that detects "hallucination" they demonstrate that hallucination can be detected by tractable probabilistic models . |
| Outcome: | The proposed model outperforms state-of-the-art methods on open-source LLMs by 20% on TruthfulQA benchmarks. |
HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Hallucination is a significant challenge for large language models, but current methods struggle when non-factual information arises in the early or mid-sequence of outputs, reducing their reliability. |
| Approach: | They propose a method that captures the full dynamics of large language models by using neural differential equations to assess the truthfulness of statements. |
| Outcome: | The proposed method achieves 14% improvement in AUC-ROC on the True-False dataset compared to state-of-the-art methods. |
Characterizing and Measuring Linguistic Dataset Drift (2023.acl-long)
Copied to clipboard
Tyler Chang, Kishaloy Halder, Neha Anna John, Yogarshi Vyas, Yassine Benajiba, Miguel Ballesteros, Dan Roth
| Challenge: | Existing metrics for dataset drift have not considered specific dimensions of linguistic drift that affect model performance. |
| Approach: | They propose three dimensions of linguistic dataset drift: vocabulary, structural, and semantic drift. |
| Outcome: | The proposed metrics are more effective than previous metrics at predicting out-of-domain model accuracies compared to popular fine-tuned embedding distances . |
IntroLM: Introspective Language Models via Prefilling-Time Self-Evaluation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to predict output quality of large language models rely on external classifiers with limited context windows and constrained representational capacity. |
| Approach: | They propose a method that enables causal language models to predict their own output quality during the prefilling phase without affecting generation using [CPX] tokens. |
| Outcome: | The proposed method outperforms existing classifiers on Qwen3-8B and DeBERTa-v3-Large models by 14% on question-answering benchmarks. |
Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to multimodal relation extraction ignore structural constraints and lack semantic expressiveness for fine-grained relation understanding. |
| Approach: | They propose a framework that reformulates multimodal relation extraction as a retrieval task driven by relation semantics. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability. |
LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models generate tokens by updating high-dimensional representations and decoding from them at each timestep. |
| Approach: | They propose a framework that allows reasoning correction and length control based on derived ideal trajectories. |
| Outcome: | The proposed model can predict correctness and length control based on ideal trajectories. |