Papers by Roy Xie
IDP Accelerator: Agentic Document Intelligence from Extraction to Compliance Validation (2026.acl-demo)
Copied to clipboard
Md Mofijul Islam, Md Sirajus Salekin, Joe King, Priyashree Roy, Vamsi Thilak Gudi, Spencer Romo, Akhil Nooney, Bob Strahan, Boyi Xie, Diego A. Socolinsky
| Challenge: | Large Language Models (LLMs) are inadequate for extracting structured insights from unstructured documents. |
| Approach: | They propose a framework enabling agentic AI for end-to-end document intelligence with four key components: DocSplit, configurable Extraction Module, and Rule Validation Module. |
| Outcome: | The proposed framework achieves 98% classification accuracy, 80% reduced processing latency, and 77% lower operational costs over legacy baselines. |
Tailoring Vaccine Messaging with Common-Ground Opinions (2024.findings-naacl)
Copied to clipboard
Rickard Stureborg, Sanxing Chen, Roy Xie, Aayushi Patel, Christopher Li, Chloe Zhu, Tingnan Hu, Jun Yang, Bhuwan Dhingra
| Challenge: | Vaccine interventions aim to answer concerns expressed about vaccination. |
| Approach: | They propose a dataset to evaluate how well responses are tailored to a common-ground opinion . they find that GPT-4-Turbo performs significantly better than others . |
| Outcome: | The proposed dataset outperforms fine tuned LLMs on the task of tailoring vaccine responses to common-ground opinions. |
Extracting Lexical Features from Dialects via Interpretable Dialect Classifiers (2024.naacl-short)
Copied to clipboard
| Challenge: | Identifying linguistic differences between dialects of a language often requires expert knowledge and meticulous human analysis. |
| Approach: | They propose a method to extract distinguishing lexical features of dialects by utilizing interpretable dialect classifiers in the absence of human experts. |
| Outcome: | The proposed method extracts key language-specific lexical features that contribute to dialectal variations. |
Over-Searching in Search-Augmented Large Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Search-augmented large language models (LLMs) excel at knowledge-intensive tasks by integrating external retrieval. |
| Approach: | They conduct a systematic evaluation of over-searching across multiple dimensions including query types, model categories, retrieval conditions, and multi-turn conversations. |
| Outcome: | The proposed model improves answer accuracy on answerable queries but harms abstention on unanswerable ones . |
ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods (2024.emnlp-main)
Copied to clipboard
| Challenge: | ReCaLL (Relative Conditional Log-Likelihood) is a membership inference attack that can detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities. |
| Approach: | They propose a membership inference attack to detect LLMs’ pretraining data by leveraging their conditional language modeling capabilities. |
| Outcome: | The proposed model achieves state-of-the-art performance on the WikiMIA dataset, even with random and synthetic prefixes, and can be further improved using an ensemble approach. |
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs (2024.findings-acl)
Copied to clipboard
Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, Jiawei Han
| Challenge: | Existing studies suggest augmenting LLMs with external text corpora to alleviate hallucination problems. |
| Approach: | They propose to augment large language models with text units retrieved from external knowledge corpora to alleviate the issue. |
| Outcome: | The proposed framework outperforms baselines on GRBench with three LLMs and shows that iterative reasoning outperformed the baselines. |
Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applications (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have markedly shifted the landscape of AI, enabling these models to tackle complex, real-world tasks through natural language instructions. |
| Approach: | They propose a benchmark which evaluates a model's susceptibility to prompt extraction attacks by employing a dual approach to evaluate the effectiveness of existing defenses and the resilience of the models. |
| Outcome: | The proposed benchmark assesses models under both defenseless and defended scenarios, employing a dual approach to evaluate the effectiveness of existing defenses and the resilience of the models. |
Adversarial Math Word Problem Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized the educational landscape due to the great improvements in their natural language generation and problem-solving capabilities. |
| Approach: | They propose a cost-effective approach to attack large language models using abstract syntax trees to generate adversarial examples that preserve the structure and difficulty of the original questions aimed for assessment. |
| Outcome: | The proposed approach significantly degrades students' math problem-solving ability on open- and closed-source LLMs. |