Papers by Puneet Mathur
DOC-RAG: ASR Language Model Personalization with Domain-Distributed Co-occurrence Retrieval Augmentation (2024.lrec-main)
Copied to clipboard
| Challenge: | Extensive experiments on three user-specific speech-to-text tasks show that DOC-RAG significantly outperforms strong baselines with an 8-15% improvement in terms of perplexity and a 4-7% reduction in terms in terms . of Word Error Rates. |
| Approach: | They propose a domain-distributed co-occurrence augmentation approach to improve automatic speech recognition of rare word patterns in unseen domains by using n-gram co-existence distributions. |
| Outcome: | Experiments on three user-specific speech-to-text tasks show that DOC-RAG outperforms baselines with an 8-15% improvement in terms of perplexity and a 4-7% reduction in terms in terms . of Word Error Rates. |
Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents (2025.emnlp-main)
Copied to clipboard
Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, Vivek Gupta, Dinesh Manocha
| Challenge: | Flowcharts are a critical tool for visualizing decision-making processes, but their non-linear structure and complex visual-textual relationships make it difficult to interpret them using LLMs. |
| Approach: | They propose a task of Fine-grained Flowchart Attribution to trace components grounding a flowchart referring LLM response. |
| Outcome: | The proposed agent mitigates visual hallucinations in LLM answers over baselines by 10–14% on a FlowExplainBench dataset. |
Test-Time Strategies for More Efficient and Accurate Agentic RAG (2026.acl-srw)
Copied to clipboard
Abhinav Sharma, Brian Zhang, Deepti Guntur, Zhiyang Zuo, Shreyas Chaudhari, Wenlong Zhao, Franck Dernoncourt, Puneet Mathur, Ryan A. Rossi, Nedim Lipka
| Challenge: | Retrieval-Augmented Generation (RAG) systems face challenges with complex, multi-hop questions. |
| Approach: | They propose to integrate contextualization module and de-duplication module to improve the accuracy of retrieved documents and to reduce the number of turns by 10.5%. |
| Outcome: | The proposed approach achieves a 5.6% increase in EM score and reduces the average number of turns by 10.5% compared to the baseline. |
SNAP-BATNET: Cascading Author Profiling and Social Network Graphs for Suicide Ideation Detection on Social Media (N19-3)
Copied to clipboard
Rohan Mishra, Pradyumn Prakhar Sinha, Ramit Sawhney, Debanjan Mahata, Puneet Mathur, Rajiv Ratn Shah
| Challenge: | Suicide is a leading cause of death among youth worldwide and currently only uses text-based cues to detect suicidal ideation. |
| Approach: | They propose a deep learning based model to extract text-based features from tweets and a novel Feature Stacking approach to combine other community-based information. |
| Outcome: | The proposed model outperforms existing models on an annotated dataset of tweets using a three-phase strategy and proposes a novel Feature Stacking approach to combine other community-based information such as historical author profiling and graph embeddings. |
A Survey on LLM-based Conversational User Simulation (2026.eacl-long)
Copied to clipboard
Bo Ni, Yu Wang, Leyao Wang, Branislav Kveton, Franck Dernoncourt, Yu Xia, Hongjie Chen, Reuben Luera, Samyadeep Basu, Subhojyoti Mukherjee, Puneet Mathur, Nesreen K. Ahmed, Junda Wu, Li Li, Huixin Zhang, Ruiyi Zhang, Tong Yu, Sungchul Kim, Jiuxiang Gu, Zhengzhong Tu, Alexa Siu, Zichao Wang, Seunghyun Yoon, Nedim Lipka, Namyong Park, Zihao Lin, Trung Bui, Yue Zhao, Tyler Derr, Ryan A. Rossi
| Challenge: | Recent advances in large language models (LLMs) have enabled high-fidelity generation of synthetic user conversation. |
| Approach: | They propose a taxonomy covering user granularity and simulation objectives . they analyze core techniques and evaluation methodologies to help them understand the latest developments . |
| Outcome: | The proposed model enables high-fidelity generation of synthetic user conversation. |
DIAGRAMS : A Review Framework for Reasoning-Level Attribution in Diagram QA (2026.acl-demo)
Copied to clipboard
Anirudh Iyengar Kaniyar Narayana Iyengar, Tampu Ravi Kumar, Manan Suri, Raviteja Bommireddy, Dinesh Manocha, Puneet Mathur, Vivek Gupta
| Challenge: | Diagram question answering (Diagram QA) requires reasoning-level attribution that links each question-answer pair to all visual regions needed to derive the answer. |
| Approach: | They propose a diagram question-answer review framework that decouples interface logic from dataset-specific JSON structures through an internal meta-schema and dataset adapters. |
| Outcome: | The proposed framework achieves 85.39% precision and 75.30% recall across six diagram QA datasets. |
DocInfer: Document-level Natural Language Inference using Optimal Evidence Selection (2022.emnlp-main)
Copied to clipboard
| Challenge: | Documentlevel NLI is an important problem for many tasks including verification of factual correctness of documents. |
| Approach: | They propose a document-level natural language inference model that builds a hierarchical document graph enriched through inter-sentence relations and performs paragraph pruning using the novel SubGraph Pooling layer. |
| Outcome: | The proposed model performs on a legal judicial reasoning task with a dataset enriched with document graphs and a proposed evidence selection algorithm. |
TIMERS: Document-level Temporal Relation Extraction (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for temporal relation extraction focus on extracting temporal relations between event pairs present in the same sentence or adjacent sentences, mostly ignoring document-level pairs. |
| Approach: | They propose a TIME, Rhetorical and Syntactic-aware model for document-level temporal relation classification in the English language that leverages rhetorical discourse features and temporal arguments from semantic role labels. |
| Outcome: | The proposed model outperforms previous methods on the TDDiscourse, TimeBank-Dense, and MATRES datasets due to its discourse-level modeling. |
From Selection to Generation: A Survey of LLM-based Active Learning (2025.acl-long)
Copied to clipboard
Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie, Junda Wu, Xintong Li, Ryan Aponte, Hanjia Lyu, Joe Barrow, Hongjie Chen, Franck Dernoncourt, Branislav Kveton, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Sungchul Kim, Zhengmian Hu, Yue Zhao, Nedim Lipka, Seunghyun Yoon, Ting-Hao Kenneth Huang, Zichao Wang, Puneet Mathur, Soumyabrata Pal, Koyel Mukherjee, Zhehao Zhang, Namyong Park, Thien Huu Nguyen, Jiebo Luo, Ryan A. Rossi, Julian McAuley
| Challenge: | Large Language Models (LLMs) have been used for selection and training of data for active learning. |
| Approach: | They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop. |
| Outcome: | The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances. |
DocScript: Document-level Script Event Prediction (2024.lrec-main)
Copied to clipboard
Puneet Mathur, Vlad I. Morariu, Aparna Garimella, Franck Dernoncourt, Jiuxiang Gu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha, Rajiv Jain
| Challenge: | Existing script event prediction frameworks such as ChatGPT and FlanT5 lack the ability to learn long-range dependencies between events. |
| Approach: | They propose a novel script event prediction task which aims to predict the next event from a candidate list of narrative events in long-form documents. |
| Outcome: | The proposed architecture can learn sequential ordering between events at the document scale. |
Structured Uncertainty guided Clarification for LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to clarifying tasks fail when user instructions are ambiguous or incomplete. |
| Approach: | They propose a principled formulation of structured uncertainty that operates directly over tool parameters and their domains. |
| Outcome: | The proposed framework improves when2call accuracy and training-time sample efficiency. |
Lizard: An Efficient Linearization Framework for Large Language Models (2026.acl-long)
Copied to clipboard
Chien Van Nguyen, Huy Huu Nguyen, Ruiyi Zhang, Hanieh Deilamsalehy, Puneet Mathur, Viet Dac Lai, Haoliang Wang, Jayakumar Subramanian, Ryan A. Rossi, Trung Bui, Nikos Vlassis, Franck Dernoncourt, Thien Huu Nguyen
| Challenge: | Existing linearization frameworks that rely on softmax attention with quadratic time and memory complexity pose significant computational and memory bottlenecks for long-context applications. |
| Approach: | They propose a linearization framework that transforms pretrained Transformer-based Large Language Models into subquadratic architectures that closely approximate softmax attention while preserving model quality. |
| Outcome: | Experiments show that the proposed framework outperforms existing methods by 9.4 - 24.5 points on the 5-shot MMLU benchmark and demonstrating superior associative recall. |
MATSA: Multi-Agent Table Structure Attribution (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Tabular data present unique challenges for attribution due to ambiguities, complex header hierarchies, and the difficulty in interpreting individual table cells without row and column context. |
| Approach: | They propose a task to generate row and column-level attributions supporting LLM-generated answers. |
| Outcome: | The proposed task outperforms baselines on tabCite and improves F1 score. |
Saliency-Aware Interpolative Augmentation for Multimodal Financial Prediction (2024.lrec-main)
Copied to clipboard
Samyak Jain, Parth Chhabra, Atula Tejaswi Neerkaje, Puneet Mathur, Ramit Sawhney, Shivam Agarwal, Preslav Nakov, Sudheer Chava, Dinesh Manocha
| Challenge: | Recent advances in the Financial AI realm have expanded the scope of data and methods they use, such as textual and audio cues from financial earnings calls, but limitations exist. |
| Approach: | They propose a Saliency-guided Hierarchical Mixup augmentation technique for multimodal financial prediction tasks. |
| Outcome: | The proposed technique outperforms state-of-the-art methods by 3-7% on financial earnings and conference call datasets. |
DocTime: A Document-level Temporal Dependency Graph Parser (2022.naacl-main)
Copied to clipboard
Puneet Mathur, Vlad Morariu, Verena Kaynig-Fittkau, Jiuxiang Gu, Franck Dernoncourt, Quan Tran, Ani Nenkova, Dinesh Manocha, Rajiv Jain
| Challenge: | Document dependency graphs (TDGs) are used to understand the temporal relations between events mentioned in a document and to improve downstream tasks such as timeline creation and time-aware summarization. |
| Approach: | They propose a temporal dependency graph parser that takes input from a text document and produces a graph that incorporates longer range dependencies. |
| Outcome: | The proposed framework outperforms existing models on three datasets and improves tasks such as timeline creation, time-aware summarization, and temporal information extraction. |
DELOC: Document Element Localizer (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to edit PDFs using natural language are ineffective at grounding the edit location effectively. |
| Approach: | They propose a system to ground PDF edit request spatially using a model to predict the edit location in the PDF. |
| Outcome: | The proposed system outperforms existing Multimodal Large Language Models and specialized models on DocEdit. |
DocEdit-v2: Document Structure Editing Via Multimodal LLM Grounding (2024.emnlp-main)
Copied to clipboard
Manan Suri, Puneet Mathur, Franck Dernoncourt, Rajiv Jain, Vlad Morariu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha
| Challenge: | Document structure editing involves manipulating localized textual, visual, and layout components in document images based on user’s requests. |
| Approach: | They propose a framework that performs end-to-end document editing by leveraging Large Multimodal Models (LMMs) by localizing edit regions of interest and disambiguating user edit requests into edit commands. |
| Outcome: | The proposed framework outperforms baselines on edit command generation (2-33%), RoI bounding box detection (12-31%), and overall document editing (1-12%) tasks. |
Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures (2021.naacl-main)
Copied to clipboard
| Challenge: | Prior work on identifying narratives related to sexual abuse disclosures did not consider this as an independent task. |
| Approach: | They propose to identify narratives related to sexual abuse disclosures as a joint modeling task that leverages their emotional attributes through multitask learning. |
| Outcome: | The proposed model leverages emotional attributes of textual conversations to identify narratives related to sexual abuse disclosures in homogeneous and heterogeneously settings. |
AnalystBench: Benchmarking professional long-form report generation with web-mined multimodal tasks (2026.findings-acl)
Copied to clipboard
Chau Minh Pham, Zichao Wang, Puneet Mathur, Alexa Siu, Akriti Jain, Aparna Garimella, Ananya B. Sai, Nedim Lipka, Mohit Iyyer, Varun Manjunatha
| Challenge: | Existing benchmarks decompose the end-to-end professional report generation into individual components. |
| Approach: | They propose a benchmarking tool that evaluates 20 real-world professional report generation tasks grounded in multimodal document collections. |
| Outcome: | The proposed model outperforms closed-source models on executive summarization tasks but drops significantly on long-horizon synthesis tasks. |
VolTAGE: Volatility Forecasting via Text Audio Fusion with Graph Convolution Networks for Earnings Calls (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to stock volatility forecasting ignore correlations between stocks. |
| Approach: | They propose to combine vocal cues with verbal and financial cue data to create a multimodal stock volatility prediction model that accounts for stock interdependence via graph convolutions. |
| Outcome: | The proposed model outperforms existing methods showing that it can predict volatility using multimodal learning. |
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for document QA for visually rich documents outperform unimodal and long-context LLMs by 12-20%. |
| Approach: | They propose a multimodal Retrieval Augmented Generation approach that integrates visual and textual retrieval with linguistic reasoning. |
| Outcome: | The proposed approach outperforms unimodal and long-context LLM benchmarks for document QA by 12-20%. |
PersonaLM: Language Model Personalization via Domain-distributed Span Aggregated K-Nearest N-gram Retrieval Augmentation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language modeling tools for automatic speech recognition (ASR) are difficult to personalize. |
| Approach: | They propose a domain-distributed Span-Aggregated K-nearest N-gram retrieval augmentation to improve language modeling for automatic speech recognition (ASR) personalization. |
| Outcome: | The proposed model outperforms baselines on Wikitext-103, UserLibri, and ASAP datasets with a 10-16% improvement in perplexity and a 5-8% reduction in word error rates. |
Speak up, Fight Back! Detection of Social Media Disclosures of Sexual Harassment (N19-3)
Copied to clipboard
| Challenge: | #MeToo movement provides platform to narrate personal experiences of sexual harassment. |
| Approach: | They propose a three-part ULMFiT architecture to tackle text subtleties in a classification task . they propose to annotate a manually annotated real-world dataset to test their approach . |
| Outcome: | The proposed model outperforms existing models that rely on handcrafted stylistic features and is more accurate than generic models. |
Multimodal Multi-Speaker Merger & Acquisition Financial Modeling: A New Task, Dataset, and Neural Baselines (2021.acl-long)
Copied to clipboard
| Challenge: | Merger and acquisition (M&A) calls provide key insights into claims made by company executives about restructuring of financial firms. |
| Approach: | They propose a baseline architecture that leverages multimodal multi-speaker input to forecast financial risk associated with M&A calls. |
| Outcome: | The proposed model performs marginally better than existing models based on BERT inputs . the proposed model is expected to be validated by the end of the year . |
MoDS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collections (2025.naacl-long)
Copied to clipboard
Nishant Balepur, Alexa Siu, Nedim Lipka, Franck Dernoncourt, Tong Sun, Jordan Lee Boyd-Graber, Puneet Mathur
| Challenge: | Query-focused summarization (QFS) gives an overview of documents to answer a query, ignoring debatable ones. |
| Approach: | They propose a multi-LLM framework that uses a Query-focused summarization approach to create balanced summaries that answer debatable queries. |
| Outcome: | The proposed framework beats SOTA by 38-59% in topic paragraph coverage and balance, based on new citation metrics. |
DocPilot: Copilot for Automating PDF Edit Workflows in Documents (2024.acl-demos)
Copied to clipboard
| Challenge: | Document workflow copilot system that can understand user intent and execute tasks accordingly to help users streamline their workflows. |
| Approach: | They propose an AI-assisted document workflow copilot system capable of understanding user intent and executing tasks accordingly. |
| Outcome: | The proposed system can understand user intent and execute tasks accordingly to help users streamline their workflows. |
GUI Agents: A Survey (2025.findings-acl)
Copied to clipboard
Dang Nguyen, Jian Chen, Yu Wang, Gang Wu, Namyong Park, Zhengmian Hu, Hanjia Lyu, Junda Wu, Ryan Aponte, Yu Xia, Xintong Li, Jing Shi, Hongjie Chen, Viet Dac Lai, Zhouhang Xie, Sungchul Kim, Ruiyi Zhang, Tong Yu, Mehrab Tanjim, Nesreen K. Ahmed, Puneet Mathur, Seunghyun Yoon, Lina Yao, Branislav Kveton, Jihyung Kil, Thien Huu Nguyen, Trung Bui, Tianyi Zhou, Ryan A. Rossi, Franck Dernoncourt
| Challenge: | Large Foundation Models (LFMs) have transformed the landscape of AI research and day-to-day life. |
| Approach: | They propose a framework that delineates GUI agents' perception, reasoning, planning, and acting capabilities. |
| Outcome: | The proposed framework delineates their perception, reasoning, planning, and acting capabilities. |
SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps (2025.findings-emnlp)
Copied to clipboard
| Challenge: | SQLSpace is a representation for text-to-SQL examples derived with minimal human intervention. |
| Approach: | They introduce SQLSpace, a human-interpretable, generalizable, compact representation for text-to-SQL examples . they demonstrate that SQLSpace enables analysis that would be difficult with raw examples alone . |
| Outcome: | The proposed representations are human-interpretable and generalizable . they are used to evaluate models with a granularity beyond overall accuracy scores . |
ChartLens: Fine-grained Visual Attribution in Charts (2025.acl-long)
Copied to clipboard
| Challenge: | MLLMs suffer from hallucinations, where generated text fails to align with visual inputs. |
| Approach: | They propose a chart attribution algorithm that uses segmentation-based techniques to identify chart objects and employs set-of-marks prompting with MLLMs for fine-grained visual attribution. |
| Outcome: | The proposed algorithm improves fine-grained attributions by 26-66% . |
DocFin: Multimodal Financial Prediction and Bias Mitigation using Semi-structured Documents (2022.findings-emnlp)
Copied to clipboard
Puneet Mathur, Mihir Goyal, Ramit Sawhney, Ritik Mathur, Jochen Leidner, Franck Dernoncourt, Dinesh Manocha
| Challenge: | Existing research focuses on textual and audio modalities of financial disclosures but ignores the rich tabular data available in financial reports. |
| Approach: | They propose to combine tabular financial data with text transcripts and audio recordings to improve stock volatility and price movement prediction by 5-12% and reduce gender bias by over 30%. |
| Outcome: | The combined data improves stock volatility and price movement prediction by 5-12% and reduces gender bias caused due to audio-based neural networks by over 30%. |