Papers by Varun Jain
The Bull and the Bear: Summarizing Stock Market Discussions (2022.lrec-1)
Copied to clipboard
| Challenge: | a dataset of 7888 reddit posts and 400 posts is used to summarize stock market topics. |
| Approach: | They curate discussions on social media platforms and construct an abstractive summarization dataset. |
| Outcome: | The proposed dataset consists of 7888 Reddit posts and summaries for 400 posts . it is robustly evaluated and will be made publicly available . |
Influence Functions for Sequence Tagging Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition, Part-of-Speech tagging, and Semantic Role Labeling are standard tasks in NLP, but there has been little work on interpretability methods for sequence taging. |
| Approach: | They propose to extend influence functions to sequence tagging tasks by identifying noisy annotations in NER corpora. |
| Outcome: | The proposed methods are able to identify noisy annotations in NER corpora and are scalable. |
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models (2021.naacl-main)
Copied to clipboard
Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, Xia Hu
| Challenge: | Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. |
| Approach: | They propose a shortcut mitigation framework to suppress NLU models from making overconfident predictions for samples with large shortcut degree. |
| Outcome: | The proposed framework suppresses the model from making overconfident predictions for samples with large shortcut degree. |
Keyphrase Prediction from Video Transcripts: New Dataset and Directions (2022.coling-1)
Copied to clipboard
Amir Pouran Ben Veyseh, Quan Hung Tran, Seunghyun Yoon, Varun Manjunatha, Hanieh Deilamsalehy, Rajiv Jain, Trung Bui, Walter W. Chang, Franck Dernoncourt, Thien Huu Nguyen
| Challenge: | Existing studies on keyphrase prediction have focused on formal texts and informal-text domains. |
| Approach: | They propose to annotate large-scale video transcripts with keyphrases from live-stream video . they propose to feed models with paragraph-level keyphrase extraction to foster future research . |
| Outcome: | The proposed model improves keyphrase prediction in live-stream video transcripts by feeding models with paragraph-level keyphrases. |
Syntopical Graphs for Computational Argumentation Tasks (2021.acl-long)
Copied to clipboard
Joe Barrow, Rajiv Jain, Nedim Lipka, Franck Dernoncourt, Vlad Morariu, Varun Manjunatha, Douglas Oard, Philip Resnik, Henning Wachsmuth
| Challenge: | adler and van Doren (1940) proposed a formalized manual process for understanding a topic based on multiple viewpoints. |
| Approach: | They propose a syntopical reading process that emphasizes comparing and contrasting viewpoints to improve topic understanding. |
| Outcome: | The proposed method outperforms approaches that do not use collection-level information. |
MAFIA: Multi-Adapter Fused Inclusive Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) are widely used in NLP for various tasks. |
| Approach: | They propose to modularly debias a pre-trained language model across multiple bias dimensions using structured knowledge and a large generative model. |
| Outcome: | The proposed model is able to debias a pre-trained language model across multiple bias dimensions in a semi-automated way. |
MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks (2024.naacl-long)
Copied to clipboard
Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
| Challenge: | Several new LLMs have been introduced necessitating their evaluation on non-English languages. |
| Approach: | They perform a thorough evaluation of the non-English capabilities of SoTA LLMs by comparing them on the same set of multilingual datasets. |
| Outcome: | The proposed model outperforms models on multilingual datasets on 22 languages including low-resource African languages. |
A Joint Model for Document Segmentation and Segment Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to text segmentation focus on document segmentation and segment labeling separately. |
| Approach: | They propose a method for jointly segmenting a document and labeling segments . they show that S-LSTM reduces segmentation error by 30% on average . |
| Outcome: | The proposed method reduces segmentation error by 30% while improving segment labeling. |
AnalystBench: Benchmarking professional long-form report generation with web-mined multimodal tasks (2026.findings-acl)
Copied to clipboard
Chau Minh Pham, Zichao Wang, Puneet Mathur, Alexa Siu, Akriti Jain, Aparna Garimella, Ananya B. Sai, Nedim Lipka, Mohit Iyyer, Varun Manjunatha
| Challenge: | Existing benchmarks decompose the end-to-end professional report generation into individual components. |
| Approach: | They propose a benchmarking tool that evaluates 20 real-world professional report generation tasks grounded in multimodal document collections. |
| Outcome: | The proposed model outperforms closed-source models on executive summarization tasks but drops significantly on long-horizon synthesis tasks. |
IGA: An Intent-Guided Authoring Assistant (2021.emnlp-main)
Copied to clipboard
Simeng Sun, Wenlong Zhao, Varun Manjunatha, Rajiv Jain, Vlad Morariu, Franck Dernoncourt, Balaji Vasan Srinivasan, Mohit Iyyer
| Challenge: | Pretrained language models have improved writing assistance functions such as autocomplete, but more complex and controllable writing assistants have yet to be explored. |
| Approach: | They build an intent-guided authoring assistant that follows fine-grained author directives by specifying different writing intents. |
| Outcome: | The proposed system generates output satisfying the author's intent and can be rephrased to their liking. |
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations that focus on performance metrics miss a key factor: increased effectiveness due to additional compute. |
| Approach: | They propose to incorporate the compute budget into evaluations to provide a more informative comparison that takes into account both performance metrics and computational cost. |
| Outcome: | The proposed framework outperforms reasoning strategies when they use comparable compute resources. |