Papers by Vipul Raheja

mEdIT: Multilingual Text Editing via Instruction Tuning (2024.naacl-long)

Copied to clipboard

Vipul Raheja, Dimitris Alikaniotis, Vivek Kulkarni, Bashar Alhafni, Dhruv Kumar

Challenge:	mEdIT is a multi-lingual extension to CoEdit for writing assistance.
Approach:	They propose to train multi-lingual large language models (LLMs) by fine-tuning them via instruction tuning.
Outcome:	The proposed model performs well on multilingual text editing benchmarks and generalizes well to new languages.

ContraDoc: Understanding Self-Contradictions in Documents with Large Language Models (2024.naacl-long)

Copied to clipboard

Jierui Li, Vipul Raheja, Dhruv Kumar

Semantic Text Processing

Challenge:	Detecting contradictions in texts is often regarded as determining relation between hypothesis and piece of premise.
Approach:	They propose a human-annotated dataset to study self-contradictions in long documents . they analyze the capabilities of four open-source and commercially available LLMs .
Outcome:	The proposed dataset outperforms open-source LLMs on document-level tasks but struggles with self-contradictions that require more nuance and context.

CoEdIT: Text Editing by Task-Specific Instruction Tuning (2023.findings-emnlp)

Copied to clipboard

Vipul Raheja, Dhruv Kumar, Ryan Koo, Dongyeop Kang

Language Models

Challenge:	We present a large language model for writing assistance that is fine-tuned on task-specific instructions.
Approach:	They propose a large language model that is fine-tuned on task-specific instructions and outputs the edited text.
Outcome:	The proposed model performs better than other state-of-the-art models on various editing benchmarks while being 60x smaller.

Adversarial Grammatical Error Correction (2020.findings-emnlp)

Copied to clipboard

Vipul Raheja, Dimitris Alikaniotis

Multilinguality

Challenge:	Experimental results show that adversarial-GEC can achieve competitive GEC quality compared to NMT-based baselines.
Approach:	They propose an adversarial approach to Grammatical Error Correction using a transformer-based model and a sentence-pair classification model.
Outcome:	The proposed approach achieves competitive GEC quality compared to baselines.

Speakerly: A Voice-based Writing Assistant for Text Composition (2023.emnlp-industry)

Copied to clipboard

Dhruv Kumar, Vipul Raheja, Alice Kaiser-Schatzlein, Robyn Perry, Apurva Joshi, Justin Hugues-Nuger, Samuel Lou, Navid Chowdhury

Multimodality

Challenge:	Speakerly TM is a voice-based writing assistance system that works across the different stages of writing.
Approach:	They propose a voice-based writing assistance system that helps users with text composition across various use cases such as emails, instant messages, and notes.
Outcome:	The proposed system can be used for email, instant messages, and notes.

Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks (2022.emnlp-main)

Copied to clipboard

Zae Myung Kim, Wanyu Du, Vipul Raheja, Dhruv Kumar, Dongyeop Kang

Syntactic Text Processing

Challenge:	Iterative text revision improves text quality by fixing grammatical errors, rephrasing for better readability or contextual appropriateness.
Approach:	They propose to build an end-to-end text revision system that can iteratively generate helpful edits by explicitly detecting editable spans with their corresponding edit intents.
Outcome:	The proposed system outperforms baselines on other text revision tasks and human evaluations.

Benchmarking Cognitive Biases in Large Language Models as Evaluators (2024.findings-acl)

Copied to clipboard

Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, Dongyeop Kang

Language Models

Challenge:	Large Language Models (LLMs) have been shown to be effective as automatic evaluators with simple prompting and in-context learning.
Approach:	They assemble 16 Large Language Models and evaluate their outputs by preference ranking . they introduce a cognitive bias benchmark to measure six different cognitive biases in LLM evaluation outputs.
Outcome:	The proposed model is biased on the CoBBLer benchmark, indicating that machine preferences are misaligned with humans.

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)

Copied to clipboard

Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou

Text Generation

Challenge:	Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work.
Approach:	They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations.
Outcome:	The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work.

Understanding Iterative Revision from Human-Written Text (2022.acl-long)

Copied to clipboard

Wanyu Du, Vipul Raheja, Dhruv Kumar, Zae Myung Kim, Melissa Lopez, Dongyeop Kang

Challenge:	This work describes IteraTeR: the first large-scale, multi-domain, edit-intention annotated corpus of iteratively revised text.
Approach:	They propose to annotate iteratively revised text using a multi-domain annotated corpus that generalizes to a variety of domains, edit intentions, revision depths, and granularities.
Outcome:	The proposed model improves automatic evaluations by integrating edit intentions with writing quality.

Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs (2024.acl-long)

Copied to clipboard

Zae Myung Kim, Kwang Lee, Preston Zhu, Vipul Raheja, Dongyeop Kang

Challenge:	Empirical findings show that although both LLMs and humans generate distinct discourse patterns influenced by specific domains, human-written texts exhibit more structural variability, reflecting the nuanced nature of human writing in different domains.
Approach:	They propose a method to leverage hierarchical parse trees and recursive hypergraphs to uncover distinctive discourse patterns in texts written by humans and LLMs.
Outcome:	The proposed method combines hierarchical parse trees and recursive hypergraphs to uncover distinctive discourse patterns in texts produced by both LLMs and humans.

Dialogue Act Classification with Context-Aware Self-Attention (N19-1)

Copied to clipboard

Vipul Raheja, Joel Tetreault

Text Classification

Challenge:	Recent work in Dialogue Act classification has treated the task as a sequence labeling problem using hierarchical deep neural networks.
Approach:	They propose a hierarchical deep neural network to model different levels of utterance and dialogue act semantics and use contextual dependencies to improve performance.
Outcome:	The proposed model improves on the Switchboard Dialogue Act Corpus while maintaining high accuracy.

SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models (2025.naacl-long)

Copied to clipboard

Margaret Mitchell, Giuseppe Attanasio, Ioana Baldini, Miruna Clinciu, Jordan Clive, Pieter Delobelle, Manan Dey, Sil Hamilton, Timm Dill, Jad Doughman, Ritam Dutt, Avijit Ghosh, Jessica Zosa Forde, Carolin Holtermann, Lucie-Aimée Kaffee, Tanmay Laud, Anne Lauscher, Roberto L Lopez-Davila, Maraim Masoud, Nikita Nangia, Anaelia Ovalle, Giada Pistilli, Dragomir Radev, Beatrice Savoldi, Vipul Raheja, Jeremy Qin, Esther Ploeger, Arjun Subramonian, Kaustubh Dhole, Kaiser Sun, Amirbek Djanibekov, Jonibek Mansurov, Kayo Yin, Emilio Villa Cueva, Sagnik Mukherjee, Jerry Huang, Xudong Shen, Jay Gala, Hamdan Al-Ali, null Tair Djanibekov, Nurdaulet Mukhituly, Shangrui Nie, Shanya Sharma, Karolina Stanczak, Eliza Szczechla, Tiago Timponi Torrent, Deepak Tunuguntla, Marcelo Viridiano, Oskar Van Der Wal, Adina Yakefu, Aurélie Névéol, Mike Zhang, Sydney Zink, Zeerak Talat

Language Models

Challenge:	Large Language Models reproduce and exacerbate social biases present in training data, and resources to quantify this issue are limited.
Approach:	They propose a multilingual parallel dataset to examine culturally-specific stereotypes that may be learned by LLMs.
Outcome:	The proposed dataset includes stereotypes from 20 regions around the world and 16 languages, spanning multiple identity categories subject to discrimination worldwide.