Papers by Raj Singh
Token Prediction as Implicit Classification to Identify LLM-Generated Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach for identifying large language models (LLMs) involved in text generation is proposed . instead of adding an additional classification layer, we reframe the classification task as a next-token prediction task . |
| Approach: | They propose a novel approach for identifying large language models involved in text generation . instead of adding an additional classification layer, they reframe the task as a next-token prediction task . |
| Outcome: | The proposed method performs exceptionally well in the text classification task . it can distinguish distinctive writing styles among various LLMs even without an explicit classifier. |
How Good is Zero-Shot MT Evaluation for Low Resource Indian Languages? (2024.acl-short)
Copied to clipboard
| Challenge: | a recent study focused on machine translation evaluation for low-resource languages . linguistic aspects that vary across languages are factors that will exacerbate the problem in low-source languages due to the reliance on extensive data resources. |
| Approach: | They propose to use multi-dimensional quality metrics and DA annotations to meta-evaluate MT evaluation metrics for low-resource languages. |
| Outcome: | The proposed evaluation metrics are based on human scores on the candidate translations of assamese, maithili, and Punjabi. |
KG-MuLQA: A Framework for KG-based Multi-Level QA Extraction and Long-Context LLM Evaluation (2026.acl-long)
Copied to clipboard
Nikita Tatarinov, Vidhyakshaya Kannan, Haricharana Srinivasa, Arnav Raj, Harpreet Singh Anand, Varun Singh, Aditya Luthra, Ravij Lade, Agam Shah, Sudheer Chava
| Challenge: | KG-MulQA extracts QA pairs at multiple complexity levels along three key dimensions: multi-hop retrieval, set operations, and answer plurality. |
| Approach: | They propose a framework that extracts QA pairs at multiple complexity levels along three key dimensions: multi-hop retrieval, set operations, and answer plurality. |
| Outcome: | The framework extracts QA pairs at multiple complexity levels along key dimensions . it enables fine-grained assessment of model performance across controlled difficulty levels. |
PhoniTale: Phonologically Grounded Mnemonic Generation for Typologically Distant Language Pairs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Vocabulary acquisition is a challenge for second-language learners when learning typologically distant languages such as English and Korean, where phonological and structural mismatches complicate vocabulary learning. |
| Approach: | They propose a cross-lingual mnemonic generation system that performs IPA-based phonological adaptation and syllable-aware alignment to retrieve L1 keyword sequence and uses LLMs to generate verbal cues. |
| Outcome: | The proposed system outperforms human-written and automated mnemonics in a short-term recall test with human participants and achieves quality comparable to human-writing mnms. |
SVeritas: Benchmark for Robust Speaker Verification under Diverse Conditions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks only evaluate a subset of potential conditions, missing others entirely. |
| Approach: | a new benchmark suite evaluates speaker verification models under a variety of stressors . a san francisco-based team evaluates models under natural and background conditions . |
| Outcome: | a new benchmark suite evaluates speaker verification models under stressors under a variety of conditions . the results show that some models perform better under stress conditions than others . |
R-BASS : Relevance-aided Block-wise Adaptation for Speech Summarization (2024.findings-naacl)
Copied to clipboard
| Challenge: | End-to-end speech summarization on long recordings is challenging because of the high computational cost. |
| Approach: | They propose a new relevance-aware block-wise adaptation method that automatically estimates block relevance based on lexical and semantic similarity between transcript and summary. |
| Outcome: | The proposed method can drop 86.3 % of blocks while maintaining comparable performance. |
Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Automatic speech recognition systems have seen remarkable improvements in recent years, but evaluation of performance remains dependent on word and character error rate (WER/CER). |
| Approach: | They investigate how distribution shifts, model size and model architecture influence hallucination error rate (HER) HER is a metric used to quantify hallucinosity in automatic speech recognition systems. |
| Outcome: | The proposed model can be used to measure hallucination errors in high-stakes domains such as healthcare, legal, and aviation. |
On the Robust Approximation of ASR Metrics (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for estimating speech recognition metrics depend on ground truth labels. |
| Approach: | They propose a label-free approach to approximating ASR performance metrics . they embed multimodal embeddings in a unified space for speech and transcription representations . |
| Outcome: | The proposed method outperforms baseline models on speech recognition benchmarks by 50%. |
Towards Noise-Tolerant Speech-Referring Video Object Segmentation: Bridging Speech and Text (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in vision-language learning have significantly advanced Human-Computer Interactions (HCI). |
| Approach: | They propose a method to align the semantic spaces between speech and text by incorporating two modules to align semantic spaces. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on AVOS benchmarks. |
CAARMA: Class Augmentation with Adversarial Mixup Regularization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Speaker verification tasks require inference of unseen classes using specialized losses. |
| Approach: | They propose a class augmentation framework that generates synthetic classes through data mixing in the embedding space. |
| Outcome: | The proposed framework improves speaker verification tasks by 8% over baseline models. |
A Computational Approach to Feature Extraction for Identification of Suicidal Ideation in Tweets (P18-3)
Copied to clipboard
| Challenge: | Suicidal ideation on social media websites is associated with higher suicide rates . suicide is the second leading cause of death among 15-29-year-olds . |
| Approach: | They propose a supervised method for detecting suicidal ideation in tweets using a dataset of manually annotated tweets. |
| Outcome: | The proposed method is compared against four baselines to validate its utility. |