Papers by Prachi Jain
Type-Sensitive Knowledge Base Inference Without Explicit Type Supervision (P18-2)
Copied to clipboard
| Challenge: | State-of-the-art knowledge base completion models make frequent errors when ranking entities that are not compatible with the type required by the relation. |
| Approach: | They propose to enhance each base factorization with two type-compatibility terms between entity-relation pairs and combine the signals in a novel manner. |
| Outcome: | The proposed model achieves 7% MRR gains over baseline models and predicts supervised types better than baseline models. |
Temporal Knowledge Base Completion: New Algorithms and Evaluation Protocols (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing TKBC models heavily overestimate link prediction performance due to imperfect evaluation mechanisms. |
| Approach: | They propose a method that integrates entities, relations and time into a uniform space . they propose improved evaluation protocols for link and time prediction . |
| Outcome: | The proposed method exploits the recurrent nature of some facts/events and temporal interactions between pairs of relations yielding state-of-the-art results. |
Joint Completion and Alignment of Multilingual Knowledge Graphs (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge graph completion are incomplete, as curators struggle to keep up with the real world. |
| Approach: | They propose a multitask approach to solve missing facts in incomplete Knowledge Graphs . they add a relation representation to the existing KG embedding scheme . |
| Outcome: | The proposed system outperforms existing models in seven languages compared to existing models . it also outperformed existing models, underscoring the value of joint alignment and completion. |
MAFIA: Multi-Adapter Fused Inclusive Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) are widely used in NLP for various tasks. |
| Approach: | They propose to modularly debias a pre-trained language model across multiple bias dimensions using structured knowledge and a large generative model. |
| Outcome: | The proposed model is able to debias a pre-trained language model across multiple bias dimensions in a semi-automated way. |
MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks (2024.naacl-long)
Copied to clipboard
Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
| Challenge: | Several new LLMs have been introduced necessitating their evaluation on non-English languages. |
| Approach: | They perform a thorough evaluation of the non-English capabilities of SoTA LLMs by comparing them on the same set of multilingual datasets. |
| Outcome: | The proposed model outperforms models on multilingual datasets on 22 languages including low-resource African languages. |
MEGA: Multilingual Evaluation of Generative AI (2023.emnlp-main)
Copied to clipboard
Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
| Challenge: | Large Large Models (LLMs) have shown impressive performance on many natural language processing tasks such as language understanding, reasoning, and language generation. |
| Approach: | They present a framework for evaluating generative LLMs in the multilingual setting and provide directions for future progress in the field. |
| Outcome: | The proposed framework evaluates generative models on 16 NLP datasets across 70 typologically diverse languages and compares them to state-of-the-art non-autoregressive models. |
A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have highlighted the existence of social biases within large vision and language models. |
| Approach: | They propose a framework for systematically evaluating gender, race, and age biases in vision-language models with respect to professions. |
| Outcome: | The proposed framework covers all supported inference modes of the recent vision-language models, including image-to-text, text-to image, and image- to-image. |