Papers by Sameer Jain
An Empirical Comparison of Instance Attribution Methods for NLP (2021.naacl-main)
Copied to clipboard
| Challenge: | Influence functions provide machinery for identifying training instances that may have led to a specific prediction, but are computationally expensive and prohibitive in many cases. |
| Approach: | They evaluate the degree to which different potential instance attribution agrees with respect to the importance of training samples. |
| Outcome: | The proposed methods exhibit desirable characteristics similar to more complex methods, but are computationally expensive. |
Combining Feature and Instance Attribution to Detect Artifacts (2022.findings-acl)
Copied to clipboard
| Challenge: | In this paper, we evaluate use of different attribution methods for aiding identification of training data artifacts. |
| Approach: | They propose hybrid methods that combine saliency maps and instance attribution methods to aid in identifying training data artifacts. |
| Outcome: | The proposed methods can be used to efficiently uncover artifacts in training data when a challenging validation set is available. |
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning (2023.findings-acl)
Copied to clipboard
Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou
| Challenge: | In-context learning-based evaluators are competitive with learned evaluation frameworks for text summarization tasks. |
| Approach: | They propose to use large language models as multi-dimensional evaluators using in-context learning to evaluate text summarization tasks. |
| Outcome: | The proposed frameworks are competitive with existing frameworks on relevance and factual consistency, the authors show . |
MEGA: Multilingual Evaluation of Generative AI (2023.emnlp-main)
Copied to clipboard
Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
| Challenge: | Large Large Models (LLMs) have shown impressive performance on many natural language processing tasks such as language understanding, reasoning, and language generation. |
| Approach: | They present a framework for evaluating generative LLMs in the multilingual setting and provide directions for future progress in the field. |
| Outcome: | The proposed framework evaluates generative models on 16 NLP datasets across 70 typologically diverse languages and compares them to state-of-the-art non-autoregressive models. |