Papers by Vatsal Raina
Is It Possible to Modify Text to a Target Readability Level? An Initial Investigation Using Zero-Shot Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Text simplification and elaboration tasks are limited to only relatively altering the readability of texts to cater to a diverse audience. |
| Approach: | They propose to generate 8 versions of a text at different readability levels using ChatGPT and Llama-2 and introduce a two-step process to generate paraphrases. |
| Outcome: | The proposed task requires the generation of 8 versions at various target readability levels for each input text. |
Efficient LLM Comparative Assessment: A Product of Experts Framework for Pairwise Comparisons (2024.emnlp-main)
Copied to clipboard
| Challenge: | LLM-as-a-judge approaches are effective but cost scales quadratically with number of candidates, which has practical limitations. |
| Approach: | They propose a Product of Expert (PoE) framework for efficient LLM Comparative Assessment where individual comparisons are considered experts that provide information on a pair’s score difference. |
| Outcome: | The proposed framework can generate score predictions that correlate well with human judgements on multiple NLG tasks with as few as 2% of comparisons. |
Finetuning LLMs for Comparative Assessment Tasks (2025.coling-main)
Copied to clipboard
| Challenge: | Automated assessment in natural language generation is a challenging task. |
| Approach: | They propose a framework for fine-tuning LLMs for comparative assessment to align the model’s output with the target distribution of comparative probabilities. |
| Outcome: | The proposed framework improves state-of-the-art performance while maintaining high performance with an efficient subset of comparisons. |
Analyzing Biases to Spurious Correlations in Text Classification Tasks (2022.aacl-short)
Copied to clipboard
| Challenge: | Often these systems exceed human performance, but there is a caveat: standard benchmarks often assume that training and evaluation data are drawn independently and identically from the same underlying distribution. |
| Approach: | They propose to exploit spurious correlations in training data to exploit these correlations . they show that even when only ‘stop’ words are available, it is possible to predict the class significantly better than random. |
| Outcome: | The proposed model can predict class significantly better when only ‘stop’ words are available at the input stage, but can degrade the ability of the system to generalize well to out-of-domain data. |
Answer Uncertainty and Unanswerability in Multiple-Choice Machine Reading Comprehension (2022.findings-acl)
Copied to clipboard
| Challenge: | Machine reading comprehension (MRC) systems focus on selecting the correct answer to a question given a context paragraph. |
| Approach: | They propose to use machine reading comprehension (MRC) to assess the ability of systems to understand natural language. |
| Outcome: | The proposed system outperforms a system built with an NOA option . the results show that the system is not confident about the NOA choice . |
An Information-Theoretic Approach to Analyze NLP Classification Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Natural language processing (NLP) tasks take either a single or multiple text elements to predict an output variable. |
| Approach: | They propose an information-theoretic framework to analyse the influence of inputs on the output of text classification tasks. |
| Outcome: | The proposed framework is available at: https://github.com/WangLuran/nlp-element-influence. |