Papers by Meet Doshi
Pretraining Language Models Using Translationese (2024.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that large language models perform well in low-resource languages . a vast majority of languages don't have comparable data as compared to English . |
| Approach: | They propose to use Translationese as synthetic data for pre-training language models for low-resource languages. |
| Outcome: | The proposed method reduces performance of LMs trained on clean data in Indian languages . the proposed model performs better in English than in other languages, but is not comparable to English. |
PUB: A Pragmatics Understanding Benchmark for Assessing LLMs’ Pragmatics Capabilities (2024.findings-acl)
Copied to clipboard
| Challenge: | Pragmatics understanding is not well studied in LLMs, but their understanding of pragmatics is lacking. |
| Approach: | They propose to use a dataset to measure LLMs' understanding of pragmatics to evaluate their models. |
| Outcome: | The proposed dataset includes 14 tasks in four pragmatics phenomena, namely; Implicature, Presupposition, Reference, and Deixis. |