Papers by Mihir Kale
TicketTalk: Toward human-level performance with end-to-end, transaction-based dialog systems (2021.acl-long)
Copied to clipboard
| Challenge: | TicketTalk dataset with 23,789 annotated dialogs is a data-driven, end-to-end approach to transaction-based dialog systems that performs at near-human levels in terms of verbal response quality and factual grounding accuracy. |
| Approach: | They propose a data-driven, end-to-end approach to transaction-based dialog systems that performs at near-human levels in terms of verbal response quality and factual grounding accuracy. |
| Outcome: | The proposed model generates verbal responses and API call predictions on a movie ticketing dialog dataset with 23,789 annotated conversations. |
nmT5 - Is parallel data still relevant for pre-training massively multilingual language models? (2021.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that cross-lingual transfer learning in pre-trained multilingual models could be improved further by incorporating parallel data. |
| Approach: | They propose to integrate parallel data into mT5 pre-training to improve results on downstream multilingual and cross-lingual tasks. |
| Outcome: | The proposed model improves cross-lingual transfer significantly in small fine-tuning datasets and small model sizes. |
XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages (2023.findings-emnlp)
Copied to clipboard
Sebastian Ruder, Jonathan Clark, Alexander Gutkin, Mihir Kale, Min Ma, Massimo Nicosia, Shruti Rijhwani, Parker Riley, Jean-Michel Sarr, Xinyi Wang, John Wieting, Nitish Gupta, Anna Katanova, Christo Kirov, Dana Dickinson, Brian Roark, Bidisha Samanta, Connie Tao, David Adelani, Vera Axelrod, Isaac Caswell, Colin Cherry, Dan Garrette, Reeve Ingle, Melvin Johnson, Dmitry Panteleev, Partha Talukdar
| Challenge: | Existing datasets are often informed by established research directions in the NLP community. |
| Approach: | They propose a benchmark to evaluate the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks. |
| Outcome: | The proposed benchmark evaluates the capabilities of language models across 88 under-represented languages over 9 key user-centric technologies including ASR, OCR, MT, and information access tasks. |
Template Guided Text Generation for Task-Oriented Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines the use of templates to generate natural language utterances for a large number of APIs. |
| Approach: | They propose a schema-guided approach which conditions the generation on a natural language schema. |
| Outcome: | The proposed method improves over strong baselines, is robust to out-of-domain inputs and shows improved sample efficiency. |
ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models (2022.tacl-1)
Copied to clipboard
Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, Colin Raffel
| Challenge: | a number of pre-trained language models use sequences of tokens corresponding to word units . token-free models that operate directly on raw text have many advantages . |
| Approach: | They propose a standard Transformer architecture that can be used to process byte sequences . they also characterize trade-offs in terms of parameter count, training FLOPs, and inference speed . |
| Outcome: | The proposed model is more robust to noise and more robust on spelling and pronunciation tasks. |
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer (2021.naacl-main)
Copied to clipboard
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel
| Challenge: | Current natural language processing pipelines often use transfer learning, where a model is pre-trained on a data-rich task before being fine-tuned on . this significantly limits their use given that roughly 80% of the world population does not speak English. |
| Approach: | They introduce a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages. |
| Outcome: | The proposed model achieves state-of-the-art on multilingual benchmarks and a simple technique to prevent accidental translation in the zero-shot setting. |
Improving Compositional Generalization with Self-Training for Data-to-Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). |
| Approach: | They propose a template-based input representation that greatly improves the model’s generalization capability. |
| Outcome: | The proposed model improves tree accuracy by 46%+ and reduces slot error rates by 73%+ over the strong baselines on SGD and Weather benchmarks. |