| Challenge: | Sanskrit word segmentation is challenging due to the issue of Sandhi . digitisation efforts have made the manuscripts available in the public domain . |
| Approach: | They propose a deep sequence to sequence model that takes only the sandhied string as input and predicts the unsandhized string. |
| Outcome: | The proposed model improves on the current state of the art by 16.79% . the system can be trained "overnight" and be used for production . |
Similar Papers
Sanskrit Sandhi Splitting using seq2(seq)2 (D18-1)
Copied to clipboard
| Challenge: | Existing methods for word splitting in Sanskrit have low accuracy as the same compound word might be broken down in multiple ways to provide syntactically correct splits. |
| Approach: | They propose a deep learning architecture called Double Decoder RNN which predicts the location of the splits with 95% accuracy and 79.5% accuracy. |
| Outcome: | The proposed model outperforms the state-of-the-art in the problem of Chinese word segmentation with 79.5% accuracy and the existing model's generalization capability. |
Sanskrit Word Segmentation Using Character-level Recurrent and Convolutional Neural Networks (D18-1)
Copied to clipboard
| Challenge: | Using end-to-end neural network models, Sanskrit is tokenized by splitting compounds and resolving phonetic merges. |
| Approach: | They propose end-to-end neural network models that tokenize Sanskrit by jointly splitting compounds and resolving phonetic merges. |
| Outcome: | The proposed models outperform the state-of-the-art for the task of splitting compounds and resolving phonetic merges. |
TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizer (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to SWS fail when encountering out-of-vocabulary tokens . lexicon driven approaches fail when dealing with out- of-vocal tokens, authors say . |
| Approach: | They propose a novel lexicon-driven shallow parser that takes into account sandhi phenomenon . they propose linguistically informed Sanskrit tokenizer module that encodes character input . |
| Outcome: | The proposed system outperforms the current state-of-the-art system by 7.2 points absolute gain on the benchmark datasets. |
One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Morphologically rich languages are notoriously challenging to process for downstream NLP applications. |
| Approach: | They propose a pretrained model for NLP applications involving the morphologically rich language Sanskrit that outperforms previous models by a considerable margin. |
| Outcome: | The proposed model outperforms tokenized models on established Sanskrit word segmentation tasks and matches the current best lexicon-based model. |
SandhiKosh: A Benchmark Corpus for Evaluating Sanskrit Sandhi Tools (L18-1)
Copied to clipboard
| Challenge: | Several important texts which are of interest to people all over the world were written in Sanskrit. |
| Approach: | They develop a Sanskrit benchmark to evaluate the completeness and accuracy of tools . they use three most prominent tools to evaluate their completeness . |
| Outcome: | The proposed tools have substantial scope for improvement and are available to researchers worldwide. |
Free as in Free Word Order: An Energy Based Model for Word Segmentation and Morphological Tagging in Sanskrit (D18-1)
Copied to clipboard
Amrith Krishna, Bishal Santra, Sasi Prasanth Bandaru, Gaurav Sahu, Vishnu Dutt Sharma, Pavankumar Satuluri, Pawan Goyal
| Challenge: | a structured prediction framework is proposed to solve word segmentation and morphological tagging tasks in a free word order language. |
| Approach: | They propose a structured prediction framework that jointly solves word segmentation and morphological tagging tasks in Sanskrit. |
| Outcome: | The proposed model outperforms the state of the art with an F-Score of 96.92 (percentage improvement of 7.06%) while using less than one tenth of the task-specific training data. |
Poetry to Prose Conversion in Sanskrit as a Linearisation Task: A Case for Low-Resource Languages (P19-1)
Copied to clipboard
| Challenge: | Obtaining the proper word ordering, called as the prose ordering, from a verse is often considered a task which requires linguistic expertise. |
| Approach: | They propose a word ordering (linearisation) task that ignores the word arrangement at the verse side. |
| Outcome: | The proposed model outperforms current models in word ordering for the translation task in Sanskrit. |
Chandomitra: Towards Generating Structured Sanskrit Poetry from Natural Language Inputs (2026.eacl-long)
Copied to clipboard
Manoj Balaji Jagadeeshan, Samarth Bhatia, Pretam Ray, Harshul Raj Surana, Akhil Rajeev P, Priya Mishra, Annarao Kulkarni, Ganesh Ramakrishnan, Prathosh Ap, Pawan Goyal
| Challenge: | Large language models are capable of creative generation tasks but prominently for high-resource languages. |
| Approach: | They propose to use large language models for structured poetry generation in Sanskrit . their constrained decoding method achieves 99.86% syntactic accuracy . |
| Outcome: | The proposed model outperforms the existing model in generating metrically valid Sanskrit poetry. |
A Benchmark and Dataset for Post-OCR text correction in Sanskrit (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Sanskrit is a classical language with 30 million manuscripts available for digitisation . however, it is considered to be low-resource when it comes to available digital resources. |
| Approach: | They propose to use a post-OCR text correction dataset to correct errors from OCR predictions from 30 different books in the Indian subcontinent. |
| Outcome: | The proposed model outperforms OCR models on graphemic and lexical levels and shows that it is more accurate than previous models. |
Automatic Speech Recognition in Sanskrit: A New Speech Corpus and Modelling Insights (2021.findings-acl)
Copied to clipboard
| Challenge: | In this paper, we propose the first large scale study of automatic speech recognition in Sanskrit . we focus on the impact of unit selection in San's ASR systems . |
| Approach: | They propose a large scale study of automatic speech recognition in Sanskrit . they propose syllable level unit selection that captures character sequences . |
| Outcome: | The proposed model captures character sequences from one vowel in the word to the next vowela. |