Sanskrit Word Segmentation Using Character-level Recurrent and Convolutional Neural Networks (D18-1)
Copied to clipboard
| Challenge: | Using end-to-end neural network models, Sanskrit is tokenized by splitting compounds and resolving phonetic merges. |
| Approach: | They propose end-to-end neural network models that tokenize Sanskrit by jointly splitting compounds and resolving phonetic merges. |
| Outcome: | The proposed models outperform the state-of-the-art for the task of splitting compounds and resolving phonetic merges. |
Similar Papers
Sanskrit Sandhi Splitting using seq2(seq)2 (D18-1)
Copied to clipboard
| Challenge: | Existing methods for word splitting in Sanskrit have low accuracy as the same compound word might be broken down in multiple ways to provide syntactically correct splits. |
| Approach: | They propose a deep learning architecture called Double Decoder RNN which predicts the location of the splits with 95% accuracy and 79.5% accuracy. |
| Outcome: | The proposed model outperforms the state-of-the-art in the problem of Chinese word segmentation with 79.5% accuracy and the existing model's generalization capability. |
Building a Word Segmenter for Sanskrit Overnight (L18-1)
Copied to clipboard
| Challenge: | Sanskrit word segmentation is challenging due to the issue of Sandhi . digitisation efforts have made the manuscripts available in the public domain . |
| Approach: | They propose a deep sequence to sequence model that takes only the sandhied string as input and predicts the unsandhized string. |
| Outcome: | The proposed model improves on the current state of the art by 16.79% . the system can be trained "overnight" and be used for production . |
TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizer (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to SWS fail when encountering out-of-vocabulary tokens . lexicon driven approaches fail when dealing with out- of-vocal tokens, authors say . |
| Approach: | They propose a novel lexicon-driven shallow parser that takes into account sandhi phenomenon . they propose linguistically informed Sanskrit tokenizer module that encodes character input . |
| Outcome: | The proposed system outperforms the current state-of-the-art system by 7.2 points absolute gain on the benchmark datasets. |
One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Morphologically rich languages are notoriously challenging to process for downstream NLP applications. |
| Approach: | They propose a pretrained model for NLP applications involving the morphologically rich language Sanskrit that outperforms previous models by a considerable margin. |
| Outcome: | The proposed model outperforms tokenized models on established Sanskrit word segmentation tasks and matches the current best lexicon-based model. |
State-of-the-art Chinese Word Segmentation with Bi-LSTMs (D18-1)
Copied to clipboard
| Challenge: | A wide variety of neural-network architectures have been proposed for the task of Chinese word segmentation. |
| Approach: | They propose a bidirectional LSTM model with standard deep learning techniques and best practices for the task of Chinese word segmentation. |
| Outcome: | The proposed model outperforms models based on standard deep learning techniques and best practices on Chinese word segmentation datasets. |
Free as in Free Word Order: An Energy Based Model for Word Segmentation and Morphological Tagging in Sanskrit (D18-1)
Copied to clipboard
Amrith Krishna, Bishal Santra, Sasi Prasanth Bandaru, Gaurav Sahu, Vishnu Dutt Sharma, Pavankumar Satuluri, Pawan Goyal
| Challenge: | a structured prediction framework is proposed to solve word segmentation and morphological tagging tasks in a free word order language. |
| Approach: | They propose a structured prediction framework that jointly solves word segmentation and morphological tagging tasks in Sanskrit. |
| Outcome: | The proposed model outperforms the state of the art with an F-Score of 96.92 (percentage improvement of 7.06%) while using less than one tenth of the task-specific training data. |
Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities (2020.coling-main)
Copied to clipboard
| Challenge: | Domain names such as openresearch are being added to a growing set of tokens that an NLP system may need to deal with. |
| Approach: | They propose a tagging model that uses characters as input to break domain names into component words . they propose taagging methods that use concatenated entity names in a large knowledge database . |
| Outcome: | The proposed model improves on concatenated entity names in a knowledge database by 33% . the proposed model can be used for a wide range of languages, including Chinese and Japanese . |
A Deep Neural Network based Approach for Entity Extraction in Code-Mixed Indian Social Media Text (L18-1)
Copied to clipboard
| Challenge: | a huge number of people use social media to express and exchange information in their own languages. |
| Approach: | They propose to use a code-mixed environment to extract higher level features from text . they use 'gadget' algorithm that automatically discovers higher level feature from text. |
| Outcome: | The proposed approach is generic and does not make use of handcrafted features or rules. |
SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposes (2023.acl-demo)
Copied to clipboard
| Challenge: | SanskritShala is a neural-based Sanskrit NLP toolkit that is available as a web-based application . |
| Approach: | They propose a neural Sanskrit NLP toolkit that facilitates linguistic analyses for word segmentation, morphological tagging, dependency parsing, and compound type identification. |
| Outcome: | The proposed toolkit reports state-of-the-art performance on benchmark datasets . it is built with easy-to-use interactive data annotation features . |
Automatic Speech Recognition in Sanskrit: A New Speech Corpus and Modelling Insights (2021.findings-acl)
Copied to clipboard
| Challenge: | In this paper, we propose the first large scale study of automatic speech recognition in Sanskrit . we focus on the impact of unit selection in San's ASR systems . |
| Approach: | They propose a large scale study of automatic speech recognition in Sanskrit . they propose syllable level unit selection that captures character sequences . |
| Outcome: | The proposed model captures character sequences from one vowel in the word to the next vowela. |