| Challenge: | Existing methods for word splitting in Sanskrit have low accuracy as the same compound word might be broken down in multiple ways to provide syntactically correct splits. |
| Approach: | They propose a deep learning architecture called Double Decoder RNN which predicts the location of the splits with 95% accuracy and 79.5% accuracy. |
| Outcome: | The proposed model outperforms the state-of-the-art in the problem of Chinese word segmentation with 79.5% accuracy and the existing model's generalization capability. |
Similar Papers
Building a Word Segmenter for Sanskrit Overnight (L18-1)
Copied to clipboard
| Challenge: | Sanskrit word segmentation is challenging due to the issue of Sandhi . digitisation efforts have made the manuscripts available in the public domain . |
| Approach: | They propose a deep sequence to sequence model that takes only the sandhied string as input and predicts the unsandhized string. |
| Outcome: | The proposed model improves on the current state of the art by 16.79% . the system can be trained "overnight" and be used for production . |
Sanskrit Word Segmentation Using Character-level Recurrent and Convolutional Neural Networks (D18-1)
Copied to clipboard
| Challenge: | Using end-to-end neural network models, Sanskrit is tokenized by splitting compounds and resolving phonetic merges. |
| Approach: | They propose end-to-end neural network models that tokenize Sanskrit by jointly splitting compounds and resolving phonetic merges. |
| Outcome: | The proposed models outperform the state-of-the-art for the task of splitting compounds and resolving phonetic merges. |
A Joint Approach to Compound Splitting and Idiomatic Compound Detection (2020.lrec-1)
Copied to clipboard
| Challenge: | Compounding is a common word-formation process in Germanic languages . high productivity and low corpus frequency of compounds increase vocabulary size . |
| Approach: | They develop a deep learning-based approach to noun compound splitting and idiomatic compound detection for the German language. |
| Outcome: | The proposed approach outperforms the current state of the art in noun compound splitting and idiomatic compound detection for the German language. |
Split and Rephrase: Better Evaluation and Stronger Baselines (P18-2)
Copied to clipboard
| Challenge: | a dataset mapping a complex sentence to a sequence of sentences conveying the same meaning is challenging in NLP. |
| Approach: | They propose a neural split and a copy-mechanism to break a complex sentence into several shorter sentences that convey the same meaning. |
| Outcome: | The proposed model outperforms the baseline model by 8.68 BLEU and further improves on the task. |
SandhiKosh: A Benchmark Corpus for Evaluating Sanskrit Sandhi Tools (L18-1)
Copied to clipboard
| Challenge: | Several important texts which are of interest to people all over the world were written in Sanskrit. |
| Approach: | They develop a Sanskrit benchmark to evaluate the completeness and accuracy of tools . they use three most prominent tools to evaluate their completeness . |
| Outcome: | The proposed tools have substantial scope for improvement and are available to researchers worldwide. |
TransLIST: A Transformer-Based Linguistically Informed Sanskrit Tokenizer (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to SWS fail when encountering out-of-vocabulary tokens . lexicon driven approaches fail when dealing with out- of-vocal tokens, authors say . |
| Approach: | They propose a novel lexicon-driven shallow parser that takes into account sandhi phenomenon . they propose linguistically informed Sanskrit tokenizer module that encodes character input . |
| Outcome: | The proposed system outperforms the current state-of-the-art system by 7.2 points absolute gain on the benchmark datasets. |
A Novel Multi-Task Learning Approach for Context-Sensitive Compound Type Identification in Sanskrit (2022.coling-1)
Copied to clipboard
Jivnesh Sandhan, Ashish Gupta, Hrishikesh Terdalkar, Tushar Sandhan, Suvendu Samanta, Laxmidhar Behera, Pawan Goyal
| Challenge: | Previously, compounding is a problem of identifying semantic relations between components of a word. |
| Approach: | They propose a multi-task learning architecture which incorporates contextual information and enriches syntactic information using morphological tagging and dependency parsing as auxiliary tasks. |
| Outcome: | The proposed architecture shows 6.1 points accuracy and 7.7 points (F1-score) absolute gain in English and Marathi languages. |
Kvistur 2.0: a BiLSTM Compound Splitter for Icelandic (2020.lrec-1)
Copied to clipboard
| Challenge: | Compounding is highly productive in Icelandic, and new compounds are constantly being created. |
| Approach: | They propose a character-based biLSTM model for splitting Icelandic compound words . the model learns how to split compound words into two parts . |
| Outcome: | The proposed model outperforms other methods on a corpus of manually split word forms. |
Grammar-based Decoding for Improved Compositional Generalization in Semantic Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have been successful in semantic parsing tasks but struggle on out-of-distribution data. |
| Approach: | They propose to use a large-scale dialogue dataset to evaluate compositional generalization of semantic parsing. |
| Outcome: | The proposed model outperforms BART- and T5-based models on the SMCalflow-CS dataset on the zero-shot learning task. |
State-of-the-art Chinese Word Segmentation with Bi-LSTMs (D18-1)
Copied to clipboard
| Challenge: | A wide variety of neural-network architectures have been proposed for the task of Chinese word segmentation. |
| Approach: | They propose a bidirectional LSTM model with standard deep learning techniques and best practices for the task of Chinese word segmentation. |
| Outcome: | The proposed model outperforms models based on standard deep learning techniques and best practices on Chinese word segmentation datasets. |