| Challenge: | a finite state transducer defines joint and conditional probability distributions over strings . a weighted finite-state transducers can only model certain functions, known as the rational relations . |
| Approach: | They propose a family of string transduction models defining joint and conditional probability distributions over pairs of strings. |
| Outcome: | The proposed models are more powerful than previous finite-state models with neural features. |
Similar Papers
Neural Induction of Finite-State Transducers (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to construct finite-state transducers by hand are difficult and require domain knowledge and significant human effort. |
| Approach: | They propose a method for automatically constructing unweighted FSTs following the hidden state geometry learned by a recurrent neural network. |
| Outcome: | The proposed method outperforms classical transducer learning algorithms by up to 87% accuracy on held-out test sets. |
Exact Hard Monotonic Attention for Character-Level Transduction (P19-1)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models with soft attention outperform monotonic models . current dominant method is the neural sequenceto-Sequency model with soft focus . |
| Approach: | They develop a hard attention sequence-to-sequence model that enforces strict monotonicity and learns alignment jointly. |
| Outcome: | The proposed model achieves state-of-the-art on grapheme-to-phoneme conversion and morphological inflection generation. |
Finite-state script normalization and processing utilities: The Nisaba Brahmic library (2021.eacl-demos)
Copied to clipboard
| Challenge: | a library for low-level processing of brahmic scripts is available for free. |
| Approach: | They propose an open-source library for efficient low-level processing of ten major South Asian Brahmic scripts. |
| Outcome: | The proposed library supports low-level processing of ten major south Asian Brahmic scripts. |
Rational Recurrences (D18-1)
Copied to clipboard
| Challenge: | Recent studies show that neural models lack strong intuitions . recent studies show connections between convolutional neural networks and weighted finite state automata (WFSAs) |
| Approach: | They show that some recurrent neural networks share a connection to weighted finite state automata (WFSAs) they define rational recurrences as recursive hidden state update functions . they propose to use these functions to write forward calculations of a finite set of WFSA's . |
| Outcome: | The proposed model outperforms two baselines on language modeling and text classification. |
Recurrent Neural Language Models as Probabilistic Finite-state Automata (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on the expressive power of recurrent neural network LMs to recognize unweighted formal languages. |
| Approach: | They propose to model a strict subset of probabilistic finite-state automata with RNNs . they show that an RNN requires left(N ||right) neurons to represent an LM . |
| Outcome: | The proposed language models can represent a strict subset of probabilistic distributions expressed by finite-state models. |
Higher-order Derivatives of Weighted Finite-state Machines (2021.acl-short)
Copied to clipboard
| Challenge: | Weighted finite-state machines (WFSMs) have a storied role in NLP . e.g., conditional random fields for part-of-speech tagging are considered special cases of WFSM. |
| Approach: | They propose an algorithm for evaluating derivatives of all orders that is faster than prior algorithms . they also find that it is faster for computing second-order expectations . |
| Outcome: | The proposed algorithm is faster than prior algorithms and can compute second-order expectations. |
Hard Non-Monotonic Attention for Character-Level Transduction (D18-1)
Copied to clipboard
| Challenge: | Character-level string-to-string transductions are an important component of NLP tasks . hard non-monotonic attention models have been used for sequence modeling tasks involving characters . |
| Approach: | They propose an exact algorithm for marginalizing over the exponential number of non-monotonic alignments between two strings. |
| Outcome: | The proposed algorithm outperforms soft attention and improves performance over stochastic approximation. |
Burmese Speech Corpus, Finite-State Text Normalization and Pronunciation Grammars with an Application to Text-to-Speech (2020.lrec-1)
Copied to clipboard
Yin May Oo, Theeraphol Wattanavekin, Chenfang Li, Pasindu De Silva, Supheakmungkol Sarin, Knot Pipatsrisawat, Martin Jansche, Oddur Kjartansson, Alexander Gutkin
| Challenge: | Using crowd-sourced speech corpus and finite-state transducer grammars, we build a text-to-speech system for Burmese, a tonal Southeast Asian language from the Sino-Tibetan family. |
| Approach: | They propose an open-source crowd-sourced multi-speaker speech corpus and finite-state grammars for performing grapheme-to-phoneme conversion for Burmese. |
| Outcome: | The proposed system performs well for Burmese in a low-resource setting. |
Benchmarking Compositionality with Formal Languages (2022.coling-1)
Copied to clipboard
| Challenge: | Compositionality is a hallmark of human language, but it is not yet fully understood . recombining known primitive concepts into larger novel combinations is elusive . |
| Approach: | They use finite-state transducers to make a dataset with controllable compositionality . they find that the models either learn the relations completely or not at all . |
| Outcome: | The proposed model learns the relation completely or not at all on large datasets. |
Neural Transition-based String Transduction for Limited-Resource Setting in Morphology (C18-1)
Copied to clipboard
| Challenge: | Morphological string transduction involves mapping one word form into another, possibly given a feature specification for the mapping. |
| Approach: | They propose a neural transition-based model that uses a simple set of edit actions for morphological transduction tasks such as reinflection and reinflation. |
| Outcome: | The proposed model outperforms state-of-the-art systems on low and medium training-set sizes and is competitive in the high-resource setting. |