Papers by David Chiang
Convergence and Diversity in the Control Hierarchy (2023.acl-long)
Copied to clipboard
| Challenge: | Weir has defined a hierarchy of language classes whose second member (L2) is generated by tree-adjoining grammars (TAG), linear indexed grammars, combinatory categorial grammars and head grammars. |
| Approach: | They propose to extend Weir's mechanism of control to give a definition of controllable pushdown automata (PDAs) they propose to use a stricter notion of equivalence to allow for finer-grained comparisons than weak equvalence. |
| Outcome: | The proposed language classes are d-weakly equivalent to Weir's original two-level grammar, but not d strongly equivalent. |
Efficiency through Auto-Sizing: Notre Dame NLP’s Submission to the WNGT 2019 Efficiency Task (D19-56)
Copied to clipboard
| Challenge: | Notre Dame Natural Language Processing group applied auto-sizing to the Transformer network to reduce the number of parameters in the model. |
| Approach: | They investigated the impact of auto-sizing on the Transformer network by applying a method to inducing sparsity in parameters. |
| Outcome: | The proposed method eliminated more than 25% of the model’s parameters while suffering a decrease of only 1.1 BLEU. |
Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention (N18-1)
Copied to clipboard
| Challenge: | Neural machine translation models learn to map from source language sentences to target language sentences via continuous-space intermediate representations. |
| Approach: | They propose an encoder with character attention which augments the (sub)word-level representation with character-level information and a decoder with multiple attentions that enable the representations from different levels of granularity to control the translation cooperatively. |
| Outcome: | The proposed model outperforms the standard word-based model, subword-based models, and strong character-based ones on translation tasks. |
BERTwich: Extending BERT’s Capabilities to Model Dialectal and Noisy Text (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models like BERT deteriorate in the face of dialect variation or noise. |
| Approach: | They propose to sandwich BERT's encoder stack between additional encoder layers trained to perform masked language modeling on noisy text. |
| Outcome: | The proposed approach promotes zero-shot transfer to dialectal text and reduces embedding space between words and noisy counterparts. |
Part-of-Speech Tagging on an Endangered Language: a Parallel Griko-Italian Resource (C18-1)
Copied to clipboard
Antonios Anastasopoulos, Marika Lekakou, Josep Quer, Eleni Zimianiti, Justin DeBenedetto, David Chiang
| Challenge: | a recent study examines POS tagging techniques on endangered languages . most natural language processing applications have been tested on only a handful of languages - a problem that is compounded by the lack of standard orthography. |
| Approach: | They evaluate POS tagging techniques on an endangered language, Griko . they use a semi-supervised method with cross-lingual transfer to achieve better accuracy . |
| Outcome: | The proposed method achieves 72.9% accuracy on a sample of 114 narratives in a language . the proposed method improves by 21 percentage points over previous methods . |
Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models are sensitive to architecture and hyperparameter settings. |
| Approach: | They incorporate architecture search into a single training run through auto-sizing . they show that auto-size can improve BLEU scores by up to 3.9 points . |
| Outcome: | The proposed algorithm improves BLEU scores on low-resource language pairs while removing one-third of the parameters from the model. |
PILA: A Historical-Linguistic Dataset of Proto-Italic and Latin (2024.lrec-main)
Copied to clipboard
| Challenge: | Historical linguists hypothesize systems of sound change to explain the evolution of language over time, but the evidence is limited. |
| Approach: | They propose a dataset that consists of roughly 3,000 pairs of forms from Proto-Italic and Latin. |
| Outcome: | The proposed dataset enables historical linguists to enhance other datasets by enhancing them with the existing datasets. |
Neural Machine Translation of Text from Non-Native Speakers (N19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) systems are known to degrade when confronted with noisy data. |
| Approach: | They propose to augment training data with sentences containing artificially-introduced grammatical errors to make the system more robust to such errors. |
| Outcome: | The proposed approach recovers 1.0 BLEU out of 2.4 BLUE lost due to grammatical errors on a set of Spanish translations of the JFLEG grammar error correction corpus. |
Efficient Algorithms for Recognizing Weighted Tree-Adjoining Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | a class of tree-adjoining languages can be characterized by various two-level formalisms controlled by semiring-weighted CFGs and PDAs. |
| Approach: | They propose semiring-weighted versions of controllable CFGs and PDAs . they also introduce a WPDA normal form that is analogous to Chomsky's normal form for CFG . |
| Outcome: | The proposed algorithms are more time-efficient than the previous ones for LIG, PAA, and EPDA. |
Tied Multitask Learning for Neural Speech Translation (N18-1)
Copied to clipboard
| Challenge: | Recent efforts in endangered language documentation focus on collecting spoken language resources . BULB project uses mobile app to collect spoken resources accompanied by spoken translations . |
| Approach: | They propose a model where the second task decoder receives information from the first task . they apply regularization that encourages transitivity and invertibility to the model . |
| Outcome: | The proposed model improves performance on low-resource speech transcription and translation tasks. |
Introducing Rhetorical Parallelism Detection: A New Task with Datasets, Metrics, and Baselines (2023.emnlp-main)
Copied to clipboard
| Challenge: | Parallelism is a common stylistic tool in rhetorical structures, but it is rarely investigated in the field of natural language processing. |
| Approach: | They propose a task of rhetorical parallelism detection to investigate its structure and meaning . they use a Latin and adapted Chinese dataset to define parallelise and define it using a family of metrics . |
| Outcome: | The proposed method achieves F1 scores on Latin and Chinese datasets. |
Syntax-Based Attention Masking for Neural Machine Translation (2021.naacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to extend transformers to source-side trees are linearized into sequences, but they are limited by positional encodings. |
| Approach: | They propose a method for extending transformers to source-side trees by using masks based on tree positions . they define a number of masks that limit self-attention based upon relationships among tree nodes . |
| Outcome: | The proposed method improves on translations from English to germany and English to english and germany by +2.1 BLEU. |
Simulating Hard Attention Using Soft Attention (2026.tacl-1)
Copied to clipboard
| Challenge: | a central element of hard attention is attention, which computes a weighted average of values from all unmasked positions. |
| Approach: | They propose transformers that can simulate hard attention by using temperature scaling and positional embeddings. |
| Outcome: | The proposed transformers can effectively focus all attention on a subset of positions. |
Languages Still Left Behind: Toward a Better Multilingual Machine Translation Benchmark (2025.emnlp-main)
Copied to clipboard
Chihiro Taguchi, Seng Mai, Keita Kurabe, Yusuke Sakai, Georgina Agyei, Soudabeh Eslami, David Chiang
| Challenge: | Multilingual machine translation (MT) benchmarks are widely used to evaluate the capabilities of modern MT systems. |
| Approach: | They propose to use a multilingual machine translation benchmark to assess the capabilities of modern machine translation systems. |
| Outcome: | The FLORES+ benchmark claims to maintain a translation quality score of over 90% . however, the data in four languages falls short of the 90% quality standard . |
Improving Lexical Choice in Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | False positives: the output layer rewards frequent words disproportionately, we argue . Falsibles: a model that learns word representations in continuous space tends to translate rare words . |
| Approach: | They propose to fix the norms of both vectors to a constant value and integrate a lexical module which is jointly trained with the rest of the model. |
| Outcome: | The proposed approach achieves improvements of up to +4.3 BLEU surpassing phrase-based translation in nearly all settings. |
Killkan: The Automatic Speech Recognition Dataset for Kichwa with Morphosyntactic Information (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for automatic speech recognition (ASR) in the endangered Kichwa language have been limited. |
| Approach: | They present Killkan, the first dataset for automatic speech recognition (ASR) in the Kichwa language, an indigenous language of Ecuador. |
| Outcome: | The proposed dataset shows that it can be used to build an automatic speech recognition system for the endangered language with reliable quality despite its small size. |
Composing Finite State Transducers on GPUs (P18-1)
Copied to clipboard
| Challenge: | Weighted finite state transducers (FSTs) are used in language processing . a GPU implementation of the composition operation is currently under development . |
| Approach: | They propose a GPU implementation of the composition operation for weighted finite state transducers. |
| Outcome: | The proposed approach achieves speedups of up to 6 times over the serial implementation and 4.5 times over OpenFST on the GPU. |
Overcoming a Theoretical Limitation of Self-Attention (2022.acl-long)
Copied to clipboard
| Challenge: | Hahn shows that for languages where acceptance depends on a single input symbol, a transformer’s classification decisions get closer and closer to random guessing as input strings get longer and longer. |
| Approach: | They propose a transformer that recognizes PARITY with perfect accuracy and a model that uses layer normalization to bring the cross-entropy of both models arbitrarily close to zero. |
| Outcome: | The proposed model can accept and reject strings with perfect accuracy and bring cross-entropy close to zero when they need to focus on a single position. |
Transformers as Transducers (2025.tacl-1)
Copied to clipboard
| Challenge: | Using finite transducers, we find that transformers can express large classes of (total functional) transductions. |
| Approach: | They extend existing RASP programming language to sequence-to-sequence transductions and introduce two new extensions. |
| Outcome: | The proposed model can express surprisingly large classes of (total functional) transductions. |
Algorithms for Weighted Pushdown Automata (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing dynamic programming algorithms for PDAs often resort to a PDA-to-CFG conversion. |
| Approach: | They propose to use a pushdown automaton to reduce the space requirements by a factor of |Gamma| or the runtime by reducing the number of states. |
| Outcome: | The proposed algorithms reduce the space requirements by a factor of |Gamma| or reduce the runtime by fewer states. |
Accelerating Sparse Matrix Operations in Neural Networks on Graphics Processing Units (P19-1)
Copied to clipboard
| Challenge: | Graphics Processing Units (GPUs) are used to train and evaluate neural networks efficiently. |
| Approach: | They propose two new GPU algorithms for multiplying a matrix by a few-hot vector and fused softmax and top-N selection. |
| Outcome: | The proposed algorithms achieve speedups over state-of-the-art parallel GPU baselines of up to 7x and 50x, respectively. |