Papers by Jaime Carbonell
Learning Rhyming Constraints using Structured Adversaries (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to text generation fail to capture higher-level structure in text, for example, rhyming patterns. |
| Approach: | They propose a method that uses a structured discriminator to learn rhyming constraints from poetry . the discriminator compares two English poetry datasets based on a learned similarity matrix . |
| Outcome: | The proposed method can learn rhyming patterns in English poetry without explicit phonetic information. |
Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to generalization to resource-rich languages are difficult . a recent study shows that word representations can be useful in low resource languages . |
| Approach: | They propose two approaches for improving generalization to low-resource languages by adapting continuous word representations using linguistically motivated subword units. |
| Outcome: | The proposed method improves generalization to low resource languages . it requires neither parallel corpora nor bilingual dictionaries and requires no parallel training . |
DeepCx: A transition-based approach for shallow semantic parsing with complex constructional triggers (D18-1)
Copied to clipboard
| Challenge: | Using DeepCx, we extend Shallow semantic parsing to include complex constructions . multi-word expressions and complex constructional arguments can express relational meanings - but little work has addressed tagging of such constructional triggers. |
| Approach: | They propose a neural-based surface construction labeling task that extends Shallow Semantic Parsing to include frames triggered by complex constructions. |
| Outcome: | The proposed system improves on the task of tagging causal language in English. |
Soft Gazetteers for Low-Resource Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Existing named entity recognition models use gazetteers to improve performance, but they are limited in coverage and do not exist in low-resource languages. |
| Approach: | They propose a method that integrates Wikipedia information into named entity models by cross-lingual entity linking. |
| Outcome: | The proposed method improves on four low-resource languages with Wikipedia . it incorporates available information from english knowledge bases into neural models . |
Towards Semi-Supervised Learning for Deep Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing methods for semantic role labeling require an immense amount of semantic-role corpora and are therefore not suitable for low-resource languages or domains. |
| Approach: | They propose a semi-supervised method that outperforms the state-of-the-art on SRL . method explicitly enforcs syntactic constraints by augmenting the training objective with a syntastic-inconsistency loss component. |
| Outcome: | The proposed method outperforms the state-of-the-art on limited SRL training corpora on CoNLL-2012 English section. |
Efficient Meta Lifelong-Learning with Limited Memory (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing natural language learning models fail to continuously learn new tasks as they are re-trained throughout their lifetime. |
| Approach: | They propose a meta-lifelong framework that combines three common lifelong learning principles . they propose to store past examples in episodic memory and replay them at training and inference time . |
| Outcome: | The proposed framework achieves state-of-the-art performance using 1% memory size and narrows the gap with multi-task learning. |
StructSum: Summarization via Structured Representations (2021.eacl-main)
Copied to clipboard
Vidhisha Balachandran, Artidoro Pagnoni, Jay Yoon Lee, Dheeraj Rajagopal, Jaime Carbonell, Yulia Tsvetkov
| Challenge: | Abstractive summarization models overfit to training corpora, lack of transparency and layout bias . authors propose incorporating latent and explicit dependencies across sentences in source document . |
| Approach: | They propose a framework based on document-level structure induction to address layout bias and lack of transparency in abstractive summarization models. |
| Outcome: | The proposed framework improves coverage of content in the source documents and generates more abstractive summaries by generating more novel n-grams. |
Domain Adaptation of Neural Machine Translation by Lexicon Induction (P19-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) is sensitive to domain shift, resulting in failure for sentences with large numbers of unknown words and lack of supervision for domain-specific words. |
| Approach: | They propose an unsupervised method which fine-tunes a pre-trained out-of-domain NMT model using a pseudo-in-domain corpus. |
| Outcome: | The proposed method improves in five domains without using in-domain parallel sentences and up to 2 BLEU over strong back-translation baselines. |
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context (P19-1)
Copied to clipboard
| Challenge: | Term memory networks (RNNs) are difficult to optimize due to gradient vanishing and explosion. |
| Approach: | They propose a neural architecture Transformer-XL that enables learning dependency beyond a fixed length without disrupting temporal coherence. |
| Outcome: | The proposed method improves state-of-the-art performance on short and long sequences and generates coherent, novel text articles with thousands of tokens. |
Improving Candidate Generation for Low-resource Cross-lingual Entity Linking (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual entity linking (XEL) do not extend well to low-resource languages with few Wikipedia pages. |
| Approach: | They propose to improve the model by combining Wikipedia references with a list of plausible candidate entities. |
| Outcome: | The proposed method yields 16.9% in Top-30 gold candidate recall compared with state-of-the-art models. |
Neural Cross-Lingual Named Entity Recognition with Minimal Resources (D18-1)
Copied to clipboard
| Challenge: | Named-entity recognition (NER) models are highly dependent on large amounts of labeled data. |
| Approach: | They propose a method that finds translations based on bilingual word embeddings . they also propose 'self-attention' which allows for a degree of flexibility with respect to word order . |
| Outcome: | The proposed method achieves state-of-the-art or competitive performance on common languages with lower resource requirements than previous approaches. |
A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models rely on large amounts of labeled data, making them challenging to extend to new, lower-resource languages. |
| Approach: | They propose a method for bootstrapping named entity recognition models in under-resourced languages . they use cross-lingual transfer learning and targeted annotation of only uncertain entities . |
| Outcome: | The proposed method achieves competitive accuracy with just one-tenth of training data. |