Papers by Mark Johnson
Open-Domain Contextual Link Prediction and its Complementarity with Entailment Graphs (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for linking knowledge graphs only use textual contexts . contextual link prediction is useful for finding context-dependent entailments . |
| Approach: | They propose a task of open-domain contextual link prediction which uses textual context and KG structure to perform link prediction. |
| Outcome: | The proposed model can ground the triples in the context of the original dataset and infer missing relations in context. |
Sources of Hallucination by Large Language Models on Inference Tasks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are claimed to be capable of Natural Language Inference (NLI) |
| Approach: | They propose to use LLMs to probe their behavior using controlled experiments. |
| Outcome: | The proposed models perform significantly worse on NLI test samples which do not conform to these biases than those which do. |
Duality of Link Prediction and Entailment Graph Induction (P19-1)
Copied to clipboard
| Challenge: | In this paper, we show that link prediction and entailment graph induction are complementary. |
| Approach: | They propose an entailment score that exploits the new facts discovered by the link prediction model and then form engorgement graphs between relations. |
| Outcome: | The proposed entailment score outperforms prior state-of-the-art results on a standard entialment dataset and the new link prediction scores show improvements over the raw link prediction score. |
Multivalent Entailment Graphs for Question Answering (2021.emnlp-main)
Copied to clipboard
Nick McKenna, Liane Guillou, Mohammad Javad Hosseini, Sander Bijl de Vroe, Mark Johnson, Mark Steedman
| Challenge: | a recent study shows that drawing inferences between open domain predicates is a necessity for true language understanding. |
| Approach: | They propose to reinterpret the Distributional Inclusion Hypothesis to model entailment between predicates of different valencies. |
| Outcome: | The proposed graphs are more useful than using the same valency evidence, the authors show . they show that drawing on evidence across valencies answers more questions than using only the same evidence. |
An adaptable task-oriented dialog system for stand-alone embedded devices (P19-3)
Copied to clipboard
Long Duong, Vu Cong Duy Hoang, Tuyen Quang Pham, Yu-Heng Hong, Vladislavs Dovgalecs, Guy Bashkansky, Jason Black, Andrew Bleeker, Serge Le Huitouze, Mark Johnson
| Challenge: | a proposed speech-based task-oriented dialogue system is built on a small embedded device . the system does not require internet connectivity because all components run locally on the device - a cost-effective solution . |
| Approach: | They propose a spoken-language end-to-end task-oriented dialogue system for small embedded devices such as home appliances. |
| Outcome: | The proposed system is based on a demo run offline on swiss raspberry pi . it eliminates privacy risks and eliminates server costs and latency . |
Neural Constituency Parsing of Speech Transcripts (N19-1)
Copied to clipboard
| Challenge: | a neural parser for transcribed speech can find EDITED disfluency nodes . this makes specialized mechanisms for parsing disfluencies unnecessary . |
| Approach: | They propose a neural self-attentive parser that finds EDITED disfluency nodes in transcribed speech. |
| Outcome: | The proposed parser finds EDITED disfluency nodes with an accuracy surpassing that of specialized systems. |
AMR dependency parsing with a typed semantic algebra (P18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representations (AMRs) are graphs which describe the predicate-argument structure of a sentence. |
| Approach: | They propose a semantic parser which parses strings into tree representations of the compositional structure of an AMR graph. |
| Outcome: | The proposed parser outperforms baselines and standard neural techniques for supertagging and dependency tree parsing. |
How to Best Use Syntax in Semantic Role Labelling (P19-1)
Copied to clipboard
| Challenge: | Existing studies on integrating external information into NLP tasks focus on word-level shallow features such as POS or chunk tags. |
| Approach: | They propose to integrate syntactic information into a neural ELMo-based SRL sequence labelling model by using a constituency representation as input features. |
| Outcome: | The proposed approach improves performance on the in-domain CoNLL’05 and CoNll’12 benchmarks. |
Improving Disfluency Detection by Self-Training a Self-Attentive Model (2020.acl-main)
Copied to clipboard
| Challenge: | Existing self-attentive parsers using contextualized word embeddings produce state-of-the-art results in joint parsing and disfluency detection. |
| Approach: | They propose to use contextualized word embeddings to train a neural model using unlabeled data to train parsers. |
| Outcome: | The proposed method produces state-of-the-art results in parsing and disfluency detection in speech transcripts. |
Disfluency Detection using Auto-Correlational Neural Networks (D18-1)
Copied to clipboard
| Challenge: | a recent study proposes an auto-correlational neural network (ACNN) that can detect disfluency in speech . the model uses a convolutional neural system and augments it with a new auto-corrector . |
| Approach: | They propose a convolutional neural network model that captures "rough copy" dependencies . the model is based on a new auto-correlation operator that capture the kinds of "rough copies" dependency . |
| Outcome: | The proposed model outperforms the baseline CNN on a disfluency detection task with a 5% increase in f-score. |
Integrating Lexical Information into Entity Neighbourhood Representations for Relation Prediction (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to predict knowledge base relations are limited by maintenance costs and text-based formats. |
| Approach: | They propose a system that can extend relational database tables with information extracted from a document corpus. |
| Outcome: | The proposed system outperforms existing methods by incorporating embeddings of text-based representations of the entities and relations. |
VnCoreNLP: A Vietnamese Natural Language Processing Toolkit (N18-5)
Copied to clipboard
| Challenge: | Using word segmenters and POS taggers, Vietnamese NLP pipelines are no longer considered SOTA models for Vietnamese. |
| Approach: | They propose a Java NLP annotation pipeline for Vietnamese that provides rich linguistic annotations. |
| Outcome: | The proposed toolkit provides rich linguistic annotations to facilitate research work on Vietnamese NLP. |
Mention Flags (MF): Constraining Transformer-based Text Generators (2021.acl-long)
Copied to clipboard
| Challenge: | Constrained decoding algorithms produce hypotheses satisfying all constraints, but they are computationally expensive and can lower the generated text quality. |
| Approach: | They propose a Mention Flag mechanism which traces whether lexical constraints are satisfied in outputs of an S2S decoder. |
| Outcome: | The proposed models maintain higher constraint satisfaction and text quality than baseline models and other constrained decoding algorithms. |
A Fast and Accurate Vietnamese Word Segmenter (L18-1)
Copied to clipboard
| Challenge: | Experimental results show that our approach outperforms previous state-of-the-art approaches in terms of accuracy and performance speed. |
| Approach: | They propose a method where rules are stored in an exception structure and new rules are only added to correct segmentation errors. |
| Outcome: | The proposed approach outperforms existing methods on Vietnamese treebank benchmarks. |
Active learning for deep semantic parsing (P18-2)
Copied to clipboard
| Challenge: | Existing methods for generating training data for semantic parsing are slow and expensive. |
| Approach: | They propose active learning for "overnight" and "natural language" parsing with a logical form . they propose several active learning strategies for overnight data collection . |
| Outcome: | The proposed approach reduces the cost of training data for deep parsing tasks by reducing the number of crowd workers required. |
End-to-End Speech Recognition and Disfluency Removal (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Disfluency detection is usually an intermediate step between an automatic speech recognition system and a downstream task. |
| Approach: | They propose to train models to directly map disfluent speech into fluent transcripts without relying on a separate disfluency detection model. |
| Outcome: | The proposed models learn to generate fluent transcripts, but their performance is slightly worse than a baseline pipeline approach consisting of an ASR system and a specialized disfluency detection model. |
Predicting accuracy on large datasets from smaller pilot data (P18-2)
Copied to clipboard
| Challenge: | obtaining training data is often the most difficult part of an NLP or ML project . obtaining data is important to estimate how much training data a system will require to achieve a target accuracy. |
| Approach: | They propose a performance extrapolation task to evaluate extrapolations on larger training sets. |
| Outcome: | The proposed method can predict accuracy on larger training datasets. |
ECOL-R: Encouraging Copying in Novel Object Captioning with Reinforcement Learning (2021.eacl-main)
Copied to clipboard
| Challenge: | Novel Object Captioning is a zero-shot Image Caption task requiring describing objects not seen in the training captions, but for which information is available from external object detectors. |
| Approach: | They propose a novel captioning model that encourages copying of object labels with reinforcement learning that encourage a copy-augmented transformer model to accurately describe the object labels. |
| Outcome: | The proposed model sets new state-of-the-art on the nocaps and held-out COCO benchmarks. |
Mastering the Craft of Data Synthesis for CodeLLMs (2025.naacl-long)
Copied to clipboard
Meng Chen, Philip Arthur, Qianyu Feng, Cong Duy Vu Hoang, Yu-Heng Hong, Mahdi Kazemi Moghaddam, Omid Nezami, Duc Thien Nguyen, Gioacchino Tangari, Duy Vu, Thanh Vu, Mark Johnson, Krishnaram Kenthapadi, Don Dharmasiri, Long Duong, Yuan-Fang Li
| Challenge: | Large language models (LLMs) have shown impressive performance in code understanding and generation. |
| Approach: | They propose a systematic review of large language models and their taxonomy and propose specialized LLMs for code-related tasks. |
| Outcome: | The proposed models have shown to be highly effective in coding tasks. |