Papers by Josiah Wang
CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection (2021.findings-acl)
Copied to clipboard
Henry Weld, Guanghao Huang, Jean Lee, Tongshu Zhang, Kunze Wang, Xinghong Guo, Siqu Long, Josiah Poon, Caren Han
| Challenge: | Existing toxic language detection models focus on the single utterance level without deeper understanding of context. |
| Approach: | They propose a dataset for in-game toxic language detection enabling joint intent classification and slot filling analysis, which is the core task of Natural Language Understanding (NLU). |
| Outcome: | The proposed framework handles utterance and token-level patterns, and rich contextual chatting history. |
VIFIDEL: Evaluating the Visual Fidelity of Image Descriptions (P19-1)
Copied to clipboard
| Challenge: | Existing methods for evaluating image description generation systems are subjective and expensive to scale. |
| Approach: | They propose a new image-aware metric for evaluating image description generation systems . it estimates the faithfulness of a generated caption with respect to the content of the actual image . |
| Outcome: | The proposed metric achieves high correlation with human judgments on two well-known datasets and is competitive with metrics that depend on and rely exclusively on human references. |
MultiSubs: A Large-scale Multimodal and Multilingual Dataset (2022.lrec-1)
Copied to clipboard
| Challenge: | a large-scale multimodal and multilingual dataset is used to facilitate research on visual grounding of words to images in their contextual usage in language. |
| Approach: | They propose a large-scale multimodal and multilingual dataset that aims to facilitate research on grounding words to images in their contextual usage in language. |
| Outcome: | The proposed dataset will facilitate research on visual grounding of words in their contextual usage in language. |
SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling (2024.eacl-long)
Copied to clipboard
| Challenge: | Visual storytelling aims to automatically generate a coherent story based on a given image sequence. |
| Approach: | They propose a framework that represents the image sequence as a graph with objects and relations that includes human action motivation and its social interaction commonsense knowledge. |
| Outcome: | The proposed framework produces stories superior across multiple metrics in terms of visual grounding, coherence, diversity, and humanness, per both automatic and human evaluations. |
Re-Temp: Relation-Aware Temporal Representation Learning for Temporal Knowledge Graph Completion (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models ignore ability to skip irrelevant snapshots according to entity-related relations in query . TKGC is difficult and even large-scale pre-trained language models such as gist ignore explicit temporal information. |
| Approach: | They propose a model that leverages explicit temporal embedding as input to skip unnecessary information for prediction. |
| Outcome: | The proposed model outperforms all state-of-the-art models on six datasets . it incorporates skip information flow after each timestamp to skip unnecessary information . |
Detect All Abuse! Toward Universal Abusive Language Detection Models (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on online abusive language detection focused on detecting a single abusive language problem in a domain, like Twitter, but none of them was successfully transferable to general ALD in different online communities. |
| Approach: | They propose a generic ALD framework that can address multiple types of ALD tasks across different domains and use a textual graph embedding to analyse the user’s linguistic behaviour. |
| Outcome: | The proposed framework surpasses the current state-of-the-art ALD algorithms across seven datasets covering multiple aspects of abusive language and different online community domains. |
RoViST: Learning Robust Metrics for Visual Storytelling (2022.findings-naacl)
Copied to clipboard
| Challenge: | Visual storytelling is the task of generating a story paragraph that describes a given image sequence. |
| Approach: | They propose 3 evaluation metrics sets that analyze which aspects we would look for in a good story . they compare their correlation with human judgement scores on a sample of machine stories . |
| Outcome: | The proposed evaluation metrics outperform other metrics on human correlation on a sample of machine stories from state-of-the-art models. |
Defoiling Foiled Image Captions (N18-2)
Copied to clipboard
| Challenge: | Existing models for vision-to-language tasks do not understand images sufficiently, despite their impressive performance. |
| Approach: | They propose to use explicit object information to solve foiled caption detection problem . they propose to replace a word in caption with a semantically similar word . |
| Outcome: | The proposed model achieves state-of-the-art on a recently published dataset with scores exceeding those achieved by humans on the task. |
Object Counts! Bringing Explicit Detections Back into Image Captioning (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to image captioning use explicit object detectors as an intermediate step, but they bypass the explicit detection phase and instead generate captions directly from image embeddings. |
| Approach: | They argue that explicit detections provide rich semantic information and can thus be used as an interpretable representation to better understand why end-to-end image captioning systems work well. |
| Outcome: | The proposed methods can be used to understand why end-to-end captioning systems work well. |
VICTR: Visual Information Captured Text Representation for Text-to-Vision Multimodal Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Existing text-to-image generation models focus on generating high resolution images and neglect understanding text descriptions. |
| Approach: | They propose a visual contextual text representation which captures rich visual semantic information of objects from text input. |
| Outcome: | The proposed visual contextual text representation improves on the state-of-the-art models. |