Papers by Josiah Poon
CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection (2021.findings-acl)
Copied to clipboard
Henry Weld, Guanghao Huang, Jean Lee, Tongshu Zhang, Kunze Wang, Xinghong Guo, Siqu Long, Josiah Poon, Caren Han
| Challenge: | Existing toxic language detection models focus on the single utterance level without deeper understanding of context. |
| Approach: | They propose a dataset for in-game toxic language detection enabling joint intent classification and slot filling analysis, which is the core task of Natural Language Understanding (NLU). |
| Outcome: | The proposed framework handles utterance and token-level patterns, and rich contextual chatting history. |
SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling (2024.eacl-long)
Copied to clipboard
| Challenge: | Visual storytelling aims to automatically generate a coherent story based on a given image sequence. |
| Approach: | They propose a framework that represents the image sequence as a graph with objects and relations that includes human action motivation and its social interaction commonsense knowledge. |
| Outcome: | The proposed framework produces stories superior across multiple metrics in terms of visual grounding, coherence, diversity, and humanness, per both automatic and human evaluations. |
Understanding Attention for Vision-and-Language Tasks (2022.coling-1)
Copied to clipboard
| Challenge: | Attention mechanism has been used in Vision-and-Language (VL) tasks to bridge the semantic gap between visual and textual clues. |
| Approach: | They conduct a comprehensive analysis on understanding the role of attention alignment by looking into attention score calculation methods and checking how it represents the visual region’s and textual token’s significance for the global assessment. |
| Outcome: | The attention score calculation methods represent visual region’s and textual token’s significance for the global assessment. |
Re-Temp: Relation-Aware Temporal Representation Learning for Temporal Knowledge Graph Completion (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models ignore ability to skip irrelevant snapshots according to entity-related relations in query . TKGC is difficult and even large-scale pre-trained language models such as gist ignore explicit temporal information. |
| Approach: | They propose a model that leverages explicit temporal embedding as input to skip unnecessary information for prediction. |
| Outcome: | The proposed model outperforms all state-of-the-art models on six datasets . it incorporates skip information flow after each timestamp to skip unnecessary information . |
Detect All Abuse! Toward Universal Abusive Language Detection Models (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on online abusive language detection focused on detecting a single abusive language problem in a domain, like Twitter, but none of them was successfully transferable to general ALD in different online communities. |
| Approach: | They propose a generic ALD framework that can address multiple types of ALD tasks across different domains and use a textual graph embedding to analyse the user’s linguistic behaviour. |
| Outcome: | The proposed framework surpasses the current state-of-the-art ALD algorithms across seven datasets covering multiple aspects of abusive language and different online community domains. |
Do Text-to-Vis Benchmarks Test Real Use of Visualisations? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks for visualisations are limited and do not reflect real-world use. |
| Approach: | They analysed visualisation code from Python, R, Javascript, and Vega to find similarities and differences between real-world and benchmark datasets. |
| Outcome: | The results show that benchmark datasets do not test the same distribution of chart types, attributes, and actions as real-world examples. |
3MVRD: Multimodal Multi-task Multi-teacher Visually-Rich Form Document Understanding (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models for visually rich document understanding do not account for the diverse carriers of document versions and their associated noises. |
| Approach: | They propose a multimodal, multi-task, multiteacher joint-grained knowledge distillation model for visually-rich form document understanding. |
| Outcome: | The proposed model outperforms baselines on a comprehensive evaluation of public datasets showing it can handle complex structures and content of visually-rich forms. |
RoViST: Learning Robust Metrics for Visual Storytelling (2022.findings-naacl)
Copied to clipboard
| Challenge: | Visual storytelling is the task of generating a story paragraph that describes a given image sequence. |
| Approach: | They propose 3 evaluation metrics sets that analyze which aspects we would look for in a good story . they compare their correlation with human judgement scores on a sample of machine stories . |
| Outcome: | The proposed evaluation metrics outperform other metrics on human correlation on a sample of machine stories from state-of-the-art models. |
Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Document Layout Analysis tasks rely on visual cues to understand documents . traditional deep learning-based methods fail to recognize the layout and components of unstructured documents based on the document structure and the boundaries of each layout region. |
| Approach: | They propose a way to harmonize and integrate heterogeneous aspects for Document Layout Analysis by using graph convolutional networks to enhance each aspect of features. |
| Outcome: | The proposed task is based on three widely used datasets: PubLayNet, FUNSD, and DocBank. |
VICTR: Visual Information Captured Text Representation for Text-to-Vision Multimodal Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Existing text-to-image generation models focus on generating high resolution images and neglect understanding text descriptions. |
| Approach: | They propose a visual contextual text representation which captures rich visual semantic information of objects from text input. |
| Outcome: | The proposed visual contextual text representation improves on the state-of-the-art models. |