Papers by Trung Bui
Copied to clipboard
| Challenge: | Recent work shows that attention mechanisms provide arguably explainable attention distributions that can help to interpret predictions. |
| Approach: | They propose a new self-attention layer where attention heads represent labels. |
| Outcome: | The proposed model obtains state-of-the-art results on the Penn Treebank and Chinese Treebank. |
Copied to clipboard
| Challenge: | Existing models for instructional video understanding struggle to understand abstract intents . identifying procedural intent within instructional videos is a challenging task . |
| Approach: | They propose to extract instructional intent from software instructional livestreams by using a multimodal cascaded cross-attention model that integrates weaker and noisier video signals with more discriminative text signals. |
| Outcome: | The proposed model improves on baseline models and compares it to existing models. |
Copied to clipboard
| Challenge: | Contextual sequence mapping is one of the fundamental problems in Natural Language Processing (NLP). |
| Approach: | They propose a new family of Recurrent Neural Networks that address contextual sequence mapping . they propose to use contextual signals to control the flow of information . |
| Outcome: | The proposed architecture outperforms existing methods on dialog problem and language model . the proposed architectures are based on a novel family of recurrent neural networks . |
Copied to clipboard
| Challenge: | Existing question-answering models do not require reasoning across sentences in the given context (passage). |
| Approach: | They propose a graph neural network that propagates information over sentences to understand information that cannot be inferred when considering sentences in isolation. |
| Outcome: | The proposed approach obtains the best performance compared to the widely used answer-selection models that do not consider the intersentential relationship. |
Copied to clipboard
| Challenge: | Input Marginalization (IM) is a method that takes the prediction difference between before-and-after an input feature (here, a token) is removed as its attribution. |
| Approach: | They propose to use a BERT-based method to replace a token with a feature to give more plausible counterfactuals. |
| Outcome: | The proposed method is effective, but the Deletion-BERT metric is biased towards IM, and the results are not convincing. |
Copied to clipboard
| Challenge: | Meeting transcripts are a promising domain for natural language tasks . lack of annotated data impedes research on other important tasks in this domain . |
| Approach: | They propose an extractive QA dataset comprising questions asked by meeting participants and corresponding responses. |
| Outcome: | The proposed dataset extracts questions asked by meeting participants and corresponding responses from transcripts. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have enabled high-fidelity generation of synthetic user conversation. |
| Approach: | They propose a taxonomy covering user granularity and simulation objectives . they analyze core techniques and evaluation methodologies to help them understand the latest developments . |
| Outcome: | The proposed model enables high-fidelity generation of synthetic user conversation. |
Copied to clipboard
| Challenge: | Current medical question answering systems have difficulty processing long, detailed and informally worded questions . a growing number of approaches attempt to enhance the processing of consumer health questions - or medical question understanding . |
| Approach: | They propose a medical question understanding and answering system with knowledge grounding and semantic self-supervision that matches a user question with a trusted medical knowledge base and retrieves a fixed number of relevant sentences from the corresponding answer document. |
| Outcome: | The proposed system retrieves more relevant answers while achieving 20 times faster. |
Copied to clipboard
| Challenge: | BERTScore and other text generation metrics do not use reference captions to evaluate image captions. |
| Approach: | They propose a new metric which does not require reference captions to evaluate image captions . they train UMIC to discriminate negative captions via contrastive learning . |
| Outcome: | The proposed metric has higher correlation than previous metrics that require multiple references. |
Copied to clipboard
| Challenge: | Existing deep learning methods for answer selection are not feature engineering or expensive external resources. |
| Approach: | They propose to use deep learning methods to analyze and predict answer quality . they use a set of candidate answers to identify which of the candidates answers the question correctly. |
| Outcome: | The proposed methods produce impressive performance without feature engineering or expensive external resources. |
Copied to clipboard
| Challenge: | Existing image captioning metrics are vulnerable to lexical perturbations, but they are not robust to such perturbations. |
| Approach: | They propose a perturbation-robust multilingual CLIPScore which is a reference-free image captioning metric for multiple languages. |
| Outcome: | The proposed metric outperforms baseline metrics in capturing lexical noise of all various perturbation types in all five languages while maintaining a strong correlation with human judgments. |
Copied to clipboard
| Challenge: | Domain-specific documents cover terminologies and specialized knowledge. |
| Approach: | They propose a domain-specific document retrieval method that embeds a document into a graph of entities and their relations into . they compare the unsupervised method with previous approaches and use it to compute relevance between queries and documents. |
| Outcome: | The proposed method outperforms baselines and fully-supervised bi-encoders in a zero-shot setting and outperformed bi-supervised approaches. |
Copied to clipboard
| Challenge: | Existing benchmarks for phrase-similarity compare phrases alone (without context) and phrases with context (with or without context). |
| Approach: | They propose to use a dataset of 28K noun phrases accompanied by their contextual Wikipedia pages to train machine phrase embeddings. |
| Outcome: | The proposed dataset improves ranking-models’ accuracy and pushes span selection models near human accuracy, which is 95% Exact Match (EM) on semantic search given a query phrase and a passage. |
Copied to clipboard
| Challenge: | Existing n-gram similarity metrics fail to discriminate the incorrect answers due to the free-form of the answer. |
| Approach: | They propose a new metric that assigns different weights to each token via keyphrase prediction to judge the correctness of GenQA. |
| Outcome: | The proposed metric has a significantly higher correlation with human judgments than existing metrics in various datasets. |
Copied to clipboard
| Challenge: | Existing models and agentic memory systems fail to adapt robustly to OAKS, demonstrating delays in state-tracking and susceptibility to distraction within streaming environments. |
| Approach: | They propose a benchmark to evaluate models' ability to adapt to changing knowledge over streaming . they use two datasets to analyze how facts evolve over time . |
| Outcome: | The proposed benchmark evaluates models in an online adaptation setting over streaming, continually updating knowledge. |
Copied to clipboard
| Challenge: | Existing studies on keyphrase prediction have focused on formal texts and informal-text domains. |
| Approach: | They propose to annotate large-scale video transcripts with keyphrases from live-stream video . they propose to feed models with paragraph-level keyphrase extraction to foster future research . |
| Outcome: | The proposed model improves keyphrase prediction in live-stream video transcripts by feeding models with paragraph-level keyphrases. |
Copied to clipboard
| Challenge: | Open-domain question answering aims at locating answers to user-generated questions in massive collections of documents. |
| Approach: | They propose an algorithm with a novel reader-retriever design that differs from both families of algorithms. |
| Outcome: | The proposed algorithm outperforms retrieval-based methods with two large-scale datasets and is state-of-the-art. |
Copied to clipboard
| Challenge: | Existing linearization frameworks that rely on softmax attention with quadratic time and memory complexity pose significant computational and memory bottlenecks for long-context applications. |
| Approach: | They propose a linearization framework that transforms pretrained Transformer-based Large Language Models into subquadratic architectures that closely approximate softmax attention while preserving model quality. |
| Outcome: | Experiments show that the proposed framework outperforms existing methods by 9.4 - 24.5 points on the 5-shot MMLU benchmark and demonstrating superior associative recall. |
Copied to clipboard
| Challenge: | Existing methods for extractive summarization lack coherence, despite improvements . a human-annotated dataset is used to improve coherency of extractive summary . |
| Approach: | They propose to use human-annotated datasets to create coherent extractive summaries . they use supervised fine-tuning and natural language user feedback to enhance coherence . |
| Outcome: | The proposed dataset shows that LLMs can produce coherent summaries with human feedback. |
Copied to clipboard
| Challenge: | Existing abstractive summarization models focus on summarizing sentences and short documents. |
| Approach: | They propose a hierarchical encoder that models the discourse structure of a document, and an attentive discourse-aware decoder to generate the summary. |
| Outcome: | The proposed model significantly outperforms state-of-the-art models on two large-scale datasets of scientific papers. |
Copied to clipboard
| Challenge: | Existing methods for multimodal summarization ignore the structure and semantics of the whole video and article. |
| Approach: | They propose a semantic-consistent cross-domain summarization model that extracts features from video and article and uses fusion methods to select representative one. |
| Outcome: | The proposed model produces high-quality multimodal summaries on three MSMO datasets. |
Copied to clipboard
| Challenge: | Existing methods address few-shot intent detection tasks from two perspectives: data augmentation and task-adaptive training with pre-trained models. |
| Approach: | They propose a few-shot intent detection schema using contrastive pre-training and fine-tuning. |
| Outcome: | The proposed method achieves state-of-the-art performance on three challenging intent detection datasets under 5-shot and 10-shot settings. |
Copied to clipboard
| Challenge: | Existing methods for keyphrase extraction are limited by the number of annotated documents. |
| Approach: | They propose a joint learning approach that uses the idea of self-distillation to extract keyphrases from unlabeled articles. |
| Outcome: | The proposed approach outperforms baseline models on two public benchmarks: Inspec and SemEval-2017. |
Copied to clipboard
| Challenge: | In-store users only need to take a picture or scan the barcode of the product of interest, and then the user can talk to the assistant about the product. |
| Approach: | They present a mobile-based intelligent shopping assistant that is designed to improve shopping experience in physical stores. |
| Outcome: | The proposed system can improve shopping experience in physical stores by leveraging advanced techniques in computer vision, speech processing, and natural language processing. |
Copied to clipboard
| Challenge: | CLIP-based classifiers rely on the prompt containing a class name that is known to the text encoder and perform poorly on new classes or the classes whose names rarely appear on the Internet. |
| Approach: | They propose to use a set of text descriptors to express a class name into a textual descriptable and match the embeddings of the detected parts to their textual ones to compute a logit score. |
| Outcome: | The proposed classifier outperforms CLIP-based classifiers on zero-shot and supervised learning settings by 88.80% and 92.20% accuracy on CUB-200 and Stanford Dogs-120. |
Copied to clipboard
| Challenge: | Current studies on image captioning focus on single image, but there are no effective models for generating relational captions for two images. |
| Approach: | They propose a language-guided image editing dataset that contains real image pairs with corresponding editing instructions. |
| Outcome: | The proposed model outperforms baseline and existing methods on two datasets. |
Copied to clipboard
| Challenge: | Existing approaches to analyzing knowledge in a corpus often focus on single factors in isolation. |
| Approach: | They propose a framework that organizes multiple contexts into independently processed groups . they classify these relationships into distracting, ambiguous, counterfactual, and duplicated . |
| Outcome: | The proposed framework outperforms existing grouping methods and single-context approaches. |
Copied to clipboard
| Challenge: | In July 2019, RoBERTa was the first to surpass a human baseline on GLUE . since then, 13 more methods have outperformed humans on the GLu leaderboard . |
| Approach: | They found that 75% to 90% of correct predictions of BERT-based classifiers remain constant after input words are randomly shuffled. |
| Outcome: | The proposed model outperforms humans on GLUE and SQuAD 2.0. |
Copied to clipboard
| Challenge: | StreamHover is a framework for annotating and summarizing livestream transcripts . the problem is that there is n't enough annotated datasets to summarize livestreams based on the informal nature of spoken language . |
| Approach: | They propose a framework for annotating and summarizing livestream transcripts using a text preview. |
| Outcome: | The proposed model generalizes better and improves over strong baselines. |
Copied to clipboard
| Challenge: | Image editing is time-consuming and requires a wide assortment of features and combinations of these features to achieve a desired effect. |
| Approach: | They propose a task-oriented dialogue system to investigate low-level instructions for NLIE . 25% of users found the system easy-to-use, resonating with their motivation . |
| Outcome: | The proposed system is easy-to-use and user-friendly. |
Copied to clipboard
| Challenge: | Existing evaluation methods for VideoMLLMs are limited to one task and fail to assess hallucinations in open-ended, free-form responses. |
| Approach: | They propose a unified framework that extracts comprehensive descriptive facts and models their semantic dependencies via a Spatio-Temporal Semantic Dependency Graph. |
| Outcome: | The proposed framework aligns more closely with human judgment than existing evaluation methods and improves factual consistency in both text and video generation. |
Copied to clipboard
| Challenge: | Recent advances in natural language processing (NLP) have led to significant breakthroughs in the field. |
| Approach: | They evaluate ChatGPT over multiple tasks with diverse languages and large datasets to provide more comprehensive information for multilingual NLP applications. |
| Outcome: | The proposed model can process and generate texts for multiple languages due to its multilingual training data. |
Copied to clipboard
| Challenge: | AutoNLU is an on-demand cloud-based system that enables users to create and edit datasets and train and test different state-of-the-art NLU models. |
| Approach: | They introduce an on-demand cloud-based system that provides an easy-to-use interface . they build powerful keyphrase extraction models that achieve state-of-the-art results . |
| Outcome: | The proposed model achieves state-of-the-art on two public benchmarks and is easy to use and use. |
Copied to clipboard
| Challenge: | Numerous parsing methods have been developed for a single sentence, while a typical human-computer interaction session or conversation is not singleturn. |
| Approach: | They propose to modify an existing scene graph given a new user's command by using graph-based sparse transformer and cross attention information fusion to improve performance. |
| Outcome: | The proposed models outperform previous systems adapted from the machine translation and graph generation literature and contribute to the research community. |
Copied to clipboard
| Challenge: | Modern image captioning models are usually trained with text similarity objectives . reference captions often describe only the most salient objects in images . |
| Approach: | They propose to use CLIP to calculate multi-modal similarity and use it as a reward function . they propose a simple finetuning strategy to improve grammar that does not require extra text annotation. |
| Outcome: | The proposed model generates more distinctive captions than the CIDEroptimized model on text-to-image retrieval and fineCapEval. |
Copied to clipboard
| Challenge: | Contrastive language-image pre-training models have demonstrated considerable success across various vision-language tasks, such as text-to-image retrieval. |
| Approach: | They propose a fine-tuning approach to enhance the representations of CLIP models for paraphrases by leveraging large language models. |
| Outcome: | The proposed model improves on baseline models across paraphrased retrieval, visual genome relation and attribution, and seven semantic textual similarity tasks. |
Copied to clipboard
| Challenge: | Existing methods for event coreference resolution use symbolic features, but they are noisy and contain errors. |
| Approach: | They propose a context-dependent gated module to adaptively control the information flows from the input symbolic features. |
| Outcome: | The proposed model achieves state-of-the-art on two datasets: ACE 2005 and KBP 2016 . |
Copied to clipboard
| Challenge: | Prior work on instruction tuning datasets combined these data types without examining their distinct effects. |
| Approach: | They investigate how training LLMs with or without context affects model behavior and performance . they find that using context-augmented data as the backbone for vision-language models reduces hallucination . |
| Outcome: | The proposed training with context-augmented data reduces hallucination and improves grounding in the visual domain. |
Copied to clipboard
| Challenge: | Large Foundation Models (LFMs) have transformed the landscape of AI research and day-to-day life. |
| Approach: | They propose a framework that delineates GUI agents' perception, reasoning, planning, and acting capabilities. |
| Outcome: | The proposed framework delineates their perception, reasoning, planning, and acting capabilities. |
Copied to clipboard
| Challenge: | Recent studies have shown that dialog-based interaction grounded in visual information is not as effective as previous VQA tasks because of its dialog history. |
| Approach: | They propose a visual dialogue subset which explicitly encodes dialog history and a NDCG benchmark of 63%. |
| Outcome: | The proposed subset (VisdialConv) of the VisdialVal set achieves state-of-the-art performance on 72 % of the data. |
Copied to clipboard
| Challenge: | Small language models (SLMs) show promise for mobile deployment, but their real world performance and applications on smartphones remain understudied. |
| Approach: | They propose a slim language model with a model size of 125M to 8B and a context length of 8B for efficient on-device processing. |
| Outcome: | The proposed model is based on a Samsung Galaxy S24 and shows comparable or superior performance. |
Copied to clipboard
| Challenge: | a corpus of image edit requests is elicited for real world images, and an annotation framework is developed . evaluators evaluate crowd-sourced annotation as a means of efficiently creating a sizable corpus at a reasonable cost. |
| Approach: | They propose a natural language interface for interacting with an image editing program . they propose an annotation framework for understanding natural language requests . |
| Outcome: | The proposed tool interprets image edit requests and maps them to actionable commands. |
Copied to clipboard
| Challenge: | Existing deep learning approaches for answer selection use word-level comparison followed by aggregation. |
| Approach: | They propose a new gated self-attention memory network for answer selection task . they combine a transfer learning technique from a large-scale online corpus to create a gated network . |
| Outcome: | The proposed model outperforms existing methods on two standard answer selection datasets: TrecQA and WikiQA. |
Copied to clipboard
| Challenge: | Existing methods for medical question understanding often fail to provide high recall in answer retrieval. |
| Approach: | They propose a multi-task learning method with data augmentation for medical question understanding that uses just one dataset to optimize for both tasks. |
| Outcome: | The proposed method outperforms existing MTL methods across 4 datasets of medical question pairs in ROUGE scores, RQE accuracy and human evaluation. |
Copied to clipboard
| Challenge: | Recent studies have shown that generative data augmentation, where synthetic samples generated from deep generative models complement the training dataset, benefit NLP tasks. |
| Approach: | They propose a Variational Hierarchical Dialog Autoencoder for modeling the complete aspects of goal-oriented dialogs using inter-connected latent variables and learns to generate coherent dialogs from the latent spaces. |
| Outcome: | The proposed model outperforms previous strong baselines on dialog response generation and user simulation tasks. |
Copied to clipboard
| Challenge: | Existing methods to detect offensive content in social media platforms are limited by the availability of labeled code-switched data. |
| Approach: | They propose a method for generating synthetic code-switched offensive content data using human-generated data and a keyword classification baseline. |
| Outcome: | The proposed algorithm can be used to generate synthetic code-switched offensive content data and train it on human-generated data. |
Copied to clipboard
| Challenge: | Community Question Answering web sites are used for non-factoid question answering . however, there is a scarcity of available datasets for this task . cnn.com's john m. sutter is releasing a dataset for why-QA . |
| Approach: | They propose a dataset of 2,854 why-question and answer(s) pairs related to Adobe Photoshop usage from five CQA web sites. |
| Outcome: | The new dataset is the first English dataset for Why-QA that focuses on a product . it can be used to build Why-Q systems, evaluate approaches and develop new models . |
Copied to clipboard
| Challenge: | Multilingual models have gained popularity for their zero-shot cross-lingual transfer learning capabilities, but their generalization ability is inconsistent for typologically diverse languages. |
| Approach: | They propose a meta-learning approach that adapts MAML to learn to adapt to new languages . they extensively evaluate two cross-lingual NLU tasks using English as source and spanish as target . |
| Outcome: | The proposed approach outperforms naive fine-tuning on cross-lingual tasks for most languages. |