Papers by Hung Bui
The Context-Dependent Additive Recurrent Neural Net (N18-1)
Copied to clipboard
| Challenge: | Contextual sequence mapping is one of the fundamental problems in Natural Language Processing (NLP). |
| Approach: | They propose a new family of Recurrent Neural Networks that address contextual sequence mapping . they propose to use contextual signals to control the flow of information . |
| Outcome: | The proposed architecture outperforms existing methods on dialog problem and language model . the proposed architectures are based on a novel family of recurrent neural networks . |
Functional Overlap Reranking for Neural Code Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | Code Large Language Models (CodeLLMs) have ushered in a new era in code generation, but selecting the best code solutions remains a challenge. |
| Approach: | They propose a new reranking strategy that quantifies the functional overlap between solution clusters to provide a better ranking strategy for code solutions. |
| Outcome: | Empirical results show that the proposed method surpasses state-of-the-art methods on the pass@1 score. |
Better Language Models of Code through Self-Improvement (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models for code (PLMCs) are pre-taught on large datasets with multi-modal objectives, but fine-tuning them requires extensive supervision and is limited by the size of the dataset provided. |
| Approach: | They propose a data augmentation framework that utilizes knowledge from the pre-training and fine-tuning stage to augment training data, which is then used for the next step. |
| Outcome: | The proposed framework significantly improves pre-trained language models’ performance in sequence-generation tasks, such as code summarization and code generation in the CodeXGLUE benchmark. |
CodeT5+: Open Code Large Language Models for Code Understanding and Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing code LLMs adopt a specific architecture or rely on a unified encoder-decoder network for downstream tasks, lacking flexibility to operate in the optimal architecture for a particular task. |
| Approach: | They propose to initialize code LLMs with frozen off-the-shelf LLM and explore instruction-tuning to align with natural language instructions. |
| Outcome: | The proposed model outperforms open-source LLMs on 20 code-related benchmarks. |