| Challenge: | Recent data augmentations for code search are at the raw-data level, which requires additional code analysis and training cost. |
| Approach: | They propose a general format of representation-level augmentation that unifies existing methods. |
| Outcome: | The proposed methods can boost the performance of code search models on a large-scale dataset. |
Similar Papers
Contrastive Learning with Keyword-based Data Augmentation for Code Search and Code Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent work on code search proposes data augmentation of queries for contrastive learning. |
| Approach: | They propose to augment query-code pairs with key words to preserve key words . they use keyDAC to fine-tune various pre-trained language models . |
| Outcome: | The proposed approach outperforms the current state-of-the-art in code search and question answering tasks. |
Exploring Data Augmentation for Code Generation Tasks (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have impacted how models are trained for programming language tasks. |
| Approach: | They propose to use augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. |
| Outcome: | The proposed methods improve translation and summarization by 6.9% and 7.5% respectively. |
CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search (2022.emnlp-main)
Copied to clipboard
Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, Nan Duan
| Challenge: | Existing code pre-training approaches often adopt (masked) language modeling as the training objective which targets on learning to predict (macked) tokens in a given code context. |
| Approach: | They propose a code-text contrastive learning model which learns function-level code semantic representations through large-scale code corpus. |
| Outcome: | The proposed model achieves new state-of-the-art with significant improvement over existing pre-trained models on eleven domain/language-specific code search tasks with six programming languages in different code granularity. |
A Survey of Data Augmentation Approaches for NLP (2021.findings-acl)
Copied to clipboard
Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, Eduard Hovy
| Challenge: | Data augmentation is a field of research that has been underexplored due to the discrete nature of language data. |
| Approach: | They present a comprehensive survey of data augmentation for NLP by summarizing the literature in a structured manner. |
| Outcome: | The proposed methods are used for popular NLP applications and tasks and highlight current challenges and directions for future research. |
SACL: Understanding and Combating Textual Bias in Code Retrieval with Semantic-Augmented Reranking and Localization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that code retrievers exhibit a strong bias towards well-documented code . |
| Approach: | They propose a framework that augments textual information with semantic information to mask specific features while preserving code functionality. |
| Outcome: | The proposed framework enhances textual information and reduces bias by augmenting code or structural knowledge with semantic information. |
Accelerating Code Search with Deep Hashing and Code Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Code search is to search reusable code snippets from source code corpus based on natural languages queries. |
| Approach: | They propose a method to accelerate code search with deep hashing and code classification by using deep hashes and code hash. |
| Outcome: | The proposed method can save 90% of retrieval time while preserving at least 99% of retrievals accuracy. |
Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search (2024.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that ReCo significantly boosts retrieval accuracy across sparse, zero-shot dense and fine-tuned dense search settings. |
| Approach: | They propose a generation-augmented retrieval framework that additionally Rewrites the Code (ReCo) within the codebase for style normalization. |
| Outcome: | The proposed method significantly boosts retrieval accuracy across sparse, zero-shot dense, and fine-tuned dense retrieval settings in diverse search scenarios. |
AugCSE: Contrastive Sentence Embedding with Diverse Augmentations (2022.aacl-main)
Copied to clipboard
| Challenge: | Similar work has shown that a single augmentation can be used to learn a robust generalpurpose representation with contrastive learning. |
| Approach: | They propose a unified framework to utilize diverse sets of data augmentations to achieve a better, general-purpose sentence embedding model. |
| Outcome: | The proposed framework achieves state-of-the-art results on downstream transfer tasks and performs competitively on semantic textual similarity tasks, using only unsupervised data. |
ReACC: A Retrieval-Augmented Code Completion Framework (2022.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that statistical language modeling with transformers can greatly improve the performance in code completion tasks. |
| Approach: | They propose a retrieval-augmented code completion framework that combines a source code retriever and an auto-regressive language model for programming language. |
| Outcome: | The proposed framework achieves state-of-the-art on CodeXGLUE benchmark. |
Contrastive Data and Learning for Natural Language Processing (2022.naacl-tutorials)
Copied to clipboard
| Challenge: | Current NLP models heavily rely on effective representation learning algorithms. |
| Approach: | This tutorial introduces contrastive learning and provides an introduction to the techniques. |
| Outcome: | This tutorial provides an introduction to the fundamentals of contrastive learning approaches and the theory behind them. |