Beyond Sequences: Two-dimensional Representation and Dependency Encoding for Code Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing code generation approaches represent code as a linear sequence of tokens, but positional encodings compromise generalization . explicit positional encoders sacrifice permutation invariance, imposes a strict order on the input sequence . |
| Approach: | They propose to represent code snippets as two-dimensional entities with explicit encodings . they propose to use dictionary learning to perform semantic matching between code lines . |
| Outcome: | The proposed model captures the hierarchical and spatial structure of code, especially the dependencies between code lines. |
Similar Papers
Hierarchical Bracketing Encodings Work for Dependency Graphs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sequence labeling (SL) is a simple yet effective paradigm for a wide range of natural language problems. |
| Approach: | They propose a new bracketing approach for dependency graph parsing that encodes graphs as sequences and n tagging actions. |
| Outcome: | The proposed approach significantly reduces label space while preserving structural information. |
Case2Code: Scalable Synthetic Data for Code Generation (2025.coling-main)
Copied to clipboard
Yunfan Shao, Linyang Li, Yichuan Ma, Peiji Li, Demin Song, Qinyuan Cheng, Shimin Li, Xiaonan Li, Pengyu Wang, Qipeng Guo, Hang Yan, Xipeng Qiu, Xuanjing Huang, Dahua Lin
| Challenge: | Large Language Models (LLMs) have shown outstanding breakthroughs in code generation. |
| Approach: | They propose a case-to-code induction task that exploits the expressiveness and correctness of programs by incorporating LLMs into their training. |
| Outcome: | The proposed task improves distribution case-to-code induction and various coding generation tasks. |
Text-to-Code Generation with Modality-relative Pre-training (2024.eacl-long)
Copied to clipboard
| Challenge: | Large pre-trained language models have been applied to programming language tasks with great success, often through further pre-training of a strictly-natural language model. |
| Approach: | They propose to map programming language modalities into the same embedding space by separating embeddable spaces between modality and modality-relative training objectives. |
| Outcome: | The proposed model can be adapted and represented differently depending on which modality they belong to and to the ultimate benefit of the downstream task. |
Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy (2021.emnlp-main)
Copied to clipboard
Colin Clement, Shuai Lu, Xiaoyu Liu, Michele Tufano, Dawn Drain, Nan Duan, Neel Sundaresan, Alexey Svyatkovskiy
| Challenge: | Statistical language modeling and translation with transformers have found many successful applications in program understanding and generation tasks. |
| Approach: | They propose an architecture-independent approach for leveraging syntactic hierarchies of source code . they use syntax trees to extract syntak hierarchical structures and integrate them into context window . |
| Outcome: | The proposed approach achieves state-of-the-art in code completion and summarization for Python in the CodeXGLUE benchmark. |
Code Generation From Flowcharts with Texts: A Benchmark Dataset and An Approach (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, researchers focus on generating codes from requirement documents. |
| Approach: | They propose to generate source code from flowcharts with texts instead of directly translating requirements into codes. |
| Outcome: | The proposed model improves on the baselines by transforming flowcharts into pseudo-code . the proposed model is based on 320 flowchartes with their corresponding source codes . |
LM2Protein: A Structure-to-Token Protein Large Language Model (2025.findings-emnlp)
Copied to clipboard
| Challenge: | RNA-binding proteins are critical for various molecular functions, relying on their precise tertiary structures. |
| Approach: | They propose a method to integrate protein 3D structural data within a sequence processing framework. |
| Outcome: | The proposed method achieves high sequence recovery in inverse folding and protein-conditioned RNA design. |
Bringing Emerging Architectures to Sequence Labeling in NLP (2026.eacl-long)
Copied to clipboard
| Challenge: | Pretrained Transformer encoders are the dominant approach to sequence labeling . however, few have been applied to sequence labels on flat or simplified tasks . |
| Approach: | They propose to use pretrained Transformer encoders to model relations across words . they find that the architectures adapt well across tagging tasks that vary in complexity . |
| Outcome: | The proposed architectures perform well across tagging tasks across languages and datasets. |
Unleashing the True Potential of Sequence-to-Sequence Models for Sequence Tagging and Structure Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Sequence-to-Sequence (S2S) models have been successful on text generation tasks . however, learning complex structures with S2S models remains challenging . |
| Approach: | They propose to use constrained decoding to model part-of-speech tagging, named entity recognition, constituency, and dependency parsing tasks with 3 lexically diverse linearization schemas and corresponding constrained coding methods. |
| Outcome: | The proposed methods outperform the state-of-the-art on four core tasks. |
ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Despite the critical role of software requirements, these criteria have not been studied actively in previous code generation works. |
| Approach: | They propose a framework that leverages in-context learning to organize and extrapolate unexpressed requirements from textual descriptions. |
| Outcome: | The proposed framework generates functional requirements from textual descriptions and extrapolates unexpressed requirements from them. |
Beyond Paragraphs: NLP for Long Sequences (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will introduce document-level representation learning techniques . document-based learning is challenging due to the limited sequence length of many models . |
| Approach: | They will provide an overview of established long sequence NLP techniques and discuss memory-saving methods that are key to processing long sequences. |
| Outcome: | The tutorial will introduce the latest and ongoing techniques for document-level representation learning. |