Papers by Yifeng Chen
Symbol tuning improves in-context learning in language models (2023.emnlp-main)
Copied to clipboard
Jerry Wei, Le Hou, Andrew Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc Le
| Challenge: | Language models are sensitive to the way that prompts are given, indicating that they are not reasoning in a robust manner. |
| Approach: | They propose to fine tune language models on in-context input-label pairs where natural language labels are replaced with arbitrary symbols. |
| Outcome: | The proposed model is much stronger at reasoning tasks and more robust to underspecified prompts than the standard model. |
Code Representation Pre-training with Complements from Program Executions (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing languages have syntactic representations of code to improve code intelligence, but they are difficult to learn from code. |
| Approach: | They propose to embed dynamic information of programs revealed by their test cases into feature representations of code as complements. |
| Outcome: | The proposed method yields 6%/19% mAP improvements over its masked language modeling counterparts. |
Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reduce sequence length rely on heuristics that break compatibility with hardware-efficient kernels like FlashAttention. |
| Approach: | They propose a method that selectively halts stabilized tokens by monitoring layer-wise update dynamics of the self-attention mechanism. |
| Outcome: | The proposed method can reduce prefill complexity while preserving model accuracy and hardware efficiency. |
FuzzAug: Data Augmentation by Coverage-guided Fuzzing for Neural Test Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using large language models to generate meaningful tests is expensive and time-consuming . |
| Approach: | They propose a data augmentation technique that incorporates valid testing semantics and diverse coverage-guided inputs into large language models. |
| Outcome: | The proposed technique improves performance over the baselines by incorporating valid testing semantics and providing diverse coverage-guided inputs. |
Beyond Logits: Aligning Feature Dynamics for Effective Knowledge Distillation (2025.acl-long)
Copied to clipboard
Guoqiang Gong, Jiaxing Wang, Jin Xu, Deping Xiang, Zicheng Zhang, Leqi Shen, Yifeng Zhang, JunhuaShu JunhuaShu, ZhaolongXing ZhaolongXing, Zhen Chen, Pengzhang Liu, Ke Zhang
| Challenge: | Knowledge distillation (KD) compresses large language models into lightweight versions called student models. |
| Approach: | They propose to align the entire feature dynamics between teacher and student models by using two additional loss terms to achieve this. |
| Outcome: | The proposed method matches the entire feature dynamics between teacher and student models rather than just the final states. |
R-PRM: Reasoning-Driven Process Reward Modeling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing Process Reward Models (PRMs) output evaluation scores directly, limiting both learning efficiency and evaluation accuracy. |
| Approach: | They propose a Reasoning-Driven Process Reward Modeling (R-PRM) which activates inherent reasoning to enhance process-level evaluation. |
| Outcome: | The proposed model outperforms baseline models on ProcessBench and PRMBench by 13.9 and 8.5 F1 scores. |
đť’®2IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction (2025.findings-naacl)
Copied to clipboard
| Challenge: | Aspect Sentiment Quad Prediction (ASQP) is an extractive task that focuses on predicting tuples of sentiment-related elements from a given text. |
| Approach: | They propose a stepwise syntax integration tuning framework that integrates syntactic structure knowledge into LLMs through a multi-step tuning process. |
| Outcome: | The proposed framework integrates syntactic structure knowledge into large language models . it decomposes the quadruple generation task into two stages . the proposed framework significantly improves state-of-the-art performance across multiple datasets . |
Understanding Programs by Exploiting (Fuzzing) Test Cases (2023.findings-acl)
Copied to clipboard
| Challenge: | Semantic understanding of programs has attracted great attention in the community . large language models (LLMs) are capable of learning contextual information from data at scale . |
| Approach: | They propose to incorporate a relationship between inputs and possible outputs into learning for achieving a deeper semantic understanding of programs. |
| Outcome: | The proposed method outperforms current state-of-the-art on two programming tasks and outperformed current state of the art by large margins. |
Syntax Matters: Towards Spoken Language Understanding via Syntax-Aware Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on SLU systems have focused on integrating syntactic information into language models. |
| Approach: | They propose a model where attention scopes are constrained based on syntactic relationships. |
| Outcome: | The proposed model improves on three datasets and can be integrated into other language models to further boost their performance. |
Content Fuzzing for Escaping Information Cocoons on Digital Social Media (2026.findings-acl)
Copied to clipboard
| Challenge: | Information cocoons restrict users’ exposure to posts with diverse viewpoints . social media platforms restrict the range of viewpoints that users encounter . |
| Approach: | They propose a confidence-guided fuzzing framework that rewrites posts while preserving their human-interpreted intent and induces different machine-inferred stance labels. |
| Outcome: | The proposed framework rewrites posts while preserving human-interpreted intent and induces different machine-inferred stance labels while maintaining semantic integrity with respect to the original content. |
Graph Explorer: Training Faithful KG Agents with Visibility-Grounded Supervision (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are strong reasoners but still hallucinate and make unreliable decisions on knowledge-intensive questions. |
| Approach: | They propose a pipeline that turns LLM into executable tool supervision without manual trace labeling. |
| Outcome: | The proposed model improves over a reproduced prompting baseline by +22.5/+16.2 points . it is based on a Graph Explorer pipeline that turns SPARQL into executable tool supervision without manual trace labeling. |