Papers by Hengzhi Pei
Fine-tuning Language Models for Joint Rewriting and Completion of Code with Potential Bugs (2024.findings-acl)
Copied to clipboard
| Challenge: | Previous work has demonstrated shortcomings of large language models of code (CodeLLMs) in completing drafty partial code with potential bugs. |
| Approach: | They propose to use large language models of code to fine-tune their models to rewrite and complete drafty partial code into functional full programs. |
| Outcome: | The proposed approach achieves superior pass rates over baselines and preserves the integrity of the original partial implementations. |
Zero-Shot Classification by Logical Reasoning on Natural Language Explanations (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show that CLORE is superior to baselines on zero-shot classification tasks. |
| Approach: | They propose a framework for classification by logically parsing and reasoning on natural language explanations. |
| Outcome: | The proposed framework outperforms baselines on zero-shot classification tasks. |
Understanding Silent Data Corruption in LLM Training (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a challenging task because of their large size and complexity. |
| Approach: | They propose to isolate and analyze the impact of SDCs on LLM training by using a cloud computing platform to access unhealthy nodes swept out of production by automated fleet management. |
| Outcome: | The proposed model training compares healthy production nodes with unhealthy nodes exhibiting SDCs at three levels: at each submodule computation, at a single optimizer step, and at . training period. |
A Concise Model for Multi-Criteria Chinese Word Segmentation with Transformer Encoder (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on multicriteria Chinese word segmentation focuses on combining multiple heterogeneous segmentation criteria into a single task. |
| Approach: | They propose a unified model for multi-criteria Chinese word segmentation which is fully-shared for all criteria. |
| Outcome: | The proposed model outperforms existing models on eight datasets with different criteria. |
T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing adversarial examples can induce arbitrary errors to the target models, but they can be exploited to estimate robustness of NLP models. |
| Approach: | They propose a target-controllable adversarial attack framework T3 to handle adversarials . they use tree-based decoders to regularize the syntactic correctness of generated text . |
| Outcome: | The proposed framework can be used to estimate the robustness of NLP models. |