Papers by Jingcheng He
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing transformer models are computationally demanding and prohibitively costly for long sequences due to the quadratic complexity of its selfattention module. |
| Approach: | They propose a transformer-based model that inherits weights from large pretrained models by removing redundancies in hidden sequences using the ready-made Fast Fourier Transform operator. |
| Outcome: | The proposed model outperforms the standard BART model on the long-range modeling benchmark LRA with significant improvements in speed and space. |
Measuring and Mitigating Local Instability in Deep Neural Networks (2023.findings-acl)
Copied to clipboard
| Challenge: | Uncertain details like random initialization can change the outputs of a trained system with potentially disastrous consequences. |
| Approach: | They propose a model stability problem by studying how the predictions of a deep neural network change as a consequence of stochasticity in the training process. |
| Outcome: | The proposed method outperforms data-agnostic methods and is 90% cheaper than the gold standard. |
Watermarking with Low-Entropy POS-Guided Token Partitioning and Z-Score-Driven Dynamic Bias for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing watermarking methods reduce the fidelity of semantics in LLMs . |
| Approach: | They propose a low-entropy token partitioning mechanism and z-score-driven dynamic bias mechanism to enhance semantics. |
| Outcome: | The proposed framework improves semantic fidelity and robustness against bias sparsity attacks. |