Papers by Zifeng Cheng
Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on prompt engineering to extract sentence embeddings from large language models (LLMs) but these models are mostly decoder-only and the earlier tokens in the sentence cannot attend to the latter, resulting in biased encoding of sentence information and cascading effects on the final decoded token. |
| Approach: | They propose a plug-and-play and training-free technique that prepends each layer’s decoded sentence embedding to the beginning of the sentence in the next layer’ s input. |
| Outcome: | The proposed technique can significantly improve the performance of existing prompt-based sentence embedding methods across different LLMs while incurring negligible additional inference cost. |
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
Aggregating Multiple Heuristic Signals as Supervision for Unsupervised Automated Essay Scoring (2023.acl-long)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) aims to evaluate the quality score of input essays without human intervention. |
| Approach: | They propose an unsupervised approach to evaluate the quality of input essays . they use multiple heuristic quality signals as pseudo-groundtruths to train a neural AES model . |
| Outcome: | The proposed approach achieves state-of-the-art performance on eight prompts of ASPA dataset compared with previous unsupervised methods . |
Focusing Condition: Inference-Time Self-Contrastive Steering Elicits Better Conditional Text Embeddings in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for extracting conditional text embeddings from large language models (LLMs) relying on prompts often fails to produce high-quality conditional embeddables, resulting in degradation of quality. |
| Approach: | They propose a plug-and-play method that constructs unconditional general text embeddings and uses them to refine conditional text embeds. |
| Outcome: | The proposed method improves performance of prompt-based methods on clustering, Semantic Textual Similarity, and triplet alignment datasets. |
Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on prompt engineering to encode the full semantics of a sentence into the embedding of the last token. |
| Approach: | They propose a technique that introduces an extra auxiliary prompt to elicit better sentence embedding . they propose to use the hidden state of the token as the sentence embedded in LLMs . |
| Outcome: | The proposed technique can improve performance of existing prompt-based methods on STS tasks and downstream classification tasks. |
A Symmetric Local Search Network for Emotion-Cause Pair Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for Emotion-cause pair extraction are not effective because of their lack of annotation. |
| Approach: | They propose a new task which aims at extracting the potential clause pairs of emotions and corresponding causes in a document. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods on the ECPE corpus. |
Improving Domain Generalization for Prompt-Aware Essay Scoring via Disentangled Representation Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing AES models are either prompt-specific or prompt-adaptive and cannot generalize well on “unseen” prompts. |
| Approach: | They propose a prompt-aware neural AES model to extract comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features. |
| Outcome: | The proposed model extracts comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features. |
AEA: Adaptive Expert Allocation Improves Sentence Embeddings from Mixture-of-Experts LLM (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve embeddings from Mixture-of-Experts models allocate a fixed number of experts uniformly across all layers and tokens, ignoring inter-layer and inter-token heterogeneity. |
| Approach: | They propose an Adaptive Expert Allocation framework that performs layer-wise and token-wise expert allocation to enhance embedding quality. |
| Outcome: | The proposed method improves embedding quality across multiple MoE models. |
ProtoCycle: Reflective Tool-Augmented Planning for Text-Guided Protein Design (2026.findings-acl)
Copied to clipboard
Yutang Ge, Guojiang Zhao, Sihang Li, Zheng Cheng, Zifeng Zhao, Hanchen Xia, Guolin Ke, Linfeng Zhang, Zhifeng Gao, Yu Guang Wang
| Challenge: | Recent deep generative models have already shown encouraging * Equal contribution. |
| Approach: | They propose to use generic instruction-tuned LLMs as direct text-to-sequence generators to achieve this goal. |
| Outcome: | Recent studies show that reflection improves sequence quality and alignment while maintaining competitive foldability. |
Multi-Prompting Decoder Helps Better Language Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to adapt Pre-trained Language Models to downstream tasks are limited by their inference APIs. |
| Approach: | They propose a multi-prompting decoding framework that query PLMs with multiple prompts . they propose to query Plms with optimal transport for hidden states and calibrated decoding for class scores . |
| Outcome: | The proposed framework achieves state-of-the-art results on multiple natural language understanding datasets under the few-shot setting. |