Papers by Zhiwei Cheng
Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on prompt engineering to extract sentence embeddings from large language models (LLMs) but these models are mostly decoder-only and the earlier tokens in the sentence cannot attend to the latter, resulting in biased encoding of sentence information and cascading effects on the final decoded token. |
| Approach: | They propose a plug-and-play and training-free technique that prepends each layer’s decoded sentence embedding to the beginning of the sentence in the next layer’ s input. |
| Outcome: | The proposed technique can significantly improve the performance of existing prompt-based sentence embedding methods across different LLMs while incurring negligible additional inference cost. |
Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reweighting data mixtures rely on manual designation with certain heuristics based on intuition or empirical results. |
| Approach: | They propose a model-based framework that learns to re-weight domains by reinforcement learning on large quantities of data mixing trajectories with corresponding feedback from an evaluation environment. |
| Outcome: | The proposed framework outperforms baselines in achieving balanced performance across source and target fields and domain spaces without retraining. |
Aggregating Multiple Heuristic Signals as Supervision for Unsupervised Automated Essay Scoring (2023.acl-long)
Copied to clipboard
| Challenge: | Automated Essay Scoring (AES) aims to evaluate the quality score of input essays without human intervention. |
| Approach: | They propose an unsupervised approach to evaluate the quality of input essays . they use multiple heuristic quality signals as pseudo-groundtruths to train a neural AES model . |
| Outcome: | The proposed approach achieves state-of-the-art performance on eight prompts of ASPA dataset compared with previous unsupervised methods . |
Focusing Condition: Inference-Time Self-Contrastive Steering Elicits Better Conditional Text Embeddings in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for extracting conditional text embeddings from large language models (LLMs) relying on prompts often fails to produce high-quality conditional embeddables, resulting in degradation of quality. |
| Approach: | They propose a plug-and-play method that constructs unconditional general text embeddings and uses them to refine conditional text embeds. |
| Outcome: | The proposed method improves performance of prompt-based methods on clustering, Semantic Textual Similarity, and triplet alignment datasets. |
Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to compress long contexts have degraded dramatically as compression ratios increase, sometimes even falling to the closed-book level. |
| Approach: | They propose a query-guided compression method that preserves key information within the compressed context. |
| Outcome: | The proposed method can consistently perform well even at high compression ratios, and offers significant benefits in terms of inference cost and throughput. |
Divergent Thinking: Escape the Homogeneity Trap in Generative Commonsense Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Generative commonsense reasoning requires models to synthesize coherent narratives that satisfy lexical constraints and commonsensical logic. |
| Approach: | They propose a framework that allows for deep semantic diversity rather than surface-level lexical variation. |
| Outcome: | The proposed framework achieves over 10% improvement in overall accuracy on NoRa and SPICE score on CommonGen-Lite. |
Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on prompt engineering to encode the full semantics of a sentence into the embedding of the last token. |
| Approach: | They propose a technique that introduces an extra auxiliary prompt to elicit better sentence embedding . they propose to use the hidden state of the token as the sentence embedded in LLMs . |
| Outcome: | The proposed technique can improve performance of existing prompt-based methods on STS tasks and downstream classification tasks. |
Rumor Detection on Twitter with Claim-Guided Hierarchical Graph Attention Networks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for rumor detection are limited to the strict relation of user responses or oversimplify the conversation structure. |
| Approach: | They propose a method that reinforces interaction of user opinions while reducing negative impact imposed by irrelevant posts. |
| Outcome: | The proposed method improves performance on three Twitter datasets and can detect rumors at early stages. |
A Symmetric Local Search Network for Emotion-Cause Pair Extraction (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for Emotion-cause pair extraction are not effective because of their lack of annotation. |
| Approach: | They propose a new task which aims at extracting the potential clause pairs of emotions and corresponding causes in a document. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods on the ECPE corpus. |
Improving Domain Generalization for Prompt-Aware Essay Scoring via Disentangled Representation Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing AES models are either prompt-specific or prompt-adaptive and cannot generalize well on “unseen” prompts. |
| Approach: | They propose a prompt-aware neural AES model to extract comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features. |
| Outcome: | The proposed model extracts comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features. |
AEA: Adaptive Expert Allocation Improves Sentence Embeddings from Mixture-of-Experts LLM (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve embeddings from Mixture-of-Experts models allocate a fixed number of experts uniformly across all layers and tokens, ignoring inter-layer and inter-token heterogeneity. |
| Approach: | They propose an Adaptive Expert Allocation framework that performs layer-wise and token-wise expert allocation to enhance embedding quality. |
| Outcome: | The proposed method improves embedding quality across multiple MoE models. |
Retrieval-Augmented Generation for Large Language Model based Few-shot Chinese Spell Checking (2025.coling-main)
Copied to clipboard
| Challenge: | Existing LLM-based Chinese spelling check methods rely on fixed prompt samples . existing methods are limited by technical bottlenecks, complex recognition environments, and individual differences . |
| Approach: | They propose a framework called RagID to provide well-chosen prompt samples . they propose to use semantic-based similarity search and iterative discriminator mechanism . |
| Outcome: | The proposed framework can provide well-chosen prompt samples and reduce overcorrection issues in Chinese spelling check tasks. |
Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learning (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing rumor detection methods are poor at detecting false rumors about breaking news or trending topics due to the lack of training data and prior knowledge. |
| Approach: | They propose an adversarial contrastive learning framework to detect false rumors by adapting features learned from well-resourced rumor data to that of the low-resource. |
| Outcome: | The proposed framework improves on two low-resource datasets and shows superior performance . it overcomes restriction of domain and/or language usage and improves robustness . |
Multi-Prompting Decoder Helps Better Language Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to adapt Pre-trained Language Models to downstream tasks are limited by their inference APIs. |
| Approach: | They propose a multi-prompting decoding framework that query PLMs with multiple prompts . they propose to query Plms with optimal transport for hidden states and calibrated decoding for class scores . |
| Outcome: | The proposed framework achieves state-of-the-art results on multiple natural language understanding datasets under the few-shot setting. |