Papers by Guangzhong Sun
Effective and Efficient Query-aware Snippet Extraction for Web Search (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to extract webpage snippets ignore contextual information of webpages, which may be sub-optimal. |
| Approach: | They propose a query-aware webpage snippet extraction method called DeepQSE that captures contextual information of webpages. |
| Outcome: | The proposed method can significantly improve the performance of DeepQSE without affecting its performance. |
Introducing Graph Context into Language Models through Parameter-Efficient Fine-Tuning for Lexical Relation Mining (2025.acl-long)
Copied to clipboard
| Challenge: | Pre-trained language models can effectively mine lexical relations between word pairs . however, graph features and semantic knowledge of pre-tried models are lacking in the task. |
| Approach: | They propose a parameter-efficient fine-tuning method which integrates graph features and semantic representations for lexical relation classification and lexic entailment tasks. |
| Outcome: | The proposed method integrates graph features and semantic representations for lexical relation mining tasks. |
On the Vulnerability of Safety Alignment in Open-Access LLMs (2024.findings-acl)
Copied to clipboard
Jingwei Yi, Rui Ye, Qisi Chen, Bin Zhu, Siheng Chen, Defu Lian, Guangzhong Sun, Xing Xie, Fangzhao Wu
| Challenge: | Large language models (LLMs) are susceptible to malicious exploitation, but are often rejected and limited harmfulness is limited. |
| Approach: | They propose two types of reverse alignment techniques: reverse supervised fine-tuning (RSFT) and reverse preference optimization (RPO). |
| Outcome: | The proposed methods can significantly enhance the success rate and harmfulness of jailbreak attacks, but they face high rejection rates and limited harmfulness. |
Structured Pruning for Large Language Models Using Coupled Components Elimination and Minor Fine-tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated powerful capabilities in natural language processing, yet their vast number of parameters poses challenges for deployment and inference efficiency. |
| Approach: | They propose a structured pruning algorithm that derives the importance of different components based on intermediate data dependencies and removes coupled components across different layers simultaneously. |
| Outcome: | The proposed algorithm reduces model size and accelerates inference without specialized operators and libraries, while maintaining its utility as versatile problem solvers. |
Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermark (2023.acl-long)
Copied to clipboard
Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, Xing Xie
| Challenge: | Large language models (LLMs) have demonstrated exceptional abilities in both text understanding and generation. |
| Approach: | They propose an Embedding Watermark method that implants backdoors on embeddings to protect copyright of large language models. |
| Outcome: | The proposed method protects the copyright of large language models without compromising service quality while minimizing the adverse impact on the original embeddings’ utility. |
Longtriever: a Pre-trained Long Text Encoder for Dense Document Retrieval (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing PLMs are infeasible for processing long documents due to computational costs and incomprehensive document understanding. |
| Approach: | They propose a retrieval model that models local semantics and global context semantics in a tightly-coupled manner. |
| Outcome: | The proposed model overcomes three core challenges of long document retrieval: substantial computational cost, incomprehensive document understanding, and scarce annotations. |
Efficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing news recommendation methods rely on centralized storage of user click behavior data, which may lead to privacy concerns and hazards. |
| Approach: | They propose a federated learning framework for privacy-preserving news recommendation . they propose aggregation of news representations and user model by a client . |
| Outcome: | The proposed framework reduces computation and communication cost on clients while keeping promising model performance. |