Papers by Guoqiang Ma
PPC-GPT: Federated Task-Specific Compression of Large Language Models via Pruning and Chain-of-Thought Distillation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are becoming more popular and are gaining widespread use in artificial intelligence. |
| Approach: | They propose a unified framework that addresses both privacy preservation and model compression in federated settings. |
| Outcome: | The proposed framework maintains competitive performance comparable to full-sized LLMs while ensuring robust privacy protection through its federated architecture. |
Repo4QA: Answering Coding Questions via Dense Retrieval on GitHub Repositories (2022.coling-1)
Copied to clipboard
| Challenge: | Stack Overflow and GitHub are open source communities that are gaining popularity . developers need to raise programming questions in coding forums and navigate to GitHub repositories . |
| Approach: | They propose a questionrepository matching task that bridges the gap between repositories and real-world coding questions. |
| Outcome: | The proposed model outperforms state-of-the-art methods on coding questions and repositories . it can find suitable coding repositoriels and bridge the gap between them . |
FedCoT: Federated Chain-of-Thought Distillation for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have emerged as a transformative force in artificial intelligence, demonstrating exceptional proficiency across various tasks. |
| Approach: | They propose a federated framework for the Chain-of-Thought distillation of knowledge from LLMs to SLMs, while adhering to privacy requirements. |
| Outcome: | The proposed framework ensures secure knowledge transfer from an LLM on a high-powered server to an SLM on resource-constrained client while adhering to privacy requirements. |
FedMKT: Federated Mutual Knowledge Transfer for Large and Small Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Recent research in large language models (LLMs) has focused on enabling clients to fine-tune their locally deployed homogeneous LLMs collaboratively or on transferring knowledge from server-based LLM to small language models at downstream clients. |
| Approach: | They propose a parameter-efficient federated mutual knowledge transfer framework for large and small language models that allows for token alignment and selective knowledge transfer between client-side LLMs and a server-side SLM. |
| Outcome: | The proposed framework enhances the performance of both LLMs and SLMs with clients' unique domain insights while preserving the server's LLM and client's unique domain insight. |
FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning Large Language Models (LLMs) suffer from a performance bottleneck . Existing approaches like Offsite-Tuning (OT) secure the LLMs IP . |
| Approach: | They propose a framework that replaces weak adapters with a unified, powerful Proxy Small Language Model (SLM) they propose 'resource-friendly' compression and 'robust optimization' to handle data heterogeneity. |
| Outcome: | Experiments show that FedProxy outperforms OT and centralized fine-tuning methods. |