Papers by Michael Shieh
InstructCoder: Instruction Tuning Large Language Models for Code Editing (2024.acl-srw)
Copied to clipboard
| Challenge: | InstructCoder is the first instruction-tuning dataset designed to adapt LLMs for general-purpose code editing. |
| Approach: | They propose to use Large Language Models to edit code based on user instructions . they use a dataset to adapt LLMs to general-purpose code editing . |
| Outcome: | The proposed model can significantly improve code editing performance compared to proprietary models . the proposed model is based on a human-written execution-based benchmark . |
Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts have identified adversarial suffixes capable of jailbreaking LLMs . however, GCG struggles with computational inefficiency, limiting further studies . |
| Approach: | They propose a two-stage transfer learning framework which decouples the search process into behavior-agnostic pre-searching and behavior-relevant post-search. |
| Outcome: | The proposed approach outperforms baseline on Llama2-chat-7b with ASRs of 43.9 (+ 22.2) and 39.0 (+ 19.5) on valid and test sets. |
Prompt Optimization via Adversarial In-Context Learning (2024.acl-long)
Copied to clipboard
Do Long, Yiran Zhao, Hannah Brown, Yuxi Xie, James Zhao, Nancy Chen, Kenji Kawaguchi, Michael Shieh, Junxian He
| Challenge: | Existing methods to optimize prompts for in-context learning are based on adversarial learning and are computationally efficient and extensible to other LLMs and tasks. |
| Approach: | They propose a method to optimize prompts for in-context learning by a generator and a discriminator. |
| Outcome: | The proposed method improves state-of-the-art prompt optimization techniques on 13 generation and classification tasks including summarization, arithmetic reasoning, machine translation, data-to-text generation, and the MMLU and big-bench hard benchmarks. |
Reasoning Robustness of LLMs to Adversarial Typographical Errors (2024.emnlp-main)
Copied to clipboard
Esther Gan, Yiran Zhao, Liying Cheng, Mao Yancan, Anirudh Goyal, Kenji Kawaguchi, Min-Yen Kan, Michael Shieh
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning using Chain-of-Thought (CoT) prompting. |
| Approach: | They develop an algorithm that iteratively samples typos for words that are important to the query and selects the edit that is most likely to succeed in attacking. |
| Outcome: | The proposed algorithm detects typographical errors in large and closed-source LLMs and shows that they are robust to them. |