Papers by Cheng-Kuang Wu
Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Using zero-shot or few-shot prompting, Large Language Models have been widely adopted in downstream applications. |
| Approach: | They propose to quantify the impact of option order and token usage on LLMs and propose mitigation strategies to enhance model performance. |
| Outcome: | The proposed mitigation strategies improve model performance and reduce the impact of token and order sensitivity on LLMs. |
None of the Above, Less of the Right Parallel Patterns in Human and LLM Performance on Multi-Choice Questions Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Multiple-choice exam questions with “None of the above” (NA) options have been extensively studied in educational testing . however, their impact on Large Language Models (LLMs) evaluation remains underexplored . |
| Approach: | They conduct systematic experiments with 28 LLMs on the MMLU benchmark to examine how NA options affect model performance and confidence calibration. |
| Outcome: | The results highlight important implications for benchmark design and raise questions about LLMs’ ability to handle uncertainty in real-world applications. |
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies demonstrate great performance gain for self-rationalization by few-shot prompting LMs with rationale-augmented exemplars. |
| Approach: | They propose to leverage explanations for small LMs to improve few-shot self-rationalization by reducing the problem of plausibility judgement to natural language inference. |
| Outcome: | The proposed approach achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric. |
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown striking ability to adapt to target tasks with a few input-output demonstrations. |
| Approach: | They propose a framework which bootstraps LMs’ intrinsic capabilities to perform zero-shot ICL. |
| Outcome: | The proposed framework outperforms baselines on 23 BIG-Bench Hard tasks on average accuracy and head-to-head comparison. |
Let Me Speak Freely? A Study On The Impact Of Format Restrictions On Large Language Model Performance. (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Structured generation is used to extract key output information from large language models (LLMs). |
| Approach: | They examine whether constraints on generation space impact LLMs’ abilities, including reasoning and domain knowledge comprehension. |
| Outcome: | The proposed model is based on a few-shot in-context learning and instruction-following capabilities. |
Fidelity-Enriched Contrastive Search: Reconciling the Faithfulness-Diversity Trade-Off in Text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Language models often generate fluent and convincing content but can lack consistency with the provided source, resulting in potential inaccuracies. |
| Approach: | They propose a new decoding method that augments the contrastive search framework with context-aware regularization terms to promote tokens that are semantically similar to the provided source while penalizing repetitiveness in the generated text. |
| Outcome: | The proposed method improves faithfulness across various language models while maintaining output diversity comparable to well-performing decoding algorithms. |
I Need Help! Evaluating LLM’s Ability to Ask for Users’ Support: A Case Study on Text-to-SQL Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | a new study examines the proactive ability of large language models to seek user support . without external feedback, many LLMs struggle to recognize their need for user support. |
| Approach: | They propose metrics to evaluate the trade-off between performance improvements and user burden . they also investigate whether LLMs can determine when to request user support . |
| Outcome: | The proposed metrics show that without external feedback, many LLMs struggle to recognize their need for user support. |