Papers by Sheng-Lun Wei
Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Using zero-shot or few-shot prompting, Large Language Models have been widely adopted in downstream applications. |
| Approach: | They propose to quantify the impact of option order and token usage on LLMs and propose mitigation strategies to enhance model performance. |
| Outcome: | The proposed mitigation strategies improve model performance and reduce the impact of token and order sensitivity on LLMs. |
Bias in the Ear of the Listener: Assessing Sensitivity in Audio Language Models Across Linguistic, Demographic, and Positional Variations (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances extend language understanding beyond text to speech, enabling unified reasoning across modalities. |
| Approach: | They construct and release a speech-augmented benchmark based on Global MMLU Lite and a data set spanning English, Chinese, and Korean. |
| Outcome: | The proposed model is robust to demographic factors but sensitive to language and option order, suggesting that speech can amplify structural biases. |
Do Before You Judge: Self-Reference as a Pathway to Better LLM Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | LLM-as-Judge frameworks are increasingly popular for AI evaluation, yet research findings on the relationship between models’ generation and judgment abilities remain inconsistent. |
| Approach: | They propose a self-reference-guided evaluation strategy that leverages a model’s own answers as references to strengthen the correlation between generation and judgment abilities. |
| Outcome: | The proposed approach strengthens the correlation between model generation and judgment abilities and provides a reliable proxy for model selection in evaluation tasks. |
No One Fits All: From Fixed Prompting to Learned Routing in Multilingual LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that translation-based prompting is not universally optimal for multilingual LLMs. |
| Approach: | They evaluate translation-based prompting across ten languages and four benchmarks . they propose a lightweight classifier that predicts whether native or translation- based prompts are optimal . |
| Outcome: | The proposed classifiers achieve statistically significant improvements over fixed prompting strategies across ten languages and four benchmarks. |
Induct-Learn: Short Phrase Prompting with Instruction Induction (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating instructions from demonstrations rely on large datasets or numerous examples, which is impractical and costly in real-world scenarios. |
| Approach: | They propose a task-level framework that induces pseudo instructions from a few demonstrations and a short phrase, adding a CoT process into existing demonstrations. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two datasets and exhibits cross-model adaptability and lower cost. |