| Challenge: | Serial position effects (SPE) are well-documented cognitive biases in human behavior. |
| Approach: | They propose to use binary choices instead of multiple choices where feasible . they also suggest limiting prompt length and placing crucial information at the beginning of prompts . |
| Outcome: | The proposed framework shows that the effects are widespread across LLMs and the proposed mitigation methods are effective. |
Similar Papers
Characterizing Positional Bias in Large Language Models: A Multi-Model Evaluation of Prompt Order Effects (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models can be influenced by various forms of biases, says a new study . positional bias affects how LLMs interpret and weigh information, the authors say . |
| Approach: | a new study examines the impact of positional bias on large language models . positional biased models prioritize items based on their position rather than content or quality . |
| Outcome: | a new study shows that LLMs prioritize items based on their position rather than content or quality . the positional bias affects how LLM interpret and weigh information, the authors say . |
Social Bias Evaluation for Large Language Models Requires Prompt Variations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have tried to evaluate and mitigate social biases accurately using limited prompts. |
| Approach: | They investigate the sensitivity of Large Language Models when changing prompt variations . they found that LLM rankings fluctuate across prompts for both task performance and social bias . |
| Outcome: | The results show that LLM rankings fluctuate when changing prompt variations . |
Can We Instruct LLMs to Compensate for Position Bias? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies reveal that position bias in large language models (LLMs) leads to difficulty in accessing information retrieved from the retriever. |
| Approach: | They propose to direct LLMs to allocate more attention towards a selected segment of the context through prompting. |
| Outcome: | The proposed approach improves the performance of large language models by promoting instruction with an exact document index. |
Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Using zero-shot or few-shot prompting, Large Language Models have been widely adopted in downstream applications. |
| Approach: | They propose to quantify the impact of option order and token usage on LLMs and propose mitigation strategies to enhance model performance. |
| Outcome: | The proposed mitigation strategies improve model performance and reduce the impact of token and order sensitivity on LLMs. |
The Impact of Inference Acceleration on Bias of LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work suggests strategies to increase inference efficiency with LLMs . however, these strategies may inadvertently lead to some side-effects. |
| Approach: | They propose to optimize inference acceleration strategies such as quantization, pruning, and caching to reduce inference cost and latency while maintaining predictive performance. |
| Outcome: | The proposed strategies reduce cost and latency while maintaining predictive performance while preserving the model size. |
Beyond Performance: Quantifying and Mitigating Label Bias in LLMs (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models exhibit undesirable preference toward predicting certain answers over others, despite their adaptability to diverse tasks. |
| Approach: | They propose a label bias calibration method that outperforms recent calibration approaches for improving performance and mitigating label bias. |
| Outcome: | The proposed method outperforms calibration approaches for improving performance and mitigating label bias. |
Cognitive Effects and Biases in Large Language Models (2026.eacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
| Approach: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
| Outcome: | This tutorial bridges psychology and NLP to clarify cognitive effects and biases in large language models. |
Do Prompt Positions Really Matter? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prompt-based learning models have a high level of interest due to their ability to perform zero-shot and fewshot tasks. |
| Approach: | They conduct the most comprehensive analysis to date of prompt position for diverse natural language processing tasks. |
| Outcome: | The proposed model is more robust than previous models and is consistent even in instruction-tuned models. |
Position Engineering: Boosting Large Language Models through Positional Information Manipulation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated significant strides towards achieving artificial general intelligence. |
| Approach: | They propose a technique termed position engineering which alters the positional information in the prompt without modifying the text itself. |
| Outcome: | The proposed technique significantly improves on the baseline in retrieval-augmented generation and in-context learning scenarios. |
What Did I Do Wrong? Quantifying LLMs’ Sensitivity and Consistency to Prompt Engineering (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have significantly improved productivity in a number of routine tasks. |
| Approach: | They propose two metrics for classification tasks, namely *sensitivity* and *consistency*, which are complementary to task performance. |
| Outcome: | The proposed metrics are complementary to task performance and can be used to guide prompt engineering and obtain LLMs that balance robustness and performance. |