Papers by Piotr Sankowski
Wait, that’s not an option: LLMs Robustness with Incorrect Multiple-Choice Options (2025.acl-long)
Copied to clipboard
| Challenge: | Using a framework that combines instruction-following with critical reasoning, we show that the ability of LLMs to override defaults when faced with invalid options is impaired by alignment techniques. |
| Approach: | They propose a framework for evaluating LLMs’ capacity to balance instruction-following with critical reasoning when presented with multiple-choice questions containing no valid answers. |
| Outcome: | The proposed framework improves models' ability to override defaults when faced with invalid options while minimizing the impact of model size and training techniques on the model. |
LLM generated responses to mitigate the impact of hate speech (2024.findings-emnlp)
Copied to clipboard
Jakub Podolak, Szymon Łukasik, Paweł Balawender, Jan Ossowski, Jan Piotrowski, Katarzyna Bakowicz, Piotr Sankowski
| Challenge: | a study aims to determine the effectiveness of large language models to counteract hate speech . it is the first real-life A/B test evaluating the effectiveness . |
| Approach: | They conduct the first real-life A/B test assessing the effectiveness of LLM-generated counter-speech. |
| Outcome: | The proposed system reduces user engagement by over 20%, the study shows . the proposed metric is based on a simple metric and is scalable to other platforms . |