Papers by Nicolaas Jedema
Measuring Retrieval Complexity in Question Answering Systems (2024.findings-acl)
Copied to clipboard
| Challenge: | a new metric, retrieval complexity (RC), measures the difficulty of answering questions. |
| Approach: | They propose a retrieval complexity metric conditioned on the completeness of retrieved documents . they propose an unsupervised pipeline to measure RC given an arbitrary retrieval system . |
| Outcome: | The proposed pipeline measures RC more accurately than alternative estimators on six challenging QA benchmarks. |
Speechworthy Instruction-tuned Language Models (2024.emnlp-main)
Copied to clipboard
Hyundong Cho, Nicolaas Jedema, Leonardo Ribeiro, Karishma Sharma, Pedro Szekely, Alessandro Moschitti, Ruben Janssen, Jonathan May
| Challenge: | Current instruction tuned language models are trained on textual preference data and therefore not aligned to speech domain. |
| Approach: | They propose to use radio-industry best practices to prompt and learn speech-based preference data to improve speech-suitability of popular instruction tuned language models. |
| Outcome: | The proposed methods achieve the best win rates in head-to-head comparisons, resulting in preferred or tied to the base model in 76.2% of comparisons on average. |