Papers by Jos Rozen
Should you marginalize over possible tokenizations? (2023.acl-short)
Copied to clipboard
| Challenge: | Autoregressive language models map token sequences to probabilities. |
| Approach: | They propose to marginalize over all tokenizations to compute the probability of a string . they find that the gap in log-likelihood is no larger than 0.5% in most cases . |
| Outcome: | The proposed algorithm shows that the gap in log-likelihood is no larger than 0.5% in most cases . but it becomes more pronounced for data with long complex words, the authors show . |
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing benchmarks for long-context LLMs focus on generic tasks that are not necessarily aligned with real-world applications. |
| Approach: | They propose to augment existing ELITR corpus by adding 271 manually crafted questions with their ground-truth answers and noisy versions of meeting transcripts altered to target different Word Error Rate levels. |
| Outcome: | The proposed benchmark augments the existing ELITR corpus by adding 271 manually crafted questions with ground-truth answers, as well as noisy versions of meeting transcripts altered to target different Word Error Rate levels. |
disco: a toolkit for Distributional Control of Generative Models (2023.acl-demo)
Copied to clipboard
| Challenge: | Pre-trained language models reproduce undesirable biases in training data and overlook patterns that are important but difficult to capture. |
| Approach: | They propose to use distributional control techniques to control the prevalence (i.e. expectations) of any features of interest in the model's outputs. |
| Outcome: | The proposed methods can control the prevalence (i.e. expectations) of features of interest in the model’s outputs. |
FaST: Feature-aware Sampling and Tuning for Personalized Preference Alignment with Limited Data (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on personalizing conversational assistants to meet specific user preferences. |
| Approach: | They propose to use a dataset to analyze a problem where only a small set of preference annotations can be collected per user. |
| Outcome: | The proposed approach leverages high-level features discovered from the data, achieving the best overall performance. |
Self-Supervised and Controlled Multi-Document Opinion Summarization (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing unsupervised methods for summarizing reviews are based on bootstrapping and require a combination of loss functions or hierarchical latent variables to ensure that the generated summaries remain on-topic. |
| Approach: | They propose a self-supervised setup that considers an individual document as a target summary for a set of similar documents. |
| Outcome: | The proposed setup makes training simpler than previous approaches by relying only on standard log-likelihood loss and mainstream models. |