Papers by Richard Khoury
Simple Agents, Biased Judges: Efficient Multi-Party Dialogue Generation & The Evaluation Gap (2026.acl-long)
Copied to clipboard
| Challenge: | Multiparty social dialogue is difficult to formalize and expensive to evaluate, especially at scale. |
| Approach: | They propose a lightweight and controllable multi-party dialoguegeneration framework as an experimental instrument for studying generation and evaluation in social interaction. |
| Outcome: | The proposed framework shows that human judgments against state-of-the-art LLM judges are consistent with human preferences for naturalness, engagingness, and overall quality in multi-party social dialogue. |
QFrBLiMP: a Quebec-French Benchmark of Linguistic Minimal Pairs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Specifically, these minimal pairs are created by manually modifying sentences extracted from an official online resource maintained by a Québec government institution. |
| Approach: | They propose to use the Quebec-French Benchmark of Linguistic Minimal Pairs to evaluate LLMs’ linguistic knowledge of prominent grammatical phenomena in Quebec-french. |
| Outcome: | The proposed corpus evaluates LLMs’ linguistic knowledge of prominent grammatical phenomena in Quebec-French. |
QFrCoLA: a Quebec-French Corpus of Linguistic Acceptability Judgments (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large and Transformer-based language models perform outstandingly in various downstream tasks, but there is limited understanding regarding how these models internalize linguistic knowledge. |
| Approach: | They propose to use a binary acceptability judgments dataset to benchmark seven language models using a standard binary acceptibility judgments framework. |
| Outcome: | The proposed dataset shows that on average, a fine-tuned Transformer-based LM outperforms other methods and that pre-trained cross-lingual LLMs do not acquire linguistic judgment capabilities during their pre-training for Quebec French. |
Idiom Understanding as a Tool to Measure the Dialect Gap (2026.findings-acl)
Copied to clipboard
| Challenge: | idiom understanding and dialect understanding are well-established benchmarks in natural language processing . a language model trained in one of these dialects will have trouble making sense of the idiomatics from the other two . |
| Approach: | They propose to combine idiom understanding and dialect understanding to test regional idiomatics . they propose to use regional ids as benchmarks for other natural language processing languages . |
| Outcome: | The proposed benchmarks are based on idiomatic and dialect understanding datasets for french and francais . the results show prestige-language proficiency does not guarantee regional dialect understanding . |
JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for text simplification focus on only one dimension: fluency, simplicity and meaning preservation. |
| Approach: | They introduce a dataset to assess legal meaning preservation between two legal texts . they also introduce sanity checks for two identical sentences . |
| Outcome: | The proposed metric shows superior correlation with human judgment compared to existing metrics. |