Papers by Corentin Kervadec
Unnatural language processing: How do language models handle machine-generated prompts? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Language model prompt optimization research has shown that semantically and grammatically well-formed manually crafted prompts are outperformed by automatically generated token sequences with no apparent meaning or syntactic structure. |
| Approach: | They propose to use machine-generated prompts to probe how models respond to input that is not composed of natural language expressions. |
| Outcome: | The proposed model outperforms human-crafted prompts on a target zero-shot task. |
Bridging Information-Theoretic and Geometric Compression in Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current language models (LMs) encode training data into finitely many variables that allow generalization to infinitely many grammatical utterances. |
| Approach: | They propose to analyze compression in language models from geometric and information-theoretic perspectives. |
| Outcome: | The proposed model can model human language in a relatively small dimension. |