Papers by Emily Cheng
Geometric Signatures of Compositionality Across a Language Model’s Lifetime (2025.acl-long)
Copied to clipboard
| Challenge: | linguistic compositionality allows atoms to locally combine to create global meaning . a rich array of meanings at the level of a phrase may be explained by simple rules of composition. |
| Approach: | They propose to relate the degree of compositionality in a dataset to the intrinsic dimension of its representations under an LM, a measure of feature complexity. |
| Outcome: | The proposed model is based on a geometric view of the compositionality of a dataset and the intrinsic dimension of its representations under an LM. |
On the Correspondence between Compositionality and Imitation in Emergent Neural Communication (2023.findings-acl)
Copied to clipboard
| Challenge: | a study examining compositionality and imitation learning in a Lewis game demonstrates that it is difficult to imitate compositional languages. |
| Approach: | They explore the link between compositionality and imitation in a Lewis game . they show that the learning algorithm used to imitate is crucial . |
| Outcome: | The proposed model improves compositionality and imitation in a Lewis game . the study shows that compositional languages are easier to imitate . |
Bridging Information-Theoretic and Geometric Compression in Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current language models (LMs) encode training data into finitely many variables that allow generalization to infinitely many grammatical utterances. |
| Approach: | They propose to analyze compression in language models from geometric and information-theoretic perspectives. |
| Outcome: | The proposed model can model human language in a relatively small dimension. |