Papers by Hiroto Kurita
Why Mean Pooling Works: Quantifying Second-Order Collapse in Text Embeddings (2026.acl-long)
Copied to clipboard
| Challenge: | Text embeddings are used across a wide range of NLP tasks, including retrieval-augmented generation. |
| Approach: | They propose a metric to quantify the collapse induced by mean pooling and a simple metric for measuring how often it occurs in real models and texts. |
| Outcome: | The proposed metric measures how often the collapse occurs in real models and texts. |
Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Embedding a sentence into a point in a highdimensional continuous space plays a foundational role in the natural language processing. |
| Approach: | They propose to use contrastive loss to fine-tune sentences by inverse word frequency . they also show that more informative words receive greater weight than less informative ones . |
| Outcome: | The proposed method improves the performance of sentence embeddings by weighing them based on information-theoretic quantities. |