Papers by Ryokan Ri
Dynamic Injection of Entity Knowledge into Dense Retrievers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Dense retrievers struggle with queries involving less-frequent entities due to limited entity knowledge. |
| Approach: | They propose a BERT-based retriever enhanced with a context-entity attention layer and dynamically updatable entity embeddings. |
| Outcome: | The proposed retriever incorporates external entity knowledge without retraining. |
EASE: Entity-Aware Contrastive Learning of Sentence Embedding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for learning sentence embeddings are fine-tuning general-purpose pretrained models with a particular training supervision. |
| Approach: | They propose a method for learning sentence embeddings via contrastive learning between sentences and related entities. |
| Outcome: | The proposed method outperforms baseline methods in multilingual settings on a variety of tasks. |
Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that pretraining with an artificial language with nesting dependency structure provides some knowledge transferable to natural language. |
| Approach: | They propose to pretrain artificial languages with structural properties that mimic natural language and then test their performance on downstream tasks. |
| Outcome: | The proposed language models show strong performance across languages and languages. |
Large Vocabulary Size Improves Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have investigated the properties of internal layers in large language models, but no studies have defined the vocabulary size. |
| Approach: | They propose a method to use a new vocabulary instead of the pre-defined one in a continual training scenario. |
| Outcome: | The proposed method outperforms the model with the pre-defined vocabulary in a continual training scenario. |
Designing the Business Conversation Corpus (D19-52)
Copied to clipboard
| Challenge: | Existing parallel corpora for machine translation of written text and monologues are limited. |
| Approach: | They propose to introduce a Japanese-English business conversation parallel corpus into machine translation training scenarios and show how it improves machine translation quality. |
| Outcome: | The proposed corpus is used in a Japanese-English business conversation training scenario and shows how it performs. |
mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for improving multilingual models only use entity information in pretraining and do not explicitly use entities in downstream tasks. |
| Approach: | They propose to leverage Wikipedia entity representations for downstream tasks . they train a multilingual language model with 24 languages with entity representation . |
| Outcome: | The proposed model outperforms word-based models in cross-lingual transfer tasks. |
Revisiting the Context Window for Cross-lingual Word Embeddings (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to mapping-based cross-lingual word embeddings are based on the assumption that the source and target embeddable spaces are structurally similar. |
| Approach: | They propose to use different context windows to evaluate bilingual word embeddings in various languages, domains, and tasks. |
| Outcome: | The size of both the source and target window improves bilingual lexicon induction, especially on frequent nouns. |
Data Augmentation with Unsupervised Machine Translation Improves the Structural Similarity of Cross-lingual Word Embeddings (2021.acl-srw)
Copied to clipboard
| Challenge: | Unsupervised cross-lingual word embedding methods learn a linear transformation matrix that maps two monolingual embeddable spaces that are separately trained with monolingual corpora. |
| Approach: | They propose a method that maps two monolingual embedding spaces that are separately trained with monolingual corpora using a pseudo-parallel corpus. |
| Outcome: | The proposed method outperforms other methods given the same amount of data and shows that using a pseudo-parallel corpus makes the source and target corpora (partially) parallel . |
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing language adaptation methods overlook the benefits of cross-lingual supervision. |
| Approach: | They propose a language adaptation tuning method that augments target corpus with English entity names and trains the model using left-to-right language modeling. |
| Outcome: | The proposed method shows significant performance gains across non-English languages. |