Papers by Shota Sasaki
A Single Linear Layer Yields Task-Adapted Low-Rank Matrices (2024.lrec-main)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) is a parameter-efficient fine-tuning method that updates initial weight matrix W0 with a delta matrix W . |
| Approach: | They propose a method that updates initial weight matrix W0 with a delta matrix W consisting of two low-rank matrices A and B. |
| Outcome: | The proposed method maintains a performance on par with LoRA despite the fact that the trainable parameters of CondLoRA are fewer than those of LoRA. |
Cross-Lingual Learning-to-Rank with Shared Representations (N18-2)
Copied to clipboard
| Challenge: | Cross-lingual information retrieval (CLIR) is a document retrieval task where the documents are written in a language different from that of the user's query. |
| Approach: | They propose a large-scale dataset derived from Wikipedia to support CLIR research in 25 languages. |
| Outcome: | The proposed model can improve the results of Swahili-English CLIR in Japanese and Japanese. |
The Impact of Integration Step on Integrated Gradients (2024.eacl-srw)
Copied to clipboard
| Challenge: | Integrated gradients (IG) are a powerful tool for explaining the internal structure of a language model. |
| Approach: | They propose to customize the step count for each instance to minimize the error. |
| Outcome: | The optimal number of steps to maintain minimal error varies from instance to instance. |
Preventing Critical Scoring Errors in Short Answer Scoring with Confidence Estimation (2020.acl-srw)
Copied to clipboard
Hiroaki Funayama, Shota Sasaki, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Masato Mita, Kentaro Inui
| Challenge: | Recent Short Answer Scoring systems use Quadratic Weighted Kappa (QWK) but it is unsatisfactory when measuring their effectiveness in actual usage. |
| Approach: | They propose a task formulation of Short Answer Scoring (SAS) that matches actual usage and extracts as many scoring predictions that are not critical scoring errors (CSEs). |
| Outcome: | The proposed system predicts scores with zero critical scoring errors (CSEs) for 50% of test data at maximum by filtering out low-reliability predictions on the basis of a certain confidence estimation. |
Subword-based Compact Reconstruction of Word Embeddings (N19-1)
Copied to clipboard
| Challenge: | Existing word-based word embeddings are based on subword information and memory-shared embeddables. |
| Approach: | They propose a method for reconstructing pre-trained word embeddings using subword information using memory-shared embedds and a variant of the key-value-query self-attention mechanism. |
| Outcome: | The proposed method can imitate well-trained word embeddings in a small fixed space while preventing quality degradation across several linguistic benchmark datasets. |