Papers by Tetsuya Sakai
Evaluating Evaluation Measures for Ordinal Classification and Ordinal Quantification (2021.acl-long)
Copied to clipboard
| Challenge: | Ordinal Classification (OC) tasks require ordinal classes, not nominal ones, to be evaluated. |
| Approach: | They use data from the SemEval and NTCIR communities to clarify evaluation measures for Ordinal Classification and Ordinal Quantification tasks. |
| Outcome: | The evaluation measures for Ordinal Classification (OC) and Ordinal Quantification (OQ) tasks are ordinal, not nominal. |
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmented Generation (2025.findings-naacl)
Copied to clipboard
Yiruo Cheng, Kelong Mao, Ziliang Zhao, Guanting Dong, Hongjin Qian, Yongkang Wu, Tetsuya Sakai, Ji-Rong Wen, Zhicheng Dou
| Challenge: | Existing research focuses on single-turn RAG, leaving a gap in addressing multi-turn conversations . a new benchmark is designed to assess RAG systems in realistic multi-turned conversations based on Wikipedia . |
| Approach: | They propose a large-scale benchmark to assess RAG systems in multi-turn contexts . CORAL includes diverse information-seeking conversations automatically derived from Wikipedia . authors propose unified framework to standardize various conversational RAG methods . |
| Outcome: | The proposed framework supports three core tasks of conversational RAG: passage retrieval, response generation, and citation labeling. |
ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models (2024.emnlp-main)
Copied to clipboard
Yuxiang Zhang, Jing Chen, Junjie Wang, Yaxin Liu, Cheng Yang, Chufan Shi, Xinyu Zhu, Zihao Lin, Hanwen Wan, Yujiu Yang, Tetsuya Sakai, Tian Feng, Hayato Yamana
| Challenge: | Currently, tool-augmented large language models (LLMs) only achieve total scores of 45.3 and 37.0, respectively, on a scale of 100. |
| Approach: | They propose a multi-level diagnostic process to assess the LLM's hallucinations through two perspectives: depth and breadth. |
| Outcome: | The proposed diagnostic process assesses the hallucinations of large language models through two perspectives: depth and breadth. |
ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval (2024.emnlp-main)
Copied to clipboard
| Challenge: | a conversational search system requires accurate interpretation of user intent from complex multi-turn contexts. |
| Approach: | They propose a dual-learning approach that adapts LLMs for retrieval via contrastive learning while enhancing the complex session understanding through masked instruction tuning. |
| Outcome: | The proposed approach outperforms existing retrieval methods on five conversational search benchmarks. |
Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective (2022.emnlp-main)
Copied to clipboard
Ping Yang, Junjie Wang, Ruyi Gan, Xinyu Zhu, Lin Zhang, Ziwei Wu, Xinyu Gao, Jiaxing Zhang, Tetsuya Sakai
| Challenge: | Existing approaches to zero-shot learning are format-agnostic and can address new learning tasks without additional training. |
| Approach: | They propose a new paradigm for zero-shot learning that is format agnostic and compatible with any format and applicable to a list of language tasks. |
| Outcome: | The proposed model shows state-of-the-art performance on several benchmarks and produces satisfactory results on tasks such as text classification and commonsense reasoning. |
MIRTT: Learning Multimodal Interaction Representations from Trilinear Transformers for Visual Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing bilinear methods focus on inter-modality information between images and questions . existing models focus on the interaction between images, questions, and images . |
| Approach: | They propose a trilinear interaction framework that incorporates attention mechanisms for capturing inter-modality and intra-modal relationships. |
| Outcome: | The proposed model outperforms bilinear models on the Visual7W Telling task and VQA-1.0 Multiple Choice task and outperformed baselines on the VQA, TDIUC and GQA datasets. |
e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent omni-modal embeddings rely heavily on implicit alignment from pretrained visionlanguage models. |
| Approach: | They propose a lightweight explicit alignment recipe that adapts off-the-shelf VLMs into robust omni-modal embedding models. |
| Outcome: | The proposed model improves on MMEB-V2 and AudioCaps with a lightweight explicit alignment recipe. |
Evaluating the Effects of Embedding with Speaker Identity Information in Dialogue Summarization (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for automatic dialogue summarization do not take into account speaker identity information, but instead use sinusoidal functions to embed speaker information at the less informative part of the position embedding. |
| Approach: | They propose to embed speaker identity information into a dialogue transcript encoder to address this issue and reduce the "who said what"-related errors. |
| Outcome: | The proposed method improves the convergence of the model in training and increases the average ROUGE scores of the generated summaries in comparison to existing methods. |
A Siamese CNN Architecture for Learning Chinese Sentence Similarity (2020.aacl-srw)
Copied to clipboard
| Challenge: | a deep neural architecture is used to learn a semantic similarity metric between two sentences . traditional methods of learning sentence similarity are based on the word level, which may not be sufficient. |
| Approach: | They propose a deep neural architecture which uses siamese convolutional neural network sharing model parameters to learn a semantic similarity metric between two sentences. |
| Outcome: | The proposed architecture outperforms baselines in similarity metrics for Chinese sentences by 8.7 points. |
LayerConnect: Hypernetwork-Assisted Inter-Layer Connector to Enhance Parameter Efficiency (2022.coling-1)
Copied to clipboard
| Challenge: | Existing parameter-efficient methods focus on reducing trainable parameters but neglect the inference speed, which limits the ability to deploy PLMs. |
| Approach: | They propose to use a hypernetwork-assisted inter-layer connector to enhance inference efficiency by tuning parameters inside a linear connector between two Transformer layers. |
| Outcome: | The proposed model reduces model parameters to 11.75% while preserving performance degradation to less than 5%. |