Papers by Pavel Denisov
Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in language modeling have led to the emergence of large language models capable ofvarious natural language processing tasks. |
| Approach: | They propose a multi-instructional training approach that integrates a large language model with a speech encoder to harness the capabilities of LLMs for speech recognition and beyond. |
| Outcome: | The proposed model can be trained and aligned with a multilingual LLM on 1900 hours of transcribed data from 139 languages. |
ADVISER: A Toolkit for Developing Multi-modal, Multi-domain and Socially-engaged Conversational Agents (2020.acl-demos)
Copied to clipboard
Chia-Yu Li, Daniel Ortega, Dirk Väth, Florian Lux, Lindsey Vanderlyn, Maximilian Schmidt, Michael Neumann, Moritz Völkel, Pavel Denisov, Sabrina Jenne, Zorica Kacarevic, Ngoc Thang Vu
| Challenge: | Existing toolkits for developing dialog systems are limited to core components and do not support multi-modal processing and social signals. |
| Approach: | They propose to use ADVISER to develop multi-modal dialog agents using multi-text and social signals. |
| Outcome: | The proposed toolkit is flexible, easy to use, and easy to extend for linguists and cognitive scientists, thereby providing a flexible platform for collaborative research. |