Learning Sparsity for Effective and Efficient Music Performance Question Answering (2025.acl-short)
Copied to clipboard
| Challenge: | Existing Music AVQA methods rely on dense and unoptimized representations, leading to inefficiencies in the isolation of key information, reduction of redundancy, and prioritization of critical samples. |
| Approach: | They propose a sparse learning framework specifically designed for Music AVQA to address these challenges. |
| Outcome: | The proposed framework reduces training time by 28.32% while maintaining accuracy while maintaining state-of-the-art performance on the Music AVQA datasets. |
Similar Papers
Learning Musical Representations for Music Performance Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for audio-visual learning fail to consider the distinctive characteristics of instruments and music. |
| Approach: | They propose to integrate multimodal interactions within the context of music data and annotate and release rhythmic and music sources in the current music datasets to enable the model to learn music characteristics. |
| Outcome: | The proposed model can learn music characteristics from the current music datasets and align its predictions with the temporal dimension. |
Music Audio-Visual Question Answering Requires Specialized Multimodal Designs (2026.findings-acl)
Copied to clipboard
Wenhao You, Xingjian Diao, Wenjun Huang, Chunhui Zhang, Keyi Kong, Weiyi Wu, Chiyu Ma, Zhongyu Ouyang, Tingxuan Wu, Ming Cheng, Soroush Vosoughi, Jiang Gui
| Challenge: | Music audio-visual question answering presents unique challenges with dense audio-visual content, intricate temporal dynamics, and the need for domain-specific knowledge. |
| Approach: | They analyze Music AVQA datasets and analyze their results to identify key design patterns . they propose concrete future directions for incorporating musical priors . |
| Outcome: | The proposed architectures are critical for success in Music AVQA, the authors argue . they suggest concrete future directions for incorporating musical priors . |
SaSR-Net: Source-Aware Semantic Representation Network for Enhancing Audio-Visual Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing AVQA methods often fail to link sound-producing objects in the video with the audio-visual information. |
| Approach: | They introduce a source-aware semantic representation network for AVQA . they use source-wise learnable tokens to capture and align audio-visual elements with the question . |
| Outcome: | The proposed model outperforms state-of-the-art models on the Music-AVQA and AVQA-Yang datasets. |
MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering (2025.naacl-industry)
Copied to clipboard
| Challenge: | Question Answering (QA) and Visual Question Answers (VQA) are well-studied problems in the language and vision domain. |
| Approach: | They propose a question-answer generation framework that learns attention across multiple sources and decodes this information for robust and unbiased answer generation. |
| Outcome: | The proposed framework can handle thousands of question types and scale to scale. |
FIGMA: Towards FIne-Grained Music retrievAl (2026.acl-long)
Copied to clipboard
| Challenge: | Existing music retrieval models fail to retrieve fine-grained musical attributes when using coarse semantic queries. |
| Approach: | They propose a multi-view contrastive architecture that captures high-level semantic context and fine-grained musical attributes within a unified representation space. |
| Outcome: | The proposed method outperforms existing CLAP-based music retrieval models on multiple benchmarks. |
Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in multimodal reasoning overlook the audio modality. |
| Approach: | They propose a large-scale audio language model for deep reasoning that leverages a multitask audio dataset. |
| Outcome: | The proposed model performs well across key benchmarks including MMAU-mini, AIR-Bench chat/foundation, and MELD. |
SparseFlow: Accelerating Transformers by Sparsifying Information Flows (2024.acl-long)
Copied to clipboard
| Challenge: | SparseFlow is an efficient method to sparsify the dense information flows within transformers. |
| Approach: | They propose a method to sparsify the dense pathways of token representations across all transformer blocks by parameterizing them to be sparse. |
| Outcome: | The proposed method reduces computational costs by half on average without compromising task accuracy. |
MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Free (2026.findings-acl)
Copied to clipboard
Yishu Lei, Shuwei He, Hu Jing, Dan Zhang, Xianlong Luo, Danxiang Zhu, Shikun Feng, Rui Liu, Jingzhou HE, Yu Sun, Hua Wu, Haifeng Wang
| Challenge: | Existing research on Large Language Models (LLMs) limited to textual input modality . acoustic information is intrinsically heterogeneous, entangling attributes such as speech, music, and environmental context. |
| Approach: | They propose a sparse Mixture-of-Experts architecture to decouple acoustic information by routing audio tokens to specialized experts. |
| Outcome: | The proposed architecture outperforms existing models on audio semantic and paralinguistic tasks while retaining shared experts for global context. |
Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization (2023.findings-eacl)
Copied to clipboard
Aishwarya Agrawal, Ivana Kajic, Emanuele Bugliarello, Elnaz Davoodi, Anita Gergely, Phil Blunsom, Aida Nematzadeh
| Challenge: | Visual question answering (VQA) is a task of answering open-ended questions about images. |
| Approach: | They evaluate two vision-and-language (V&L) models under different settings . they find they tend to learn to solve the benchmark rather than the skills required by VQA . |
| Outcome: | The proposed models exhibit poor generalization under out-of-distribution settings. |
Jamendo-MT-QA: A Benchmark for Multi-Track Comparative Music Question Answering (2026.findings-acl)
Copied to clipboard
Junyoung Koh, Jaeyun Lee, Soo Yong Kim, Gyu Hyeong Choi, Jung In Koh, Jordan Phillips, Yeonjin Lee, Min Song
| Challenge: | Existing benchmarks for music question answering do not systematically evaluate reasoning across tracks. |
| Approach: | They propose a dataset and benchmark for multi-track comparative question answering . they construct 36,519 comparative QA items over 12,173 track pairs . |
| Outcome: | The proposed dataset and benchmark for multi-track comparative question answering is based on the Jamendo-QA dataset. |