AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, personal AI assistants on the phone and AR glasses can assist our daily life in addressing our questions like "how to adjust the date for this watch?" |
| Approach: | They propose a task that asks a question about affordance of items in our daily life . they construct a dataset that contains 3.2k multimodal questions on 1.6k video segments . |
| Outcome: | The proposed task outperforms baseline methods while still having room for improvement in the future. |
Similar Papers
GazeVQA: A Video Question Answering Dataset for Multiview Eye-Gaze Task-Oriented Collaborations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the use of exocentric and egocentric videos in video question answering are focusing on eye-gaze information. |
| Approach: | They propose a task-oriented VQA dataset that captures eye-gaze information . they propose assisting models that ground the perceptual input into semantic information based on three different answer types . |
| Outcome: | The proposed model can ground the perceptual input into semantic information while reducing ambiguities. |
Multi-Tenant Optimization For Few-Shot Task-Oriented FAQ Retrieval (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Business-specific Frequently Asked Questions (FAQs) are important part of many taskoriented dialog systems today. |
| Approach: | They propose a method to scale multi-tenant FAQ applications in real-world context . they use query-Question similarity and few-shot intent detection techniques . |
| Outcome: | The proposed approach scales multi-tenant FAQ applications in real-world context . it uses fine-tuning of the last layer in sentence Bi-Encoders along with tenant-specific weight switching. |
EgoMemory: Memory-Augmented Personalized Retrieval for Long-Context Egocentric Video (2026.findings-acl)
Copied to clipboard
Yuanmin Tang, Jue Zhang, Xiaoting Qin, Jing Yu, Meikang Qiu, Gaopeng Gou, Gang Xiong, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Wu
| Challenge: | Existing egocentric video datasets do not support the personalization and long-context reasoning required for episodic memory retrieval. |
| Approach: | They propose a benchmark framework that uses MLLMs and reflective Chain-of-Thought to ground user queries in personal memory explicitly. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on three benchmarks . it can be used to generate detailed target video descriptions in long-context contexts based on user-specific object annotations enriched with user-specified object annotation data . |
Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation (P18-1)
Copied to clipboard
| Challenge: | Recent advances on abstractive summarization have allowed substantial improvements in the quality of the model, but there is still scope for improvement. |
| Approach: | They propose novel multi-task architectures with high-level layer-specific sharing across multiple encoder and decoder layers of the three tasks and soft-sharing mechanisms. |
| Outcome: | The proposed model improves on the CNN/DailyMail and Gigaword datasets and on the DUC-2002 transfer setup. |
Interactive Query-Assisted Summarization via Deep Reinforcement Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing systems that can perform interactive summarization cannot ingest the full document set or operate at sufficient speed for interactivity. |
| Approach: | They propose two deep reinforcement learning models for interactive summarization task . they use interactive session state and history to refrain from redundancy . |
| Outcome: | The proposed model improves informativeness while preserving positive user experience. |
From Short Video to Clickable Search: RLVR-Enabled Listwise Query Suggestion with Retrieval-Augmented Context (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing methods to generate short-video bottom-bar queries are largely retrieval-based. |
| Approach: | They propose to reformulate the task as one-shot list generation, producing multiple queries per video . they also build multi-query ground truth from exposure and CTR logs, and redesign offline evaluation . |
| Outcome: | The proposed system yields strong offline and online improvements . it is deployed on Kuaishou to serve hundreds of millions of users daily . |
SA-DETR:Span Aware Detection Transformer for Moment Retrieval (2025.coling-main)
Copied to clipboard
| Challenge: | Moment Retrieval aims to locate video segments related to text. |
| Approach: | They propose a method that leverages the importance of instance related span anchors . they initialize span anchor using instance related fuse token and supervise them with GT labels . |
| Outcome: | The proposed method achieves competitive results on QVHighlights, Charades-STA and TACoS. |
ViLL-E: Video LLM Embeddings for Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Video Large Language Models excel at video understanding tasks where outputs are textual . however, they underperform specialized embedding-based models in Retrieval tasks . |
| Approach: | They propose a video-LLM-based model with an embedding generation mechanism that allows the model to "think longer" for complex videos and stop early for easy ones. |
| Outcome: | The proposed model outperforms specialized embedding-based models in video understanding tasks while remaining competitive on VideoQA tasks. |
DPDV: Dual-Pathway and Dual-View Representation Learning for Bridging Information Asymmetry in Text-Video Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for text-based person anomaly search fail to address the pose-semantic gap . asymmetric cross-modal information poses a challenge to accurately establishing retrieval relationships . |
| Approach: | They propose a video retrieval framework that partitions visual features into two categories based on relevance to the text query and performs effective interaction. |
| Outcome: | The proposed framework achieves leading retrieval performance on five benchmark datasets. |
AlanaVLM: A Multimodal Embodied AI Foundation Model for Egocentric Video Understanding (2024.findings-emnlp)
Copied to clipboard
Alessandro Suglia, Claudio Greco, Katie Baker, Jose Part, Ioannis Papaioannou, Arash Eshghi, Ioannis Konstas, Oliver Lemon
| Challenge: | Current Vision-Language Models (VLMs) focus on third-person view videos, neglecting the richness of egocentric perceptual experience. |
| Approach: | They propose to use the Egocentric Video Understanding Dataset (EVUD) to train VLMs on video captioning and question answering tasks specific to egocentric videos. |
| Outcome: | The proposed model outperforms open-source models including strong Socratic models using GPT-4 as a planner by 3.6% and outperformed Claude 3 and Gemini Pro Vision 1.0. |