Papers with TSG
Rethinking the Video Sampling and Reasoning Strategies for Temporal Sentence Grounding (2022.findings-emnlp)
Copied to clipboard
Jiahao Zhu, Daizong Liu, Pan Zhou, Xing Di, Yu Cheng, Song Yang, Wenzheng Xu, Zichuan Xu, Yao Wan, Lichao Sun, Zeyu Xiong
| Challenge: | Existing methods for temporal sentence grounding ignore two crucial issues . 1) Boundary-bias: the video downsampling process may lose these two frames . 2) Reasoning-biases: such incorrect new boundary frames lead to the reasoning bias . |
| Approach: | They propose a siamese sampling mechanism to generate additional contextual frames . they use a reasoning strategy to learn the inter-relationship among these frames a . |
| Outcome: | Extensive experiments demonstrate the effectiveness of a new siamese sampling network on three challenging datasets. |
An Adaptive Prompt Generation Framework for Task-oriented Dialogue System (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing black-box large language models (LLMs) have excellent performance in task-oriented dialogue (TOD) tasks, but obtaining suitable prompts for specific tasks is challenging. |
| Approach: | They propose a black-box large language model that generates domain and slot information in the belief state, which serves as prior knowledge for subsequent prompt generation. |
| Outcome: | The proposed framework outperforms existing prompting methods on the MultiWOZ 2.0 dataset. |
Grammar-based Decoding for Improved Compositional Generalization in Semantic Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have been successful in semantic parsing tasks but struggle on out-of-distribution data. |
| Approach: | They propose to use a large-scale dialogue dataset to evaluate compositional generalization of semantic parsing. |
| Outcome: | The proposed model outperforms BART- and T5-based models on the SMCalflow-CS dataset on the zero-shot learning task. |
Learning to Focus on the Foreground for Temporal Sentence Grounding (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for temporal sentence grounding do not capture subtle details of small objects. |
| Approach: | They propose a detection-free framework for temporal sentence grounding that learns to locate foreground regions related to the query in consecutive frames. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three challenging datasets. |
Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on temporal sentence grounding rely on expensive video-query paired annotations . despite this, there are no ground-truth annotations in the current work . |
| Approach: | They propose to use paired video-query and segment boundary annotations to generate temporal sentence grounding without training. |
| Outcome: | The proposed model outperforms existing unsupervised methods and beats supervised ones on two challenging datasets. |
Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamic Audio-Visual Scenarios (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Audio-visual question answering requires multistep spatio-temporal reasoning over multimodal contexts. |
| Approach: | They propose a new target-aware joint spatio-temporal grounding network for audio-visual question answering . the proposed system integrates audio-vision fusion and question-awful temporal grounding into one module . |
| Outcome: | The proposed method over existing state-of-the-art methods is effective over existing methods . it can focus on audio-visual cues relevant to the query subject by utilizing explicit semantics from the question . |
Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to learn effective alignment between vision and language features are insufficient in practice due to complicated multi-step reasoning. |
| Approach: | They propose an iterative alignment network which iterates inter- and intra-modal features within multiple steps for more accurate grounding. |
| Outcome: | The proposed model performs better than the state-of-the-arts on three challenging benchmarks. |