Papers by Te-Lin Wu
HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing taxonomies have limited coverage due to expensive manual curation process. |
| Approach: | They propose an algorithm that expands existing taxonomies to preserve their structure in a more expressive hyperbolic embedding space and learns to represent concepts and their relations with a hyperbolical Graph Neural Network. |
| Outcome: | The proposed algorithm outperforms baseline models with representation learning in a Euclidean feature space and achieves state-of-the-art performance on the taxonomy expansion benchmarks. |
LegalDiscourse: Interpreting When Laws Apply and To Whom (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in NLP and information retrieval have already enabled practical applications. |
| Approach: | They propose a 'discourse' taxonomy for span-and-relation parsing of legal texts . they use a dataset of 602 state-level law paragraphs with 3,715 discourse spans and 1,671 relations to investigate the increase in liquor licenses and decrease in applicable laws. |
| Outcome: | The proposed model performs poorly at span identification and relation classification, but lags far below human level. |
VDebugger: Harnessing Execution Feedback for Debugging Visual Programs (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Visual programs are executable code generated by large language models to address visual reasoning problems. |
| Approach: | They propose a critic-refiner framework that localizes and debugs visual programs by tracking execution step by step. |
| Outcome: | The proposed framework detects and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy. |
ACQUIRED: A Dataset for Answering Counterfactual Questions In Real-Life Videos (2023.emnlp-main)
Copied to clipboard
Te-Lin Wu, Zi-Yi Dou, Qingyuan Hu, Yu Hou, Nischal Chandra, Marjorie Freedman, Ralph Weischedel, Nanyun Peng
| Challenge: | despite its importance, there are few datasets that cover multimodal counterfactual reasoning . a dataset focusing on this area is limited because of its limited coverage over synthetic environments . |
| Approach: | They develop a video question answering dataset that provides questions on multimodal reasoning . they ask questions about counterfactual hypotheses over visual events . |
| Outcome: | The proposed dataset shows a significant performance gap between models and humans . it provides questions that span physical, social, and temporal dimensions . |
SIMMC-VR: A Task-oriented Multimodal Dialog Dataset with Situated and Immersive VR Streams (2023.acl-long)
Copied to clipboard
Te-Lin Wu, Satwik Kottur, Andrea Madotto, Mahmoud Azab, Pedro Rodriguez, Babak Damavandi, Nanyun Peng, Seungwhan Moon
| Challenge: | Existing models lack a large-scale benchmark to capture userâassistant interactions . et al., 2022: 145-160. |
| Approach: | They propose a video-grounded task-oriented dialog dataset that captures real-world AI-assisted user scenarios in VR. |
| Outcome: | The proposed dataset captures real-world AI-assisted user scenarios in VR. |
Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on active object grounding from an egocentric perspective is focusing on localizing and tracking active objects that undergo major state change as a result of human actions/interactions to the environment without being told exactly what/where to ground. |
| Approach: | They propose to use a narrated egocentric video dataset to localize and track active objects that undergo major state change as a result of human actions/interactions to the environment without being told exactly what/where to ground. |
| Outcome: | The proposed framework leads to 54% improvement in standard metrics on the TREK-150-OPE-Det localization + tracking task, and >7% improvement in all standard metrics. |
Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals (2022.acl-long)
Copied to clipboard
| Challenge: | Current machine learning methods are incapable of efficiently utilizing multimodal information. |
| Approach: | They propose to use text-and-image alignment to improve machine learning's performance on multimodal event sequencing. |
| Outcome: | The proposed models perform significantly worse than humans on multimodal event sequencing than humans. |
Learning Action Conditions from Instructional Manuals for Instruction Understanding (2023.acl-long)
Copied to clipboard
| Challenge: | a weakly supervised task is proposed to extract mentions of preconditions and postconditions of actions from instructional manuals. |
| Approach: | They propose a task dubbed action condition inference which extracts mentions of preconditions and postconditions of actions from instructional manuals. |
| Outcome: | The proposed approach improves on the existing models, but still far behind human performance. |
Character-centric Story Visualization via Visual Planning and Token Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Story visualization is a task that requires machines to understand long text inputs and produce a globally consistent image sequence that illustrates the contents of the story. |
| Approach: | They propose to augment VQ-VAE with a text-to-visual-token (transformer) architecture to enable multiple image generation based on a complete story. |
| Outcome: | The proposed method excels at preserving characters and produces higher quality image sequences compared with baselines. |