Papers by Song-Chun Zhu
v-HUB: A Benchmark for Video Humor Understanding from Vision and Sound (2026.acl-long)
Copied to clipboard
Zhengpeng Shi, Yanpeng Zhao, Jianqun Zhou, Yuxuan Wang, Qinrong Cui, Wei Bi, Song-Chun Zhu, Bo Zhao, Zilong Zheng
| Challenge: | Humor enriches our daily lives and appears in many forms, from jokes and cartoons to comedies and viral videos. |
| Approach: | They introduce a video humor understanding benchmark to test their ability to understand humor from visual cues. |
| Outcome: | The proposed video humor understanding benchmark is based on a collection of short videos . it features rich annotations and a study of environmental sound that can enhance humor . |
SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on identifying entities' relations from the semantics of dialogues-they utilize either the attention mechanism or a refined token graph to locate informative words. |
| Approach: | They propose a sequential structure prediction task to incrementally parse SocAoG for dynamic inference upon any incoming utterance. |
| Outcome: | Empirical results show that the proposed model infers social relations more accurately than the state-of-the-art methods. |
LangSuit·E: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have shown inspiring achievements in constructing autonomous agents that rely on language descriptions as inputs. |
| Approach: | They propose a flexible and simulation-free testbed that simulates 6 representative embodied tasks in textual embodies. |
| Outcome: | The proposed testbed offers adaptability to diverse environments without multiple simulation engines and allows easy customization of communication and action strategies. |
GRICE: A Grammar-based Dataset for Recovering Implicature and Conversational rEasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | a grammar-based dialogue dataset, GRICE, is designed to bring implicature into pragmatic reasoning in conversations . implicature recovery is a key component of open-ended dialogue reasoning . |
| Approach: | They propose a grammar-based dialogue dataset to bring implicature into pragmatic reasoning . they use a hierarchical grammar model to generate the entire dataset . |
| Outcome: | The proposed model shows a significant performance gap between baseline methods and human models . the model shows an overall performance boost in conversational reasoning . |
ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection (2025.findings-acl)
Copied to clipboard
Jiaqi Li, Xinyi Dong, Yang Liu, Zhizhuo Yang, Quansen Wang, Xiaobo Wang, Song-Chun Zhu, Zixia Jia, Zilong Zheng
| Challenge: | ReflectEvo-460k is a large-scale, comprehensive, self-generated reflection dataset with broadened instructions and diverse multi-domain tasks. |
| Approach: | They propose a pipeline that iteratively generates self-reflection for self-training and a large-scale reflection dataset with broadened instructions and diverse multi-domain tasks. |
| Outcome: | The proposed pipeline improves Llama-3 reasoning ability by up to 71.2% and Mistral by upto 44.4%. |
Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for solving geometric problems are either small in scale or not publicly available. |
| Approach: | They propose a large-scale benchmark for geometric problem solving using formal language and symbolic reasoning. |
| Outcome: | The proposed approach parses geometry problems into formal language and performs symbolic reasoning step by step. |
RulE: Knowledge Graph Reasoning with Rule Embedding (2024.findings-acl)
Copied to clipboard
| Challenge: | Knowledge graph reasoning is an important problem for knowledge graphs. |
| Approach: | They propose a framework that leverages logical rules to enhance KG reasoning by learning rule embeddings from existing triplets and first-order rules. |
| Outcome: | The proposed framework outperforms existing embedding-based and rule-based methods on multiple benchmarks. |
Structured Attention for Unsupervised Dialogue Structure Induction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using structured attention, a model can learn dialogue structure in unsupervised fashion. |
| Approach: | They propose to incorporate structured attention layers into a Variational Recurrent Neural Network model with discrete latent states to learn dialogue structure in an unsupervised fashion. |
| Outcome: | The proposed model learns semantic structures similar to templates used to generate a dialogue corpus on two-party datasets and on multi-party dialogues, disentangling dialogues without human annotation. |
SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for Multimodal Large Language Models (MLLMs) have been lacking due to the rich nature of social interaction. |
| Approach: | They propose a video benchmark to evaluate MLLMs' capabilities across social scene understanding, social state reasoning, and social dynamics prediction. |
| Outcome: | The proposed benchmarks evaluate MLLMs' capabilities across social scene understanding, social state reasoning, and social dynamics prediction tasks. |
CrossVQA: Scalably Generating Benchmarks for Systematically Testing VQA Generalization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing visual question answering models leverage spurious biases and take shortcuts to improve performance. |
| Approach: | They propose a semi-automatic framework for generating disentangled shifts by introducing a controllable visual question-answer generation module that generates highly-relevant question-announcer pairs with the desired dataset style. |
| Outcome: | The proposed framework generates highly-relevant and diverse question-answer pairs with the desired dataset style. |
JurisBench: A Deep Benchmark for Assessing Large Language Models in Professional Legal Practice (2026.acl-long)
Copied to clipboard
Ziang Chen, Guannan Li, Fanlin Ji, Yipeng Kang, Jiaqi Li, Muhan Zhang, Yangtao Zhang, Li Tianjiao, Jiannan Wang, Xin Guo, Song-Chun Zhu, Bin Ling
| Challenge: | Existing legal benchmarks evaluate isolated tasks or exam-style questions, failing to capture the procedural interdependencies and adjudicative rigor inherent in professional practice. |
| Approach: | They propose a vertical, depth-oriented, domain-specific benchmark to evaluate Large Language Models (LLMs) in Chinese civil litigation. |
| Outcome: | The proposed benchmarks show that large language models exhibit an "illusion of competence" the results highlight a critical gap between fluent linguistic output and judicial reliability . |
Mind the Context: The Impact of Contextualization in Neural Module Networks for Grounding Visual Referring Expressions (2021.emnlp-main)
Copied to clipboard
| Challenge: | Prior implementations of NMN use pre-defined and fixed textual inputs in their module instantiation. |
| Approach: | They propose to parameterize the module arguments to reduce the number of modules in NMN by up to 75% without any loss in performance. |
| Outcome: | The proposed model outperforms the state-of-the-art model on CLEVR-Ref+ dataset with +8.1% improvement in accuracy and +4.3% on full test set. |
Words Aren’t Enough, Their Order Matters: On the Robustness of Grounding Visual Referring Expressions (2020.acl-main)
Copied to clipboard
| Challenge: | Visual referring expression recognition is a task that requires natural language understanding in the context of an image. |
| Approach: | They propose to use contrastive learning and multi-task learning to increase the robustness of ViLBERT, the current state-of-the-art model for this task. |
| Outcome: | The proposed methods are 12% to 23% lower in performance than the established progress for this task. |
Enhancing LLM-Based Social Bot via an Adversarial Learning Framework (2025.emnlp-main)
Copied to clipboard
| Challenge: | Social media platforms provide an ideal testbed for large language models that exhibit human-like behavior. |
| Approach: | They propose an LLM-based social **Bot that enhances human-like generative capabilities through an adversarial learning framework. |
| Outcome: | The proposed framework generates human-like content aligned with diverse user profiles . it exhibits strong social responsiveness, more accurately modeling opinion dynamics . |