Papers by Xiujie Song
A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) are hardly comprehensively evaluated for their cognitive abilities. |
| Approach: | They propose to evaluate high-level cognitive abilities of Large Vision-Language Models (LVLMs) using images with rich semantics. |
| Outcome: | The proposed evaluation benchmark consists of 251 images along with comprehensive annotations. |
One Battle After Another: Probing LLMs’ Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for instruction-following in multi-topic dialogues are limited to a fixed number of turns, susceptible to saturation and failing to account for users’ interactive experience. |
| Approach: | They propose a framework featuring a three-layer tracking mechanism and a query synthesis agent to mimic sequential user behaviors. |
| Outcome: | The proposed framework outperforms existing benchmarks in the evaluation of instruction following in multi-topic dialogues and demonstrates deficiencies in failure recovery and fine-grained instruction following. |
Transferable and Efficient: Unifying Dynamic Multi-Domain Product Categorization (2023.acl-industry)
Copied to clipboard
| Challenge: | e-commerce platforms are encountering increasingly complex product categorization scenarios . multiple business domains correspond to different category taxonomies, with different depths and distinct literal expressions of category names. |
| Approach: | They propose a taxonomy-agnostic framework that calculates semantic relatedness between product titles and category names in the vector space. |
| Outcome: | The proposed framework outperforms strong baselineson three dynamic multi-domain product categorization tasks. |