Papers by Jiacheng Liu

From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation (2026.findings-acl)

Challenge:	Existing research to improve CoT efficiency falls into three categories, each with distinct limitations.
Approach:	They propose a training-free framework that addresses both dimensions of CoT reasoning by applying a progressive precision reduction strategy coupled with an entropy-based confidence mechanism for adaptive termination.
Outcome:	Empirical results show that the proposed framework achieves 11.3 efficiency gain without compromising accuracy.

Copied to clipboard

Pujun Zheng, Jiacheng Yao, Jinquan Zheng, Chenyang Gu, Guoxiu He, Jiawei Liu, Yong Huang, Tianrui Guo, Wei Lu

Robustness in NLP

Challenge:	Large language models (LLMs) are currently used to evaluate scientific papers by assigning an absolute score to each paper independently.
Approach:	They propose a comparison-native framework for paper evaluation that integrates comparison into both data construction and model learning.
Outcome:	The proposed framework achieves an average relative improvement of 21.8% over the strong baseline DeepReview-14B, while exhibiting robust generalization to five previously unseen datasets.

RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems (2026.findings-acl)

Copied to clipboard

Jiacheng Liu, Zichen Tang, Zhongjun Yang, Xinyi Hu, Xueyuan Lin, Linwei Jia, Ruofei Bai, Rongjin Li, Shiyao Peng, Haocheng Gao, Haihong E

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators (2025.findings-acl)

Challenge:	Existing tools to generate structured content for research tasks are limited in their ability to generate high-quality roadmaps.
Approach:	They propose a benchmark to evaluate the ability of large language models (LLMs) to generate high-quality roadmaps for solving complex research problems.
Outcome:	The proposed system can improve LLMs’ ability for roadmap generation while saving 84% of the time required by human experts.

Copied to clipboard

Jianling Li, ShangZhan Li, Zhenye Gao, Qi Shi, Yuxuan Li, Zefan Wang, Jiacheng Huang, WangHaojie WangHaojie, Jianrong Wang, Xu Han, Zhiyuan Liu, Maosong Sun

BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook (2026.acl-long)

Challenge:	Triton is a high-level Python-like programming language for building efficient GPU kernels.
Approach:	They propose a TritonBench benchmark that provides a comprehensive evaluation of Tritonic operators on widely deployed GPUs.
Outcome:	The proposed benchmarks show that current LLMs struggle to generate efficient Triton operators on widely deployed GPUs aligned with industry applications.

Copied to clipboard

Hao Gu, Lujun Li, Hao Wang, Lei Wang, Zheyu Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Sirui Han, Yike Guo

Text Clustering

Challenge:	Recent sparsity-aware binarization approaches can achieve sub-1-bit compression, but they face performance degradation, mask-management overhead, and limited hardware compatibility.
Approach:	They propose a binary quantization framework that leverages binary pattern clustering and weight transformation to overcome performance degradation and mask-management overhead.
Outcome:	The proposed framework achieves state-of-the-art compression (1.11–0.7 bits) it maintains high performance with only a 3.1% accuracy drop in zero-shot benchmarks while delivering a 1.6 speedup over FP16.

Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verification (2024.findings-acl)

Copied to clipboard

Soumya Sanyal, Tianyi Xiao, Jiacheng Liu, Wenya Wang, Xiang Ren

OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens (2025.acl-demo)

Challenge:	Existing evidence that humans make numerous inferences to understand discourse and text is not fully understood.
Approach:	They propose to use textual inference datasets with multi-sentence premises to solve the entailment verification problem.
Outcome:	The proposed model outperforms GPT-3.5 and rivals GPL-4 in EV tasks.

Copied to clipboard

Jiacheng Liu, Taylor Blanton, Yanai Elazar, Sewon Min, Yen-Sung Chen, Arnavi Chheda-Kothary, Huy Tran, Byron Bischoff, Eric Marsh, Michael Schmitz, Cassidy Trier, Aaron Sarnat, Jenna James, Jon Borchardt, Bailey Kuehl, Evie Yu-Yen Cheng, Karen Farley, Taira Anderson, David Albright, Carissa Schoenick, Luca Soldaini, Dirk Groeneveld, Rock Yuren Pang, Pang Wei Koh, Noah A. Smith, Sophie Lebrecht, Yejin Choi, Hannaneh Hajishirzi, Ali Farhadi, Jesse Dodge

FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging (2025.acl-long)

Challenge:	tracing language models' outputs back to training data is a problem because they are trained on text corpora with trillions of tokens . existing methods for tracers have not been scaled to work within this multi-trillion-token setting .
Approach:	They propose a system that traces language models' outputs verbatim back to training data . OLMOTRACE retrieves documents from the model's training data that contain exact matches .
Outcome:	The proposed system can find verbatim matches between LM output and training data . it can be used to explore fact checking, hallucination, and creativity of language models .

Copied to clipboard

Zichen Tang, Haihong E, Ziyan Ma, Haoyang He, Jiacheng Liu, Zhongjun Yang, Zihua Rong, Rongjin Li, Kun Ji, Qing Huang, Xinyang Hu, Yang Liu, Qianhe Zheng

Discourse-Aware Neural Extractive Text Summarization (2020.acl-main)

Challenge:	Compared to existing benchmarks, FinanceReasoning provides three key advancements: (1) credibility; (2) comprehensiveness; (3) numerical precision; (4) complexity; (5) complexity; and (6) complexity.
Approach:	They propose a benchmark to evaluate the reasoning capabilities of large reasoning models (LRMs) in financial numerical reasoning problems.
Outcome:	The proposed benchmark exceeds existing benchmarks in 67.8% of financial concepts and formulas and is credible, comprehensive, and challenging.

Copied to clipboard

Jiacheng Xu, Zhe Gan, Yu Cheng, Jingjing Liu

Information Extraction & Text Mining

Challenge:	Recent studies have shown that sentence-based extractive models result in redundant or uninformative phrases in the extracted summaries.
Approach:	They propose a discourse-aware neural summarization model that extracts sub-sentential discourse units as candidates for extractive selection on a finer granularity.
Outcome:	Experiments show that the proposed model outperforms state-of-the-art models on popular summarization benchmarks.

STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent Framework (2025.findings-acl)

Copied to clipboard

Wenhao Liu, Zhenyi Lu, Xinyu Hu, Jerry Zhang, Dailin Li, Jiacheng Cen, Huilin Cao, Haiteng Wang, Yuhan Li, Xie Kun, Dandan Li, Pei Zhang, Chengbo Zhang, Yuxiang Ren, Xiaohong Huang, Yan Ma

Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Index (2025.emnlp-main)

Challenge:	Existing datasets suffer from outdated and insufficient challenging content, neglecting human-like reasoning, and limited reliability due to single-LLM generation.
Approach:	They propose a human-in-the-loop, multi-agent data generation framework that integrates reasoning-dense filters, multiagent collaboration, and human mathematicians’ evaluations to ensure the reliability and quality of the dataset.
Outcome:	The proposed framework improves accuracy and quality of the 2,000-synthesized datasets by integrating reasoning-dense filters, multi-agent collaboration, and human mathematicians’ evaluations.

Copied to clipboard

Hao Xu, Jiacheng Liu, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi

Information Retrieval

Challenge:	Modern language models are trained on text data downsampled from massive text corpora like Common Crawl.
Approach:	They propose an efficient and scalable system that can make petabyte-level text corpora searchable by using the FM-index data structure.
Outcome:	The proposed system indexes 83TB of Internet text in 99 days with a single 128-core CPU node (or 19 hours if using 137 such nodes).

Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs (2026.acl-long)

Copied to clipboard

Binxing Xu, Hao Gu, Lujun Li, Hao Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Xintong Yang, Chao Li, Sirui Han, Yike Guo

Green & Sustainable NLP

Challenge:	Existing approaches to training LLMs at ultra-low precisions suffer from convergence instability and substantial training costs.
Approach:	They propose a progressive QAT framework with outlier channel splitting to address these issues . they use nested structure of integer quantization grids to enable a "train once, deploy any precision" paradigm .
Outcome:	The proposed framework outperforms baselines on both Llama2/3 and W2A16, with an 11 speedup over BF16.

LLMSurgeon: Diagnosing Data Mixture of Large Language Models (2026.acl-long)

Copied to clipboard

Yaxin Luo, Jiacheng Cui, Xiaohan Zhao, Xinyi Shang, Jiacheng Liu, Xinyue Bi, Zhaoyi Li, Zhiqiang Shen

Phrase Grounding by Soft-Label Chain Conditional Random Field (D19-1)

Challenge:	a lack of transparency in large language models makes auditing their "digital DNA" difficult.
Approach:	They propose a framework that casts DMS as an inverse problem under label-shift assumption . they propose LLMScan, a recipe-verifiable evaluation suite built from open-source LLMs .
Outcome:	The proposed framework casts DMS as an inverse problem under label-shift assumption . compared with existing frameworks, it recovers domain mixtures with high fidelity .

Copied to clipboard

Jiacheng Liu, Julia Hockenmaier

Unexpected Phenomenon: LLMs’ Spurious Associations in Information Extraction (2024.findings-acl)

Challenge:	Existing methods to ground entities depend on inference or non-differentiable losses.
Approach:	They propose a phrase grounding task that grounds entities to corresponding regions in an image . they use neural chain Conditional Random Fields to model dependencies among regions .
Outcome:	The proposed method is based on a dataset of the Flickr30k Entities dataset.

Copied to clipboard

Weiyan Zhang, Wanpeng Lu, Jiacheng Wang, Yating Wang, Lihan Chen, Haiyun Jiang, Jingping Liu, Tong Ruan

Information Extraction & Text Mining

Challenge:	Information extraction (IE) tasks require a limited number of example instructions to achieve effective performance.
Approach:	They propose two strategies to find spurious associations in large language models (LLMs) they use forward label extension and backward label validation to leverage extended labels to improve model performance.
Outcome:	The proposed methods improve performance on Chinese and English datasets and 9.55%, 11.42%, and 21.27% in F1 scores on SciERC, ACE05, and DuEE datasets.

Aggregating Crowd of LLMs for Cost-Effective Data Annotation (2026.findings-eacl)

Copied to clipboard

Jiacheng Liu, Xiaofeng Hou

Generated Knowledge Prompting for Commonsense Reasoning (2022.acl-long)

Challenge:	Recent advances in Large Language Models (LLMs) have shown promise for automated data annotation, yet reliance on expensive commercial models like GPT-4 limits accessibility.
Approach:	They propose to build a crowd of LLMs which aggregates annotations from multiple sLLMs using label aggregation algorithms.
Outcome:	The proposed approach outperforms individual sLLMs and human crowd labels yields superior results compared to either method alone.

Copied to clipboard

Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi

Beyond Benchmarks: A Capability-Based Maturity Model for Systematic AI Integration in Hospitals (2026.findings-acl)

Challenge:	Existing methods for commonsense reasoning rely on high-quality knowledge, but they are often dominated by large-scale pretrained models that are fine-tuned on a target benchmark.
Approach:	They develop generated knowledge prompting which generates knowledge from a language model and provides it as additional input when answering a question.
Outcome:	The proposed method improves state-of-the-art models on four commonsense reasoning tasks.

Copied to clipboard

Rui Li, Xiaofen WU, Mingqian Liu, Xiaoxia Song, Xu Xiangjun, Jiacheng Qiao, Zhuang Boqin, Xu Chen

Challenge:	Current Large Language Models (LLMs) excel in standardized tests focused on medical knowledge recall, but not in real-world healthcare scenarios.
Approach:	They propose a "capability-based hospital AI Maturity Model" framework that categorizes capabilities into distinct maturity levels . medical artificial intelligence is currently at a critical transition stage from technical verification to deep clinical integration .
Outcome:	The proposed model provides a clear, stepwise evolutionary path for hospitals from foundational infrastructure construction to ubiquitous intelligence.

CrossWeigh: Training Named Entity Tagger from Imperfect Annotations (D19-1)

Copied to clipboard

Zihan Wang, Jingbo Shang, Liyuan Liu, Lihao Lu, Jiacheng Liu, Jiawei Han

Information Extraction & Text Mining

Challenge:	Named entity recognition (NER) models can identify labels in 5.38% of test sentences . a framework to handle label mistakes during NER model training is proposed .
Approach:	They propose a framework to manually correct label mistakes in named entity recognition (NER) they aim to improve the accuracy of models by re-evaluating popular models on corrected test sets .
Outcome:	The proposed framework can detect label mistakes in 5.38% of test sentences . the proposed framework improves on three datasets with a high-performance model .

ICG: Improving Cover Image Generation via MLLM-based Prompting and Personalized Preference Alignment (2025.emnlp-main)

Copied to clipboard

Zhipeng Bian, Jieming Zhu, Qijiong Liu, Wang Lin, Guohao Cai, Zhaocheng Du, Jiacheng Sun, Zhou Zhao, Zhenhua Dong

Responsible & Trustworthy NLP

Challenge:	Large language models and diffusion models have opened new possibilities for AI-generated content . personalized cover image generation remains underexplored despite its critical role in boosting user engagement on digital platforms.
Approach:	They propose a framework that integrates MLLM-based prompting with personalized preference alignment to generate high-quality, contextually relevant covers.
Outcome:	The proposed framework improves image quality, semantic fidelity, and personalization, leading to stronger user appeal and offline recommendation accuracy in downstream tasks.

QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch (2026.findings-acl)

Copied to clipboard

Hao Gu, Hao Wang, Jiacheng Liu, Lujun Li, Qiyuan Zhu, Bei Liu, Binxing Xu, Lei Wang, Xintong Yang, Sida Lin, Sirui Han, Yike Guo

Challenge:	Recent work has shown that reinforcement learning with simple rule-based reward functions (RLVR) can induce emergent reasoning behaviors and yield gains in challenging domains such as math problem solving.
Approach:	They propose a rollout-alignment-quantization-aware RL which aligns training-side forward with the quantized rollout to minimize mismatch.
Outcome:	The proposed approach outperforms quantized-rollout training by +5.5 on Qwen3-30B-A3B MoE for math problems while maintaining low-bit throughput.

Crystal: Introspective Reasoners Reinforced with Self-Feedback (2023.emnlp-main)

Copied to clipboard

Jiacheng Liu, Ramakanth Pasunuru, Hannaneh Hajishirzi, Yejin Choi, Asli Celikyilmaz

AEGIS: A Holistic Benchmark for Evaluating Forensic Analysis of AI-Generated Academic Images (2026.acl-long)

Challenge:	Existing knowledge-augmented reasoning methods fail to capture the introspective nature of knowledge required in commonsense reasoning.
Approach:	They propose a method to develop an introspective commonsense reasoner that introspects for knowledge statements related to the given question and makes an informed prediction.
Outcome:	The proposed method outperforms standard supervised finetuning and chain-of-thought distilled methods and enhances the transparency of the commonsense reasoning process.

Copied to clipboard

Bo Zhang, Tzu-Yen Ma, Zichen Tang, Junpeng Ding, Zirui Wang, Yizhuo Zhao, Peilin Gao, Zijie Xi, Zixin Ding, Haiyang Sun, Haocheng Gao, Yuan Liu, Liangjia Wang, Yiling Huang, Yujie Wang, Yuyue Zhang, Ronghui Xi, Yuanze Li, Jiacheng Liu, Zhongjun Yang, Haihong E

Unleashing Spatial Reasoning in Multimodal Large Language Models via Textual Representation Guided Reasoning (2026.acl-long)

Challenge:	AEGIS examines whether current models can effectively audit AI-generated images in academic papers.
Approach:	They propose a holistic benchmark for forensic analysis of AI-Generated academic ImageS that reveals limitations in academic image forensics.
Outcome:	AEGIS compared with existing benchmarks on seven academic categories and features key advances in forensic analysis.

Copied to clipboard

Jiacheng Hua, Yishu Yin, Yuhang Wu, Tai Wang, Yifei Huang, Miao Liu

Improving the Transformer Translation Model with Document-Level Context (D18-1)

Challenge:	Existing Multimodal Large Language Models struggle with 3D spatial reasoning as they fail to construct structured abstractions of the 3D environment depicted in video inputs.
Approach:	They propose a prompting method that induces MLLMs to generate 3D representations as reasoning traces for more accurate spatial question answering.
Outcome:	Extensive experiments on VSI-Bench and OST-Bech show that TRACE improves over prior prompting strategies across a diverse range of MLLM backbones.

Copied to clipboard

Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang, Yang Liu

Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning (2026.acl-long)

Challenge:	Existing models for document-level context translation ignore documentlevel context.
Approach:	They propose a document-level context encoder to represent document- level context and integrate it into the Transformer model.
Outcome:	Experiments on NIST Chinese-English and IWSLT French-English datasets show that the proposed translation model outperforms the Transformer model significantly.

Copied to clipboard

Junpeng Ding, Zichen Tang, Haihong E, Mengyuan Ji, Yang Liu, Haolin Tian, Haiyang Sun, Pengqi Sun, Yang Xu, Yichen Liu, Haocheng Gao, Zijie Xi, Ruomeng Jiang, Peizhi Zhao, Rongjin Li, Yuanze Li, Jiacheng Liu, Zhongjun Yang, Jintong Chen, Siying Lin

LoRAExit: Empowering Dynamic Modulation of LLMs in Resource-limited Settings using Low-rank Adapters (2024.findings-emnlp)

Challenge:	Xu and Peng, 2025) . . SPUR is a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images.
Approach:	They propose to use 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images to evaluate the visual perception of multimodal large language models (MLLMs) . they also propose to utilize cross-panel relation understanding to evaluate MLLM’s ability to decipher intricate cross-panel relations.
Outcome:	The proposed model is based on 4,264 question-answering pairs derived from 1,084 expert-curated images.

Copied to clipboard

Jiacheng Liu, Peng Tang, Xiaofeng Hou, Chao Li, Pheng-Ann Heng

Responsible & Trustworthy NLP

Challenge:	Large Language Models (LLMs) have exhibited remarkable performance across various natural language processing tasks, but deployment on resource-limited settings remains a challenge.
Approach:	They propose a dynamic inference architecture that leverages low-rank adaptors for efficient deployment of LLMs.
Outcome:	The proposed architecture significantly improves performance when deployed on resource-limited settings.

Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering (2022.emnlp-main)

Copied to clipboard

Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, Yejin Choi

Question Answering

Challenge:	Recent research shows that relevant knowledge can provide useful context for commonsense tasks.
Approach:	They propose a method that learns to generate contextually relevant knowledge in response to given questions.
Outcome:	The proposed method shows consistent gains over 9 commonsense benchmarks.

An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint (2025.emnlp-main)

Copied to clipboard

Yi Sun, Han Wang, Jiaqiang Li, Jiacheng Liu, Xiangyu Li, Hao Wen, Yizhen Yuan, Huiwen Zheng, Yan Liang, Yuanchun Li, Yunxin Liu

Low-Resource NLP

Challenge:	Large Language Models (LLMs) are a powerful tool for test-time scaling, but they are often used under time constraints.
Approach:	They propose to use LLMs to make models think before answering questions . they also use self-correction and best-of-N decoding to encourage deeper thinking .
Outcome:	The proposed models are able to achieve higher inference accuracy with extra inference computation under time constraints.

Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statements (2023.emnlp-main)

Copied to clipboard

Jiacheng Liu, Wenya Wang, Dianzhuo Wang, Noah Smith, Yejin Choi, Hannaneh Hajishirzi

Commonsense Reasoning

Challenge:	Despite the advances of language models, they still produce text that contains trivial commonsense errors.
Approach:	They propose a general-purpose commonsense statement verification model that learns to estimate the plausibility of declarative statements based on commonsensical knowledge.
Outcome:	The proposed model outperforms existing models that can be repurposed for commonsense verification, even including GPT-3.5/ChatGPT/GPT-4.

Transfer Knowledge from Natural Language to Electrocardiography: Can We Detect Cardiovascular Disease Through Language Models? (2023.findings-eacl)

Copied to clipboard

Jielin Qiu, William Han, Jiacheng Zhu, Mengdi Xu, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao

Data to Defense: The Role of Curation in Aligning Large Language Models Against Safety Compromise (2025.emnlp-main)

Challenge:	Recent advances in Large Language Models (LLMs) have shown powerful ability in various downstream applications.
Approach:	They propose an approach for cardiovascular disease diagnosis and automatic ECG diagnosis report generation.
Outcome:	The proposed approach generates high-quality cardiac diagnosis reports and achieves competitive zero-shot classification performance even compared with supervised baselines.

Copied to clipboard

Xiaoqun Liu, Jiacheng Liang, Luoxi Tang, Muchao Ye, Weicheng Ma, Zhaohan Xi

TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)

Challenge:	Recent studies have identified a vulnerability in large language models (LLMs) during customization.
Approach:	They propose an adaptive data curation approach that allows any text to be curated to enhance its effectiveness in counteracting harmful samples during customization.
Outcome:	The proposed approach reduces compromising effects and generates 100% safe responses.

Copied to clipboard

Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang

Robustness in NLP

Challenge:	Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction.
Approach:	They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack.
Outcome:	The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses.

Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions (2026.findings-acl)

Copied to clipboard

Zhongbin Guo, Zhen Yang, Yushan Li, Xinyue Zhang, Wenyu Gao, Jiacheng Wang, Chengzhi Li, Xiangrui Liu, Ping Jian

Enhancing Agentic Textual Graph Retrieval with Synthetic Stepwise Supervision (2026.acl-long)

Challenge:	Existing advances in Spatial Intelligence rely on vision-Language Models . however, a critical question remains: does spatial understanding originate from visual encoders?
Approach:	They propose to evaluate the SI performance of Large Language Models without pixel-level input.
Outcome:	The proposed benchmark challenges large language models to perform symbolic reasoning rather than visual pattern matching.

Copied to clipboard

Ge Chang, Jinbo Su, Jiacheng Liu, Pengfei Yang, Yuhao Shang, Huiwen Zheng, Hongli Ma, Yan Liang, Yuanchun Li, Yunxin Liu