Papers by Peng Qian

AT²PO: Agentic Turn-based Policy Optimization via Tree Search (2026.acl-long)

Challenge:	composition of pre-training datasets for large language models remains undisclosed . current methods for evaluating data quality are limited by single-dimensional evaluation or redundancy-focused strategies.
Approach:	They propose a multi-dimensional data selection method that integrates dimensions with existing quality metrics through learned optimal weightings.
Outcome:	The proposed method doubles convergence speed for 1.3B model models and improves downstream task performance by 3.23%.

Copied to clipboard

Zefang Zong, Dingwei Chen, Yang Li, Qi Yi, Bo Zhou, Chengming Li, BO Qian, Peng Chen, Jie Jiang

Information Retrieval

Challenge:	Recent advances in large language models (LLMs) have catalyzed the development of autonomous agents capable of executing complex, multi-turn tasks.
Approach:	They propose a framework for agentic reinforcement learning that integrates turn-level tree search with tree search to address key challenges.
Outcome:	The proposed framework addresses key challenges: limited exploration diversity, sparse credit assignment, and misaligned policy optimization.

MAKAR: a Multi-Agent framework based Knowledge-Augmented Reasoning for Grounded Multimodal Named Entity Recognition (2025.emnlp-main)

Copied to clipboard

Xinkui Lin, Yuhui Zhang, Yongxiu Xu, Kun Huang, Hongzhang Mu, Yubin Wang, Gaopeng Gou, Li Qian, Li Peng, Wei Liu, Jian Luan, Hongbo Xu

Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)

Challenge:	Existing methods for GMNER fail to address semantic ambiguity caused by polysemy and long-tail distribution of datasets.
Approach:	They propose a framework for Grounded Multimodal Named Entity Recognition that leverages a Multimodal Large Language Model to address semantic ambiguity.
Outcome:	Extensive experiments show that the proposed framework outperforms existing methods on two benchmark datasets.

Copied to clipboard

Ethan Wilcox, Peng Qian, Richard Futrell, Ryosuke Kohita, Roger Levy, Miguel Ballesteros

When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes (2022.naacl-main)

Challenge:	Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it.
Approach:	They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training.
Outcome:	The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts.

Copied to clipboard

Mycal Tucker, Tiwalayo Eisape, Peng Qian, Roger Levy, Julie Shah

FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning (2026.acl-long)

Challenge:	Recent studies show that models encode syntactic information redundantly . this allows researchers to boost models' performance by injecting syntaktic information into embeddings .
Approach:	They propose a new probe design that guides probes to consider all syntactic information present in embeddings.
Outcome:	The proposed model improves performance by injecting syntactic information into models.

Copied to clipboard

Zhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan, Hachem Madmoun, Fan Zhang, Debopriyo Banerjee, Georgi Nenkov Georgiev, Xueqing Peng, Lingfei Qian, Jimin Huang, Jinyan Su, Aaryamonvikram Singh, Rui Xing, Rania Elbadry, Chen Xu, Haonan Li, Fajri Koto, Ivan Koychev, Tanmoy Chakraborty, Yuxia Wang, Salem Lahlou, Veselin Stoyanov, Sophia Ananiadou, Preslav Nakov

Reasoning

Challenge:	Existing benchmarks emphasize final numerical answers while neglecting intermediate reasoning steps.
Approach:	They propose a symbolic benchmark for verifiable Chain-of-Thought evaluation in finance . FINCHAIN spans 58 topics across 12 financial domains and three difficulty levels .
Outcome:	The proposed benchmark aims to bridge symbolic reasoning and factual verification.

SyntaxGym: An Online Platform for Targeted Evaluation of Language Models (2020.acl-demos)

Copied to clipboard

Jon Gauthier, Jennifer Hu, Ethan Wilcox, Peng Qian, Roger Levy

Syntactic Text Processing

Challenge:	SyntaxGym is an online platform and open-source framework for targeted syntactic evaluation of neural network language models.
Approach:	They propose to make targeted syntactic evaluations accessible to both experts in NLP and linguistics and reproducible across computing environments.
Outcome:	The proposed framework is reproducible across computing environments and standardized following the norms of psycholinguistic experimental design.

CoCoID: Learning Contrastive Representations and Compact Clusters for Semi-Supervised Intent Discovery (2022.emnlp-industry)

Copied to clipboard

Qian Cao, Deyi Xiong, Qinlong Wang, Xia Peng

Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs (2024.acl-long)

Challenge:	Existing approaches to intent discovery cluster novel intents with prior knowledge from intent-labeled data in a semi-supervised way.
Approach:	They propose a semi-supervised intent discovery framework CoCoID with two components . they propose to discriminate user utterance representation learning and intra-cluster knowledge distillation .
Outcome:	The proposed framework outperforms state-of-the-art intent discovery models by over 1.4 ACC and ARI points and 1.1 NMI points across four datasets.

Copied to clipboard

Zhiwei Cao, Qian Cao, Yu Lu, Ningxin Peng, Luyang Huang, Shanbo Cheng, Jinsong Su

What if This Modified That? Syntactic Interventions with Counterfactual Embeddings (2021.findings-acl)

Challenge:	Existing methods to compress long contexts have degraded dramatically as compression ratios increase, sometimes even falling to the closed-book level.
Approach:	They propose a query-guided compression method that preserves key information within the compressed context.
Outcome:	The proposed method can consistently perform well even at high compression ratios, and offers significant benefits in terms of inference cost and throughput.

Copied to clipboard

Mycal Tucker, Peng Qian, Roger Levy

Representation Learning

Challenge:	Prior art aims to uncover meaningful properties within model representations, but it is unclear how faithfully such probes portray information that the models actually use.
Approach:	They propose a technique for generating counterfactual embeddings within models . they produce evidence that some models use a tree-distancelike representation of syntax .
Outcome:	The proposed technique produces evidence that some models use tree-distancelike representations of syntax in downstream prediction tasks.

Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection (2026.findings-acl)

Copied to clipboard

Zhiwei Liu, Yupeng Cao, Yuechen Jiang, Mohsinul Kabir, Polydoros Giannouris, Chen Xu, Ziyang Xu, Tianlei Zhu, Md. Tariquzzaman, Triantafillos Papadopoulos, Yan Wang, Lingfei Qian, Xueqing Peng, Zhuohan Xie, Ye Yuan, Saeed Almheiri, Abdulrazzaq Alnajjar, Ming-Bin Chen, Harry Stuart, Paul Thompson, Prayag Tiwari, Alejandro Lopez-Lira, Xue Liu, Jimin Huang, Sophia Ananiadou

INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent (2025.acl-long)

Challenge:	Existing research on LLM biases has focused on direct questioning or general-purpose settings . pronounced behavioral biase despite their growing deployment in financial analysis, forecasting, and decision support.
Approach:	They propose a benchmark to evaluate behavioral biases of large language models in MFMD . they use a multilingual financial misinformation dataset to integrate these with misinformation claims .
Outcome:	The proposed benchmark evaluates behavioral biases of large language models across economic scenarios.

Copied to clipboard

Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.p. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow, Qianqian Xie

Dual-Gated Fusion with Prefix-Tuning for Multi-Modal Relation Extraction (2023.findings-acl)

Challenge:	Recent advances have underscored the potential of large language model (LLM)-based agents in financial decision-making.
Approach:	They propose to evaluate LLM agents using 13 different LLMs as backbone models across various market environments and tasks.
Outcome:	The proposed framework assesses the reasoning and decision-making capabilities of 13 different LLMs across various market environments and tasks.

Copied to clipboard

Qian Li, Shu Guo, Cheng Ji, Xutan Peng, Shiyao Cui, Jianxin Li, Lihong Wang

Evaluating Large Language Models on Controlled Generation Tasks (2023.emnlp-main)

Challenge:	Existing methods for multi-modal relation extraction lack useful visual information.
Approach:	They propose a novel multi-modal relation extraction framework to capture deeper correlations of text, entity pair, and image/objects.
Outcome:	The proposed framework captures the deeper correlations of text, entity pair, and image/objects, and extracts useful information.

Copied to clipboard

Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wieting, Nanyun Peng, Xuezhe Ma

Controlled Evaluation of Grammatical Knowledge in Mandarin Chinese Language Models (2021.emnlp-main)

Challenge:	Recent studies have looked into the ability of large language models in various benchmark tasks, including question generation, reading comprehension, multilingual and etc. However, few studies investigate the controllability of large languages.
Approach:	They propose to compare large language models with state-of-the-start finetuned smaller models to find that large language model controls are comparable to smaller models.
Outcome:	The proposed model can meet hard constraints and perform better than state-of-the-art models.

Copied to clipboard

Yiwen Wang, Jennifer Hu, Roger Levy, Peng Qian

Structural Supervision Improves Learning of Non-Local Grammatical Dependencies (N19-1)

Challenge:	Prior work has shown that structural supervision helps English language models learn generalizations about syntactic phenomena such as subject-verb agreement.
Approach:	They train LSTMs, Recurrent Neural Network Grammars, Transformer language models, and Transformer-parameterized generative parsing models on Mandarin Chinese datasets.
Outcome:	The proposed models learn aspects of Mandarin Chinese grammar that assess syntactic and semantic relationships.

Copied to clipboard

Ethan Wilcox, Peng Qian, Richard Futrell, Miguel Ballesteros, Roger Levy

CoRE: A Fine-Grained Code Reasoning Benchmark Beyond Output Prediction (2026.findings-acl)

Challenge:	State-of-the-art LSTM language models learn sequential contingencies with some success . LS models fail to learn other non-local grammatical dependencies, however .
Approach:	They compare LSTM language models with RNNGs to examine grammatical dependencies . they find that hierarchical supervision improves learning of non-local dependencies.
Outcome:	The proposed model outperforms the existing model on non-local dependencies and learns many of the Island Constraints on the filler-gap dependency.

Copied to clipboard

Jun Gao, Yun Peng, Qian Qiao, Changhai Zhou, Yuhua Zhou, Shiyang Zhang, Shichao Weng, Zhenchang Xing, Xiaoxue Ren

Comparing the Evaluation and Production of Loophole Behavior in Humans and Large Language Models (2023.findings-emnlp)

Challenge:	Existing code reasoning benchmarks evaluate final output correctness under a single implementation.
Approach:	They propose a Code Reasoning benchmark that evaluates code reasoning through implementation invariance and process transparency.
Outcome:	The proposed benchmarks lack implementation invariance and process transparency . they observe superficial execution where models arrive at correct outputs without reasoning .

Copied to clipboard

Sonia Murthy, Kiera Parece, Sophie Bridgers, Peng Qian, Tomer Ullman

MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)

Challenge:	a recent study shows that loophole-seeking is frequent and intuitive in children . a large number of models capture the pragmatic understanding required for loopholes, says a researcher .
Approach:	a study compares large language models to humans to examine loophole behavior . they found that models struggle to recognize humor in creative exploitation of loopholes .
Outcome:	a study compares state-of-the-art models to humans to examine loophole behavior in humans . a large language model can generate loopholes, but only two are capable of generating them .

Copied to clipboard

Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie

Rethinking Cross-Subject Data Splitting for Brain-to-Text Decoding (2025.emnlp-main)

Challenge:	Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models.
Approach:	They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts.
Outcome:	The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings .

Copied to clipboard

Congchi Yin, Qian Yu, Zhiwei Fang, Changping Peng, Piji Li

Text Generation

Challenge:	Recent studies have successfully decoded natural language from non-invasive brain signals . current dataset splitting methods suffer from data leakage problem .
Approach:	They propose a right cross-subject data splitting criterion without data leakage for decoding fMRI and EEG signal to text.
Outcome:	The proposed method overfits and overestimates brain-to-text decoding models.

Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study (2025.coling-main)

Copied to clipboard

Bowen Li, Wenhan Wu, Ziwei Tang, Lin Shi, John Yang, Jinyang Li, Shunyu Yao, Chen Qian, Binyuan Hui, Qicheng Zhang, Zhiyin Yu, He Du, Ping Yang, Dahua Lin, Chao Peng, Kai Chen

Neural language models as psycholinguistic subjects: Representations of syntactic state (N19-1)

Challenge:	Existing benchmarks focused on simplified or isolated aspects of coding, ignoring the full spectrum of programming challenges.
Approach:	They propose a case study that examines the performance of large language models across the entire software development lifecycle with four programming languages, multiple domains, and carefully designed and verified metrics for each task.
Outcome:	The proposed model performs across the entire software development lifecycle, including design, environment setup, implementation, acceptance testing, and unit testing.

Copied to clipboard

Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, Roger Levy

Flexible Generation from Fragmentary Linguistic Input (2022.acl-long)

Challenge:	a recent study examines the extent to which neural network language models reflect incremental representations of syntactic state . we examine neural network model behavior on sentences chosen to probe specific aspects of the learned representations .
Approach:	They employ experimental methodologies developed in psycholinguistics to study syntactic representation in the human mind.
Outcome:	The proposed models are trained on large datasets and only sensitive to subtle cues . the results raise questions about the accuracy of the models and their performance .

Copied to clipboard

Peng Qian, Roger Levy

Challenge:	dominant paradigm for high-performance models in novel language tasks is direct specialization via training from scratch or fine-tuning large pre-trained models.
Approach:	They propose a new model that makes it possible to infer human behavior through basic computational motifs.
Outcome:	The proposed model outperforms direct-specialization models in three evaluations and performs comparable to human models.

CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory (2025.acl-long)

Copied to clipboard

Weichen Zhang, Chen Gao, Shiquan Yu, Ruiying Peng, Baining Zhao, Qian Zhang, Jinqiang Cui, Xinlei Chen, Yong Li

Structural Guidance for Transformer Language Models (2021.acl-long)

Challenge:	Existing ground VLN agents struggle in aerial VLLN due to the lack of predefined navigation graphs and the exponentially expanding action space in long-horizon exploration.
Approach:	They propose a large language model-empowered aerial VLN agent that decomposes the long-horizon task into sub-goals with different semantic levels.
Outcome:	The proposed method achieves state-of-the-art performance with significant improvement in continuous city environments.

Copied to clipboard

Peng Qian, Tahira Naseem, Roger Levy, Ramón Fernandez Astudillo

A Systematic Assessment of Syntactic Generalization in Neural Language Models (2020.acl-main)

Challenge:	Pre-trained Transformer language models have proven remarkably successful in learning generic transferable linguistic representations without resorting to data intensive pre-training.
Approach:	They propose to combine a generative parsing and a structural scaffolding idea to guide the model's representation via additional structure loss that separates the incremental constituency parse.
Outcome:	The proposed models achieve impressive perplexity results on language modelling datasets, perform well on grammatical judgments, and provide useful linguistic representations that benefit a wide range of downstream tasks.

Copied to clipboard

Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, Roger Levy

SPEAK: Spiking Neurons as an Entropy-Aware Tokenizer for Large Language Models (2026.acl-long)

Challenge:	Existing work on syntactic knowledge models has not provided a clear picture of the properties required to produce proper syntaktic generalizations.
Approach:	They propose to evaluate syntactic knowledge of language models by varying model architectures . they find substantial differences in syntaktic generalization performance by model architecture .
Outcome:	The proposed model architectures outperform other architectures on a set of 34 English-language syntactic test suites.

Copied to clipboard

Ming Chen, Wenyao Li, Chao Liang, Shi Gu, Peng Lin, De Ma, Huajin Tang, Qian Zheng, Gang Pan

Representation of Constituents in Neural Language Models: Coordination Phrase as a Case Study (D19-1)

Challenge:	Existing tokenizers fail to explicitly leverage historical tokenization results . large language models (LLMs) have demonstrated remarkable effectiveness across NLP tasks .
Approach:	They propose a tokenizer that integrates spiking neurons to explicitly leverage historical tokenization results.
Outcome:	The proposed tokenizer leverages historical tokenization results, but does not selectively leverage history based on contextual relevance.

Copied to clipboard

Aixiu An, Peng Qian, Ethan Wilcox, Roger Levy

SolEval: Benchmarking Large Language Models for Repository-level Solidity Smart Contract Generation (2025.emnlp-main)

Challenge:	Existing studies have focused on the ability of neural models to compute and employ phrase-level features attached to a set of words, such as subject number or whquestion words.
Approach:	They examine whether models can represent constituent-level features, using coordinated noun phrases as a case study.
Outcome:	The proposed model can combine gender and gender features to drive downstream expectations, while having less success with gender agreement.

Copied to clipboard

Zhiyuan Peng, Xin Yin, Rui Qian, Peiqin Lin, YongKang Liu, Hao Zhang, Chenhao Ying, Yuan Luo

AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following (2026.acl-long)

Challenge:	Existing methods focus on Python and Java, neglecting Solidity, the programming language for Ethereum smart contracts.
Approach:	They construct a repository-level benchmark for Solidity to evaluate the performance of LLMs on Ethereum.
Outcome:	The proposed benchmarks show that the best performing LLM achieves only 26.29% Pass@10, highlighting room for improvement in Solidity code generation.

Copied to clipboard

Yun He, Wenzhe Li, Hejia Zhang, Songlin Li, Karishma Mandyam, Sopan Khosla, Yuanhao Xiong, Nanshu Wang, Xiaoliang Peng, Beibin Li, Shengjie Bi, Shishir G Patil, Qi Qi, Shengyu Feng, Julian Katz-Samuels, Richard Yuanzhe Pang, Sujan Kumar Gonugondla, Hunter Lang, Yue Yu, Yundi Qian, Maryam Fazel-Zarandi, Licheng Yu, Amine Benhalloum, Hany Hassan Awadalla, Manaal Faruqui

Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance (2025.emnlp-main)

Challenge:	Recent advances in large language models (LLMs) have shown impressive performance on a range of tasks, yet advanced instruction following (IF) remains a significant challenge.
Approach:	They propose a benchmark that features over 1,600 prompts and expert-curated rubrics that assess LLMs’ ability to follow complex, multi-turn, and system-level instructions.
Outcome:	The proposed framework improves instruction-following abilities of large language models, achieving a 6.7% gain on AdvancedIF and strong results on public benchmarks.

Copied to clipboard

Xueqing Peng, Triantafillos Papadopoulos, Efstathia Soufleri, Polydoros Giannouris, Ruoyu Xiang, Yan Wang, Lingfei Qian, Jimin Huang, Qianqian Xie, Sophia Ananiadou