Papers by Ye Jia
One Battle After Another: Probing LLMs’ Limits on Multi-Turn Instruction Following with a Benchmark Evolving Framework (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for instruction-following in multi-topic dialogues are limited to a fixed number of turns, susceptible to saturation and failing to account for users’ interactive experience. |
| Approach: | They propose a framework featuring a three-layer tracking mechanism and a query synthesis agent to mimic sequential user behaviors. |
| Outcome: | The proposed framework outperforms existing benchmarks in the evaluation of instruction following in multi-topic dialogues and demonstrates deficiencies in failure recovery and fine-grained instruction following. |
QSpell 250K: A Large-Scale, Practical Dataset for Chinese Search Query Spell Correction (2025.naacl-industry)
Copied to clipboard
| Challenge: | Chinese Search Query Spell Correction is a task designed to identify and correct typographical errors within queries. |
| Approach: | They propose a large-scale benchmark specifically developed for Chinese Query Spell Correction. |
| Outcome: | The proposed benchmark covers a broad range of topics, including formal entities, everyday colloquialisms and idiomatic expressions. |
CVSS Corpus and Massively Multilingual Speech-to-Speech Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing work on speech-to-speech translation (S2ST) systems rely on text representation, but they are text-centric. |
| Approach: | They introduce a massively multilingual-to-English speech-tospeech translation corpus . they synthesize the translation text from the Common Voice speech corpus and CoVoST 2 into English . |
| Outcome: | The proposed corpus outperforms existing models on CoVoST 2 by 5.8 BLEU . the proposed model outperformed the previous state-of-the-art model without extra data . |
Estimating Large Language Model Capabilities without Labeled Test Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models have shown impressive ability to perform in-context learning from only a few examples, but their accuracy varies widely from task to task. |
| Approach: | They propose a method that trains a meta-model using LLM confidence scores as features to perform ICL accuracy estimation. |
| Outcome: | The proposed method improves over baselines across 7 out of 12 settings and achieves the same accuracy as evaluating on 40 sampled examples per task. |
Spec-o3: A Tool-Augmented Vision-Language Agent for Rare Celestial Object Candidate Vetting via Automated Spectral Inspection (2026.acl-long)
Copied to clipboard
| Challenge: | Spec-o3 is a tool-augmented vision-language agent that performs astronomer-aligned spectral inspection. |
| Approach: | They propose a tool-augmented vision-language agent that performs astronomer-aligned spectral inspection via interleaved multimodal chain-of-thought reasoning. |
| Outcome: | Spec-o3 outperforms traditional visual inspection methods on rare-object inspection tasks. |
Modeling discourse cohesion for discourse parsing via memory network (P18-2)
Copied to clipboard
| Challenge: | Existing approaches to discourse parsing focus on studying the semantic and syntactic aspects of EDU pairs, but they do not address long span dependencies. |
| Approach: | They propose a new transition-based discourse parser that takes discourse cohesion into account by using memory networks. |
| Outcome: | The proposed method outperforms traditional features and improves performance on the RST discourse treebank. |
A Self-Evolving LLM Agent Framework for Role-Based Norm Compliance in Healthcare (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing systems treat roles as static prompts and rely on one-shot safety filters . a self-evolving LLM agent is proposed that learns from role-based social experience . |
| Approach: | They propose a self-evolving LLM agent that learns from role-based social experience and explicitly models communicator-level individual traits informed by prior communication questionnaires and clinical literature. |
| Outcome: | The proposed agent learns from role-based social experience and models communicator-level individual traits informed by prior communication questionnaires and clinical literature. |
How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that large language models can be used to predict performance on new configurations. |
| Approach: | They investigate the predictability of large language model capabilities by using BIG-bench . they find a subset of BIG-Bench tasks as informative as BIG-bnch Hard . |
| Outcome: | The proposed model achieves an R2 score greater than 95% on BIG-bench . the model is 3 smaller than BIG-Bench Hard, and the model performs better on the full set. |
Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression (2026.findings-acl)
Copied to clipboard
Jingyu Peng, Maolin Wang, Nan Wang, Jiatong Li, Yuchen Li, Yuyang Ye, Wanyu Wang, Pengyue Jia, Kai Zhang, Xiangyu Zhao
| Challenge: | Despite advances in aligning LLMs with human values, current safety mechanisms remain vulnerable to jailbreak attacks. |
| Approach: | They propose a black-box jailbreak method that uses logical expression translation to bypass LLM safety mechanisms. |
| Outcome: | The proposed method exploits the distributional gap between alignment data and logic-expressed inputs while preserving the underlying semantic intent and readability while evading safety constraints. |