Papers by Letian Peng
C-World: A Computer Use Agent Environment Creator (2026.acl-long)
Copied to clipboard
Ziqiao Xi, Shuang Liang, Qi Liu, Jiaqing Zhang, Letian Peng, Fang Nan, Meshal Nayim, Tianhui Zhang, Rishika Mundada, Lianhui Qin, Biwei Huang, Kun Zhou
| Challenge: | C-World enables users to build agent environments on demand. |
| Approach: | They propose a system that enables users to build agent environments on demand. |
| Outcome: | The proposed system outperforms baselines on 119k samples and achieves Spearman = 0.883 ranking correlation with real execution. |
Controllable Data Augmentation for Few-Shot Text Mining with Chain-of-Thought Attribute Manipulation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for data augmentation generate new examples wildly without proper control, which hinders the usefulness of the proposed approach. |
| Approach: | They propose a chain-of-thought attribute manipulation approach that generates new data from existing examples by tweaking in the user-provided attribute. |
| Outcome: | The proposed approach generates new data from existing examples by tweaking in the user-provided, task-specific attribute, e.g., sentiment polarity or topic in movie reviews. |
Learn from Failure: Fine-tuning LLMs with Trial-and-Error Data for Intuitionistic Propositional Logic Proving (2024.acl-long)
Copied to clipboard
Chenyang An, Zhibo Chen, Qihao Ye, Emily First, Letian Peng, Jiayun Zhang, Zihan Wang, Sorin Lerner, Jingbo Shang
| Challenge: | Recent advances in Automated Theorem Proving have shown the effectiveness of leveraging a (large) language model that generates tactics (i.e. proof steps) to search through proof states. |
| Approach: | They propose to use a large language model that generates tactics to search through proof states. |
| Outcome: | The proposed model solves more unseen theorems with lower trial searches than the current model, which only learns from failed attempts. |
Text Grafting: Near-Distribution Weak Supervision for Minority Classes in Text Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work generates pseudo labels by mining texts similar to the class names from the raw corpus, but there is a high risk that LLMs cannot generate in-distribution data, leading to ungeneralizable classifiers. |
| Approach: | They propose to use LLMs to generate pseudo labels by mining masked templates from corpus . they then use state-of-the-art LLM to synthesize near-distribution texts falling into minority classes . |
| Outcome: | The proposed framework improves on the previous methods for extremely weak-supervised text classification. |
Less than One-shot: Named Entity Recognition via Extremely Weak Supervision (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named entity recognition (NER) problem is performed under extremely weak supervision . XWS setting is considered weaker than 1-shot since example entity is given in context-free way . |
| Approach: | They propose a method that uses extremely weak supervision to train named entity recognition models. |
| Outcome: | The proposed method outperforms the state-of-the-art few-shot methods with 1-shot supervision and ChatGPT annotations significantly. |
Contextualized Semantic Distance between Highly Overlapped Texts (2023.findings-acl)
Copied to clipboard
| Challenge: | Conventional semantic metrics are based on word representations and are vulnerable to disturbance of overlapped components with similar representations. |
| Approach: | They propose a mask-and-predict strategy to evaluate the semantic distance between the overlapped sentences using words in the longest common sequence as neighboring words and use masked language modeling to predict their positions. |
| Outcome: | The proposed method outperforms the state-of-the-art in domain adaption by a huge margin. |
The Price of Format: Diversity Collapse in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Instruction-tuned large language models employ structured templates to enforce format consistency during inference. |
| Approach: | They fine-tune instruction-tuning large language models with structured templates and evaluate their results across three axes: downstream task performance, alignment behavior, and output diversity. |
| Outcome: | The proposed model generates semantically similar outputs even under high temperature sampling and structural tokens in templates significantly constrain the model’s output space. |
Deriving Character Logic from Storyline as Codified Decision Trees (2026.acl-long)
Copied to clipboard
| Challenge: | Existing behavioral profiles are unstructured, weakly validated, and unusable . existing models are weakly valid, leading to brittle agent behavior . Using codified decision trees, we show that CDT outperforms previous methods . |
| Approach: | They propose a data-driven framework that induces an executable decision structure from narrative data. |
| Outcome: | The proposed framework outperforms human-written profiles and prior profiles on multiple benchmarks. |
Incubating Text Classifiers Following User Instruction with Nothing but LLM (2024.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we aim to generate text classification data given arbitrary class definitions . Traditional supervised text classification fine-tunes models on expensive human annotation . |
| Approach: | They propose a framework that can generate text classification data given arbitrary class definitions . they use instruction-to-data mappings and in-context augmentation to refine the framework . |
| Outcome: | The proposed framework outperforms existing methods on benchmarks and training data generation by prompt engineering. |
Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM’s Nest (2025.acl-long)
Copied to clipboard
| Challenge: | Massive high-quality data, both pre-training raw texts and post-training annotations, have been carefully prepared to incubate advanced large language models (LLMs). |
| Approach: | They propose to reframe next-token prediction into extraction for tokens already present in the context of LLMs by reframing next-tongue prediction into IE models. |
| Outcome: | The proposed model learns 102.6M extractive data converted from pre-training and post-training data with better performance than existing pre-trained IE models. |
Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection (2026.acl-long)
Copied to clipboard
Yuwei Zhang, Wenhao Yu, Shangbin Feng, Yifan Zhu, Letian Peng, Jayanth Srinivasa, Gaowen Liu, Jingbo Shang
| Challenge: | Existing knowledge injection benchmarks for large language models lack standardized testing grounds. |
| Approach: | They propose a knowledge injection benchmark that leverages recently-added and expert-curated facts from Wikipedia’s “Did You Know...” entries. |
| Outcome: | The proposed framework improves reliability accuracy by 29.1%. |
ULTRABENCH: Benchmarking LLMs under Extreme Fine-grained Text Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks evaluate models on only a few attributes, typically fewer than five . a new benchmark evaluates large language models under dense, multi-attribute constraints . |
| Approach: | They propose a benchmark that evaluates large language models under dense, multi-attribute constraints. |
| Outcome: | The proposed benchmark evaluates large language models under dense, multi-attribute constraints. |
Answer is All You Need: Instruction-following Text Embedding via Answering the Question (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for encoding instruction information fail to be sensitive to clearer criteria like “evaluate similarity based on emotion” . instead, we propose a different approach, which treats the instruction as a “question” about the input text and encodes the expected answers to obtain the representation accordingly. |
| Approach: | They propose a text embedder that captures characteristics of texts specified by user instructions clarifying the similarity criterion. |
| Outcome: | The proposed model improves instruction-following capabilities when applied to large language models and encoder-based LMs. |