Papers by Philip Li
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Improving Cross-Domain Low-Resource Text Generation through LLM Post-Editing: A Programmer-Interpreter Approach (2024.findings-eacl)
Copied to clipboard
| Challenge: | Large pre-trained language models such as GPT-3.5 and GPT-4 have gained significant attention in natural language research due to limited computational resources or inaccessible parameters. |
| Approach: | They propose a neural programmer-interpreter approach that preserves the domain generalization ability of LLMs while editing their output. |
| Outcome: | The proposed framework significantly improves GPT-3.5’s performance in logical form-to-text conversion and low-resource machine translation, surpassing other state-of-the-art (SOTA) LLM post-editing methods in cross-domain settings. |
HiURE: Hierarchical Exemplar Contrastive Learning for Unsupervised Relation Extraction (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to extract relational feature signals from natural language sentences use self-supervised clustering and classification that cause gradual drift problems. |
| Approach: | They propose a framework that derives hierarchical signals from relational feature space using cross hierarchy attention and effectively optimizes relation representation of sentences under exemplar-wise contrastive learning. |
| Outcome: | The proposed framework can extract the relationship between entities from natural language sentences without prior knowledge on relation scope or distribution. |
Stronger, Lighter, Better: Towards Life-Long Attribute Value Extraction for E-Commerce Products (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models for attribute value extraction struggle for parameter efficiency and reliability due to data contamination and catastrophic forgetting. |
| Approach: | They propose to decouple product type and attribute to promote de-contamination and parameter efficiency while scaling up. |
| Outcome: | The proposed model achieves state-of-the-art performance with affordable parameter size, least historical knowledge forgetting, and greatest robustness against noises. |
LLM Jailbreak Detection for (Almost) Free! (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for detecting jailbreak prompts entail significant computational costs . |
| Approach: | They propose a free jailbreak detection method which scales logits by temperature to detect jailbreak prompts . |
| Outcome: | The proposed method detects jailbreak prompts with no additional computational costs. |
CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents (2025.findings-acl)
Copied to clipboard
Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li
| Challenge: | Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexity of constructing tasks and evaluators. |
| Approach: | They propose a cross-environment agent benchmark framework that integrates graph-based evaluation and task generation methods. |
| Outcome: | The proposed framework supports multiple devices and can be easily extended to any environment with a Python interface. |
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System (2025.acl-long)
Copied to clipboard
Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, Nanqing Dong
| Challenge: | Recent AI methods have shown promise in tasks such as hypothesis generation and experimental design, but they fail to replicate the collaborative nature of real-world scientific practices. |
| Approach: | They propose a virtual scientific system that mimics the collaborative nature of scientific research by organizing a team of agents to generate, evaluate, and refine research ideas. |
| Outcome: | The proposed system outperforms the state-of-the-art method in producing new scientific ideas and offers valuable insights to guide future research. |
Reimagining Safety Alignment with An Image (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to large language models face inefficiency, fragility, or architectural constraints, resulting in inefficient performance and heightened over-refusal in cross-modal tasks. |
| Approach: | They propose an optimization-driven visual prompt framework that enhances security and reduces over-refusal at the same time. |
| Outcome: | The proposed framework enhances security and reduces over-refusal while maintaining robust safety while reducing unnecessary denials. |
The Best of Both Worlds: Combining Human and Machine Translations for Multilingual Semantic Parsing with Active Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Prior studies have focused on translating utterances from high-resource languages to low-resourced languages. |
| Approach: | They propose an active learning approach that exploits the strengths of both human and machine translations by iteratively adding small batches of human translations into the machine-translated training set. |
| Outcome: | The proposed approach reduces errors and bias in the translated data, resulting in higher parser accuracies than the current model trained on machine translations. |
ETC: Encoding Long and Structured Inputs in Transformers (2020.emnlp-main)
Copied to clipboard
Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, Li Yang
| Challenge: | Existing models for natural language processing (NLP) have been challenging to scale attention to longer inputs. |
| Approach: | They propose an extended Transformer construction architecture that scales attention to longer inputs by combining global-local attention with relative position encodings and a "Contrastive Predictive Coding" objective. |
| Outcome: | The proposed architecture scales attention to longer inputs and encodes structured inputs. |
HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for hierarchical text classification do not consider statistical constraint on label representations learned by structure encoder. |
| Approach: | They propose a new hierarchical text classification model called HTCInfoMax which incorporates two modules to improve the model's representations. |
| Outcome: | The proposed model can model the interaction between each text sample and its ground truth labels explicitly which filters out irrelevant information. |
No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languages (2024.emnlp-main)
Copied to clipboard
Youssef Mohamed, Runjia Li, Ibrahim Ahmad, Kilichbek Haydarov, Philip Torr, Kenneth Church, Mohamed Elhoseiny
| Challenge: | Traditionally, vision research focused on unambiguous class labels, whereas ArtELingo emphasizes diversity of opinions over languages and cultures. |
| Approach: | They propose a vision-language benchmark that spans 28 languages and encompasses approximately 200,000 annotations. |
| Outcome: | The proposed benchmark spans 28 languages and encompasses approximately 200,000 annotations . the challenge is to build machine learning systems that assign emotional captions to images . |
Teams of LLM Agents can Exploit Zero-Day Vulnerabilities (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing frameworks for LLM agents fail to exploit real-world zero-day vulnerabilities . prior work has shown that simple agents can hack mock "capture-the-flag" websites . |
| Approach: | They propose a system of agents with a planning agent that can launch subagents to exploit real-world vulnerabilities. |
| Outcome: | The proposed framework improves over previous frameworks by up to 4.3 on 14 real-world vulnerabilities. |
Evaluating Robustness of Generative Search Engine on Adversarial Factoid Questions (2024.findings-acl)
Copied to clipboard
Xuming Hu, Xiaochuan Li, Junzhe Chen, Yinghui Li, Yangning Li, Xiaoguang Li, Yasheng Wang, Qun Liu, Lijie Wen, Philip Yu, Zhijiang Guo
| Challenge: | Existing large language models (LLMs)-backed generative search engines may not always be accurate. |
| Approach: | They propose to evaluate the robustness of retrieval-augmented generation in a realistic and high-risk setting where adversaries have only black-box system access. |
| Outcome: | The proposed model exhibits higher susceptibility to factual errors compared to LLMs without retrieval. |
Mastering the Craft of Data Synthesis for CodeLLMs (2025.naacl-long)
Copied to clipboard
Meng Chen, Philip Arthur, Qianyu Feng, Cong Duy Vu Hoang, Yu-Heng Hong, Mahdi Kazemi Moghaddam, Omid Nezami, Duc Thien Nguyen, Gioacchino Tangari, Duy Vu, Thanh Vu, Mark Johnson, Krishnaram Kenthapadi, Don Dharmasiri, Long Duong, Yuan-Fang Li
| Challenge: | Large language models (LLMs) have shown impressive performance in code understanding and generation. |
| Approach: | They propose a systematic review of large language models and their taxonomy and propose specialized LLMs for code-related tasks. |
| Outcome: | The proposed models have shown to be highly effective in coding tasks. |
Hierarchical Bi-Directional Self-Attention Networks for Paper Review Rating Recommendation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for review rating prediction ignore hierarchies among data . paper review rating predictions are important for improving paper review process . |
| Approach: | They propose a Hierarchical bi-directional self-attention Network framework for paper review rating prediction and recommendation . they leverage hierarchical structure of paper reviews with three levels of encoders . |
| Outcome: | The proposed approach can be used to make an effective decision-making tool for the academic paper review process. |
Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning (2026.acl-long)
Copied to clipboard
Zelin Tan, Hejia Geng, Xiaohang Yu, Mulei Zhang, Guancheng Wan, Yifan Zhou, Qiang He, Xiangyuan Xue, Heng Zhou, Yutao Fan, Zhong-Zhi Li, Zaibin Zhang, Guibin Zhang, Chen Zhang, Zhenfei Yin, Philip Torr, Lei Bai
| Challenge: | elucidating scaling laws for large language models (LLMs) during pre-training remains unexplored. |
| Approach: | They characterize how model scale, data, and compute interact during pre-training . they find that large models consistently demonstrate superior compute and data efficiency . |
| Outcome: | The proposed scaling laws offer practical guidance for scaling reasoning capabilities through reinforcement learning post-training. |
Behavioral Consistency Validation for LLM Agents: An Analysis of Trading-Style Switching through Stock-Market Simulation (2026.findings-acl)
Copied to clipboard
Zeping Li, Guancheng Wan, Keyang Chen, Yu Chen, Yiwen Zhao, Philip Torr, Guangnan Ye, Zhenfei Yin, Hongfeng Chai
| Challenge: | Recent studies have applied Large Language Models (LLMs) as agents in financial stock market simulations to test if micro-level behaviors aggregate into macro-level phenomena. |
| Approach: | They propose four alignment metrics and use Mann–Whitney U tests to compare agents’ style-switching behavior with financial theory. |
| Outcome: | The proposed model is only partially consistent with financial theory. |
Can an Individual Manipulate the Collective Decisions of Multi-Agents? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that coordinated multi-agent systems exhibit enhanced decision-making and reasoning abilities through collaboration. |
| Approach: | They propose a framework that simulates agent interactions within a multi-agent system to generate adversarial samples and use them to manipulate the target agent in the target system. |
| Outcome: | The proposed framework generates adversarial samples that are used to manipulate the target agent in the target system, misleading the system’s decision-making process. |
Multi-grained Named Entity Recognition (P19-1)
Copied to clipboard
| Challenge: | Existing approaches treat Named Entity Recognition (NER) as a sequence labeling task. |
| Approach: | They propose a framework for Multi-Grained Named Entity Recognition where multiple entities or entity mentions in a sentence could be non-overlapping or totally nested. |
| Outcome: | The proposed framework outperforms current state-of-the-art frameworks by 4.4% in terms of the F1 score among nested/non-overlapping NER tasks. |
Joint Slot Filling and Intent Detection via Capsule Neural Networks (P19-1)
Copied to clipboard
| Challenge: | Existing models that label slots and detect intent do not preserve hierarchical relationship between words, slots, and intents. |
| Approach: | They propose a capsule-based neural network model which performs slot filling and intent detection via a dynamic routing-by-agreement schema. |
| Outcome: | The proposed model performs better than existing models and existing models on real-world datasets. |