Papers by Lin Ai
LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces (2026.findings-acl)
Copied to clipboard
Yukang Feng, Jianwen Sun, Zelai Yang, Jiaxin Ai, Chuanhao Li, Zizhen Li, Fanrui Zhang, Kang He, Rui Ma, Jifan Lin, Jie Sun, Yang Xiao, Sizhuo Zhou, Wenxiao Wu, Yiming Liu, Pengfei Liu, Shenglin Zhang, Kaipeng Zhang
| Challenge: | Existing benchmarks for agentic programming in long-horizon command-line interface tasks are limited by short task horizons, data contamination from GitHub scraping, and a lack of fine-grained evaluation metrics. |
| Approach: | They propose a benchmark to evaluate agentic capabilities across long-horizon command-line interface tasks. |
| Outcome: | The proposed benchmarks cover four engineering categories: from scratch, feature addition, bug fixing, and refactoring. |
Defending Against Social Engineering Attacks in the Age of LLMs (2024.emnlp-main)
Copied to clipboard
Lin Ai, Tharindu Kumarage, Amrita Bhattacharjee, Zizhou Liu, Zheng Hui, Michael Davinroy, James Cook, Laura Cassani, Kirill Trapeznikov, Matthias Kirchner, Arslan Basharat, Anthony Hoogs, Joshua Garland, Huan Liu, Julia Hirschberg
| Challenge: | Existing research has developed frameworks to understand human-to-human CSE attacks. |
| Approach: | They propose a modular defense pipeline that improves detection at both the message and conversation levels. |
| Outcome: | The proposed model can be exploited to facilitate chat-based social engineering attacks and generate high-quality CSE content, but their detection capabilities are suboptimal, leading to increased operational costs for defense. |
NovAScore: A New Automated Metric for Evaluating Document Level Novelty (2025.coling-main)
Copied to clipboard
Lin Ai, Ziwei Gong, Harshsaiprasad Deshpande, Alexander Johnson, Emmy Phung, Ahmad Emami, Julia Hirschberg
| Challenge: | Recent research has focused on identifying text that introduces new, previously unknown information, but has seen a decline in novelty detection due to the rise of large language models. |
| Approach: | They propose a novel automated metric for evaluating document-level novelty that aggregates the novelty and salience scores of atomic information and provides high interpretability and a detailed analysis of a document's novelty. |
| Outcome: | The proposed metric scores high on the TAP-DLND 1.0 dataset and a human-annotated dataset. |
SMARTMiner: Extracting and Evaluating SMART Goals from Low-Resource Health Coaching Notes (2025.findings-emnlp)
Copied to clipboard
Iva Bojic, Qi Chwen Ong, Stephanie Hilary Xinyi Ma, Lin Ai, Zheng Liu, Ziwei Gong, Julia Hirschberg, Andy Hau Yan Ho, Andy W. H. Khong
| Challenge: | SMARTMiner extracts specific, measurable, attainable, relevant, time-bound (SMART) goals from unstructured health coaching notes. |
| Approach: | They propose a framework for extracting and evaluating specific, measurable, attainable, relevant, time-bound (SMART) goals from unstructured health coaching notes. |
| Outcome: | The framework extracts behavior change goal spans and categorizes their SMARTness. |
Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension (2024.emnlp-main)
Copied to clipboard
| Challenge: | Extractive Machine Reading Comprehension (MRC) is a challenging field in the field of Natural Language Processing. |
| Approach: | They propose a Question-Attended Span Extraction module to address the limitations of generative approaches for extractive machine reading comprehension (MRC) . module significantly enhances performance of pre-trained generative language models, enabling them to surpass the extractive capabilities of advanced Large Language Models (LLMs) |
| Outcome: | The QASE module surpasses state-of-the-art models in few-shot settings. |
Akan Cinematic Emotions (ACE): A Multimodal Multi-party Dataset for Emotion Recognition in Movie Dialogues (2025.findings-acl)
Copied to clipboard
David Sasu, Zehui Wu, Ziwei Gong, Run Chen, Pengyuan Shi, Lin Ai, Julia Hirschberg, Natalie Schluter
| Challenge: | Akan Cinematic Emotions (AkaCE) is the first multimodal emotion dialogue dataset for an African language . it contains 385 emotion-labeled dialogues and 6162 utterances across audio, visual, and textual modalities, along with word-level prosodic prominence annotations. |
| Approach: | They propose to use AkaCE to analyze African cinematic emotions using word-level prosodic prominence annotations. |
| Outcome: | The Akan Cinematic Emotions (AkaCE) dataset addresses the significant lack of resources for low-resource languages in emotion recognition research. |
Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuances (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that Large Language Models possess a form of emotional intelligence, capable of interpreting emotional stimuli in text. |
| Approach: | They propose a method that translates speech characteristics into natural language descriptions and integrates them into LLMs to perform multimodal emotion analysis via text prompts. |
| Outcome: | The proposed method outperforms baseline models that require structural modifications on two datasets showing significant improvements in emotion recognition accuracy. |
PropaInsight: Toward Deeper Understanding of Propaganda in Terms of Techniques, Appeals, and Intent (2025.coling-main)
Copied to clipboard
Jiateng Liu, Lin Ai, Zizhou Liu, Payam Karisani, Zheng Hui, Yi Fung, Preslav Nakov, Julia Hirschberg, Heng Ji
| Challenge: | Existing research on propaganda detection does not capture the motives behind the content or its broader impact. |
| Approach: | They propose a framework that dissects propaganda into techniques, arousal appeals, and underlying intent. |
| Outcome: | The proposed framework improves performance in a wide range of scenarios and can be used to identify and categorize propaganda techniques. |
TeamFusion: Supporting Open-ended Teamwork with Multi-Agent Systems (2026.acl-long)
Copied to clipboard
| Challenge: | Many group decisions are open-ended, and aggregation approaches suppress minority perspectives . team members must surface hidden assumptions, discuss disagreements, negotiate acceptable trade-offs . |
| Approach: | They propose a multi-agent system that instantiates a proxy agent for each team member . they also conduct a structured discussion to elicit agreements and disagreements . |
| Outcome: | The proposed system outperforms direct aggregation on two teamwork tasks . it can judge how well individual views are represented in team decisions and consensually good deliverables . |
LegalAgentBench: Evaluating LLM Agents in Legal Domain (2025.acl-long)
Copied to clipboard
Haitao Li, Junjie Chen, Jingli Yang, Qingyao Ai, Wei Jia, Youfeng Liu, Kai Lin, Yueyue Wu, Guozhi Yuan, Yiran Hu, Wuyue Wang, Yiqun Liu, Minlie Huang
| Challenge: | Existing general-domain benchmarks do not capture complexity of real-world judicial cognition and decision-making. |
| Approach: | They propose a benchmark specifically designed to evaluate LLM Agents in the legal domain. |
| Outcome: | The proposed benchmark includes 17 corpora from real-world legal scenarios and provides 37 tools for interacting with external knowledge. |
Half-S: Halving the Scale for Near-Lossless 4-Bit LLM Training (2026.findings-acl)
Copied to clipboard
Jinyang Du, Ruihao Gong, Linghan Ai, Zining Wang, Yunke Peng, Yao Wang, Lei Yan, null Wxuefei, Yaoyuan Wang, Jinyang Guo, Dahua Lin, Xianglong Liu
| Challenge: | Existing 4-bit training pipelines rely on max-scaling, which causes representation collapse . despite this, there are limitations in the accuracy of 4-bit LLM training . |
| Approach: | They propose a scaling strategy that uses half-scaling as a hardware-friendly default . they propose fp4 support that allows for a faster scaling of large language models . |
| Outcome: | The proposed scaling strategy narrows the gap between theoretical optimum and BF16 while maintaining the efficiency benefits of 4-bit training. |
A Survey on Open Information Extraction from Rule-based Model to Large Language Model (2024.findings-emnlp)
Copied to clipboard
Liu Pai, Wenyang Gao, Wenjie Dong, Lin Ai, Ziwei Gong, Songfang Huang, Li Zongsheng, Ehsan Hoque, Julia Hirschberg, Yue Zhang
| Challenge: | Open Information Extraction (OpenIE) is a key NLP task aimed at extracting structured information from unstructured text sources. |
| Approach: | They propose to categorize OpenIE into rule-based, neural, and pre-trained large language models and discuss each within a chronological framework. |
| Outcome: | The paper categorizes OpenIE approaches into rule-based, neural, and pre-trained large language models, discussing each within a chronological framework. |
A Review of Incorporating Psychological Theories in LLMs (2026.eacl-long)
Copied to clipboard
Zizhou Liu, Ziwei Gong, Lin Ai, Zheng Hui, Run Chen, Colin Wayne Leach, Michelle R. Greene, Julia Hirschberg
| Challenge: | a holistic review systematically integrating psychology across the LLM lifecycle remains missing. |
| Approach: | They examine how psychological theories can inform stages of LLM development . they highlight current trends and gaps in how psychological theory is applied . |
| Outcome: | The authors highlight current trends and gaps in how psychological theories are applied . they argue that psychological insights have shaped pivotal NLP breakthroughs . |