Papers by Fan Lin

49 papers
S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models (2026.acl-long)

Copied to clipboard

Challenge: Existing benchmarks rely heavily on text-based evaluation and largely ignore paralinguistic cues such as prosody, emotion, and speaker traits.
Approach: They propose a speech-native benchmark for evaluating instruction-following S2S models with explicit assessment of both semantic understanding and paralinguistic expression.
Outcome: The proposed system enables more natural, robust, and human-aligned speech agents.
Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities (2025.findings-acl)

Copied to clipboard

Challenge: Using large language models, large language model models can be used to evaluate reasoning abilities in context-rich scenarios.
Approach: They construct datasets for both propositional logic and abductive logic reasoning with four difficulty levels across 12 distinct domains based on Wikipedia categorization and those with purely abstract variables.
Outcome: The proposed model can be used to benchmark LLMs in real-world scenarios, but not in context-rich scenarios.
SHARP: Unlocking Interactive Hallucination via Stance Transfer in Role-Playing LLMs (2025.findings-acl)

Copied to clipboard

Challenge: Existing studies on social interactions neglect hallucination while struggling with poor generalizability and implicit character fidelity judgments.
Approach: They propose a generalizable and explicit paradigm for uncovering interactive patterns of Large Language Models across diverse worldviews by defining interactive hallucination through stance transfer and SHARP, a benchmark built by extracting relations from commonsense knowledge graphs.
Outcome: The proposed paradigm is generalizable and explicit and demonstrates its effectiveness and stability.
FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)

Copied to clipboard

Challenge: Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity.
Approach: They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models.
Outcome: The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models.
Q-TOD: A Query-driven Task-oriented Dialogue System (2022.emnlp-main)

Copied to clipboard

Challenge: Existing pipelined task-oriented dialogue systems have difficulties adapting to unseen domains . end-to-end systems are plagued by large-scale knowledge bases in practice .
Approach: They propose a query-driven task-oriented dialogue system that extracts dialogue context information into a natural language query.
Outcome: The proposed system outperforms strong baselines and establishes a new state-of-the-art performance on three publicly available task-oriented dialogue datasets.
LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding (2025.findings-acl)

Copied to clipboard

Challenge: Existing methods for adapting LLMs to streaming rely on expensive re-encoding or limited scalability.
Approach: They propose a group position encoding paradigm built on batch architectures to enhance consistency between streaming and batch modes.
Outcome: The proposed method outperforms existing methods on cross-lingual and cross-modal tasks.
CrowdAgent: Multi-Agent Managed Multi-Source Annotation System (2025.emnlp-demos)

Copied to clipboard

Challenge: Recent approaches to annotate data focus on labeling, but lack holistic process control . a novel system that integrates task assignment, data annotation, and quality/cost management is needed .
Approach: They propose a multi-agent system that integrates task assignment, data annotation, and quality/cost management.
Outcome: The proposed system automates human management by using a collaborative multi-agent system.
Youling: an AI-assisted Lyrics Creation System (2020.emnlp-demos)

Copied to clipboard

Challenge: Recent studies have focused on a single pass of lyrics generation with little human intervention.
Approach: They propose an AI-assisted lyrics creation system that supports one pass full-text generation and interactive generation modes.
Outcome: The proposed system supports full-text generation and interactive generation modes . it also provides a revision module which enables users to revise undesired lyrics repeatedly.
PreP-OCR: A Complete Pipeline for Document Image Restoration and Enhanced OCR Accuracy (2025.acl-long)

Copied to clipboard

Challenge: Existing pipelines that combine document image restoration with semantic-aware post-OCR correction can improve text extraction from degraded images.
Approach: They propose a two-stage pipeline that combines document image restoration with semantic-aware post-OCR correction to enhance both visual clarity and textual consistency.
Outcome: The proposed pipeline reduces character error rates by 63.9-70.3% on 13,831 pages of real historical documents in English, French, and Spanish compared to OCR on raw images.
Do VLMs Have a Moral Backbone? A Study on the Fragile Morality of Vision-Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Vision-Language Models (VLMs) have advanced multimodal learning, driving progress in cross-modal reasoning.
Approach: They propose to examine moral robustness of vision-language models by analyzing their moral stances under multimodal perturbations.
Outcome: The proposed model-agnostic multimodal perturbations expose VLMs to a variety of moral vulnerabilities, including a sycophancy trade-off where stronger instruction-following models are more susceptible to persuasion.
Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis (2022.emnlp-main)

Copied to clipboard

Challenge: Existing pre-trained language representation models (PLMs) capture sentiment information from word-level while under-considering sentence-level information.
Approach: They propose a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks that enhance the PLM’s knowledge about sentiment words.
Outcome: The proposed model achieves state-of-the-art on various sentence-level and aspect-level sentiment classification benchmarks.
Transition-based Directed Graph Construction for Emotion-Cause Pair Extraction (2020.acl-main)

Copied to clipboard

Challenge: Existing methods to extract emotions and causes from unannotated text are pipelined, causing error propagation.
Approach: They propose to transform a task into a procedure of parsing-like directed graph construction . they propose to generate a directed graph with labeled edges based on a sequence of actions .
Outcome: The proposed method outperforms the state-of-the-art methods by 6.71% (p0.01) in F1 measure.
CoMoL: Efficient Mixture of LoRA Experts via Dynamic Core Space Merging (2026.findings-acl)

Copied to clipboard

Challenge: Existing PEFT methods suffer from limited parameter efficiency and coarse-grained adaptation due to proliferation of LoRA experts and instance-level routing.
Approach: They propose a new MoE-LoRA framework that incorporates expert diversity, parameter efficiency, and fine-grained adaptation.
Outcome: The proposed framework outperforms existing methods on multiple tasks while maintaining parameter efficiency.
Diagnosing Failures in Large Language Models’ Answers: Integrating Error Attribution into Evaluation Framework (2025.findings-acl)

Copied to clipboard

Challenge: Existing evaluation models lack error attribution capability due to their proprietary nature.
Approach: They propose a misattribution framework with 6 primary and 15 secondary categories to facilitate in-depth analysis.
Outcome: The proposed framework is based on a dataset specifically designed for error attribution, along with the corresponding scores and feedback.
SrDetection: A Self-Referential Framework for Data Leakage Detection in Code Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Existing methods for evaluating code large language models assume access to proprietary training corpora or use external reference sets with manually tuned, non-generalizable thresholds.
Approach: They propose a framework for self-referential leakage detection for gray-box and black-box settings.
Outcome: The proposed framework improves average F1 by 21.52 points in the gray-box setting and 14.46 points in black-box settings over strong baselines.
An Empirical Study of Clinical Note Generation from Doctor-Patient Encounters (2023.eacl-main)

Copied to clipboard

Challenge: Medical doctors spend 52 to 102 minutes per day writing clinical notes from patient encounters.
Approach: They propose to use a new dataset to generate automated and manual clinical notes from doctor-patient conversations in a clinical setting.
Outcome: The proposed model could reduce the time spent writing clinical notes from doctor-patient conversations in a clinical setting.
PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation (2022.findings-aacl)

Copied to clipboard

Challenge: Experimental results show PLATO-XL achieves state-of-the-art results across multiple conversational tasks.
Approach: They propose to train PLATO-XL models with up to 11 billion parameters, trained on Chinese and English social media conversations.
Outcome: The proposed model achieves state-of-the-art on multiple conversational tasks, verifying its potential as a foundation model of conversational AI.
Towards IP Intelligence: Benchmarking Large Language Models on Intellectual Property Knowledge and Practice (2026.findings-acl)

Copied to clipboard

Challenge: Existing datasets and benchmarks focus only on patents or cover limited aspects of the IP field, lacking alignment with real-world scenarios.
Approach: They propose a bilingual IP task taxonomy and a large-scale bilingual benchmark to evaluate LLMs in real-world IP practice.
Outcome: The proposed model achieves only 75.8% accuracy, indicating room for improvement . open-source IP and law-oriented models lag behind closed-source general-purpose models .
BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis (2024.emnlp-demo)

Copied to clipboard

Challenge: Recent advances in large language models have demonstrated impressive reasoning capabilities, indicating their potential to serve as the foundation for agents.
Approach: They propose a detailed emulation system that combines large vision-language model and multi-agent system to emulate dynamic interactions between multiple agents over a period of time.
Outcome: The proposed system combines large vision-language model and multi-agent system to emulate dynamic interactions between agents and their environments over a period of time.
X-LeBench: A Benchmark for Extremely Long Egocentric Video Understanding (2025.findings-emnlp)

Copied to clipboard

Challenge: Existing benchmark datasets focus on short to moderately long videos, leaving a substantial gap in evaluating extensive, ultra-long egocentric video recordings.
Approach: X-LeBench is a benchmark dataset designed to evaluate long egocentric video recordings . it uses a life-logging pipeline to produce realistic, coherent daily plans .
Outcome: X-LeBench is a new benchmark dataset designed to evaluate long-form egocentric video understanding . the approach produces realistic, coherent daily plans aligned with real-world video data .
Towards Transferable Personality Representation Learning based on Triplet Comparisons and Its Applications (2025.emnlp-main)

Copied to clipboard

Challenge: Existing methods for personality analysis treat corpus as a single unit for classification, but this approach presents several challenges.
Approach: They propose a task paradigm for text-based personality representation learning that uses a triplet personality trend comparison dataset to learn single-sentence personality embeddings with desirable metric properties.
Outcome: The proposed model significantly boosts performance across various applications, including personality detection, personality retrieval, and emotion translation prediction.
CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation (2025.findings-acl)

Copied to clipboard

Challenge: Medical quality control indicators are essential to assess the qualifications of healthcare institutions for medical services.
Approach: They propose a Chinese electronic medical records-based dataset for MQCIC and propose CF-IR method that disentangles clinical fact verification and inferential rule reasoning actions.
Outcome: The proposed method outperforms Chain-of-Thought methods on 20 representative LLMs, covering general and medical models.
VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation (2024.emnlp-main)

Copied to clipboard

Challenge: Existing video metrics are lagging behind in providing reliable scores over generated videos due to lack of large-scale human-annotated dataset.
Approach: They propose to use VideoFeedback to train a human-annotated multi-aspect score over 37.6K synthesized videos from 11 existing video generative models.
Outcome: The proposed model outperforms the prior best metrics by 50 points in the test.
Cogito: A Cognitive Agentic Framework Driven by Dynamic Graph of Thoughts for Financial Report Generation (2026.findings-acl)

Copied to clipboard

Challenge: Existing approaches to financial report generation are insufficient to handle dynamic uncertainties of real-world financial environments.
Approach: They propose a cognitively grounded agentic framework for professional financial report generation that is driven by Dynamic Graph of Thoughts and a social collaboration mechanism to facilitate coordinated agent interaction.
Outcome: The proposed framework is based on a dynamic reasoning model and social collaboration mechanism.
AgentRM: Enhancing Agent Generalization with Reward Modeling (2025.acl-long)

Copied to clipboard

Challenge: Existing LLM-based agents have strong performance on held-in tasks, but their generalizability to unseen tasks remains poor.
Approach: They propose a reward-based generalizable reward model to guide the policy model for effective test-time search.
Outcome: The proposed agentRM outperforms existing agents on held-in tasks by 8.8 points on average.
Improving Factual Consistency of News Summarization by Contrastive Preference Optimization (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent advances in news summarization have created problems with “hallucinations” that are factually inconsistent with the source text.
Approach: They propose to disentangle LLMs’ propensities to generate faithful and fake content by adopting a probing-based specific training method to improve their capacity of distinguishing two types of propensity.
Outcome: The proposed method disentangles LLMs’ propensities to generate faithful and fake content and improves their ability to distinguish between two types of propensity.
Multimodal Language Models See Better When They Look Shallower (2025.emnlp-main)

Copied to clipboard

Challenge: Existing studies show that multimodal large language models extract visual features from the final layers of a pretrained Vision Transformer.
Approach: They propose a feature fusion method that strategically incorporates shallower layers . they propose MLLMs that extract visual features from the final layers of a pretrained Vision Transformer .
Outcome: The proposed method outperforms deep layers on fine-grained visual tasks . it is the first comprehensive study of visual layer selection for MLLMs .
TranSHER: Translating Knowledge Graph Embedding with Hyper-Ellipsoidal Restriction (2022.emnlp-main)

Copied to clipboard

Challenge: Existing knowledge graph embedding methods restrict entities on hyper-ellipsoid surfaces, resulting in suboptimal knowledge graph completion.
Approach: They propose a score function that leverages relation-specific translations between head and tail entities to relax constraints on hyper-ellipsoid surfaces.
Outcome: The proposed method achieves state-of-the-art performance on link prediction and generalizes well to datasets in different domains and scales.
HAD: HAllucination Detection Language Models Based on a Comprehensive Hallucination Taxonomy (2026.acl-industry)

Copied to clipboard

Challenge: relying on large language models for information has raised concerns about reliability and accuracy of outputs.
Approach: They propose a hallucination taxonomy with 11 categories for various NLG tasks and propose HAllucination Detection models which integrate hallucinism detection, span-level identification, and correction into a single inference process.
Outcome: The proposed models outperform baselines on HaluEval, FactCHD, and FaithBench, confirming their robustness and versatility.
CFBench: A Comprehensive Constraints-Following Benchmark for LLMs (2025.acl-long)

Copied to clipboard

Challenge: Existing evaluations of Large Language Models (LLMs) focus on fragmented constraints or narrow scenarios, but they overlook the comprehensiveness and authenticity of constraints from the user’s perspective.
Approach: They propose a Chinese Comprehensive Constraints Following Benchmark for LLMs that compiles constraints from real-world instructions and constructs a systematic framework for constraint types.
Outcome: The proposed framework integrates multi-dimensional assessment criteria with requirement prioritization, covering various perspectives of constraints, instructions, and requirement fulfillment.
STINMatch: Semi-Supervised Semantic-Topological Iteration Network for Financial Risk Detection via News Label Diffusion (2023.emnlp-main)

Copied to clipboard

Challenge: Commercial news provides rich semantics and timely information for automated financial risk detection.
Approach: They propose a semi-supervised Semantic-Topological Iteration Network, STINMatch, along with a news-enterprise knowledge graph to endorse the risk detection enhancement.
Outcome: The proposed model outperforms existing models in terms of generalization and semantics and annotation.
Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning (2025.acl-long)

Copied to clipboard

Challenge: Representation Fine-tuning (ReFT) is a proposed method for improving parameter efficiency . however, it yields suboptimal performance, as fixed-position representations have uncertain impact on outputs .
Approach: They propose a method that fine-tunes critical representations in a low-rank linear subspace while freezing the base model.
Outcome: The proposed method improves accuracy of LLaMA-2-7B and ReFT by 18.2 and 3.8 on GSM8K.
A Knowledge Regularized Hierarchical Approach for Emotion Cause Analysis (D19-1)

Copied to clipboard

Challenge: Emotion cause analysis aims to identify the reasons behind emotions . previous models focus on learning architecture with local textual information .
Approach: They propose a method to extract emotion cause with hierarchical neural model and knowledge-based regularizations by sentiment lexicon and common knowledge.
Outcome: The proposed method outperforms baselines on two public datasets in different languages and outperformed competitive baselines by 2.08%.
Contextual Domain Classification with Temporal Representations (2021.naacl-industry)

Copied to clipboard

Challenge: Existing studies that incorporate context in SLU have focused on domains where context is limited to a few minutes.
Approach: They propose temporal representations that combine wall-clock second difference and turn order offset information to utilize both recent and distant context in a novel large-scale setup.
Outcome: The proposed model reduces 13.04% of classification errors compared to baseline . previous studies have focused on domains where context is limited to a few minutes .
Benchmarking for Domain-Specific LLMs: A Case Study on Academia and Beyond (2025.findings-emnlp)

Copied to clipboard

Challenge: Comp-Comp is an iterative benchmarking framework grounded in the principles of comprehensiveness and compactness.
Approach: They propose a benchmark framework that incorporates the principle of comprehensiveness and compactness.
Outcome: The proposed framework is domain-agnostic and adaptable to a wide range of specialized fields.
Text-to-ES Bench: A Comprehensive Benchmark for Converting Natural Language to Elasticsearch Query (2025.acl-long)

Copied to clipboard

Challenge: Recent research on text-to-Query has explored using large language models to convert user query intent to executable code.
Approach: They propose a novel semantic parsing task that leverages large language models to generate domain-specific language and post-processing code to support multi-index Elasticsearch queries.
Outcome: The proposed model outperforms DeepSeek-R1 on the large Elasticsearch Dataset (LED) and BirdES datasets.
Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation (2025.acl-long)

Copied to clipboard

Challenge: Large Language Models (LLMs) have produced significant advances in the field of recommender systems.
Approach: They propose to retrieve up-to-date structure information from the knowledge graph to augment recommendations by leveraging external knowledge sources.
Outcome: Experiments on a large dataset show that the proposed method is effective in enhancing LLM-based recommendations.
UEGP: Unified Expert-Guided Pre-training for Knowledge Rekindle (2024.findings-naacl)

Copied to clipboard

Challenge: Existing paradigms for pre-training and fine-tuning have limitations . knowledge rekindle aims to break through performance upper bounds of experts without introducing additional annotated data.
Approach: They propose a new paradigm for pre-training and fine-tuning that aims to re-incorporate the fine- tuned expert model into the training cycle and break through performance upper bounds of experts.
Outcome: The proposed model breaks through performance upper bounds of experts without additional annotated data.
HoWToBench: Holistic Evaluation for LLM’s Capability in Human-level Writing using Tree of Writing (2026.acl-long)

Copied to clipboard

Challenge: Evaluating the writing capabilities of large language models remains a significant challenge due to the multidimensional nature of writing skills and the limitations of existing metrics.
Approach: They propose to model the aggregation weights of sub-features in a tree-structured workflow and propose a Chinese writing benchmark that mitigates biases.
Outcome: The proposed tree-of-writing (ToW) measures the writing capabilities of large language models (LLMs) in Chinese and shows that it mitigates biases and achieves a *0.93* Pearson correlation with human judgments.
EmojiPrompt: Generative Prompt Obfuscation for Privacy-Preserving Communication with Cloud-based LLMs (2025.naacl-long)

Copied to clipboard

Challenge: Recent advances in Large Language Models (LLMs) have substantially expanded their applicability across diverse fields, such as personalized recommendations, health report analysis, and financial decision-making.
Approach: They propose a generative transformation paradigm that obfuscates user data with linguistic and non-linguistic elements before submitting it to cloud-based LLMs.
Outcome: The proposed paradigm obfuscates user private data while maintaining performance compared to the unobflated version.
HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs (2026.findings-acl)

Copied to clipboard

Challenge: High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly.
Approach: They define the first HSS domain system covering 14 mainstream fields and introduce HSS-Synth.
Outcome: the proposed pipeline outperforms 14 leading baselines on 16 benchmarks.
TeachMaster: Generative Teaching via Code (2026.acl-industry)

Copied to clipboard

Challenge: Existing methods for creating video content are limited by high costs and slow update cycles.
Approach: They propose a paradigm shifting educators from manual creators to high-level directors who focus on pedagogical intents while agents handle execution.
Outcome: The proposed framework reduces production costs to 0.3% of traditional course videos and provides a robust solution for scalable education.
Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integration (2024.findings-acl)

Copied to clipboard

Challenge: Existing LMs undergo task-agnostic pertaining, but task-specific pretraining has gained prominence.
Approach: They propose retrieval augmented pretraining and task-specific pretraining for DG . they propose to refine language model pretraining to align it more closely with downstream task .
Outcome: The proposed method improves the performance of multiple-choice questions by integrating knowledge graphs and language models.
A Survey on Foundation Language Models for Single-cell Biology (2025.acl-long)

Copied to clipboard

Challenge: Existing single-cell foundation language models are based on pre-trained and large language models.
Approach: They review the development of single-cell foundation language models . they discuss data tokenization strategies and pre-training paradigms .
Outcome: The proposed models have shown remarkable performance in a variety of single-cell data analysis tasks.
Ambiguous Learning from Retrieval: Towards Zero-shot Semantic Parsing (2023.acl-long)

Copied to clipboard

Challenge: Existing neural semantic parsers require a large amount of training data which is expensive and difficult to obtain.
Approach: They propose a framework for a supervised retrieval system based on pretrained language models . they propose ambiguous supervision to improve the precision and coverage of the task .
Outcome: The proposed approach outperforms state-of-the-art zero-shot parsing methods in ambiguous supervision.
Hate Speech Detection Based on Sentiment Knowledge Sharing (2021.acl-long)

Copied to clipboard

Challenge: Existing methods for hate speech detection are stereotyped and biased . et al., a paper examining the effectiveness of multitask learning in hate speech recognition tasks .
Approach: They propose a hate speech detection framework based on sentiment knowledge sharing . they extract affective features of the target sentence and use sentiment features from external resources .
Outcome: The proposed model can detect hate speech over two public datasets.
KuiLeiXi: a Chinese Open-Ended Text Adventure Game (2021.acl-demo)

Copied to clipboard

Challenge: Recent advances in pre-trained language models have made it possible to generate human-like text.
Approach: They propose to integrate an open-ended text adventure game in Chinese, named KuiLeiXi, where players interact with the AI until the plot goals are reached.
Outcome: The proposed game lacks incentives and relies on players to explore on their own.
IntelliCockpitBench: A Comprehensive Benchmark to Evaluate VLMs for Intelligent Cockpit (2025.findings-acl)

Copied to clipboard

Challenge: Visual Question Answering (VQA) is a key task in vehicular systems.
Approach: They propose a benchmark that encompasses diverse automotive scenarios . they use images from front, side, and rear cameras, various road types, weather conditions, and interior views .
Outcome: The proposed benchmark includes images from front, side, and rear cameras, various road types, weather conditions, and interior views.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations