Papers by Haowei Lin
CriticBench: Benchmarking LLMs for Critique-Correct Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | CriticBench is a benchmark designed to assess LLMs’ abilities to critique and refine their reasoning across a variety of tasks. |
| Approach: | They propose a benchmark to assess LLMs' ability to critique and correct reasoning across a variety of tasks. |
| Outcome: | The proposed benchmark examines the performance of 17 large language models in generation, critique, and correction reasoning. |
FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Score (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for detecting out-of-distribution instances are empirical . state-of the-art methods for OOD detection are suboptimal since they only estimate in-distance density pout(x). |
| Approach: | They propose a method that measures the “OOD-ness” of a test case x through the likelihood ratio between out-distribution mathcal Pout and in-division mathcal Pin. |
| Outcome: | The proposed method improves existing methods on popular benchmarks and establishes a new SOTA on popular NLP benchmarks. |
Continual Training of Language Models for Few-Shot Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work on applying large language models (LMs) achieves impressive performance in many NLP applications. |
| Approach: | They propose to continuously post-train an LM with unlabeled domains to expand its knowledge without forgetting previous skills. |
| Outcome: | The proposed system improves few-shot end-task learning in these domains. |
Ensemble Privacy Defense for Knowledge-Intensive LLMs against Membership Inference Attacks (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are the foundation of modern natural language processing, powering applications across diverse domains. |
| Approach: | They propose a model-agnostic defense framework which aggregates and evaluates the outputs of a knowledge-injected LLM, a base LLM and a dedicated judge model to enhance resistance against membership inference attacks. |
| Outcome: | The proposed framework reduces MIA success by up to 27.8% for SFT and 526.3% for RAG compared to inference-time baseline while maintaining answer quality. |
CoreCodeBench: Decoupling Code Intelligence via Fine-Grained Repository-Level Tasks (2026.acl-long)
Copied to clipboard
Lingyue Fu, Hao Guan, Bolun Zhang, Haowei Yuan, Yaoming Zhu, Lin Qiu, ZongYu Wang, Xuezhi Cao, Xunliang Cai, Weiwen Liu, Weinan Zhang, Yong Yu
| Challenge: | Existing large language models for software engineering rely on coarse-grained pass rates obscuring specific cognitive bottlenecks. |
| Approach: | They propose a repository-level benchmark that dissects coding capabilities through atomized tasks. |
| Outcome: | The proposed framework achieves a 78.55% validity yield, surpassing the 31.7% retention rate of SWE-bench-Verified. |
Adapting a Language Model While Preserving its General Knowledge (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing DA-training methods do not explicitly identify what knowledge should be preserved and what should be changed by the domain corpus. |
| Approach: | They propose to use an unlabeled corpus of aparticular domain to train a pre-trained general-purpose language model to adapt the LM so that end-tasks in the domain can give improved performances. |
| Outcome: | The proposed method improves the performance of pre-trained general-purpose language models by contrasting the representations of the general and the full (both general and domain knowledge) to learn an integrated representation with both general and specific knowledge. |