Papers by Kai Shu
Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing claims verification models rely on annotated data, which is expensive to create at a large scale. |
| Approach: | They propose a model that can verify complex claims without annotated data . they leverage the in-context learning ability of Large Language Models to translate a claim into a First-Order-Logic clause . |
| Outcome: | The proposed model outperforms baseline models on three datasets . it performs well on the datasets, and the results are published online. |
WALNUT: A Benchmark on Semi-weakly Supervised Learning for Natural Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on weak supervision for NLU focus on a specific task or simulate weak supervision signals from ground-truth labels. |
| Approach: | They propose a benchmark to advocate and facilitate research on weak supervision for NLU . they use document-level and token-level prediction tasks as examples . |
| Outcome: | The proposed benchmark advocates and facilitates research on weak supervision for NLU tasks. |
Taxonomy-Guided Zero-Shot Recommendations with LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to deploy large language models (LLMs) into RecSys have limited prompt length, unstructured item information, and un-constrained generation of recommendations. |
| Approach: | They propose a taxonomy-guided recommendation framework that empowers LLMs with category information in a systematic approach. |
| Outcome: | The proposed framework significantly improves recommendation quality compared to zero-shot approaches. |
Measuring Sycophancy of Language Models in Multi-turn Dialogues (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior research on sycophancy has focused on single-turn factual correctness, overlooking the dynamics of real-world interactions. |
| Approach: | They propose a new evaluation suite that assesses sycophantic behavior in multi-turn, free-form conversational settings. |
| Outcome: | The proposed evaluation suite measures how quickly a model conforms to the user and how frequently it shifts its stance under sustained user pressure. |
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge (2025.emnlp-main)
Copied to clipboard
Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, Huan Liu
| Challenge: | Recent advances in Large Language Models (LLMs) inspire the "LLM-as-a-judge" paradigm . traditional methods of assessment and evaluation fail in dynamic and open-ended scenarios . |
| Approach: | They propose a paradigm where LLMs are leveraged to perform scoring, ranking, or selection for machine learning evaluation scenarios. |
| Outcome: | The proposed model-based judgment and evaluation paradigms are based on large language models and are compared to the current model-driven evaluation paradigm. |
PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on prompt-based few-shot tuning focus on deriving proper label words with a verbalizer or generating prompt templates to elicit semantics from PLMs. |
| Approach: | They propose a framework that leverages label semantics for prompt-based tuning. |
| Outcome: | The proposed framework improves on few-shot text classification tasks by leveraging label semantics and data augmentation. |
Authorship Attribution for Neural Text Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning have enabled the generation of realistic artifacts . however, the qualities of texts generated by these models are better, often confusing classifiers if they are not real. |
| Approach: | They propose to use neural network-based language models to generate realistic texts . they investigate the authorship attribution problem in three versions of a text . |
| Outcome: | The proposed models generate texts that are difficult to distinguish from human-written ones . the results show that most generators still generate texts significantly different from human ones compared to other models . |
Piecing It All Together: Verifying Multi-Hop Multimodal Claims (2025.coling-main)
Copied to clipboard
| Challenge: | Existing claim verification datasets often do not require systems to perform complex reasoning or effectively interpret multimodal evidence. |
| Approach: | They propose a task that requires models to reason over multiple pieces of evidence . they construct a large-scale dataset comprising 15k multi-hop claims paired with multimodal evidence - generated and refined using large language models with additional input from human feedback. |
| Outcome: | The proposed method is based on human performance benchmarks and human reasoning hops. |
When Safety Alignment Fails to Generalize: Probing with Language Game Jailbreaks (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing safety alignment methods rely on fixed or narrow transformation schemes to generalize . existing methods based on fixed and narrow transformations are often inadequate . |
| Approach: | They propose a framework for discovering and refining language game-based jailbreaks to probe alignment generalization. |
| Outcome: | The proposed framework allows controlled exploration of alignment behavior across closely related linguistic variants. |
Can Large Language Models Identify Authorship? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional capacity for reasoning and problem-solving, but their potential in authorship analysis remains under-explored. |
| Approach: | They propose to integrate explicit linguistic features into LLMs to provide explanations into their reasoning processes. |
| Outcome: | The proposed models demonstrate their ability to perform zero-shot, end-to-end authorship verification effectively and provide explainability through explicit linguistic features. |
ConQRet: A New Benchmark for Fine-Grained Automatic Evaluation of Retrieval Augmented Computational Argumentation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for evaluating RAArg are costly and lack long, complex arguments and real-world evidence. |
| Approach: | They propose to use multiple fine-grained LLM judges to evaluate RAArg using a new benchmark that features long and complex human-authored arguments on debated topics. |
| Outcome: | The proposed methods provide better and more interpretable assessments than traditional single-score metrics and even previously reported human crowdsourcing. |