Papers by Jifan Yu
Distantly Supervised Course Concept Extraction in MOOCs with Academic Discipline (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to extract knowledge concepts from MOOCs are noisy and incomplete because of the limited dictionary and diverse MOOC. |
| Approach: | They propose to automatically extract course concepts using distant supervision to eliminate the heavy work of human annotations. |
| Outcome: | The proposed framework outperforms state-of-the-art methods with 7% absolute improvement in F1 score. |
Subgraph Retrieval Enhanced Model for Multi-hop Knowledge Base Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing retrieval methods for knowledge base question answering are either heuristic or interwoven with the reasoning, causing reasoning on the partial subgraphs. |
| Approach: | They propose a subgraph retrieval framework that decouples the retrieval from the subsequent reasoning process and trains subgraphs for easier reasoning. |
| Outcome: | The proposed framework improves retrieval and QA performance over existing methods. |
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have developed watermarking algorithms which restrict the generation process to leave an invisible trace for watermark detection. |
| Approach: | They propose a benchmarking procedure that compares different methods to ensure consistent watermarking strength and jointly evaluates their generation and detection performance. |
| Outcome: | The proposed benchmark compares 4 open-source watermarks on 2 LLMs under 2 watermarking strengths and observes the common struggles for current methods on maintaining the generation quality. |
Evaluating Generative Language Models in Information Extraction as Subjective Question Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Modern large language models (LLMs) perform poorly in elementary tasks like relation extraction and event extraction due to two issues in conventional evaluation methods. |
| Approach: | They propose a method to evaluate large language models by incorporating a human annotation schema. |
| Outcome: | The proposed evaluation method improves matching between model outputs and golden labels. |
Program Transfer for Answering Complex Questions over Knowledge Bases (2022.acl-long)
Copied to clipboard
| Challenge: | Program induction for complex questions over knowledge bases relies on a large number of parallel question-program pairs for the given KB, but the gold program annotations are usually lacking, making learning difficult. |
| Approach: | They propose an approach to leverage program annotations on rich KBs as external supervision signals to aid program induction for low-resourced KB. |
| Outcome: | The proposed approach outperforms SOTA methods on ComplexWebQuestions and WebQuestionSP. |
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks focus on pairwise matching, ignoring robustness . current models exhibit frustrating degradation, with a maximum drop of 23.43 F1 score . |
| Approach: | They propose a benchmark that simulates the evaluation of open information extraction models in the real world . they perform experiments on typical models published in the last decade and a representative large language model . |
| Outcome: | The proposed model is rated robust on a knowledge-invariant clique with different syntactic and expressive forms. |
Exploring the Cognitive Knowledge Structure of Large Language Models: An Educational Diagnostic Assessment Approach (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on LLMs evaluation with exams are lacking in cognitive research on their overall knowledge structure. |
| Approach: | They conduct an evaluation using a human test dataset based on Bloom Taxonomy to reveal the knowledge structures of Large Language Models and gain insights of their cognitive capabilities. |
| Outcome: | The proposed model can pass AP, SAT, and Leetcode exams, but lacks the cognitive power to perform on human exams. |
Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making (2021.acl-long)
Copied to clipboard
| Challenge: | Entity Matching (EM) aims at recognizing entity records that denote the same real-world object. |
| Approach: | They propose a novel EM framework that consists of Heterogeneous Information Fusion and Key Attribute Tree Induction to decouple feature representation from matching decision. |
| Outcome: | The proposed framework outperforms SOTA EM models on 6 public datasets and 3 industrial datasets. |
KoRC: Knowledge Oriented Reading Comprehension Benchmark for Deep Text Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for deep text understanding have encountered two major limitations . most require human annotation of knowledge, which leads to limited knowledge coverage . |
| Approach: | They propose a benchmark to help readers understand a document with prior knowledge . they use massive knowledge bases to guide annotators and large language models to construct knowledgable questions . |
| Outcome: | The proposed benchmarks have limited knowledge coverage and use choices or spans as answers, which results in narrow answer space. |
ExpanRL: Hierarchical Reinforcement Learning for Course Concept Expansion in MOOCs (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing methods for concept expansion in MOOCs are inefficient because of the diversity of MOOC courses and rapid updates. |
| Approach: | They propose an end-to-end hierarchical reinforcement learning (HRL) model for concept expansion in MOOCs that employs a two-level mechanism of seed selection and concept expansion. |
| Outcome: | The proposed model improves on nine real MOOC datasets and maintains competitive performance under different settings. |
SimPBL: A Multi-Agent Framework for Project-Based Learning (2026.acl-long)
Copied to clipboard
Daniel Zhang-Li, Joy Jia Yin Lim, Binglin Liu, Shangqing Tu, Zijun Yao, Hao Peng, Jifan Yu, Haoxuan Li, Zhanxin Hao, Ye He, Zekun Li, Jiangyi Wang, Lei Hou, Bin Xu, Xin Cong, Zhiyuan Liu, Huiqin Liu, Yu Zhang, Juanzi Li
| Challenge: | Existing LLMs provide partial assistance without modeling these roles, and overly comprehensive help can reduce learner autonomy. |
| Approach: | They propose a multi-agent framework with an orchestrator agent that provides adaptive scaffolding from interaction logs and collaborator agents that support project work through boundary-aware collaboration. |
| Outcome: | The proposed framework improves learner examination scores by 14% . it is based on a multi-agent framework with an orchestrator agent . |
HOSMEL: A Hot-Swappable Modularized Entity Linking Toolkit for Chinese (2022.acl-demo)
Copied to clipboard
| Challenge: | Existing studies have explored the use of entity linking (EL) in downstream tasks. |
| Approach: | They propose a modularized entity linking toolkit for easy task adaptation. |
| Outcome: | The proposed toolkit achieves significantly better accuracy and less time and spaceconsumption than existing methods. |
FFAEval: Evaluating Dialogue System via Free-For-All Ranking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluation metrics for open-domain dialogue systems show poor correlation with human assessment. |
| Approach: | They propose a free-for-all human evaluation framework that shares dialogue history with annotators for multi-turn scoring. |
| Outcome: | The proposed framework achieves a strong correlation with human assessment on English and Chinese dialogue systems. |
MOOCCube: A Large-scale Data Repository for NLP Applications in MOOCs (2020.acl-main)
Copied to clipboard
Jifan Yu, Gan Luo, Tong Xiao, Qingyang Zhong, Yuquan Wang, Wenzheng Feng, Junyi Luo, Chenyu Wang, Lei Hou, Juanzi Li, Zhiyuan Liu, Jie Tang
| Challenge: | Massive open online courses (MOOCs) are a popular educational platform for advanced research. |
| Approach: | They propose to use MOOCCube to build a large-scale data repository of over 700 MOOC courses, 100k concepts, 8 million student behaviors with an external resource. |
| Outcome: | The proposed datasets show that they can facilitate research in MOOCs. |
Beyond Self-Report: Bridging the Intention-Behavior Gap in Critical Thinking Assessment via Interpretable Multi-Agent System (2026.acl-long)
Copied to clipboard
Zekun Li, Jifan Yu, Haoxuan Li, Ye He, Daniel Zhang-Li, Shangqing Tu, Joy Jia Yin Lim, Yikun Jiang, Jiaxin Yuan, Yu Zhang
| Challenge: | Accurate assessment of critical thinking is limited by the Intention Behavior Gap in psychology . evaluators that measure self-reported competence are limited by multiagent architectures . |
| Approach: | They propose a framework that operationalizes cognitive assessment into an interpretable multi-agent workflow with Assessment Chain-of-Thought. |
| Outcome: | The proposed framework aligns better with human expert ratings than gold-standard inventories on large-scale simulations and human participants. |
Dynamic Scaling of Unit Tests for Code Reward Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing large language models struggle to produce accurate responses on the first attempt for complex reasoning tasks like code generation. |
| Approach: | They propose a lightweight yet effective unit test generator that scales unit tests based on problem difficulty. |
| Outcome: | The proposed approach significantly improves performance on three benchmarks. |
Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking (2024.acl-long)
Copied to clipboard
| Challenge: | Existing factuality detection methods are not effective for large language models (LLMs). |
| Approach: | They propose a probing model that trains on offline consistency checking results. |
| Outcome: | The proposed model reduces the computational burden of generating multiple responses by online consistency verification and improves on factuality detection and question answering benchmarks. |
Simulating Classroom Education with LLM-Empowered Agents (2025.naacl-long)
Copied to clipboard
Zheyuan Zhang, Daniel Zhang-Li, Jifan Yu, Linlu Gong, Jinchang Zhou, Zhanxin Hao, Jianxiao Jiang, Jie Cao, Huiqin Liu, Zhiyuan Liu, Lei Hou, Juanzi Li
| Challenge: | Initial studies have focused on task-specific, independent LLM-empowered agents, but the potential of LLMs within a multi-agent collaborative framework for classroom simulation with real user participation remains unexplored. |
| Approach: | They propose a multi-agent classroom simulation teaching framework that recognizes representative class roles and introduces a novel class control mechanism for automatic classroom teaching. |
| Outcome: | The proposed framework can simulate dynamic learning environment for users with active teacher-student and student-studente interactions. |
TableLLM: Enabling Tabular Data Manipulation by LLMs in Real Office Usage Scenarios (2025.findings-acl)
Copied to clipboard
Xiaokang Zhang, Sijia Luo, Bohan Zhang, Zeyao Ma, Jing Zhang, Yang Li, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, Jifan Yu, Shu Zhao, Juanzi Li, Jie Tang
| Challenge: | TableLLM is a robust large language model capable of handling tabular data manipulation tasks. |
| Approach: | They propose a distant supervision method for training which includes a reasoning process extension strategy and a cross-way validation strategy. |
| Outcome: | The proposed model has 8 billion parameters and is capable of handling tabular data tasks. |
UPER: Boosting Multi-Document Summarization with an Unsupervised Prompt-based Extractor (2022.coling-1)
Copied to clipboard
| Challenge: | Multi-Document Summarization (MDS) uses the extract-then-abstract paradigm, which extracts a relatively short meta-document and then feeds it into the deep neural networks to generate an abstract. |
| Approach: | They propose to use pre-trained language models to calculate document and keyword’s perplexity to boost other metrics for evaluating a document’s salience. |
| Outcome: | The proposed method can be applied as a plug-in to boost other metrics for evaluating a document’s salience, thus improving the subsequent abstract generation. |
CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing inference scaling methods rely heavily on the quality of candidate responses . however, they are unable to produce correct answers when all candidates are flawed . |
| Approach: | They propose a CoT-based inference scaling strategy that leverages CoT reasoning to synthesize superior answers by analyzing complementary information from multiple candidate responses. |
| Outcome: | The proposed method improves performance on four benchmark datasets with seven policy models. |
From Knowing to Teaching: Scaffolding Pedagogical Decisions for LLM Agent (2026.acl-long)
Copied to clipboard
Yucheng Wang, Shen Yang, Jifan Yu, Haoxuan Li, Joy Jia Yin Lim, Daniel Zhang-Li, Huiqin Liu, Lei Hou, Juanzi Li, Bin Xu
| Challenge: | Large language models produce content lacking pedagogical depth when asked to generate lessons . |
| Approach: | They propose a framework that allows teachers to select content according to pedagogical intent and sequence topics so foundations precede applications. |
| Outcome: | The framework achieves 67.8% win rate in human evaluation and 79.6% in LLM-based evaluation against eight baselines. |
CharacterGLM: Customizing Social Characters with Large Language Models (2024.emnlp-industry)
Copied to clipboard
Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Pei Ke, Guanqun Bi, Libiao Peng, JiaMing Yang, Xiyao Xiao, Sahand Sabour, Xiaohan Zhang, Wenjing Hou, Yijia Zhang, Yuxiao Dong, Hongning Wang, Jie Tang, Minlie Huang
| Challenge: | Character-based dialogue systems (CharacterDial) allow users to customize social characters for social interactions. |
| Approach: | They will collect a large-scale Chinese corpus of characters with diverse categories and behaviors and develop CharacterGLM models to address these challenges. |
| Outcome: | Experiments show that CharacterGLM outperforms most popular open- and closed-source LLMs and performs comparable to GPT-4. |
VisKoP: Visual Knowledge oriented Programming for Interactive Knowledge Base Question Answering (2023.acl-demo)
Copied to clipboard
Zijun Yao, Yuanyong Chen, Xin Lv, Shulin Cao, Amy Xin, Jifan Yu, Hailong Jin, Jianjun Xu, Peng Zhang, Lei Hou, Juanzi Li
| Challenge: | Existing knowledge base question answering systems that parse natural language questions into knowledge oriented program language (KoPL) . |
| Approach: | They propose a knowledge base question answering system that integrates human into the loop to edit and debug queries. |
| Outcome: | The proposed system can debug and edit knowledge base questions on a million-entity-level . it provides auto-completion for its knowledge base schema and user interaction can fix a large portion of wrong KoPL programs to acquire the correct answer. |
LM-Interview: An Easy-to-use Smart Interviewer System via Knowledge-guided Language Model Exploitation (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Semi-structured interviews are a crucial method of data acquisition in qualitative research. |
| Approach: | They propose a semi-structured interview system that automates interview preparation, analysis and control by interviewers. |
| Outcome: | Experimental results show that LM-Interview performs comparable to human interviewers . the system can be used to analyze semi-structured interviews without interviewers' involvement . |
Course Concept Expansion in MOOCs with External Knowledge and Interactive Game (P19-1)
Copied to clipboard
| Challenge: | Existing methods to expand course concepts in MOOCs suffer from semantic drifts and lack of knowledge guidance. |
| Approach: | They propose to use a boundary search method to search for new concepts via external knowledge base and then use heterogeneous features to verify the results. |
| Outcome: | The proposed method improves on the datasets from Coursera and XuetangX. |
Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies have explained to what extent LLMs extract conflicting knowledge from the provided text, but they neglect the necessity to reason with conflicting information. |
| Approach: | They construct a dataset for knowledge conflict resolution examination in the form of question answering that divides reasoning with conflicting knowledge into three levels. |
| Outcome: | The proposed dataset enables analysis of reasoning with conflicting knowledge in the form of question answering. |
Learn to Not Link: Exploring NIL Prediction in Entity Linking (2023.findings-acl)
Copied to clipboard
| Challenge: | Entity linking models have been successful in capturing semantic features, but the NIL prediction problem has not been addressed. |
| Approach: | They propose an entity linking dataset that categorizes mentions linking to NIL into Missing Entity and Non-Entity Phrases. |
| Outcome: | The proposed dataset categorizes mentions linking to NIL into Missing Entity and Non-Entity Phrase categories and ensures the presence of mentions by human annotation and entity masking. |
A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing dialogue systems have demonstrated impressive performance conducting fluent and natural-sounding conversations, but they are plagued by the Knowledge Hallucination problem. |
| Approach: | They propose a method that exploits the dialogue-knowledge interaction to reduce hallucination by using external knowledge resources to generate more informative responses. |
| Outcome: | The proposed method reduces hallucination without disrupting other dialogue performance while keeping adaptive to different generation models. |