Papers by Junliang He
BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Currently, pre-trained language model (PLM) based metrics are widely adopted in text generation tasks. |
| Approach: | They propose to use PLMs to encode stereotypical societal biases in PLM-based metrics . they show that popular metrics exhibit higher social bias than traditional metrics based on 6 attributes . |
| Outcome: | The proposed method shows that PLM-based metrics exhibit higher social bias than traditional metrics on 6 attributes. |
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline (2022.naacl-main)
Copied to clipboard
Xiangyang Liu, Tianxiang Sun, Junliang He, Jiawen Wu, Lingling Wu, Xinyu Zhang, Hao Jiang, Zhao Cao, Xuanjing Huang, Xipeng Qiu
| Challenge: | Rather than pursuing the reachless SOTA accuracy, researchers are focusing on model efficiency and usability. |
| Approach: | They propose an evaluation and a public leaderboard for efficient NLP models that depicts the Pareto Frontier for various language understanding tasks. |
| Outcome: | The proposed model outperforms or performs on par with SOTA compressed and early exiting models. |
CAMIEval: Enhancing NLG Evaluation through Multidimensional Comparative Instruction-Following Analysis (2025.naacl-long)
Copied to clipboard
| Challenge: | Evaluating the quality of texts generated by language models has always been a challenging task in natural language processing (NLP). |
| Approach: | They propose a multidimensional comparative evaluation method based on instruction-following that combines relevance, factuality, and adherence with a concrete Chain-of-Thoughts process to enhance the accuracy of evaluations. |
| Outcome: | The proposed method outperforms existing methods in correlation with human evaluations on two NLG evaluation benchmarks. |
FiNE: Filtering and Improving Noisy Data Elaborately with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Currently, there are two mainstream methods for improving data integrity: data filtering and data augmentation. |
| Approach: | They propose a method to improve data integrity by combining data filtering and data augmentation with LLMs. |
| Outcome: | The proposed method surpasses the open-source chat version on HalluQA by 8.45 on the open source version. |
Making Large Language Models Better Reasoners with Orchestrated Streaming Experiences (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that large language models can perform complex reasoning tasks without labeled data and unlabeled data. |
| Approach: | They propose a framework for solving reasoning tasks that store answers in a streaming experience pool and orchestrate helpful questions from the pool to assist itself in answering new questions. |
| Outcome: | The proposed framework can self-improve as it answers reasoning questions . it stores all answered reasoning questions and their reasoning steps in a streaming experience pool . |