Papers by Liqiang Jing
FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) lack the capacity to handle multimodal inputs effectively. |
| Approach: | They introduce a reference-free and fine-grained evaluation metric that measures the faithfulness of the generated free-form answers from large vision-language models. |
| Outcome: | The proposed metric measures the faithfulness of free-form answers from large vision-language models. |
Multi-source Semantic Graph-based Multimodal Sarcasm Explanation Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies can only identify sarcastic post but could not give concrete explanation for why it is sarkastic. |
| Approach: | They propose a multimodal sarcasm explanation scheme that generates a sentence for a social post to explain why it contains sarkasis. |
| Outcome: | The proposed model outperforms existing methods on a public dataset. |
FIFA: Unified Faithfulness Evaluation Framework for Text-to-Video and Video-to-Text Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluation methods for VideoMLLMs are limited to one task and fail to assess hallucinations in open-ended, free-form responses. |
| Approach: | They propose a unified framework that extracts comprehensive descriptive facts and models their semantic dependencies via a Spatio-Temporal Semantic Dependency Graph. |
| Outcome: | The proposed framework aligns more closely with human judgment than existing evaluation methods and improves factual consistency in both text and video generation. |
LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research (2025.emnlp-main)
Copied to clipboard
Shuo Yan, Ruochen Li, Ziming Luo, Zimu Wang, Daoyang Li, Liqiang Jing, Kaiyu He, Peilin Wu, Juntong Ni, George Michalopoulos, Yue Zhang, Ziyang Zhang, Mian Zhang, Zhiyu Chen, Xinya Du
| Challenge: | Large language model (LLM) agents have demonstrated remarkable potential in advancing scientific discovery, but their capability in reproducing code from research papers remains underexplored. |
| Approach: | They propose to evaluate LLM agents' ability to reproduce scientific research papers by analyzing code reproduction tasks from 23 research papers published in top-tier NLP venues. |
| Outcome: | The proposed benchmark systematically evaluates the capability of large language model (LLM) agents on code reproduction from Language Modeling Research. |
FIHA: Automated Fine-grained Hallucinations Evaluations in Large Vision Language Models with Davidson Scene Graphs (2025.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to large vision-language models rely on costly annotations and are not comprehensive in terms of evaluating all aspects. |
| Approach: | They propose an automated method which can access LVLMs hallucination in an LLM-free and annotation-free way and model the dependency between different types of halluciNations. |
| Outcome: | The proposed model can model the dependency between different types of hallucinations and generate Q&A pairs on any image dataset at minimal cost. |