Papers by Jingming Yao
Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Detecting AI-generated poetry is difficult due to distinctive characteristics of modern Chinese poetry. |
| Approach: | They propose a benchmark for detecting AI-generated modern Chinese poetry . they use a high-quality dataset and systematic performance assessments . |
| Outcome: | The proposed benchmark is based on a high-quality dataset of 800 poems written by six professional poets and 41,600 poems generated by four mainstream LLMs. |
Poller: Are LLMs Suitable for Evaluating Poetry Understanding Task? (2026.findings-acl)
Copied to clipboard
| Challenge: | Traditional methods for poetry evaluation are expensive and unsuitable for large-scale data. |
| Approach: | They propose a method leveraging Large Language Models to evaluate poetry understanding tasks using Large Language models. |
| Outcome: | The proposed method reduces the evaluation error between LLMs and humans by adopting the poet's perspective. |
What is the Best Way for ChatGPT to Translate Poetry? (2024.acl-long)
Copied to clipboard
| Challenge: | Despite promising results, our analysis reveals persistent issues in the translations generated by ChatGPT that warrant attention. |
| Approach: | They propose an Explanation-Assisted Poetry Machine Translation method which leverages monolingual poetry explanation as a guiding information for the translation process. |
| Outcome: | The proposed method outperforms traditional translation methods of ChatGPT and the existing online systems in English-Chinese poetry translation. |
Can ChatGPT Really Understand Modern Chinese Poetry? (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies have focused on poetry generation and translation, but their scope has been limited to evaluation and analysis of experimental results without addressing fundamental issues of comprehension. |
| Approach: | They propose a framework for evaluating ChatGPT's understanding of modern poetry . they evaluated the interpretations of unpublished modern Chinese poems by different poets . |
| Outcome: | The proposed framework is based on the evaluation of unpublished poems by poets and shows that its interpretations align with the original poets’ intents in over 73% of the cases. |