Papers by Qisheng Liao
Fumbling in Babel: An Investigation into ChatGPT’s Language Identification Ability (2024.findings-naacl)
Copied to clipboard
| Challenge: | ChatGPT is a powerful NLP tool but its language identification abilities are unclear. |
| Approach: | They compile a benchmark comprising 670 languages representing 23 language families spoken in five continents and compare their language identification abilities to ChatGPT's (both GPT-3.5 and GPT-4) performance. |
| Outcome: | The proposed model performs poorly on African languages, while GPT-3.5 and GPT-4 perform poorly on English, Afrikaans, Arabic, Indonesian, Italian, Mandarin Chinese, and several more. |
The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing instruction tuned large language models (LLMs) struggle to understand cross-lingual sociopragmatic meaning (SM) lack of comprehensive investigation into their ability to understand SM is partly due to SM not being adequately represented in any of the existing benchmarks. |
| Approach: | They evaluate the performance of instruction tuned large language models (LLMs) on a multilingual benchmark specifically designed for SM understanding. |
| Outcome: | The proposed benchmark outperforms instruction tuned large language models on a wide range of tasks but falls behind task-specific finetuned models. |
FRAPPE: FRAming, Persuasion, and Propaganda Explorer (2024.eacl-demo)
Copied to clipboard
Ahmed Sajwani, Alaa El Setohy, Ali Mekky, Diana Turmakhan, Lara Hassan, Mohamed El Zeftawy, Omar El Herraoui, Osama Afzal, Qisheng Liao, Tarek Mahmoud
| Challenge: | FRAPPE is a linguistic analysis, persuasion, and propaganda-based news analysis system that analyzes articles for genre, framings, and persulasion techniques. |
| Approach: | They propose a FRAming, Persuasion, and Propaganda Explorer system that analyzes articles for genre, framings, and use of persuation techniques. |
| Outcome: | FRAPPE analyzes articles for genre, framings, and use of persuasion techniques . it also draws comparisons between persulasion and framping strategies adopted by a diverse pool of news outlets and countries across multiple languages for different topics . |