Papers by Jonathan Zheng
NEO-BENCH: Evaluating Robustness of Large Language Models with Neologisms (2024.acl-long)
Copied to clipboard
| Challenge: | Prior work on temporal language change observed degradation when finetuning on older text and evaluating on newer data and named entities. |
| Approach: | They construct a benchmark to evaluate LLMs’ ability to generalize to neologisms with various natural language understanding tasks and model perplexity. |
| Outcome: | The proposed model performs better in downstream tasks and with later knowledge cutoff dates than models with earlier knowledge cut off dates. |
Stanceosaurus: Classifying Stance Towards Multicultural Misinformation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing corpora focus on misinformation spreading within western countries. |
| Approach: | They present a new corpus of tweets annotated with stance towards 250 misinformation claims. |
| Outcome: | The proposed method achieves 53.1 F1 on Hindi and 50.4 F1 in Arabic without any target-language fine-tuning. |