Papers with SLING
SLING: Sino Linguistic Evaluation of Large Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Using pre-trained language models, we find that the accuracy of LMs is far below human performance. |
| Approach: | They propose a benchmark of Sino LINGuistics which consists of 38K sentence pairs in Mandarin Chinese grouped into 9 high-level linguistic phenomena. |
| Outcome: | The proposed model performs better on local phenomena than hierarchical models and has a strong gender and number bias. |