Papers by Franklin Ma
LingGym: How Far Are LLMs from Thinking Like Field Linguists? (2025.emnlp-main)
Copied to clipboard
| Challenge: | LINGGYM is a benchmark that evaluates LLMs’ capacity for meta-linguistic reasoning using Interlinear Glossed Text and grammatical descriptions extracted from 18 typologically diverse reference grammars. |
| Approach: | They propose a benchmark that evaluates LLMs’ capacity for meta-linguistic reasoning using Interlinear Glossed Text and grammatical descriptions extracted from 18 typologically diverse reference grammars. |
| Outcome: | The proposed model can generalize linguistic inference across low-resource languages and structures not seen during training. |