Papers by Peiyang Song
In-Context Learning May Not Elicit Trustworthy Reasoning: A-Not-B Errors in Pretrained Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in artificial intelligence have led to the creation of highly capable large language models (LLMs) that can perform tasks in a human-like manner, but lack infant-level cognitive abilities in certain areas. |
| Approach: | They designed a text-based multi-choice QA scenario similar to the A-Not-B error to test their inhibitory control abilities. |
| Outcome: | The proposed model shows that state-of-the-art LLMs perform well with in-context learning but make errors and show a drop of as many as 83.3% in reasoning tasks when the context changes trivially. |
Creative and Context-Aware Translation of East Asian Idioms with GPT-4 (2024.findings-emnlp)
Copied to clipboard
| Challenge: | figurative language is a challenge for human translators, who often choose a context-aware translation . a set of commonly used idioms condenses its figurativ meaning into a few characters . |
| Approach: | They evaluate whether GPT-4 can generate high-quality translations using Pareto-optimal prompting strategies that outperform translation engines from Google and DeepL. |
| Outcome: | The proposed translations outperform translation engines from Google and DeepL at low cost. |