G-IdiomAlign: A Gloss-Pivoted Benchmark for Cross-Lingual Idiom Alignment (2026.acl-long)
Copied to clipboard
| Challenge: | Existing tools for cross-lingual idiom-to-idiom equivalence evaluation are limited . figurative meanings are non-compositional and culturally grounded, making literal mappings unreliable. |
| Approach: | They propose a gloss-pivoted benchmark where each idiom is anchored by an English gloss from Wiktionary. |
| Outcome: | The proposed benchmark is based on a dictionary-anchored English idiom . a bias to literal translation is a dominant failure mode across diverse LLMs, the study shows . |
Similar Papers
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
How to Align Multiple Signed Language Corpora for Better Sign-to-Sign Translations? (2025.naacl-long)
Copied to clipboard
| Challenge: | despite the growing need for advanced signing technologies, signed language resources remain scarce. |
| Approach: | They propose a linguistically informed alignment algorithm that matches instances between signed languages . they compare similarities and differences across three signed languages to develop a model . |
| Outcome: | The proposed algorithm performs well on automatic metrics for sign-to-sign translation and generation. |
TokAlign: Efficient Vocabulary Adaptation via Token Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Tokenization is a foundational step for Large Language Models (LLMs) but low compression rate of vanilla tokenizers decelerates training and inference process. |
| Approach: | They propose a method to replace the vocabulary of Large Language Models (LLMs) by learning a one-to-one mapping matrix for token IDs. |
| Outcome: | The proposed method significantly improves multilingual text compression rates and vocabulary initialization for Large Language Models. |
Hierarchical Mapping for Crosslingual Word Embedding Alignment (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing strategies that map word embeddings into a crosslingual space are biased towards the choice of the pivot language. |
| Approach: | They propose to map any two languages into a different middle space by learning mappings across languages in a hierarchical way. |
| Outcome: | The proposed strategy significantly improves vocabulary induction scores in all existing benchmarks and in a new non-English–centered benchmark. |
Creative and Context-Aware Translation of East Asian Idioms with GPT-4 (2024.findings-emnlp)
Copied to clipboard
| Challenge: | figurative language is a challenge for human translators, who often choose a context-aware translation . a set of commonly used idioms condenses its figurativ meaning into a few characters . |
| Approach: | They evaluate whether GPT-4 can generate high-quality translations using Pareto-optimal prompting strategies that outperform translation engines from Google and DeepL. |
| Outcome: | The proposed translations outperform translation engines from Google and DeepL at low cost. |
CHENGYU-BENCH: Benchmarking Large Language Models for Chinese Idiom Understanding and Use (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks focus on narrow tasks such as multiple-choice cloze tests, isolated translation, or simple paraphrasing. |
| Approach: | They propose a benchmark to measure Chinese idioms' cultural and contextual nuances . they evaluate 2,937 human-verified examples covering 1,765 common idiomes . |
| Outcome: | The proposed benchmarks achieve 95% accuracy on Evaluative Connotation, but only 85% on Appropriateness and 40% top-1 accuracy in Open Cloze. |
PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit reasonable multilingual abilities, despite predominantly English-centric pretraining. |
| Approach: | They propose a framework that establishes multilingual alignment prior to language model pretraining and preserves this alignment using a code-switching strategy during pretraining. |
| Outcome: | Experiments in a synthetic English to English-Clone setting show that PreAlign outperforms standard multilingual joint training in language modeling, zero-shot cross-lingual transfer, and cross-linguistic knowledge application. |
Deep Generative Model for Joint Alignment and Word Representation (N18-1)
Copied to clipboard
| Challenge: | EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. |
| Approach: | They exploit translation as a distributional context and embed words as posterior probability densities, rather than point estimates, which allows them to compare words in context using a measure of overlap between distributions. |
| Outcome: | The proposed model performs on a range of lexical semantics tasks and achieves competitive results on benchmarks including natural language inference, paraphrasing, and text similarity. |
AlignBench: Benchmarking Chinese Alignment of Large Language Models (2024.acl-long)
Copied to clipboard
Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, Jie Tang
| Challenge: | Effective evaluation of alignment for emerging Chinese LLMs is still significantly lacking, calling for real-scenario grounded, open-ended, challenging and automatic evaluations tailored for alignment. |
| Approach: | They propose a multi-dimensional benchmark for evaluating LLMs’ alignment in Chinese with 8 main categories, 683 real-scenario rooted queries and corresponding human verified references. |
| Outcome: | The benchmark uses a human-in-the-loop data curation pipeline, 683 real-scenario rooted queries and human verified references. |
Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | despite lack of explicit cross-lingual training data, multilingual models can achieve cross-linguistic transfer. |
| Approach: | They find alignment is significantly correlated with cross-lingual transfer . they advocate for further research on realignment methods for smaller models . |
| Outcome: | The proposed method outperforms XLM-R Large in POS-tagging between English and Arabic by +15.8 accuracy. |