Papers by Shimin Tao
The “Knowledge–Behavior Gap” in Cultural Taboo Safety of Large Language Models (2026.acl-long)
Copied to clipboard
Ying He, Sihang Jiang, Xingzhou Chen, Zhouhong Gu, Yiwei Gu, Minggui HE, Shimin Tao, null Mahongxia, Yanghua Xiao
| Challenge: | Existing cultural benchmarks assess cultural knowledge or values biases, but ignore cultural taboos. |
| Approach: | They propose a benchmark to evaluate and improve the cultural taboo safety of large language models. |
| Outcome: | The proposed benchmark spans 77 countries and regions, and includes over 2,020 taboos. |
DeReA: Improving Idiom Translation with Detect-Retrieve-Arbitrate Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to idiom translation are limited by the constraints of static parametric memory and retrieval noise . idiomatic expressions are non-compositional units where figurative meanings diverge from literal interpretations . |
| Approach: | They propose a detect-retrieve-arbitrate framework that detects idiomatic spans by reasoning over semantic conflicts between literal and contextual meanings. |
| Outcome: | The proposed framework improves GPT-5-mini and Emerging Slang datasets on various model scales. |
SmartSpanNER: Making SpanNER Robust in Low Resource Scenarios (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is one of the most fundamental tasks in natural language processing. |
| Approach: | They propose a method which introduces a Named Entity Head (NEH) prediction task to SpanNER and performs multi-task learning together with task of span classification. |
| Outcome: | The proposed method improves the robustness of SpanNER in low resource scenarios on the CoNLL03, Few-NERD, GENIA and ACE05 benchmark datasets. |
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation (2024.emnlp-main)
Copied to clipboard
Yuan Ge, Yilun Liu, Chi Hu, Weibin Meng, Shimin Tao, Xiaofeng Zhao, Mahong Xia, Zhang Li, Boxing Chen, Hao Yang, Bei Li, Tong Xiao, JingBo Zhu
| Challenge: | Existing methods for instruction data selection have limitations such as relying on fragile external APIs, being affected by biases in GPT models, or reducing the diversity of the selected instruction dataset. |
| Approach: | They propose an industrial-friendly, expert-aligned and diversity-preserved instruction data selection method: Clustering and Ranking (CaR). |
| Outcome: | The proposed method outperforms Alpaca's existing methods by 32.1% in GPT-4 evaluations. |
Capture Human Disagreement Distributions by Calibrated Networks for Natural Language Inference (2022.findings-acl)
Copied to clipboard
| Challenge: | Previously, it's common to disregard it as noise or as a sign of poor-quality data, as their annotations are heavily based on personal experience and opinions. |
| Approach: | They propose to capture the human disagreement distribution from the perspective of model calibration. |
| Outcome: | The proposed model can achieve competitive performance when well-calibrated, on divergence scores between predictive probability and the true human opinion distribution, and the accuracy. |
Improved Pseudo Data for Machine Translation Quality Estimation with Constrained Beam Search (2023.emnlp-main)
Copied to clipboard
Xiang Geng, Yu Zhang, Zhejian Lai, Shuaijie She, Wei Zou, Shimin Tao, Hao Yang, Jiajun Chen, Shujian Huang
| Challenge: | evaluating the quality of machine translation outputs becomes increasingly essential with the rapid development of machine language (MT). |
| Approach: | They propose to generate pseudo data using the MT model with constrained beam search (CBSQE) they propose to preserve the reference parts with high MT probabilities as correct translations . |
| Outcome: | The proposed model outperforms strong baselines in both supervised and unsupervised settings. |
M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models (2025.findings-emnlp)
Copied to clipboard
Jiaxin Guo, Daimeng Wei, Yuanchang Luo, Hengchao Shang, Zongyao Li, Jinlong Yang, Zhanglin Wu, Zhiqiang Rao, Shimin Tao, Hao Yang
| Challenge: | a new ensemble decoding approach enhances the performance of Large Language Models. |
| Approach: | They propose a multi-prompt ensemble decoding approach to enhance LLM performance . they submit n variations of prompts with X to LLMs in batch mode to decode and derive probability distributions . |
| Outcome: | The proposed method improves pass@k rates, LENS metrics and BLEU scores on diverse NLP tasks. |
Modeling Consistency Preference via Lexical Chains for Document-level Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that consistency preference for lexical chains reduces lexical translation inconsistency . Lexical translation consistency is a common discourse phenomenon . |
| Approach: | They propose a consistency-aware model which captures consistency context . they then define consistency-tailored latent variables which guide translation of corresponding sentences . |
| Outcome: | The proposed model significantly improves translation performance in ChineseEnglish and FrenchEnglish translation tasks. |
Part Represents Whole: Improving the Evaluation of Machine Translation System Using Entropy Enhanced Metrics (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing machine translation metrics have poor correlations with human assessments . entropy-based evaluations are often limited to a limited number of samples . |
| Approach: | They propose a fast and unsupervised approach to enhance machine translation metrics using entropy by introducing sentence-level difficulty. |
| Outcome: | The proposed method outperforms existing metrics on five sub-tracks in the WMT19 Metrics shared tasks. |
DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translators (2024.emnlp-main)
Copied to clipboard
| Challenge: | Concatenating large language models are adapted to context-aware neural machine translation in a concatenated way . a recent paradigm shift has been witnessed in discourse-related challenges such as zero pronoun translation . |
| Approach: | They propose an alternative adaptation approach to make large language models discriminately model and utilize inter- and intra-sentence contexts. |
| Outcome: | The proposed approach outperforms concatenation mode and improves performance in discourse modeling. |
Lexical Translation Inconsistency-Aware Document-Level Translation Repair (2023.findings-acl)
Copied to clipboard
| Challenge: | Experimental results show document-level translation repair improves translation consistency but still suffers from lexical translation inconsistency due to the lack of inter-sentence context. |
| Approach: | They propose a document-level translation repair model to model translation inconsistency via automatic post-editing. |
| Outcome: | The proposed model improves translation quality and lexical consistency on document-level translation datasets. |
Collective Human Opinions in Semantic Textual Similarity (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing benchmarks for semantic textual similarity (STS) use averaged human ratings as gold standard. |
| Approach: | They propose to use a Chinese sentence-to-sentence dataset to study collective human opinions in semantic textual similarity (STS) neither a scalar nor a single Gaussian fits a set of observed judgments adequately, they argue . |
| Outcome: | The proposed dataset does not capture disagreements on individual instances, but rather the confidence over the aggregate dataset. |
Neighbors Are Not Strangers: Improving Non-Autoregressive Translation under Low-Frequency Lexical Constraints (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to lexically constrained neural machine translation suffer from high latency. |
| Approach: | They propose a plug-in algorithm for non-autoregressive translation for this problem . they propose ACT to familiarize the model with the source-side context of constraints . |
| Outcome: | The proposed model improves over the backbone constrained NAT model in constraint preservation and translation quality, especially for rare constraints. |
Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance (2025.emnlp-main)
Copied to clipboard
Yilun Liu, Minggui He, Feiyu Yao, Yuhe Ji, Shimin Tao, Jingzhou Du, Justin Li, Jian Gao, Zhang Li, Hao Yang, Boxing Chen, Osamu Yoshie
| Challenge: | Existing solutions for text-to-image synthesis are sensitive on textual prompts, posing a challenge for novice users. |
| Approach: | They propose a dialogue-based TIS prompt generation model that emphasizes user experience for novice users. |
| Outcome: | The proposed model emphasizes user experience for novice users . it improves user-centricity score while maintaining a competitive quality of synthesized images. |
Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement (2025.acl-long)
Copied to clipboard
| Challenge: | Recent research has shown that large language models (LLMs) can enhance translation quality through self-refinement. |
| Approach: | They propose to extend translation refinement from sentence-level to document-level by using document-to-document (Doc2Doc) translations. |
| Outcome: | The proposed method improves translation quality across ten translation tasks with LLaMA-3-8B-Instruct and Mistral-Nemo-Instru. |
The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Models (2026.acl-long)
Copied to clipboard
Yilun Liu, Chunguang Zhao, Mengyao Piao, Lingqi Miao, Shimin Tao, Minggui HE, Chenxin Liu, Zhang Li, null Mahongxia, Jiaxin Guo, Chen Liu, Liqun Deng, Jiansheng Wei, Xiaojun Meng, Fanyi Du, Daimeng Wei, Yanghua Xiao
| Challenge: | Existing multilingual evaluation benchmarks neglect cultural nuances and lack language coverage in subjective tasks. |
| Approach: | They propose a framework that categorizes evaluation tasks into three cultural layers and nine cognitive sub-layers. |
| Outcome: | The proposed framework surpasses prior coverage by up to 111% on 20+ LLMs. |
Evaluation Dataset for Lexical Translation Consistency in Chinese-to-English Document-level Translation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on document-level neural machine translation (NMT) assume that all repeated source words should be translated consistently. |
| Approach: | They construct a test set of 310 bilingual news articles to evaluate lexical translation consistency. |
| Outcome: | The proposed test sets show that translation consistency is consistent across multiple languages. |