Papers by Tianyi Tang
Learning to Imagine: Visually-Augmented Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for natural language generation are pre-trained on text-only corpora, resulting in visual commonsense. |
| Approach: | They propose a method that makes pre-trained language models learn to imagine for visually-augmented natural language generation. |
| Outcome: | The proposed method is compatible with Transformer-based architecture. |
Context-Tuning: Learning Contextualized Prompts for Natural Language Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have shown that pretrained language models (PLMs) lack sufficient consideration of input semantics to generate natural language. |
| Approach: | They propose a continuous prompting approach to fine-tune PLMs for natural language generation by modeling an inverse generation process from output to input. |
| Outcome: | The proposed method fine-tunes only 0.12% of the parameters while maintaining good performance. |
ChatMap: Mining Human Thought Processes for Customer Service Chatbots via Multi-Agent Collaboration (2025.findings-acl)
Copied to clipboard
Xinyi Jiang, Tianyi Hu, Yuheng Qin, Guoming Wang, Zhou Huan, Kehan Chen, Gang Huang, Rongxing Lu, Siliang Tang
| Challenge: | Existing methods for enhancing dialogue performance rely on summarizing behavior . e-commerce chatbots need to align their dialogue strategies with human behavior to achieve coherent, human-like conversations with customers. |
| Approach: | They propose a method to extract core patterns from dialogue data and integrate them into models by mining service thought processes using a multi-agent aPproach. |
| Outcome: | The proposed method outperforms manual methods and outperfies baselines on Taobao in China. |
Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate impressive multilingual capability, but their performance varies substantially across different languages. |
| Approach: | They propose a generic template prompt that stimulates cross-lingual and logical reasoning skills to enhance task performance across languages. |
| Outcome: | The proposed method improves multilingual capability across languages and covers high-resource and low-resourced languages. |
RMTBench: Benchmarking LLMs Through Multi-Turn User-Centric Role-Playing (2025.findings-emnlp)
Copied to clipboard
Hao Xiang, Tianyi Tang, Yang Su, Bowen Yu, An Yang, Fei Huang, Yichang Zhang, Yaojie Lu, Hongyu Lin, Xianpei Han, Jingren Zhou, Junyang Lin, Le Sun
| Challenge: | Existing benchmarks focus on character-centric approach and fail to reflect real-world applications. |
| Approach: | RMTBench is a user-centric bilingual role-playing benchmark featuring 80 diverse characters and over 8,000 dialogue rounds. |
| Outcome: | RMTBench features 80 diverse characters and over 8,000 dialogue rounds. |
MVP: Multi-task Supervised Pre-training for Natural Language Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) have achieved remarkable success in natural language generation tasks. |
| Approach: | They propose to use a large-scale natural language generation corpus to pre-train a text generation model MVP in a supervised manner. |
| Outcome: | The proposed model outperforms BART and Flan-T5 on 13 out of 17 datasets and outperformed BART by 9.3% and FlaN-T5. |
Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models (2026.acl-long)
Copied to clipboard
Binghai Wang, Yantao Liu, Yuxuan Liu, Tianyi Tang, Shenzhi Wang, Chang Gao, Chujie Zheng, Yichang Zhang, Le Yu, Shixuan Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Bowen Yu, Fei Huang, Junyang Lin
| Challenge: | Recent studies observe a phenomenon where reward models achieve high accuracy on static datasets but fail to generalize effectively during RLHF. |
| Approach: | They propose a method that combines rationale consistency with outcome accuracy to improve performance on RM-Bench and JudgeBench. |
| Outcome: | The proposed method surpasses baselines on RM-Bench and JudgeBench by an average of 5% and improves creative writing tasks by 7%. |
PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)
Copied to clipboard
Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei
| Challenge: | Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. |
| Approach: | They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios. |
| Outcome: | The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics. |
BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing long context models suffer from performance decline when the input text exceeds their length limit. |
| Approach: | They propose a multi-task long context benchmark to evaluate LLMs' long context ability using 10 datasets from 5 different NLP tasks. |
| Outcome: | The proposed model covers 5 domains and core capacities of large language models. |
Zero-shot Visual Question Answering with Language Model Feedback (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge-based visual question answering are based on pre-trained language models. |
| Approach: | They propose a language model guided captioning approach that leverages a pre-trained language model to generate captions for an image to help answer a visual question. |
| Outcome: | The proposed method outperforms several competing methods on the knowledge-based VQA task and achieves comparable results to a fine-tuned VLP model. |
Learning to Transfer Prompts for Text Generation (2022.naacl-main)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) have made remarkable progress in text generation tasks via fine-tuning. |
| Approach: | They propose a prompt-based method that learns source prompts and transfers them as target prompts to perform target generation tasks. |
| Outcome: | The proposed method can be used to perform text generation tasks in a transferable setting. |
Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models (2024.acl-long)
Copied to clipboard
Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, Ji-Rong Wen
| Challenge: | Despite the impressive multilingual capabilities demonstrated by LLMs, the understanding of how these abilities develop and function remains nascent. |
| Approach: | They propose a novel detection method to pinpoint language-specific neurons within LLMs by selectively activating or deactivating these neurons. |
| Outcome: | The proposed method can “steer” the output language of LLMs by selectively activating or deactivating language-specific neurons. |
TextBox 2.0: A Text Generation Library with Pre-trained Language Models (2022.emnlp-demos)
Copied to clipboard
Tianyi Tang, Junyi Li, Zhipeng Chen, Yiwen Hu, Zhuohao Yu, Wenxun Dai, Wayne Xin Zhao, Jian-yun Nie, Ji-rong Wen
| Challenge: | TextBox 2.0 focuses on the use of pre-trained language models (PLMs) to generate text. |
| Approach: | They propose a library that integrates pre-trained language models into 13 common text generation tasks and 83 datasets. |
| Outcome: | The proposed library covers 13 common text generation tasks and their corresponding datasets and incorporates 45 PLMs covering general, translation, Chinese, dialogue, controllable, distilled, prompting, and lightweight PLM. |
LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)
Copied to clipboard
Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen
| Challenge: | a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented. |
| Approach: | They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs). |
| Outcome: | The proposed library is based on extensive experiments in a variety of evaluation settings. |
Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References (2024.naacl-long)
Copied to clipboard
Tianyi Tang, Hongyuan Lu, Yuchen Jiang, Haoyang Huang, Dongdong Zhang, Xin Zhao, Tom Kocmi, Furu Wei
| Challenge: | Existing evaluation benchmarks with limited references may not accurately reflect the quality of the model’s hypotheses. |
| Approach: | They propose a method to enrich evaluation benchmarks by diversifying the expression of a single reference into multiple high-quality ones to cover the semantic space of the reference sentence as much as possible. |
| Outcome: | The proposed method can enhance evaluation benchmarks by diversifying the expression of reference into multiple high-quality ones to cover the semantic space of the reference sentence as much as possible. |
The Web Can Be Your Oyster for Improving Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models encode a large amount of knowledge, but knowledge is frozen at the time of training, and the models become static and limited by training data. |
| Approach: | They propose an adaptive search engine assisted learning method that can self-evaluate the confidence level of PLM’s predictions and adaptively determine when to refer to the web for more data. |
| Outcome: | The proposed model outperforms retrieval-augmented methods on 16 knowledge-intensive tasks on a wide range of knowledge-related tasks. |
Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for KG-to-text generation are based on pretrained language models. |
| Approach: | They propose to automatically generate a text that describes the facts in knowledge graph (KG) they leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation. |
| Outcome: | The proposed model outperforms all comparison methods on fully-supervised and fewshot settings. |
TextBox: A Unified, Modularized, and Extensible Framework for Text Generation (2021.acl-demo)
Copied to clipboard
Junyi Li, Tianyi Tang, Gaole He, Jinhao Jiang, Xiaoxuan Hu, Puzhao Xie, Zhipeng Chen, Zhuohao Yu, Wayne Xin Zhao, Ji-Rong Wen
| Challenge: | TextBox is an open-source text generation framework that is modularized and extensible. |
| Approach: | They propose to provide a unified, modularized, and extensible text generation framework that implements 21 text generation models on 9 benchmark datasets. |
| Outcome: | The proposed framework implements 21 models on 9 benchmark datasets and is available under the Apache License 2.0 license. |
ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models (2022.naacl-main)
Copied to clipboard
Junyi Li, Tianyi Tang, Zheng Gong, Lixin Yang, Zhuohao Yu, Zhipeng Chen, Jingyuan Wang, Xin Zhao, Ji-Rong Wen
| Challenge: | Recent years have featured a trend towards Transformer based pretrained language models (PLMs) in natural language processing systems. |
| Approach: | They propose to use four evaluation dimensions to evaluate ten widely-used PLMs . they find that pretrained language models are good at different ability tests . |
| Outcome: | The results show that pretrained language models are good at different ability tests and have excellent transferability between tasks. |
ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text generation use auto-regressive (AR) methods, but inefficient inference is a problem. |
| Approach: | They propose an efficient and effective PLM to explicitly model the token dependency during NAR text generation. |
| Outcome: | The proposed model outperforms existing models on three text generation tasks while achieving 10 times faster inference speedup. |