Papers by Guangtao Zeng
Unsupervised Non-transferable Text Classification (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train a good deep learning model require labeled data for the target domain which can be difficult to obtain. |
| Approach: | They propose an unsupervised non-transferable learning method that does not require annotated target domain data and introduce a secret key component for recovering the model’s access to the target domain. |
| Outcome: | The proposed method reduces model generalization ability in specific target domains while still recovering access to the target domain. |
Sailor: Open Language Models for South-East Asia (2024.emnlp-demo)
Copied to clipboard
| Challenge: | Large language models (LLMs) rely on English data for training, but are often not comparable across other languages. |
| Approach: | They propose to develop a family of open language models for SEA languages . they use BPE dropout, aggressive data cleaning and deduplication to improve model robustness . |
| Outcome: | The proposed models perform well across four benchmarks, including commonsense reasoning, question answering, reading comprehension and examination. |
Tailored Primitive Initialization is the Secret Key to Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) has emerged as a powerful paradigm for improving the reasoning capabilities of large language models. |
| Approach: | They propose a pipeline that automatically discovers thinking token patterns with reasoning primitives and curates SFT datasets to prepare LLMs for RL. |
| Outcome: | The proposed pipeline outperforms baseline methods on mathematical and logical reasoning benchmarks on RL tasks. |
MedDialog: Large-scale Medical Dialogue Datasets (2020.emnlp-main)
Copied to clipboard
Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie
| Challenge: | telemedicine is a medical practice that provides patient care remotely using video conferencing tools. |
| Approach: | They build large-scale medical dialogue datasets to facilitate research . they pretrain several models on the Chinese MedDialog dataset and compare their performance . |
| Outcome: | The proposed datasets show that models trained on MedDialog can generate doctor-like medical dialogues. |
On the Generation of Medical Dialogs for COVID-19 (2021.acl-short)
Copied to clipboard
Meng Zhou, Zechen Li, Bowen Tan, Guangtao Zeng, Wenmian Yang, Xuehai He, Zeqian Ju, Subrato Chakravorty, Shu Chen, Xingyi Yang, Yichen Zhang, Qingyang Wu, Zhou Yu, Kun Xu, Eric Xing, Pengtao Xie
| Challenge: | under the pandemic of COVID-19, people experiencing COVI D19-related symptoms have a pressing need to consult doctors. |
| Approach: | They develop a medical dialog system that can provide COVID19-related consultations . they use two dialog datasets containing conversations between doctors and patients . |
| Outcome: | The proposed system can provide COVID19-related consultations, but is too small compared with general-domain dialog datasets. |
One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Parameter-efficient transfer learning methods can be expensive in storage when applied to broader ranges of tasks. |
| Approach: | They propose a method that enables efficient sharing of a single PETL network across layers and tasks. |
| Outcome: | The proposed method outperforms other methods with 10% parameters required by the latter on various downstream tasks. |
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models (2023.emnlp-main)
Copied to clipboard
Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan
| Challenge: | Recent work has shown that language models (LMs) have strong multi-step (i.e., procedural) reasoning capabilities. |
| Approach: | They propose a mechanistic interpretation of language models for multi-step reasoning tasks by introducing a new probing approach that recovers the reasoning tree from the model’s attention patterns. |
| Outcome: | The proposed model implicitly embeds a reasoning tree resembling the correct reasoning process within it, and detects the information from the model’s attention patterns for most examples. |