Papers by Xiaoyong Du
TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities (2023.acl-demo)
Copied to clipboard
Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Gou, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan
| Challenge: | Several pre-training models of different modalities are showing a rising trend of homogeneity in their model structures. |
| Approach: | They propose a toolkit that supports pre-training models of different modalities. |
| Outcome: | The proposed toolkit can match the performance of the original implementations on text, vision, and audio benchmarks. |
Double-Edged Sword or Sharp Tool? Designing and Evaluating Triadic LLM-Teacher Collaboration for K-12 Writing at Scale (2026.findings-acl)
Copied to clipboard
| Challenge: | a large-scale empirical dataset involving 57,954 essays from 10,195 students across 120 schools over two years is presented in this paper. |
| Approach: | They propose a triadic collaboration system that supports K-12 writing learning . they propose linguistic expansion as a pedagogical gatekeeper and bridge . |
| Outcome: | The proposed system improves writing quality through a strategic labor division . authors find that excessive linguistic expansion yields diminishing marginal utility . |
PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Table-based fact verification has attracted a lot of attention recently due to the lack of datasets that can be used to pre-train language models to be aware of common table operations. |
| Approach: | They propose a table-based fact verification tool that pre-trains language models to be aware of common table operations such as aggregating a column or comparing tuples. |
| Outcome: | The proposed method outperforms previous methods on two table-based fact verification datasets TabFact and SEM-TAB- FACTS. |
UER: An Open-Source Toolkit for Pre-training Models (D19-3)
Copied to clipboard
Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, Xiaoyong Du
| Challenge: | Existing work on pre-training models have shown that it is important to use a framework to deploy various pre- training models efficiently. |
| Approach: | They propose an assemble-on-demand pre-training toolkit that assembles pre-trained models on demand and encapsulates them with rich modules. |
| Outcome: | The proposed framework can reproduce state-of-the-art models or develop models that remain unexplored. |
Enabling Agents to Communicate Entirely in Latent Space (2026.acl-long)
Copied to clipboard
Zhuoyun Du, Runze Wang, Huiyu Bai, Zouying Cao, Xiaoyong Zhu, Yu Cheng, Bo Zheng, Wei Chen, Haochao Ying
| Challenge: | Natural language is the de facto communication medium for LLM-based agents, but it presents a fundamental constraint . natural language downsampling limits the depth and nuance of information that can be transmitted . et al.: inter-agent latent space communication is a promising paradigm for solving complex tasks . |
| Approach: | They propose a paradigm that leverages the last hidden states of an LLM as a representation of its thought for direct communication. |
| Outcome: | The proposed paradigm outperforms fine-tuned chain-of-thought prompting and single-agent baselines even across heterogeneous models. |
Analogical Reasoning on Chinese Morphological and Semantic Relations (P18-2)
Copied to clipboard
| Challenge: | Analogical reasoning is effective in capturing linguistic regularities. |
| Approach: | They propose to use Chinese lexical knowledge to build an analogical reasoning task using a large dataset. |
| Outcome: | The proposed dataset proves to be reliable benchmark for evaluating Chinese word embeddings. |