Papers by Jaesung Tae
TESS: Text-to-Text Self-Conditioned Simplex Diffusion (2024.eacl-long)
Copied to clipboard
Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew Peters, Arman Cohan
| Challenge: | Existing models for diffusion generation are expensive and discrete, resulting in a large number of diffusion steps to generate text. |
| Approach: | They propose a text diffusion model that is fully non-autoregressive and employs a new form of self-conditioning and applies the diffusion process on the logit simplex space rather than the learned embedding space. |
| Outcome: | The proposed model outperforms state-of-the-art non-autoregressive models, requires fewer diffusion steps with minimal drop in performance, and is competitive with pretrained autoregressive sequence-to-sequence models. |
What Language Model to Train if You Have One Million GPU Hours? (2022.findings-emnlp)
Copied to clipboard
Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
| Challenge: | Recent years have seen the advent of large language models characterized by emergent capabilities arising from sheer scale alone. |
| Approach: | They propose to use a multilingual model to compare performance to the English-only model by ablation at the billion-parameter scale. |
| Outcome: | The proposed model is based on a multilingual model and its performance against the English-only model. |
TESS 2: A Large-Scale Generalist Diffusion Language Model (2025.acl-long)
Copied to clipboard
| Challenge: | Existing instruction-following diffusion models are predominantly trained using an autoregressive paradigm. |
| Approach: | They propose a general instruction-following diffusion language model that outperforms contemporary instruction-tuned diffusion models and matches and sometimes exceeds strong autoregressive (AR) models. |
| Outcome: | The proposed model outperforms and sometimes exceeds existing autoregressive (AR) models on a number of tasks. |
Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies (2023.findings-emnlp)
Copied to clipboard
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir Radev
| Challenge: | In-context learning (ICL) is a new approach to natural language processing tasks that rely on large language models to make predictions based on context . recent studies have shown that neural symbolic design is the preferred choice for question answering systems because of its limited working memory and unreliable long-term memory. |
| Approach: | They propose to extend in-context learning to question answering tasks that utilize structured knowledge sources and to explore various prompt design strategies for employing LLMs. |
| Outcome: | The proposed approach outperforms the state-of-the-art system by 2.5 points and the best fine-tuned system by 5.1 points on the Spider dataset. |
Surfer100: Generating Surveys From Web Resources, Wikipedia-style (2022.lrec-1)
Copied to clipboard
Irene Li, Alex Fabbri, Rina Kawamura, Yixin Liu, Xiangru Tang, Jaesung Tae, Chang Shen, Sally Ma, Tomoe Mizutani, Dragomir Radev
| Challenge: | Recent work on Wikipedia page generation focuses on generating the initial leading paragraph of a page, while recent pretrained language models improve upon both extractive and abstractive steps of previous models. |
| Approach: | They propose a pretrained language model that can be combined to generate Wikipedia-style summaries with sections using 100 reference human-collected surveys. |
| Outcome: | The proposed approach is compared with existing methods with 100 human-collected surveys. |