Papers by Ting-Rui Chiang
Semantically-Aligned Equation Generation for Solving and Reasoning Math Word Problems (N19-1)
Copied to clipboard
| Challenge: | Existing methods to solve math word problems require accurate natural language understanding to bridge texts and math expressions. |
| Approach: | They propose a neural approach to automatically solve math word problems by operating symbols according to their semantic meanings in texts. |
| Outcome: | The proposed model outperforms state-of-the-art models and the best non-retrieval-based models over 10% accuracy in a Math23K dataset. |
Pelican Soup Framework: A Theoretical Framework for Language Model Capabilities (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing theoretical frameworks for large language models (LLMs) do not explain how pretraining leads to in-context learning. |
| Approach: | They propose a theoretical framework that allows LLMs to generalize to unseen instructions and perform in-context learning even when verbalizers are irrelevant to the task. |
| Outcome: | The proposed framework can be used to analyze LLMs' ability to perform in-context learning . it can be applied to linguistic, psychology, and philosophy tasks . |
DialCrowd 2.0: A Quality-Focused Dialog System Crowdsourcing Toolkit (2022.lrec-1)
Copied to clipboard
| Challenge: | DialCrowd 2.0 helps requesters obtain higher quality data from human intelligence tasks. |
| Approach: | They propose to use DialCrowd 2.0 to help requesters obtain higher quality data . they aim to improve the way requesters present tasks and facilitate effective communication with workers. |
| Outcome: | The proposed toolkit enables requesters to obtain higher quality data by presenting tasks more clearly and facilitating effective communication with workers. |
The Distributional Hypothesis Does Not Fully Explain the Benefits of Masked Language Model Pretraining (2023.emnlp-main)
Copied to clipboard
| Challenge: | Despite the rise of the prompting paradigm with the scaling breakthrough of very large language models, understanding the mechanism of model fine-tuning remains an important endeavor. |
| Approach: | They analyze the masked language modeling pretraining objective function from the perspective of the Distributional Hypothesis and examine whether the distributional property leads to better sample efficiency and better generalization capability of pretrained models. |
| Outcome: | The proposed model pretraining objective function improves sample efficiency and generalization capability but does not explain the generalization ability of natural language models. |
Breaking Down Multilingual Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Multilingual training is an essential ingredient in machine translation systems . but it has different effects in different multilingual settings, such as many-to-one, one-tomany and many- to-many learning . |
| Approach: | They compare multilingual training settings with encoders and decoders initialized by multilingual learning . they find important attention heads for each language pair and compare their correlations during inference . |
| Outcome: | The proposed models outperform the best models for high-resource languages and one-to-many models for low-resourced languages. |
On Retrieval Augmentation and the Limitations of Language Model Training (2024.naacl-short)
Copied to clipboard
| Challenge: | Recent efforts to improve the performance of language models (LMs) have focused on scaling up model and training data size, though with steep accompanying energy and compute resource costs. |
| Approach: | They propose to augment a language model with k-nearest neighbors retrieval on its training data to reduce its perplexity. |
| Outcome: | The proposed model reduces storage costs by over 25x compared to traditional retrieval methods for GPT-2 and Mistral 7B . |
The Rotary Position Embedding May Cause Dimension Inefficiency in Attention Heads for Long-Distance Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | We hypothesize that the wide range of rotation angles may prevent LLMs from utilizing certain dimensions. |
| Approach: | They propose to use the Rotary Position Embedding (RoPE) for long context modeling . they hypothesize that the wide range of rotation angles may prevent LLMs from utilizing those dimensions. |
| Outcome: | The proposed model may not be useful for long-context modeling. |