Papers by Eric Tang
RWKV: Reinventing RNNs for the Transformer Era (2023.findings-emnlp)
Copied to clipboard
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Gv, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan Wind, Stanisław Woźniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
| Challenge: | recurrent neural networks struggle to match the performance of Transformers due to limitations in parallelization and scalability. |
| Approach: | They propose a model architecture that combines the efficient parallelizable training of transformers with the efficient inference of RNNs. |
| Outcome: | The proposed model performs on par with similarly sized RNNs, suggesting future work can leverage this architecture to create more efficient models. |
GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoning (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to solve geometric problems are dependent on handcraft rules and limited on small-scale datasets. |
| Approach: | They propose a Geometric Question Answering dataset with 5,010 geometric problems with corresponding annotated programs to illustrate the solving process. |
| Outcome: | The proposed method is significantly lower than human performance on the proposed dataset than on a publicly available dataset. |
Target-Guided Open-Domain Conversation (P19-1)
Copied to clipboard
| Challenge: | a new study aims to improve opendomain chat systems by integrating goals and strategy into the system. |
| Approach: | They propose a structured approach that introduces coarse-grained keywords to control intended content of system responses and attains smooth conversation transition through turn-level supervised learning. |
| Outcome: | The proposed system produces meaningful and effective conversations significantly better than other approaches. |
Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | a high prompt sensitivity has been widely accepted as a core limitation of large language models . a recent study suggests that prompt senescence may be an artifact of evaluation processes . |
| Approach: | They examine whether prompt sensitivity is an inherent weakness or an artifact of evaluation . they find that heuristic evaluation methods overlook semantically correct responses . large language models have achieved remarkable success across a wide range of tasks . |
| Outcome: | The proposed model is more robust to prompt templates than previously thought . the authors show that prompt sensitivity may be an artifact of evaluation rather than a flaw . |
Language Models Can Easily Learn to Reason from Demonstrations (2025.findings-emnlp)
Copied to clipboard
Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G Patil, Matei Zaharia, Joseph E. Gonzalez, Ion Stoica
| Challenge: | Large reasoning models (LRMs) tackle complex problems by following long chain-of-thoughts (Long CoT) however, the training techniques and data requirements to elicit Long CoT remain poorly understood. |
| Approach: | They propose to use data-efficient supervised fine-tuning and parameter-efficient low-rank adaptation to elicit Long CoT reasoning. |
| Outcome: | The proposed model can learn Long CoT reasoning through data-efficient supervised fine-tuning and parameter-efficient low-rank adaptation. |
BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to construct knowledge graphs are limited to a small set of relations due to manual cost or restrictions in text corpus. |
| Approach: | They propose to automatically construct knowledge graphs (KGs) of diverse new relations from pretrained language models that accept knowledge queries with prompts. |
| Outcome: | The proposed framework extracts knowledge of over 400 new relations from pretrained language models, including RoBERTaNet, with minimal input of a relation definition and a few shot of example entity pairs. |