Papers by Thuat Nguyen
MTA: Multi-Granular Trajectory Alignment for Large Language Model Distillation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reducing the computational cost of large language models (LLMs) focus on minimizing the divergence between the output probability distributions of the teacher and the student, which limits knowledge transfer. |
| Approach: | They propose a framework that aligns teacher and student representations along their layer-wise transformation trajectory. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on teacher–student layers. |
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing instruction-tuned open-source LLMs have only been instruction- tuned for English and a few popular languages, thus hindering their accessibility to many other languages in the world. |
| Approach: | They propose a framework that uses supervised fine-tuning and reinforcement learning from human feedback to improve the accessibility of large language models. |
| Outcome: | The proposed framework enables the evaluation of generative LLMs in multiple languages. |
CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages (2024.lrec-main)
Copied to clipboard
Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai, Hieu Man, Nghia Trung Ngo, Franck Dernoncourt, Ryan A. Rossi, Thien Huu Nguyen
| Challenge: | Existing training datasets for large language models are often not fully disclosed. |
| Approach: | They propose a multilingual dataset with 6.3 trillion tokens in 167 languages . they use a pipeline of multiple stages to achieve the best quality for model training . |
| Outcome: | The proposed dataset is cleaned and deduplicated to achieve the best quality for model training . lack of transparency has hindered research on attributing and addressing hallucination and bias issues . 6.3 trillion tokens in 167 languages are used to train multilingual LLMs . |
Introducing a New Dataset for Event Detection in Cybersecurity Texts (2020.emnlp-main)
Copied to clipboard
| Challenge: | a large amount of text data is produced to report and discuss cyber vulnerabilities . detecting cybersecurity events is necessary to keep us informed about the fast growing number of such events reported in text. |
| Approach: | They propose a dataset characterizing the manual annotation for 30 important cybersecurity event types and a large dataset to develop deep learning models. |
| Outcome: | The proposed dataset characterizes the manual annotation for 30 important event types and supports the modeling of document-level information to improve the performance. |