Papers by Sung Hwang
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) are proficient in natural language processing tasks, but their deployment is limited by extensive parameter sizes and computational demands. |
| Approach: | They propose a method to enhance computational efficiency in large language models by 4-bit weight and 8-bit activation quantization. |
| Outcome: | The proposed techniques significantly boost task accuracies to levels comparable with full-precision models. |
Rationale-Guided Retrieval Augmented Generation for Medical Question Answering (2025.naacl-long)
Copied to clipboard
Jiwoong Sohn, Yein Park, Chanwoong Yoon, Sihyeon Park, Hyeon Hwang, Mujeen Sung, Hyunjae Kim, Jaewoo Kang
| Challenge: | Large language models (LLMs) struggle with hallucinations and outdated knowledge. |
| Approach: | They propose a retrieval-augmented generation framework for enhancing the reliability of RAG in biomedical contexts. |
| Outcome: | The proposed framework outperforms the previous best medical RAG model by up to 5.6% across three medical question-answering benchmarks. |
CR-COPEC: Causal Rationale of Corporate Performance Changes to learn from Financial Reports (2023.findings-emnlp)
Copied to clipboard
Ye Chun, Sunjae Kwon, Kyunghwan Sohn, Nakwon Sung, Junyoup Lee, Byoung Seo, Kevin Compher, Seung-won Hwang, Jaesik Choi
| Challenge: | CR-COPEC is a large-scale domain-adaptation causal sentence dataset to detect financial performance changes of corporate. |
| Approach: | They propose a large-scale domain-adaptation causal sentence dataset to detect financial performance changes of corporate. |
| Outcome: | The proposed dataset can be used by investors and analysts without having to read through all the documents. |
Knowledge-Augmented Language Model Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent Language Models (LMs) generate factually incorrect answers to queries . authors propose to augment LMs with knowledge retrieved from external source . |
| Approach: | They propose to augment LMs with knowledge retrieved from external sources . they validate the output and knowledge of the knowledge-augmented LM with a separate verifier . |
| Outcome: | The proposed model can generate factually incorrect answers on multiple questions . a verifier detects retrieval errors and can correct them by retrieving new knowledge or generating new text . |
Cold-Start Aware User and Product Attention for Sentiment Classification (P18-1)
Copied to clipboard
| Challenge: | Existing models do not deal with cold-start problem typical in review websites. |
| Approach: | They propose a Hybrid Contextualized Sentiment Classifier that uses word encoder and Cold-Start Aware Attention to pool word vectors. |
| Outcome: | The proposed model performs significantly better on famous datasets despite having less complexity and can be trained much faster. |
Test-Time Self-Adaptive Small Language Models for Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent instruction-finetuned large language models (LMs) have shown notable performances in various tasks, such as question-answering. |
| Approach: | They propose to use unlabeled test data to transfer smaller language models with limited knowledge. |
| Outcome: | The proposed strategy shows significant performance improvements on benchmark QA datasets with higher robustness across diverse prompts, enabling LMs to stay stable. |
Co-training and Co-distillation for Quality Improvement and Compression of Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge Distillation (KD) compresses expensive pre-trained language models . however, most smaller models fail to surpass performance of larger model . |
| Approach: | They propose a framework that co-trains two models while mutually distilling knowledge to improve performance and inference speed together. |
| Outcome: | The proposed framework outperforms the original larger model by 1.66 on the GLUE benchmark. |