Papers by Chanjun Park
Copied to clipboard
| Challenge: | Korean pretrained language models struggle to generate short sentences with a given condition based on compositionality and commonsense reasoning. |
| Approach: | They propose a Korean text-generation dataset for Korean generative commonsense reasoning and language model evaluation using a semi-automatic dataset construction approach. |
| Outcome: | The proposed dataset is available at http://aihub.or.kr/opendata/korea-university. |
Copied to clipboard
| Challenge: | Recent machine translation systems overcome language barriers for a wide range of users, yet they carry the risk of catastrophic meaning deviations. |
| Approach: | They introduce a culture-aware "Politeness" type for detecting critical translation errors . they also provide multiclass labels for critical error detection and critical error type classification . |
| Outcome: | Empirical results show that the proposed method outperforms baselines in both tasks. |
Copied to clipboard
| Challenge: | Creating high-quality datasets for large language models often relies on resource-intensive, GPU-accelerated models for quality filtering, making the process time-consuming and costly. |
| Approach: | They propose a framework that operates entirely on CPUs to streamline the processes of dataset extraction, filtering, and curation. |
| Outcome: | The proposed framework reduces preparation time and costs while maintaining high data quality while enhancing the applicability of LLMs in specialized contexts. |
Copied to clipboard
| Challenge: | Existing evaluations of the open-domain question answering task focus solely on whether the model provides the correct answer. |
| Approach: | They propose to examine the phenomenon of discrepancies in abilities across two distinct tasks—QA and evidence selection—when performed simultaneously. |
| Outcome: | The proposed framework and resources examines the ability of large language models to perform two distinct tasks simultaneously, from the perspective of task alignment. |
Copied to clipboard
| Challenge: | Byte Pair Encoding (BPE) is an effective approach in machine translation across several languages, but it is prone to over-segmentation in Korean, an agglutinative and morphologically rich language. |
| Approach: | They propose a new method that incorporates long words into the Korean vocabulary by strategically preserving morphological information and reducing semantic confusion. |
| Outcome: | The proposed method outperforms BPE and surpasses state-of-the-art morpheme-aware tokenization methods. |
Copied to clipboard
| Challenge: | Existing methods for dense retrieval in multilingual environments encode language identity alongside semantics. |
| Approach: | They propose a method that trains on pooled embeddings to remove language-identity signal directly in vector space. |
| Outcome: | The proposed method improves ranking quality and cross-language coverage across multiple languages with especially strong gains for script-distinct languages. |
Copied to clipboard
| Challenge: | Recent large language models (LLMs) demonstrate multilingual abilities, yet they are English-centric due to dominance of English in training corpora. |
| Approach: | They propose to use a synthetic English-korean CS question-answering dataset to investigate this potential. |
| Outcome: | The proposed model can activate, identify and leverage knowledge for reasoning in low-resource languages. |
Copied to clipboard
| Challenge: | Existing chatbots lack realistic practice scenarios for English learners . existing platforms employ hand-crafted and patternmatching rules, limiting communication ability and responding appropriately to out-of-situation utterances. |
| Approach: | They propose a real-world situational dialogue-based chatbot for English education . it generates appropriate responses in various real-life situations while providing accurate feedback . |
| Outcome: | The proposed chatbot generates appropriate responses in various real-life situations while providing accurate feedback to learners. |
Copied to clipboard
| Challenge: | Large language models (LLMs) have improved IE, but their potential for ATE has not been explored. |
| Approach: | They propose a retrieval-based prompting strategy that selects demonstrations according to syntactic rather than semantic similarity in a few-shot setting. |
| Outcome: | The proposed method improves performance on three specialized ATE benchmarks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) often retain outdated or incorrect information from pre-training, which undermines their reliability. |
| Approach: | They propose a conflict-free model editing framework that selectively removes outdated knowledge from LLMs to improve their accuracy and reliability. |
| Outcome: | The proposed framework improves both editing accuracy and model reliability when applied to existing editing methods. |
Copied to clipboard
| Challenge: | Evalverse is a library that unifies disparate evaluation tools into a single, user-friendly framework. |
| Approach: | They propose to integrate existing evaluation frameworks into a single, user-friendly framework that enables individuals with limited knowledge of artificial intelligence to request LLM evaluations and receive detailed reports. |
| Outcome: | The proposed framework can be used by individuals with limited knowledge of artificial intelligence to request and receive LLM evaluations and receive detailed reports. |
Copied to clipboard
| Challenge: | Language models are striving to grasp commonsense reasoning, but they are lacking in Korean commons- ense benchmarks. |
| Approach: | They present a fine-grained benchmark dataset focused on Korean commonsense reasoning that includes multiple-choice questions across seven error categories. |
| Outcome: | The proposed datasets show that LLMs struggle with Korean commonsense reasoning . human accuracy benchmarked at approximately 85%, while GPT-4’s performance lags at about 74%, and other LLM models demonstrate an average accuracy of around 42%. |
Copied to clipboard
| Challenge: | Conventional evaluation metrics for automatic speech recognition systems produce a singular aggregate score, which is insufficient for understanding specific system vulnerabilities. |
| Approach: | They propose to introduce the Korean Error Explainable Benchmark Dataset for ASR and Post-processing (KEBAP) this method enables a more balanced assessment encompassing speech recognition accuracy and user readability. |
| Outcome: | The proposed method enables a more balanced assessment encompassing speech recognition accuracy and user readability. |
Copied to clipboard
| Challenge: | Existing benchmarks for evaluating Large Language Models are limited to the English language. |
| Approach: | They introduce the Open Ko-LLM Leaderboard and Ko-H5 Benchmark as tools for evaluating Large Language Models in Korean using private test sets. |
| Outcome: | The proposed evaluation framework is well integrated in the Korean LLM community. |
Copied to clipboard
| Challenge: | Recent efforts in LLM alignment focus on instance-wise supervision, costing substantial . ZEBRA binarizes response pairs by evaluating the quality and similarity of their origin models . |
| Approach: | They propose a model behavior-wise zero-annotation framework that binarizes preference data . ZEBRA binarized response pairs by evaluating the quality and similarity of their origin models . |
| Outcome: | The proposed framework achieves comparable alignment performance to instance-supervised methods . |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been criticized for perpetuating stereotypes against diverse groups based on race, sexual orientation, and other attributes. |
| Approach: | They devised a set of prompts that reflect major societal issues in Korea and assign varied personas to both ChatGPT and GPT-4 to assess the toxicity of the generated sentences. |
| Outcome: | The proposed model produces twice the level of toxic content as ChatGPT and GPT-4 under certain conditions. |
Copied to clipboard
| Challenge: | Large Language Models are often judged by their scores on standard benchmarks, yet such scores often overstate real capability since they mask the mix of skills a task actually demands. |
| Approach: | They propose a diagnostic framework that decomposes benchmark performance into ten cognitively grounded abilities and computes an Ability Impact Score (AIS) AIS quantifies how much each ability contributes to a model’s success on a given benchmark. |
| Outcome: | The proposed framework decomposes performance into ten cognitively grounded abilities and computes an Ability Impact Score (AIS) that quantifies how much each ability contributes to a model’s success on a given benchmark. |
Copied to clipboard
| Challenge: | Korean morphological variations present unique opportunities and challenges in natural language processing (NLP), necessitating an advanced understanding of morpheme-based sentence construction. |
| Approach: | They propose a method to replicate morphological transformations inherent in Korean sentences based on lexical and functional morphemes through generative data augmentation. |
| Outcome: | The proposed method improves performance in Korean multiple classification datasets without incurring external data usage. |
Copied to clipboard
| Challenge: | Existing safety evaluations may overlook the inherent weaknesses of Large Language Models, despite their benefits. |
| Approach: | They propose a benchmark to evaluate the robustness of Large Language Models under extreme conditions. |
| Outcome: | The proposed approach evaluates the fairness of large language models under extreme conditions. |
Copied to clipboard
| Challenge: | Recent studies have focused on the restoration and translation of historical languages. |
| Approach: | They propose to use two different stimuli to priming ancient-Korean NMT . they confirm the possibility of developing a human-centric model based on cognitive science . |
| Outcome: | The proposed model can be used to translate historical Korean documents using neural machine translation. |
Copied to clipboard
| Challenge: | Recent studies using pretrain-finetuning approach have achieved state-of-the-art (SOTA) performance in many natural language processing tasks. |
| Approach: | They propose a new tokenization method that combines morphology-considered subword tokenization and vocabulary methods to address this limitation. |
| Outcome: | The proposed method can be used without modifying the model structure. |
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) post-processing requires substantial amounts of data, requiring expensive phonetic transcription experts. |
| Approach: | They propose a "Hyper-BTS" dataset that is five times larger than prior studies . they propose criteria for categorizing error types within ASR post-processing . |
| Outcome: | The proposed method can generate ASR inputs from clean text using a text-to-speech system. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated proficiency in code generation and comprehension across multiple programming languages. |
| Approach: | They propose a parameter-localized subset of LLMs that facilitates coding capabilities. |
| Outcome: | The proposed model significantly improves performance on coding tasks while preserving non-coding functionalities. |
Copied to clipboard
| Challenge: | Rapid advancements in large language models have highlighted the need for robust evaluation frameworks that assess their core capabilities. |
| Approach: | They propose two benchmarks to assess core capabilities of large language models . current benchmarks for Thai focus mainly on traditional NLP tasks . |
| Outcome: | The proposed benchmarks are based on evaluations of various LLMs with multi-lingual capabilities and are publicly available to encourage further research and development for Thai LLM. |
Copied to clipboard
| Challenge: | Existing evaluation methods often fail to produce objective results and favor high similarity to the ground-truth question-answer pairs. |
| Approach: | They propose an alternative approach to evaluate question-answer generation using Generative Interpretation (GI) GI outperforms existing evaluation methods in terms of human alignment . |
| Outcome: | The proposed approach outperforms existing evaluation methods in human alignment and shows comparable performance with GPT3.5, only with BART-large. |
Copied to clipboard
| Challenge: | Maintaining consistent character personas remains a significant challenge due to variability in information extraction. |
| Approach: | They propose a framework to dynamically reconstruct character personas through Character Persona Training. |
| Outcome: | The proposed framework is evaluated through Big Five personality evaluations and creative tasks, in which characters generate original narratives. |
Copied to clipboard
| Challenge: | Existing methods to efficiently filter large web corpora require GPU resources. |
| Approach: | They propose an ensemble approach that leverages two contrasting KenLMs to filter large web corpora. |
| Outcome: | The proposed method significantly reduces noisy content while preserving high-quality content compared to the traditional KenLM training method. |
Copied to clipboard
| Challenge: | Existing studies of critical error detection lack content addressing the causes of catastrophic errors. |
| Approach: | They propose a dataset that introduces the attributes of error explanation and correction regarding critical errors. |
| Outcome: | The proposed dataset reduces time costs and mitigates human annotation bias. |
Copied to clipboard
| Challenge: | Retrieval-Augmented Generation (RAG) systems are limited in their evaluation due to the intricate interplay between retrieval and generation components. |
| Approach: | They propose a Question Answering Question Answerer dataset specifically designed for RAG evaluation that integrates external, non-parametric knowledge retrieved by a retrieval pool of 37,800 entries. |
| Outcome: | The proposed dataset consists of 7,560 curated instances mapped to a retrieval pool of 37,800 entries, enabling an efficient evaluation of both retrieval and generation tasks. |
Copied to clipboard
| Challenge: | Automatic post-editing (APE) is a research field that aims to correct errors in translated sentences regardless of the utilized machine translation system. |
| Approach: | They propose a method for automatically generating APE data based on a noising scheme from a parallel corpus. |
| Outcome: | The proposed method shows that depending on the type of noise, the noising scheme-based APE data generation may lead to inferior performance. |
Copied to clipboard
| Challenge: | Existing studies on the Open Ko-LLM Leaderboard have been limited to five months . this limited analysis of the Open LLM Leaderboard provides a more comprehensive understanding of the progress in developing large language models. |
| Approach: | They conduct a longitudinal study over eleven months to address limitations of previous studies . they analyze 1,769 models over this period to provide a more comprehensive understanding . |
| Outcome: | The study extends observation period of the Open Ko-LLM Leaderboard to eleven months . primary questions are: What are the specific challenges in improving LLM performance? |
Copied to clipboard
| Challenge: | Open Ko-LLM Leaderboard has been instrumental in benchmarking Korean Large Language Models . however, the leaderboard has faced significant limitations over time due to its academic nature . |
| Approach: | They propose an improved version of the Open Ko-LLM Leaderboard to improve benchmarking . original benchmarks replaced with new tasks that align with real-world capabilities . four new native Korean benchmarks are introduced to better reflect distinct characteristics of Korean language . |
| Outcome: | The proposed framework improves the Open Ko-LLM Leaderboard2 benchmark suite. |
Copied to clipboard
| Challenge: | LLMs are prone to generate responses to instruction-formatted statements in an instinctive manner, rather than comprehending the underlying user intention within the given instructions. |
| Approach: | They propose to use an instruction-following capability benchmark to evaluate LLMs' instruction understanding capability. |
| Outcome: | The proposed benchmark analyzes the instruction understanding capability of large language models with four instruction candidates and a single candidate. |
Copied to clipboard
| Challenge: | FreeTalky is a deep learning-based foreign language learning platform for people who experience anxiety dealing with foreign languages. |
| Approach: | They propose a deep learning-based foreign language learning platform called FreeTalky . it employs a humanoid robot NAO and various deep learning models . |
| Outcome: | The proposed system provides personalized learning based on persona dialogue and grammar error correction, and also helps alleviate xenoglossophobia by replacing the real human in the conversation with a NAO robot, through human evaluation. |
Copied to clipboard
| Challenge: | a novel MT pipeline that considers the intra-data relation is proposed . previous MT systems have demonstrated relatively low performance, making them hardly utilized as another data source. |
| Approach: | They propose a new MT pipeline that considers the intra-data relation . they propose CS and IT to enhance the intra data relation based on a data point . |
| Outcome: | The proposed pipeline improves translation quality and training data compared with the existing approach . it yields better training data and better translation quality than previous approaches . |
Copied to clipboard
| Challenge: | Existing software packages are expensive and difficult to use, and only provide simple functions. |
| Approach: | They propose an AI-based AAC system called PicTalky that can improve communication skills for children with language disabilities. |
| Outcome: | The proposed system improves communication skills and language comprehension abilities for children with language disabilities. |
Copied to clipboard
| Challenge: | Large language models excel in generating coherent and contextually rich outputs, but their capacity to handle long-form contexts is limited by fixed-length position embeddings. |
| Approach: | They propose a method that enables the efficient processing long-form sequences beyond the model’s length limit through recurrent compression without retraining the entire model. |
| Outcome: | The proposed method significantly improves LLM’s ability to manage extended contexts, making it well-suited for tasks that require both comprehensive context understanding and query relevance. |
Copied to clipboard
| Challenge: | a problem of information sparsity in QA tasks is causing fragmentation of textual data . highlighting entity-AWare Knowledge (HAWK) framework can be used to address this problem . |
| Approach: | a framework is proposed to highlight key information in a context and structuralize it in an entity-aware manner. |
| Outcome: | a proposed framework improves QA tasks with long contexts by highlighting key information in a context . the framework achieves a 27.6-point F1 score increase and an average win rate of 76.75% . |
Copied to clipboard
| Challenge: | A sparse Mixture-of-Experts architecture has emerged as a highly scalable solution for instruction tuning. |
| Approach: | They propose a mixture-of-Clustered-Experts (MoCE) architecture that allows expert specialization . they evaluate the mechanism on a set of benchmarks and show its superiority . |
| Outcome: | The proposed approach outperforms existing models and benchmarks on instruction tuning scenarios with significant input heterogeneity. |
Copied to clipboard
| Challenge: | Existing datasets and methods for detecting hate speech are limited by resource-intensive nature and only focus on the primary language. |
| Approach: | They propose a Korean Hate Speech Counter Punch (KHSCP) method that generates fact-based responses to hate speech in the Korean language and propose to use existing resources to overcome data scarcity. |
| Outcome: | The proposed method can overcome data scarcity in low-resource environments by leveraging existing resources. |
Copied to clipboard
| Challenge: | Retrieval-augmented generation (RAG) is a key framework in natural language processing . however, the effectiveness of RAG is often hindered by coreferential complexity in retrieved documents . |
| Approach: | They investigate how entity coreference affects document retrieval and generative performance in RAG-based systems. |
| Outcome: | The proposed model improves QA performance and retrieval relevance and contextual understanding. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly requiring precision and accuracy in alignment tuning. |
| Approach: | They propose a stepwise DPO technique that partitions available preference datasets incrementally rather than utilizing entire dataset simultaneously. |
| Outcome: | The proposed technique improves the accuracy of reference models and the overall performance of the final model. |
Copied to clipboard
| Challenge: | Existing studies have shown that the performance of large language models is insufficient for non-English data, such as Korean. |
| Approach: | They propose a framework that integrates evidential information from external sources into the prompt for the Korean GEC task. |
| Outcome: | The proposed framework extracts salient phrases from the given source and retrieves non-parametric knowledge based on these phrases. |
Copied to clipboard
| Challenge: | despite its high utility, there are limitations concerning manual QE data creation. |
| Approach: | They propose to generate a Korean-English QE dataset that is fully automatic . they find that the algorithm is more accurate and faster than manual QE . |
| Outcome: | The proposed datasets show that they scale up to 1.58M and 6.58M, respectively, and show that the results are significantly better when compared to the previous datasets. |
Copied to clipboard
| Challenge: | Existing approaches to enhance mathematical reasoning and problem-solving abilities of Large Language Models (LLMs) despite their remarkable performance across domains, a notable challenge persists in the realm of mathematical reasoning. |
| Approach: | They propose a sequential learning approach that integrates the Chain-of-Thought and the Program-ofThough. |
| Outcome: | The proposed approach achieves state-of-the-art (SOTA) performance by integrating CoT and PoT learning. |
Copied to clipboard
| Challenge: | Existing text chunking methods neglect complex and long industrial document structures, causing information loss and reduced answer quality. |
| Approach: | They propose a multimodal chunking pipeline that detects document regions and extracts text from them via OCR. |
| Outcome: | Extensive tests show that MultiDocFusion improves retrieval precision by 8–15% and ANLS QA scores by 2–3% compared to baselines. |