Papers by Jang Kim
Copied to clipboard
| Challenge: | voluminous historical records are difficult to fully utilize since they are written in ancient languages and some parts are damaged over time. |
| Approach: | They propose a multi-task learning approach to restore and translate historical documents using a self-attention mechanism. |
| Outcome: | The proposed approach improves the accuracy of the translation task over baselines without multi-task learning. |
Copied to clipboard
| Challenge: | Existing safety evaluations rely on coarse success rates and domain-specific setups, making it difficult to diagnose why and where these models fail. |
| Approach: | They propose a framework for systematically evaluating the physical safety of LLMs in embodied decision making. |
| Outcome: | The proposed framework assesses the physical safety of LLMs in embodied decision making. |
Copied to clipboard
| Challenge: | Reinforcement learning with verifiable rewards and Reinforced Learning from internal feedback fail to benefit from test-time compute due to entropy collapse and the resulting loss of reasoning diversity. |
| Approach: | They propose a strategy that assigns each generated token a redistribution score and applies selective KL regularization to only the top 5% of tokens under this score. |
| Outcome: | The proposed model improves on both RLVR and RLIF models on math reasoning benchmarks, showing that targeted entropy control at a vanishingly small subset of tokens is sufficient to sustain reasoning diversity and effective test-time scaling. |
Copied to clipboard
| Challenge: | End-to-end dialogue systems with monolithic neural architecture are often trained with input-output utterances without taking into account the entire annotations available in the corpus. |
| Approach: | They propose an end-to-end neural architecture for goal-oriented dialogue systems that addresses both challenges . they propose a modular architecture where modules are optimized individually . |
| Outcome: | The proposed system achieved the top position in the human evaluation task . it is based on a neural architecture that can be integrated with external systems . |
Copied to clipboard
| Challenge: | Spurious correlations are patterns that appear in datasets but do not represent genuine relationships. |
| Approach: | They propose a more general form of counterfactual data augmentation that tackles multiple biases . they propose 'CoBA' that decomposes text into subject-predicate-object triples and modifies them to disrupt spurious correlations. |
| Outcome: | The proposed framework reduces biases and strengthens out-of-distribution resilience. |
Copied to clipboard
| Challenge: | Existing evaluation frameworks for multimodal large language models suffer from limitations . modality shortcuts and biased reasoning paths are common in such models . |
| Approach: | a new benchmark evaluates omni-modal multi-hop reasoning using 6,144 questions . authors propose OMHBench to address these limitations by comparing modalities . |
| Outcome: | OMHBench evaluates omni-modal multi-hop reasoning on 6,144 questions with balanced reasoning paths . evaluation of 13 state-of-the-art models shows performance gap exists between MLLMs and open-source models . |
Copied to clipboard
| Challenge: | Existing evaluation methods do not assess whether large language models fully utilize contextual information. |
| Approach: | They introduce a new metric to assess LLMs' ability to fully utilize contextual information. |
| Outcome: | The proposed benchmark comprises 1,986 test instances spanning four long-context tasks with high IC scores in the domains of books, debates, medicine, and law. |
Copied to clipboard
| Challenge: | Existing evaluations of the open-domain question answering task focus solely on whether the model provides the correct answer. |
| Approach: | They propose to examine the phenomenon of discrepancies in abilities across two distinct tasks—QA and evidence selection—when performed simultaneously. |
| Outcome: | The proposed framework and resources examines the ability of large language models to perform two distinct tasks simultaneously, from the perspective of task alignment. |
Copied to clipboard
| Challenge: | Current benchmarks typically compare system outputs against baselines, but this method yields lower reliability than direct comparison. |
| Approach: | They propose to integrate tournament structure on top of head-to-head comparison. |
| Outcome: | The proposed model achieves higher reliability with fewer comparisons even with smaller datasets or weaker judges. |
Copied to clipboard
| Challenge: | open-domain chatbots focus on short single-session dialogue, neglecting the potential need for understanding contextual information in multiple consecutive sessions. |
| Approach: | They propose a 1M multi-session dialogue dataset for integrating time intervals and speaker relationships into a long-term conversation setup. |
| Outcome: | The proposed model can generate coherent responses according to time intervals and speaker relationships with high user engagement without contradiction in a long-term conversation setup. |
Copied to clipboard
| Challenge: | Recent advances in Vision Language Models (VLMs) focus on aligning images with short descriptive captions. |
| Approach: | They propose a method that combines VLMs with Large Language Models to efficiently align images with long texts without additional text pairs. |
| Outcome: | The proposed method bridges the gap between VLM and LLM without additional image-long text pairs. |
Copied to clipboard
| Challenge: | Existing knowledge distillation methods for autoregressive large language models (LLMs) are not effective for reducing generation quality, but they can be useful for real-time applications. |
| Approach: | They propose a self-distillation framework that allows for effective KD without external teacher . they propose to use two modes of knowledge distillation to determine distillation direction . |
| Outcome: | The proposed framework outperforms existing methods with external teachers on instruction-following tasks. |
Copied to clipboard
| Challenge: | Featured snippets are a compressed excerpt that contains the answer to a user's query . knowledge-snippet is a useful tool for generating information retrieval services such as google. |
| Approach: | They propose to automatically extract query-knowledge snippet pairs from structured HTML documents using a new Language Model. |
| Outcome: | The proposed framework improves the quality of generated knowledge-snippets in real-world environments. |
Copied to clipboard
| Challenge: | a lack of comprehensive comparison between GR and Dual Encoders in IR systems is limiting . GR is more adaptable to evolving knowledge (4–11%), robust in learning knowledge with temporal information, and efficient in terms of inference FLOPs (x2), indexing time (x6) and storage footprint (x4) |
| Approach: | They propose to use autoregressive language models to perform information retrieval (IR) their results highlight the potential of GR for future use in practical IR systems . |
| Outcome: | The proposed model is more adaptable to evolving knowledge (4–11%), robust in learning knowledge with temporal information, efficient inference FLOPs (x2), indexing time (x6), and storage footprint (x4) compared to the most common model, Dual Encoder (DE). |
Copied to clipboard
| Challenge: | a well-formulated benchmark allows objective and precise evaluation of diverse models. |
| Approach: | They propose a benchmark for Korean balanced evaluation of significant tasks that requires advanced Korean linguistic knowledge. |
| Outcome: | The proposed benchmarks are based on five Korean-language downstream tasks . the data is annotated by humans and thoroughly reviewed to guarantee high data quality. |
Copied to clipboard
| Challenge: | Existing methods rely on linear sequential operations to solve First-Order Logic queries. |
| Approach: | They propose a model-agnostic approach that fully integrates the context of the query graph. |
| Outcome: | The proposed method improves performance on two datasets by 19.5%. |
Copied to clipboard
| Challenge: | Prompt tuning has achieved remarkable progress in vision–language models, but its generalization ability in ALMs remains underexplored. |
| Approach: | They propose a plug-and-play framework that regularizes the prompt embedding space . they propose introducing a semantic expansion loss with margin constraints that promote compactness . |
| Outcome: | The proposed framework regularizes the prompt embedding space by incorporating semantic neighbors generated by large language models. |
Copied to clipboard
| Challenge: | Recent advances in large language models have drawn attention for their potential to automate and optimize processes across diverse sectors. |
| Approach: | They propose a specialized LLM for plant construction engineering that delivers optimized responses to plant engineers by leveraging enriched domain knowledge. |
| Outcome: | The proposed model delivers optimized responses to plant engineers by leveraging enriched domain knowledge. |
Copied to clipboard
| Challenge: | a recent study using LLMs has relied on outdated datasets and limited generalization ability on unseen texts. |
| Approach: | They construct a large-scale dataset of political discourse and use it to make three judgments . they identify distinct patterns and demonstrate tendencies of label agreement using a leave-one-out strategy. |
| Outcome: | The proposed approach is applicable in real-world settings with inherent constraints. |
Copied to clipboard
| Challenge: | Existing LLMs are limited in their ability to incorporate feedback from an environment. |
| Approach: | They propose an LLM agent that consists of an Actor and a Critic. |
| Outcome: | The proposed agent outperforms existing LLMs on benchmark environments and shows that it can generate diverse trajectories and pick the most rewarding trajectory. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models have led to innovations in various domains such as education, healthcare, and finance, while raising serious concerns that they can be easily misused for malicious purposes. |
| Approach: | They identify specific neurons (“aggression neurons”) closely related to the expression of aggression and analyze how manipulating them affects the model’s overall aggression. |
| Outcome: | The proposed model outputs show that manipulating neurons can increase aggression by up to 33% in all models and even more extreme when they are concentrated in certain layers. |
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) plays a pivotal role in medical Natural Language Processing (NLP) yet, there is no open-source medical NER dataset specifically for Korean. |
| Approach: | They used ChatGPT to construct an open-source Korean NER dataset . they found 20% increase in medical NER performance compared to general Korean ner datasets. |
| Outcome: | The KBMC dataset shows an impressive 20% increase in medical NER performance compared to models trained on general Korean NER datasets. |
Copied to clipboard
| Challenge: | Recent advances in large language models have enabled molecular reasoning for property prediction. however, toxicity arises from complex biological mechanisms, necessitating mechanistic reasoning for reliable prediction. |
| Approach: | They propose a benchmark that evaluates organ-level toxicity reasoning across multiple organs . they find strong predictive performance does not necessarily imply reliable reasoning . |
| Outcome: | The proposed benchmark evaluates toxicity prediction performance and reasoning quality across LLMs. |
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) demonstrate remarkable reasoning capabilities comparable to human cognitive abilities. |
| Approach: | They propose a sentence-level rationale reduction framework that leverages likelihood-based criteria to selectively remove redundant reasoning sentences while preserving reasoning capabilities. |
| Outcome: | The proposed framework improves performance by 7.71% while reducing token generation by 19.87% compared to models trained with complete reasoning paths. |
Copied to clipboard
| Challenge: | Existing retrieval-augmented generation approaches struggle with query complexity, propagated reasoning errors, or rely on incomplete or noisy retrieval. |
| Approach: | a unified retrieval-augmented generation framework is developed to address query complexity . the framework decomposes queries into semantically coherent sub-queries . it explicitly verifies retrieved sub-facts and adaptively refines queries based on identified knowledge gaps. |
| Outcome: | a new framework improves answer completeness and reliability by decomposing queries into coherent sub-queries . the framework explicitly verifies retrieved sub-facts and adaptively refines queries based on identified knowledge gaps. |
Copied to clipboard
| Challenge: | Existing methods to manage topic shifts within on-topic dialogues are limited in their ability to generate training datasets. |
| Approach: | They propose a data generation framework that automatically generates conversational question-answering datasets with natural topic transitions by leveraging relationships between entities in a knowledge graph. |
| Outcome: | The proposed framework generates conversational question-answering datasets with natural topic transitions and proves its effectiveness in generating dialogues with topic shifts. |
Copied to clipboard
| Challenge: | Consistency within a document is a crucial feature indicative of its quality . low-quality documents often lack internal consistency or contain content unrelated to headlines . |
| Approach: | They propose a hierarchical graph convolutional network that detects internal inconsistencies within a document and incongruences between the title and body. |
| Outcome: | The proposed model outperforms existing models on the inconsistency dataset and on the publicly available incongruent-related dataset. |
Copied to clipboard
| Challenge: | Recent research shows word embeddings have strong gender biases in embeddable spaces . a proposed method can be used to debiase word embeds without loss of semantic information . |
| Approach: | They propose a latent disentanglement method with a siamese auto-encoder structure with an adapted gradient reversal layer to debiase word embeddings. |
| Outcome: | The proposed method can preserve semantic information during debiasing while minimizing loss of semantic information for extrinsic NLP tasks. |
Copied to clipboard
| Challenge: | Recent studies show that adding a instruction tuning stage to training large language models can improve zero-shot task generalization. |
| Approach: | They propose a method that retrieves promptspecific source prompt embeddings from training instances . they train soft prompt embeds for each prompt through prompt tuning and store the samples . |
| Outcome: | The proposed method outperforms hard prompts on unseen tasks by 2.39% points and outperformed 10 out of 11 datasets. |
Copied to clipboard
| Challenge: | Existing research defines “grounding” as having the correct answer, which does not ensure the reliability of the entire response. |
| Approach: | They propose a stricter definition of grounding: fully utilizes the necessary knowledge from the provided context and stays within the limits of that knowledge. |
| Outcome: | The proposed model can be ground on external contexts and maintain its correct answer. |
Copied to clipboard
| Challenge: | According to psychological and neuroscientific research, a high-stress environment can restrict attentional resources and intensify negative affect, thereby impairing the ability to understand emotions. |
| Approach: | They constructed a large-vision language model that combines race, gender, and age group and used the Pretend prompt technique to induce LVLMs to interpret others’ emotions. |
| Outcome: | The results suggest that the effects of high-stress and demographic attributes identified in human research may also be reflected in LVLMs. |
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit a high degree of alignment with human behavior based on their robust capabilities for natural language understanding and generation. |
| Approach: | They developed a dataset of 5K fictional avatar images for assignment as visual personas to large language models (LLMs) and analyzed their negotiation behaviors based on the visual traits depicted in these images. |
| Outcome: | The proposed model exhibited aggressive negotiation behaviors when the opponent’s image appeared less aggressive than their own, and less aggressive negotiation behavior when the opposing image appeared more aggressive. |
Copied to clipboard
| Challenge: | Existing models that ground knowledge and persona at the same time are limited, leading to hallucination and a passive way of using personas. |
| Approach: | They propose a conversational agent that grounds external knowledge and persona simultaneously and a retrieval augmented generation model that generates utterances with lesser hallucination and more engagingness. |
| Outcome: | The proposed agent generates the utterance with lesser hallucination and more engagingness utilizing retrieval augmented generation with knowledge-persona enhanced query. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have led to significant improvements in various service domains, including search, recommendation, and chatbot applications. |
| Approach: | They propose a framework for developing scalable, controllable, and reliable AI-driven agents that can be applied to real-world applications. |
| Outcome: | The proposed framework bridges the gap between academic research and real-world application, and enables scalable, controllable, and reliable AI-driven agents. |
Copied to clipboard
| Challenge: | Existing methods to inject lexical features into self-attention mechanisms have shown remarkable performance across various downstream tasks in NLP. |
| Approach: | They propose to inject lexical features into the self-attention mechanism of Transformer-based models by injecting lexicon-based Sentiment Lexical Attention into the attention scores throughout the training process. |
| Outcome: | The proposed method shows significant performance improvements on the NSMC sentiment classification benchmark and is able to perform in out-of-domain tasks. |
Copied to clipboard
| Challenge: | Knowledge distillation (KD) is widely used for transferring capabilities from proprietary models to efficient open-source counterparts. |
| Approach: | They propose a method that constructs a geometric target distribution in logit space to emphasize agreement between the teacher and the student. |
| Outcome: | Experiments show that the proposed method outperforms supervised fine-tuning and existing on-policy baselines. |
Copied to clipboard
| Challenge: | Existing studies highlight that large language models are receptive to external information that contradicts their parametric knowledge, but little research has been conducted on the direct impact of instruction-tuning on this phenomenon. |
| Approach: | They examine how instruction-tuning influences LLMs' susceptibility to misinformation, particularly in knowledge conflict situations. |
| Outcome: | The proposed model is more user-oriented and more likely to accept misinformation when it is presented by the user. |
Copied to clipboard
| Challenge: | Language models with less than 100B parameters perform poorly on chain-of-thought reasoning . we aim to equip smaller LMs with the step-by-step reasoning capability . |
| Approach: | They propose to equip smaller LMs with the step-by-step reasoning capability by tuning with CoT rationales. |
| Outcome: | The proposed dataset outperforms large LMs on 4 domain-specific tasks even with demonstrations . |
Copied to clipboard
| Challenge: | Large Vision Language Models suffer from hallucinations, attributing incorrect or misleading features to images. |
| Approach: | They propose a test-time approach that recalibrates the influence of blind tokens . they identify blind token by analyzing layer-wise attention distributions over image tokens. |
| Outcome: | The proposed approach reduces hallucinations in large vision language models . it uses a contrastive decoding strategy to balance the influence of blind tokens . |
Copied to clipboard
| Challenge: | a new framework for automated essay scoring is needed to achieve multi-perspective understanding and judgment. |
| Approach: | They propose a roundtable essay scoring framework that performs precise and human-aligned scoring under a zero-shot setting. |
| Outcome: | The proposed framework outperforms previous zero-shot AES approaches by enabling collaboration among agents with diverse evaluation perspectives. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown significant performance, approaching human perception levels. |
| Approach: | They propose an approach that sketches molecular structures for reasoning by explicitly incorporating key structural features into the model. |
| Outcome: | The proposed framework improves molecular understanding through extensive experiments. |
Copied to clipboard
| Challenge: | Existing frameworks for grounding pretrained language models as task planners are challenging due to their intricate entanglement with domain knowledge. |
| Approach: | They propose a framework that leverages the hierarchical nature of semantic skills to ground them in different domains. |
| Outcome: | The proposed framework is effective in 300 cross-domain EIF scenarios. |
Copied to clipboard
| Challenge: | Existing models for low-resource languages struggle with domain-specific terms and lack of expert annotators for dataset creation. |
| Approach: | They propose a method for collecting low-resource language pairs from industrial domains using a large language model and neural machine translation framework. |
| Outcome: | The proposed model performs poorly on DaCoM-created datasets with up to 53.7 BLEURT points difference depending on domain inclusion. |
Copied to clipboard
| Challenge: | Using semiparametric token-sequence co-supervision, language models are trained using a finite parametric vocabulary space. |
| Approach: | They propose a semiparametric token-sequence co-supervision training method that leverages supervision from two different supervisions. |
| Outcome: | The proposed method outperforms models trained via each supervision independently and shows that it encourages a broader generalization capability across the model. |
Copied to clipboard
| Challenge: | Molecular string representations such as SMILES and SELFIES are becoming a standard format for applying large language models (LLMs) however, molecular strings follow complex syntactic rules for encoding molecules, which LLMs struggle to interpret. |
| Approach: | They propose a framework that parses SMILES into clean and deterministic tasks to promote graph-level molecular comprehension. |
| Outcome: | The proposed framework improves structural comprehension and competes with the baseline on the Mol-Instructions benchmark. |
Copied to clipboard
| Challenge: | Recent advances have introduced GER frameworks that utilize LLMs to refine ASR outputs. |
| Approach: | They propose a framework that allows a large language model to compose independent N-best hypotheses from separate automatic speech recognition (ASR) and visual speech recognition models. |
| Outcome: | The proposed framework achieves 57.7% error rate gain over standard ASR baseline, compared to single-stream approaches that achieve only 10% gain. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) absorb social and cultural biases embedded in vast web-scale corpora and are increasingly deployed in high-stakes domains such as hiring, education, and law. |
| Approach: | They propose a framework to investigate counterfactual unfairness through humor by observing how the model’s responses change when we swap who speaks and who is addressed while holding other factors constant. |
| Outcome: | The proposed framework covers humor generation refusal, speaker intention inference, and relational/societal impact prediction tasks. |
Copied to clipboard
| Challenge: | Recent advances of large language models (LLMs) have enabled them to provide long and detailed responses by leveraging their parametric knowledge. |
| Approach: | They introduce a dataset of 10K hierarchical sets of information-seeking subqueries that define structured boundaries for outline creation and evaluation in C2 scenarios. |
| Outcome: | The proposed model generates higher-quality outlines that better fulfill diverse user needs when trained with alignment techniques like DPO. |
Copied to clipboard
| Challenge: | Existing evaluation datasets for external knowledge-based VQA lack a capability to determine which passage is useful for answering queries. |
| Approach: | They propose a visual question answering benchmark for vision language models based on retrieval augmented generation (RAG) the proposed benchmark includes five input passages, a capability lacking in previous research. |
| Outcome: | The proposed benchmark includes five input passages and is validated using the state-of-the-art Llama3-based VLM, the Llava-Llamama-3 model. |
Copied to clipboard
| Challenge: | Existing question answering systems mainly focus on text data, but few Korean datasets exist . a dataset for table question answering is written in English, but it lacks Korean-specific datasets . |
| Approach: | They construct Korean-specific datasets for table question answering using crowd-sourced workers . they then fine-tune the model with these datasets and report the evaluation results . |
| Outcome: | The proposed model is based on Korean datasets and is publicly available . the model is evaluated against other datasets from Korean question answering systems . |
Copied to clipboard
| Challenge: | Question Answering (QA) with large language models has shown impressive performance, yet hallucinations persist . incorporating LQC into QA pipelines reduces hallucinosities while preserving strong answer quality. |
| Approach: | They propose a small classification model that detects verification-required queries before the LLM generates a potentially faulty answer. |
| Outcome: | The proposed model reduces hallucinations while maintaining strong answer quality. |
Copied to clipboard
| Challenge: | Multimodality has been explored in multi-party and multi-session conversations, but task-specific constraints have hindered its seamless integration into dynamic, natural conversations. |
| Approach: | They propose a multimodal conversation dataset and a model with multimodal memory retrieval to equip chatbots with "eyes and ears" they aim to integrate multimodality into chatbot interactions by integrating visual and auditory inputs into the chatbot. |
| Outcome: | The proposed model demonstrates the ability to engage in long-term conversations with multiple speakers in complex, real-world-like settings, effectively processing visual and auditory inputs to understand and respond appropriately. |
Copied to clipboard
| Challenge: | Recent studies show that large language models and vision large language model (VLLMs) possess EI and the ability to understand emotional stimuli in the form of text and images. |
| Approach: | They analyze the key elements affecting the emotion prediction performance of VLLMs in conversational contexts. |
| Outcome: | The proposed model performance was compared with other models in a conversational context. |
Copied to clipboard
| Challenge: | Recent dialogue systems exhibit an inability to replicate dynamic, continuous, long-term interactions involving multiple partners. |
| Approach: | They propose a multi-session dialogue system that builds on real-world interactions by integrating deep layered interactions and widening conversation networks. |
| Outcome: | The proposed system is based on a dataset of 6 consecutive dialogue episodes with four speakers (one main speaker and three partners) appearing in each episode. |
Copied to clipboard
| Challenge: | Existing time series forecasting approaches reduce them to numerical predictors that bypass their strengths or allow direct forecast generation that destabilizes predictions in non-stationary settings. |
| Approach: | They propose a framework that decouples semantic reasoning from quantitative prediction. |
| Outcome: | The proposed framework decouples semantic reasoning from quantitative prediction. |
Copied to clipboard
| Challenge: | Recent DPOs introduce additional hyperparameters, reducing feasibility for LLM fine-tuning. |
| Approach: | They propose an algorithm that regularizes the reward against a reference policy without extra hyperparameters to address suboptimal outcomes. |
| Outcome: | The proposed algorithm outperforms baseline algorithms with the same hyperparameter complexity while maintaining training simplicity. |
Copied to clipboard
| Challenge: | Existing QE models exhibit systematic gender bias, especially in gender-ambiguous contexts. |
| Approach: | They propose a multi-agent-based, fairness-aware QE framework that mitigates gender bias in both gender-ambiguous and gender-explicit scenarios. |
| Outcome: | The proposed framework mitigates gender bias in gender-ambiguous and gender-explicit scenarios while maintaining the strengths of existing models. |
Copied to clipboard
| Challenge: | Using auxiliary functions to implement functions is important for instruction-tuned models because it reduces the implementation difficulty of a target function compared to implementing them from scratch. |
| Approach: | They propose several ways to provide auxiliary functions to the models by adding them to the query or providing a response prefix to incorporate the ability to utilize auxiliary function with the instruction following capability. |
| Outcome: | The proposed models outperform the recent powerful language models, gpt-4o, in the code generation task. |
Copied to clipboard
| Challenge: | Large reasoning models (LRMs) externalize explicit reasoning traces before producing the answer, yet suffer from overthinking challenge. |
| Approach: | They propose a framework that enables large reasoning models to self-determine when to reason by training only the initial root token via group relative reward and group-wise advantages. |
| Outcome: | The proposed framework reduces training overhead and VRAM usage by focusing on the root token . it learns difficulty-aware adaptive thinking at just 2% of the training compute of prior methods. |
Copied to clipboard
| Challenge: | Chain-of-thought prompting generates an explanation before the final prediction, but its performance is affected by the factual accuracy of the explanation. |
| Approach: | They propose a toolkit for annotating the factual correctness of generated explanations and collecting revision data of wrong explanations. |
| Outcome: | The proposed toolkit is publicly available at https://github.com/SeungoneKim/CoTEVer. |
Copied to clipboard
| Challenge: | Adaptive multi-agent systems (MAS) are increasingly adopted as solutions to complex problems. |
| Approach: | They conduct extensive empirical study on adaptive multi-agent systems . they find they are prone to topological overfitting and exhibit illusory coordination . authors urge prioritization of generalization in MAS development and evaluation . |
| Outcome: | a new study shows adaptive multi-agent systems are prone to overfitting and lack coordination . the findings highlight the need to prioritize generalization in MAS development . |
Copied to clipboard
| Challenge: | a recent study evaluated language models using abstract evaluation criteria that lack the flexibility and granularity of human assessment. |
| Approach: | They propose a benchmark to evaluate nine distinct language models' capabilities . they use instance-specific evaluation criteria to mirror human evaluation . |
| Outcome: | The proposed benchmark evaluates nine distinct capabilities of language models across 77 tasks. |
Copied to clipboard
| Challenge: | Large language models (LLMs) are renowned for their extensive linguistic knowledge and strong generalization capabilities, but their high computational demands make them unsuitable for resource-constrained environments. |
| Approach: | They propose a framework that integrates a single frozen layer from an LLM into a SLM and fine-tunes the combined model for specific tasks. |
| Outcome: | The proposed framework improves performance across a range of natural language processing tasks, including both natural language understanding and generation. |
Copied to clipboard
| Challenge: | Existing models that generate news subheadings rely on topical headline information to capture topical knowledge from the article. |
| Approach: | They propose a model that uses topical headline information to generate news subheadings using masked headline tokens. |
| Outcome: | The proposed model outperforms the comparative models on three news datasets written in two languages and performs robustly on a small dataset and various masking ratios. |
Copied to clipboard
| Challenge: | a recent development of spoken dialogue systems has enabled deep learning to achieve state-of-the-art performance. |
| Approach: | They propose a Python-based domain-independent, open-source toolkit for spoken dialogue systems. |
| Outcome: | The proposed toolkit extends OpenDial's Java-based architecture and provides new functions for neural dialogue state tracking and action planning. |
Copied to clipboard
| Challenge: | Current retrieval-augmented generation methods struggle with complex multi-hop reasoning, relying on unstructured semantic matching that lacks the logical structure needed to systematically guide retrieval. |
| Approach: | They propose a framework that elevates retrieval to structured, program-guided reasoning by combining three stages of program-type selection and evidence accumulation. |
| Outcome: | Evaluated on five benchmarks including HotPotQA, 2WikiMultihopQA, ARC-Challenge, MMLU-Pro, and MedQA with various LLMs, PROGRAM achieves state-of-the-art performance with up to 24% relative improvement on HotPtQA and 13.2% on MedQA over strong baselines including FLARE, ProbTree and Self-RAG. |
Copied to clipboard
| Challenge: | Existing function-calling benchmarks focus on single-turn interactions but ignore complexity of real-world scenarios. |
| Approach: | They propose a framework that constructs practical function-calling datasets by synthesizing conversations through a tool graph that maintains dependencies across rounds. |
| Outcome: | The proposed framework synthesizes conversations through a tool graph that maintains dependencies across rounds and a multi-agent system with distinct personas to enhance dialogue naturalness. |
Copied to clipboard
| Challenge: | Word-level adversarial attacks have shown success in NLP, decreasing performance of transformer-based models with smaller perturbation rate. |
| Approach: | They propose a dataset for four popular attack methods on four datasets and four models to encourage further research in this field. |
| Outcome: | The proposed baseline has the highest auc on 29 out of 30 dataset-attack-model combinations. |
Copied to clipboard
| Challenge: | Existing multilingual embedding models often struggle to capture cross-lingual alignment during training. |
| Approach: | They propose a novel loss function that leverages an English passage as a bridge to strengthen alignments between target language and English. |
| Outcome: | The proposed model improves retrieval performance across cross-lingual scenarios while minimizing performance degradation in English. |
Copied to clipboard
| Challenge: | Backchannel (BC) is a short and quick reaction signal of a listener to a speaker's utterances. |
| Approach: | They propose a model that utilizes lexical information in utterances to enhance backchannel (BC) prediction. |
| Outcome: | The proposed model showed 14.24% performance improvement compared to baseline in the four BC categories: continuer, understanding, empathic response, and No BC. |
Copied to clipboard
| Challenge: | Existing approaches to adapt language models for multilingual reasoning tasks require multilingual supervision. |
| Approach: | They propose a zero-shot approach to adapt language models for multilingual reasoning tasks without multilingual supervision by bridging two models by introducing minimal trainable parameters between them. |
| Outcome: | The proposed approach significantly improves multilingual reasoning capabilities on low-resource languages. |
Copied to clipboard
| Challenge: | Language Models (LMs) become outdated as the world changes, a phenomenon called temporal misalignment. |
| Approach: | They propose a lifelong benchmark that utilizes the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
| Outcome: | The proposed benchmark can be trained on the difference between consecutive snapshots of English Wikipedia and English Wikidata for training and evaluation. |
Copied to clipboard
| Challenge: | Existing methods to detect out-of-distribution (OOD) samples are overconfident for real-world language applications. |
| Approach: | They propose a method that constructs a surrogate OOD dataset by sequentially masking tokens related to ID classes. |
| Outcome: | The proposed method can train a rejection network with ID and diverse outlier samples but requires additional data collection overhead. |
Copied to clipboard
| Challenge: | Recent studies have addressed intricate phonological phenomena in French, relying on extensive linguistic knowledge or a significant amount of sentence-level pronunciation data. |
| Approach: | They propose a grapheme-to-phoneme and post-lexical processing approach to address French phonological phenomena using sentence-level pronunciation data. |
| Outcome: | The proposed approach mitigates the lack of extensive labeled data and serves as a feasible solution for addressing French phonological phenomena even under resource-constrained environments. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are becoming powerful agentic systems . generic benchmarks fail to assess realistic, non-English performance . |
| Approach: | They propose to evaluate five core agentic capabilities: Reasoning, Planning, Action (tool-use), Retrieval-Augmented Generation, and Instruction Following. |
| Outcome: | The evaluations reveal significant performance disparities between models that employ explicit reasoning and those that do not. |
Copied to clipboard
| Challenge: | Structured reasoning and tool-grounded molecular optimization are still underexplored. |
| Approach: | They propose a framework for molecular optimization that leverages tool-guided reasoning and role-specialized LLM agents. |
| Outcome: | a new framework outperforms existing LLM frameworks on 15 of 23 tasks. |
Copied to clipboard
| Challenge: | Recent studies show that language models can perform diverse downstream NLP tasks in a zero-shot manner without any gradient updates. |
| Approach: | They show that updating pretrained LMs with Gradient Ascent Post-training enhances zero-shot generalization capabilities without any task-specific fine-tuning. |
| Outcome: | The proposed method can be comparable to 2-3x times larger LMs across 12 different NLP tasks. |
Copied to clipboard
| Challenge: | Legal QA benchmarks focus on case law, overlooking statute-centric regulatory reasoning . relevant evidence is distributed across hierarchically linked documents, creating statutory retrieval gap . |
| Approach: | They propose a structure- and safety-aware benchmark for statute-centric legal QA . the benchmark assesses whether models can retrieve hierarchically fragmented evidence . |
| Outcome: | The proposed benchmark evaluates whether models can retrieve hierarchically fragmented evidence and safely abstain when statutory context is insufficient. |
Copied to clipboard
| Challenge: | Existing studies focus on capturing implicit relations between entities, but humans use explicit information chains to find the relations between two entities. |
| Approach: | They propose a cross-document relation extraction task that captures the relation between two entities mentioned in different documents. |
| Outcome: | The proposed model outperforms baseline models in the CodRED task. |
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have raised concerns about inference costs, increasing the need for research into model compression. |
| Approach: | They propose a method that utilizes prompt tuning to enable generative language models to transfer student-friendly knowledge. |
| Outcome: | Extensive experiments on instruction-following datasets show that PromptKD achieves state-of-the-art performance while adding only 0.0007% of the teacher’s parameters as prompts. |
Copied to clipboard
| Challenge: | PLEX is a lottery-ticket based parameter-efficient fine-tuning method that adapts large language models to well-supported and underrepresented programming languages (PLs) in pretraining. |
| Approach: | They propose a lottery-ticket based parameter-efficient fine-tuning method that adapts large language models to well-supported and underrepresented programming languages (PLs) |
| Outcome: | The proposed method achieves state-of-the-art performance among PEFT methods while maintaining competitive results with reduced computational overhead. |
Copied to clipboard
| Challenge: | Existing studies on open-domain dialogue systems that allow free topics are challenging . however, non-English dialogue systems suffer from reproducing the performance of English dialogue systems . |
| Approach: | They propose to use English knowledge to improve the performance of open-domain dialogue systems . they construct a Korean-English T5 language model and develop a knowledge-grounded Korean dialogue model . |
| Outcome: | The proposed model improves even when only English knowledge is given . the model is built with a pre-trained language model and a knowledge-grounded Korean dialogue model . |