Papers by Jongyoon Song
Entity-level Factual Adaptiveness of Fine-tuning based Abstractive Summarization Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Abstractive summarization models generate factually inconsistent content when parametric knowledge conflicts with knowledge in the input document. |
| Approach: | They propose a method to enhance factual adaptiveness while achieving factual consistency on original datasets. |
| Outcome: | The proposed method improves factual adaptiveness while achieving factual consistency on original datasets. |
Rare Tokens Degenerate All Tokens: Improving Neural Text Generation via Adaptive Gradient Gating for Rare Token Embeddings (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have determined that the learned token embeddings of large-scale neural language models are degenerated to be anisotropic with a narrow-cone shape. |
| Approach: | They propose a method to degenerate the learning gradient for rare token embeddings by gating the specific part of the gradient for all tokens during training stage. |
| Outcome: | The proposed method improves the performance of the models but lacks the training dynamics needed to solve the representation degeneration problem. |
Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Context (2025.findings-naacl)
Copied to clipboard
| Challenge: | Multi-hop reasoning requires multi-step reasoning based on supporting documents within a given context. |
| Approach: | They propose a method that prompts the model by repeatedly presenting the context. |
| Outcome: | The proposed method improves the F1 score by 30%p on multi-hop QA tasks and increases accuracy by 70%p on a synthetic task. |
Model Intrinsic Features of Fine-tuning based Text Summarization Models for Factual Consistency (2023.findings-acl)
Copied to clipboard
| Challenge: | a summarization model with relatively low factual consistency is more likely to model summaries that are not conditional to the documents. |
| Approach: | They analyze the model intrinsic features by varying the fine-tuning objectives and datasets. |
| Outcome: | The proposed models have a high inductive bias for factual consistency and are more likely to model summaries that are not conditional to the documents. |
AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate (2021.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive neural machine translation models suffer from the multi-modality problem . aligNART leverages full alignment information to explicitly reduce the modality of the target distribution . |
| Approach: | They propose an alignment decomposition method which explicitly reduces the modality of the target distribution. |
| Outcome: | The proposed model outperforms previous models that focus on modality reduction on two translation tasks. |
TRUEBench: Can LLM Response Meet Real-world Constraints as Productivity Assistant? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks fail to evaluate large language models' instruction-following capabilities . current benchmarks lack multilinguality, implicit constraints and multi-turn dialogue . |
| Approach: | a new benchmark is designed to evaluate large language models' instruction-following capabilities . the benchmark features input prompts across 12 languages and includes inter-instance multilingual instructions . |
| Outcome: | a new benchmark for large language models (LLMs) is designed to assess their performance in real-world settings. |
Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment (2025.naacl-long)
Copied to clipboard
Sangwon Yu, Jongyoon Song, Bongkyu Hwang, Hoyoung Kang, Sooah Cho, Junhwa Choi, Seongho Joe, Taehee Lee, Youngjune Gwon, Sungroh Yoon
| Challenge: | Experimental results show that large language models exhibit a negative bias in binary decision tasks . hallucination is a factor that degrades reliability of LLMs . |
| Approach: | They propose a negative attention score to systematically and quantitatively formulate negative bias by using a parameter-efficient fine-tuning technique. |
| Outcome: | The proposed method reduces the gap between precision and recall caused by negative bias while preserving generalization abilities. |