About
My research makes large language models efficient to train and run, and keeps LLMs and the agents built on them reliable under distribution shift.
I am a Ph.D. student in the Department of Statistics and Data Science at Yonsei University, advised by Prof. Kyungwoo Song. I work on large language models (LLMs) and LLM agents from two directions: making LLMs train more stably and run more efficiently, and making LLMs and the agents built on them stay reliable when the data or environment changes.
Before my Ph.D., I received my M.S. in Artificial Intelligence from the University of Seoul, advised by Prof. Jiyoung Jung, and my B.S. in Electrical and Semiconductor Engineering from Pukyong National University.
- Papers
- 14
- 10 conference, 4 journal
- First-author papers
- 7
- first or co-first author
- Workshop papers
- 6
- 2 oral presentations
- Years in research
- 4+
- since starting my M.S. in 2022
Now ongoing projects
-
Causally transportable LLM agents
In sequential decision environments with known causal structure, I measure how well the decisions an agent learns from source-domain experience hold up under domain shift. I am designing agents that, at each step, combine causal action effects learned in the source domain with their belief about what has changed in the target domain.
-
On-premise LLM agent for vehicle fault diagnosis
Building an on-premise LLM agent on open-source models, covering knowledge extraction from documents, memory construction and retrieval, and automotive safety and quality question-answering datasets.
Publications 21
equal contribution
-
MIDUS: Memory-Infused Depth Up-Scaling
arXiv preprint
Efficient LLMs Paper
-
Bounded Hyperbolic Tangent: A Stable and Efficient Alternative to Pre-Layer Normalization in Large Language Models
International Conference on Machine Learning (ICML), 2026
Replaces pre-layer normalization with a bounded activation. Across pretraining and post-training of 374M to 3B parameter models, it trains 1.6% faster and generates tokens with 1.77% higher throughput than RMSNorm on average, while keeping language understanding and reasoning performance.
-
Dissecting Causal Mechanism Shifts via FANS: Function And Noise Separation
International Conference on Machine Learning (ICML), 2026
-
RRD: Routing-and-Residual Distillation for Efficient MoE Recovery in Large Language Models
ICML 2026 Workshop on High-dimensional Learning Dynamics
Converts a dense LLM into a sparse Mixture-of-Experts model. Turning off 25% of Qwen2.5-7B's original FFN per token, it raises mean zero-shot accuracy on five benchmarks by 3.1 points over the strongest dense-to-MoE baseline, using only 40M training tokens.
Efficient LLMs -
CBPE: Causal-aware Blood Pressure Estimation with Multimodal Data
ACM Symposium on Applied Computing (SAC), 2026
Models latent confounders across physiological signals, clinical records and medical text to estimate blood pressure.
Causality Code -
Graph Perceiver IO: A General Architecture for Graph Structured Data
Pattern Recognition (PR), 2026
Representation learning Paper
-
CCL: Causal-aware In-context Learning for Out-of-Distribution Generalization
Neural Information Processing Systems (NeurIPS), 2025
Learns causal representations that stay invariant across domains and selects in-context demonstrations that share causal features with the query, improving out-of-distribution generalization.
-
Exploring the Potential of Foundation Models as Reliable AI Contact Centers
KDD 2025 Datasets and Benchmarks Track
A benchmark that evaluates whether foundation models can serve as reliable AI contact centers.
LLM evaluation & applications Paper -
Flat Posterior Does Matter For Bayesian Model Averaging
Conference on Uncertainty in Artificial Intelligence (UAI), 2025
Distribution shift Paper -
LBC: Language-Based-Classifier for Out-Of-Variable Generalization
Nations of the Americas Chapter of the ACL (NAACL), 2025
Distribution shift Paper -
Multi-query frequency prompting for physiological signal domain adaptation
Knowledge-Based Systems (KBS), 2025
Distribution shift Paper -
GFML: Gravity Function for Metric Learning
Engineering Applications of Artificial Intelligence (EAAI), 2025
Representation learning Paper -
A Double-Edged Sword: Benchmarking the Trade-off Between Bias Mitigation and Helpfulness of LLM Guardrails in Finance
FinAI Workshop at CIKM 2025
LLM evaluation & applications
-
CED: Comparing Embedding Differences for Detecting Out-of-Distribution and Hallucinated Text
Findings of EMNLP, 2024
-
Perturb-and-Compare Approach for Detecting Out-of-Distribution Samples in Constrained Access Environments
European Conference on Artificial Intelligence (ECAI), 2024
Distribution shift Paper -
TC-Llama 2: Fine-tuning LLM for Technology and Commercialization Applications
Journal of Big Data, 2024
LLM evaluation & applications Paper -
Exploring the Potential of Foundation Models as Reliable AI Contact Centers
IJCAI 2024 Workshop on Trustworthy Interactive Decision-Making with Foundation Models
LLM evaluation & applications -
LBC: Language-Based-Classifier for Out-Of-Variable Generalization
IJCAI 2024 Workshop on Trustworthy Interactive Decision-Making with Foundation Models
Distribution shift -
H-Consistency Bounded Critic for Enhancing Retrieval-Augmented Generation
The First International OpenKG Workshop at IJCAI 2024
LLM evaluation & applications -
Flat Posterior For Bayesian Model Averaging
The Sixth Data Science Meets Optimisation (DSO) Workshop at IJCAI 2024
Distribution shift
-
Geodesic Multi-Modal Mixup for Robust Fine-tuning
Neural Information Processing Systems (NeurIPS), 2023
Distribution shift Paper
-
Bounded Hyperbolic Tangent: A Stable and Efficient Alternative to Pre-Layer Normalization in Large Language Models
International Conference on Machine Learning (ICML), 2026
Replaces pre-layer normalization with a bounded activation. Across pretraining and post-training of 374M to 3B parameter models, it trains 1.6% faster and generates tokens with 1.77% higher throughput than RMSNorm on average, while keeping language understanding and reasoning performance.
-
Dissecting Causal Mechanism Shifts via FANS: Function And Noise Separation
International Conference on Machine Learning (ICML), 2026
-
CBPE: Causal-aware Blood Pressure Estimation with Multimodal Data
ACM Symposium on Applied Computing (SAC), 2026
Models latent confounders across physiological signals, clinical records and medical text to estimate blood pressure.
Causality Code -
CCL: Causal-aware In-context Learning for Out-of-Distribution Generalization
Neural Information Processing Systems (NeurIPS), 2025
Learns causal representations that stay invariant across domains and selects in-context demonstrations that share causal features with the query, improving out-of-distribution generalization.
-
Exploring the Potential of Foundation Models as Reliable AI Contact Centers
KDD 2025 Datasets and Benchmarks Track
A benchmark that evaluates whether foundation models can serve as reliable AI contact centers.
LLM evaluation & applications Paper -
Flat Posterior Does Matter For Bayesian Model Averaging
Conference on Uncertainty in Artificial Intelligence (UAI), 2025
Distribution shift Paper -
LBC: Language-Based-Classifier for Out-Of-Variable Generalization
Nations of the Americas Chapter of the ACL (NAACL), 2025
Distribution shift Paper -
CED: Comparing Embedding Differences for Detecting Out-of-Distribution and Hallucinated Text
Findings of EMNLP, 2024
-
Perturb-and-Compare Approach for Detecting Out-of-Distribution Samples in Constrained Access Environments
European Conference on Artificial Intelligence (ECAI), 2024
Distribution shift Paper -
Geodesic Multi-Modal Mixup for Robust Fine-tuning
Neural Information Processing Systems (NeurIPS), 2023
Distribution shift Paper
-
Graph Perceiver IO: A General Architecture for Graph Structured Data
Pattern Recognition (PR), 2026
Representation learning Paper -
Multi-query frequency prompting for physiological signal domain adaptation
Knowledge-Based Systems (KBS), 2025
Distribution shift Paper -
GFML: Gravity Function for Metric Learning
Engineering Applications of Artificial Intelligence (EAAI), 2025
Representation learning Paper -
TC-Llama 2: Fine-tuning LLM for Technology and Commercialization Applications
Journal of Big Data, 2024
LLM evaluation & applications Paper
-
RRD: Routing-and-Residual Distillation for Efficient MoE Recovery in Large Language Models
ICML 2026 Workshop on High-dimensional Learning Dynamics
Converts a dense LLM into a sparse Mixture-of-Experts model. Turning off 25% of Qwen2.5-7B's original FFN per token, it raises mean zero-shot accuracy on five benchmarks by 3.1 points over the strongest dense-to-MoE baseline, using only 40M training tokens.
Efficient LLMs -
A Double-Edged Sword: Benchmarking the Trade-off Between Bias Mitigation and Helpfulness of LLM Guardrails in Finance
FinAI Workshop at CIKM 2025
LLM evaluation & applications -
Exploring the Potential of Foundation Models as Reliable AI Contact Centers
IJCAI 2024 Workshop on Trustworthy Interactive Decision-Making with Foundation Models
LLM evaluation & applications -
LBC: Language-Based-Classifier for Out-Of-Variable Generalization
IJCAI 2024 Workshop on Trustworthy Interactive Decision-Making with Foundation Models
Distribution shift -
H-Consistency Bounded Critic for Enhancing Retrieval-Augmented Generation
The First International OpenKG Workshop at IJCAI 2024
LLM evaluation & applications -
Flat Posterior For Bayesian Model Averaging
The Sixth Data Science Meets Optimisation (DSO) Workshop at IJCAI 2024
Distribution shift
-
MIDUS: Memory-Infused Depth Up-Scaling
arXiv preprint
Efficient LLMs Paper
No papers match these filters.
Background
Education
- Mar 2024 – Feb 2028 (expected) Ph.D. in Statistics and Data Science Yonsei University Advisor: Kyungwoo Song
- Mar 2022 – Feb 2024 M.S. in Artificial Intelligence University of Seoul Advisor: Jiyoung Jung · Co-advisor: Kyungwoo Song
- Mar 2014 – Feb 2021 B.S. in Electrical and Semiconductor Engineering Pukyong National University
Industry and institute collaborations
- 2025 – present Hyundai Motor Group On-premise LLM agent for vehicle fault diagnosis
- 2025 Upstage AI BHyT, evaluated across LLM pretraining and post-training
- 2024 Electronics and Telecommunications Research Institute (ETRI) Multimodal blood-pressure estimation (CBPE)
Past projects
-
BHyT: a bounded activation in place of pre-layer normalization
Developed BHyT and evaluated it across LLM pretraining and post-training, from 374M to 3B parameters.
See the paper -
Multimodal blood-pressure estimation
Modeled latent confounders across physiological signals, clinical records and medical text; led to CBPE.
See the paper
Writing All posts →
- 인과 그래프로 "조립 가능한" 의사결정 환경 만들기: SCDM과 SCDP The Design and Composition of Structural Causal Decision Processes
- 환경이 바뀌었다면, 과거 경험의 action label을 고쳐서 재사용하자 Cross-Domain Offline Policy Adaptation via Selective Transition Correction
- Multi-turn RL은 왜 잘되다가 무너질까: RAGEN의 Echo Trap RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
- 실패의 원인과 수정할 위치를 함께 찾는다: ModularRSI ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement