Portrait of Hoyoon Byun

Hoyoon Byun

Ph.D. student, Yonsei University
Statistics and Data Science

What I work on

Each dot is a paper ( first author, co-author) or a project (). Work on two topics sits where their circles overlap. Pick a circle or a keyword to filter the list.

About

My research makes large language models efficient to train and run, and keeps LLMs and the agents built on them reliable under distribution shift.

I am a Ph.D. student in the Department of Statistics and Data Science at Yonsei University, advised by Prof. Kyungwoo Song. I work on large language models (LLMs) and LLM agents from two directions: making LLMs train more stably and run more efficiently, and making LLMs and the agents built on them stay reliable when the data or environment changes.

Before my Ph.D., I received my M.S. in Artificial Intelligence from the University of Seoul, advised by Prof. Jiyoung Jung, and my B.S. in Electrical and Semiconductor Engineering from Pukyong National University.

Papers
14
10 conference, 4 journal
First-author papers
7
first or co-first author
Workshop papers
6
2 oral presentations
Years in research
4+
since starting my M.S. in 2022

Now ongoing projects

  • Since 2026 CausalityDistribution shift

    Causally transportable LLM agents

    In sequential decision environments with known causal structure, I measure how well the decisions an agent learns from source-domain experience hold up under domain shift. I am designing agents that, at each step, combine causal action effects learned in the source domain with their belief about what has changed in the target domain.

  • Since 2025 Hyundai Motor Group LLM evaluation & applications

    On-premise LLM agent for vehicle fault diagnosis

    Building an on-premise LLM agent on open-source models, covering knowledge extraction from documents, memory construction and retrieval, and automotive safety and quality question-answering datasets.

Publications 21

equal contribution

Preprint 1 paper
  1. arXiv Preprint

    MIDUS: Memory-Infused Depth Up-Scaling

    Taero Kim, Hoyoon Byun, Youngjun Choi, Sungrae Park, Kyungwoo Song

    arXiv preprint

    Efficient LLMs Paper
2026 5 papers
  1. ICML 2026 Conference

    Bounded Hyperbolic Tangent: A Stable and Efficient Alternative to Pre-Layer Normalization in Large Language Models

    Hoyoon Byun, Youngjun Choi, Taero Kim, Sungrae Park, Kyungwoo Song

    International Conference on Machine Learning (ICML), 2026

    Replaces pre-layer normalization with a bounded activation. Across pretraining and post-training of 374M to 3B parameter models, it trains 1.6% faster and generates tokens with 1.77% higher throughput than RMSNorm on average, while keeping language understanding and reasoning performance.

    Efficient LLMs Paper Code
  2. ICML 2026 Conference

    Dissecting Causal Mechanism Shifts via FANS: Function And Noise Separation

    Gyeongdeok Seo, Jaeyoon Shim, Mingyu Kim, Hoyoon Byun, Yonghan Jung, Kyungwoo Song

    International Conference on Machine Learning (ICML), 2026

    CausalityDistribution shift Paper
  3. ICML 2026 W Workshop

    RRD: Routing-and-Residual Distillation for Efficient MoE Recovery in Large Language Models

    Hoyoon Byun, Kangjun Noh, Somin Kim, Heedong Kim, Jaeyoon Shim, Sungjun Lim, Youngjun Choi, Kyungwoo Song

    ICML 2026 Workshop on High-dimensional Learning Dynamics

    Converts a dense LLM into a sparse Mixture-of-Experts model. Turning off 25% of Qwen2.5-7B's original FFN per token, it raises mean zero-shot accuracy on five benchmarks by 3.1 points over the strongest dense-to-MoE baseline, using only 40M training tokens.

    Efficient LLMs
  4. SAC 2026 Conference

    CBPE: Causal-aware Blood Pressure Estimation with Multimodal Data

    Hoyoon Byun, Sumin Park, Jinho Kang, Hyeonjeong Lee, Minseong Kim, Doosik Kim, Kwang-Yong Kim, Kyungwoo Song

    ACM Symposium on Applied Computing (SAC), 2026

    Models latent confounders across physiological signals, clinical records and medical text to estimate blood pressure.

    Causality Code
  5. PR Journal

    Graph Perceiver IO: A General Architecture for Graph Structured Data

    Seyun Bae, Hoyoon Byun, Changdae Oh, Yoon-Sik Cho, Kyungwoo Song

    Pattern Recognition (PR), 2026

    Representation learning Paper
2025 7 papers
  1. NeurIPS 2025 Conference

    CCL: Causal-aware In-context Learning for Out-of-Distribution Generalization

    Hoyoon Byun, Gyeongdeok Seo, Joonseong Kang, Taero Kim, Jihee Kim, Kyungwoo Song

    Neural Information Processing Systems (NeurIPS), 2025

    Learns causal representations that stay invariant across domains and selects in-context demonstrations that share causal features with the query, improving out-of-distribution generalization.

    CausalityDistribution shift Paper Code
  2. KDD 2025 Conference

    Exploring the Potential of Foundation Models as Reliable AI Contact Centers

    Hoyoon Byun, Minhoi Park, Seolah Kim, Eunbi Kim, Kyungwoo Song

    KDD 2025 Datasets and Benchmarks Track

    A benchmark that evaluates whether foundation models can serve as reliable AI contact centers.

    LLM evaluation & applications Paper
  3. UAI 2025 Conference

    Flat Posterior Does Matter For Bayesian Model Averaging

    Sungjun Lim, Jeyoon Yeom, Sooyon Kim, Hoyoon Byun, Jinho Kang, Yohan Jung, Jiyoung Jung, Kyungwoo Song

    Conference on Uncertainty in Artificial Intelligence (UAI), 2025

    Distribution shift Paper
  4. NAACL 2025 Conference

    LBC: Language-Based-Classifier for Out-Of-Variable Generalization

    Kangjun Noh, Baekryun Seong, Hoyoon Byun, Youngjun Choi, Sungjin Song, Kyungwoo Song

    Nations of the Americas Chapter of the ACL (NAACL), 2025

    Distribution shift Paper
  5. KBS Journal

    Multi-query frequency prompting for physiological signal domain adaptation

    Jinho Kang*, Hoyoon Byun*, Taero Kim, Jiyoung Jung, Kyungwoo Song

    Knowledge-Based Systems (KBS), 2025

    Distribution shift Paper
  6. EAAI Journal

    GFML: Gravity Function for Metric Learning

    Hoyoon Byun, Sungjun Lim, Kyungwoo Song

    Engineering Applications of Artificial Intelligence (EAAI), 2025

    Representation learning Paper
  7. CIKM 2025 W Workshop

    A Double-Edged Sword: Benchmarking the Trade-off Between Bias Mitigation and Helpfulness of LLM Guardrails in Finance

    Hoyoon Byun, et al.

    FinAI Workshop at CIKM 2025

    LLM evaluation & applications
2024 7 papers
  1. EMNLP 2024 Conference Findings

    CED: Comparing Embedding Differences for Detecting Out-of-Distribution and Hallucinated Text

    Hakyung Lee, Keon-Hee Park, Hoyoon Byun, Jeyoon Yeom, Jihee Kim, Gyeong-Moon Park, Kyungwoo Song

    Findings of EMNLP, 2024

    Distribution shiftLLM evaluation & applications Paper
  2. ECAI 2024 Conference

    Perturb-and-Compare Approach for Detecting Out-of-Distribution Samples in Constrained Access Environments

    Hee-young Lee*, Hoyoon Byun*, Changdae Oh, Jinyeong Bak, Kyungwoo Song

    European Conference on Artificial Intelligence (ECAI), 2024

    Distribution shift Paper
  3. J. Big Data Journal

    TC-Llama 2: Fine-tuning LLM for Technology and Commercialization Applications

    Jeyoon Yeom, Hakyung Lee, Hoyoon Byun, Yewon Kim, Jeongeun Byun, Yunjeong Choi, Sungjin Kim, Kyungwoo Song

    Journal of Big Data, 2024

    LLM evaluation & applications Paper
  4. IJCAI 2024 W Workshop Oral

    Exploring the Potential of Foundation Models as Reliable AI Contact Centers

    Hoyoon Byun, Minhoi Park, Seolah Kim, Eunbi Kim, Kyungwoo Song

    IJCAI 2024 Workshop on Trustworthy Interactive Decision-Making with Foundation Models

    LLM evaluation & applications
  5. IJCAI 2024 W Workshop Oral

    LBC: Language-Based-Classifier for Out-Of-Variable Generalization

    Kangjun Noh, Baekryun Seong, Hoyoon Byun, Youngjun Choi, Kyungwoo Song

    IJCAI 2024 Workshop on Trustworthy Interactive Decision-Making with Foundation Models

    Distribution shift
  6. IJCAI 2024 W Workshop

    H-Consistency Bounded Critic for Enhancing Retrieval-Augmented Generation

    Seonggyun Lee, Hoyoon Byun, Kyungwoo Song

    The First International OpenKG Workshop at IJCAI 2024

    LLM evaluation & applications
  7. IJCAI 2024 W Workshop

    Flat Posterior For Bayesian Model Averaging

    Sungjun Lim, Jeyoon Yeom, Sooyon Kim, Hoyoon Byun, Jinho Kang, Yohan Jung, Jiyoung Jung, Kyungwoo Song

    The Sixth Data Science Meets Optimisation (DSO) Workshop at IJCAI 2024

    Distribution shift
2023 1 paper
  1. NeurIPS 2023 Conference

    Geodesic Multi-Modal Mixup for Robust Fine-tuning

    Changdae Oh, Junhyuk So, Hoyoon Byun, YongTaek Lim, Minchul Shin, Jong-June Jeon, Kyungwoo Song

    Neural Information Processing Systems (NeurIPS), 2023

    Distribution shift Paper

Background

Education

  1. Mar 2024 – Feb 2028 (expected) Ph.D. in Statistics and Data Science Yonsei University Advisor: Kyungwoo Song
  2. Mar 2022 – Feb 2024 M.S. in Artificial Intelligence University of Seoul Advisor: Jiyoung Jung · Co-advisor: Kyungwoo Song
  3. Mar 2014 – Feb 2021 B.S. in Electrical and Semiconductor Engineering Pukyong National University

Industry and institute collaborations

  1. 2025 – present Hyundai Motor Group On-premise LLM agent for vehicle fault diagnosis
  2. 2025 Upstage AI BHyT, evaluated across LLM pretraining and post-training
  3. 2024 Electronics and Telecommunications Research Institute (ETRI) Multimodal blood-pressure estimation (CBPE)

Past projects

  • 2025 Upstage AI Efficient LLMs

    BHyT: a bounded activation in place of pre-layer normalization

    Developed BHyT and evaluated it across LLM pretraining and post-training, from 374M to 3B parameters.

    See the paper
  • 2024 ETRI Causality

    Multimodal blood-pressure estimation

    Modeled latent confounders across physiological signals, clinical records and medical text; led to CBPE.

    See the paper

Writing All posts →