KIAS Center for AI and Natural Sciences 2026 Winter Workshop

Training accurate dynamical surrogates for chaotic systems is difficult as their trajectories are extremely sensitive to initial conditions and model parameters, making exact future forecasts fundamentally impossible. Yet chaotic systems possess ergodicity and time-invariant measures, which give rise to well-defined global statistical properties. A properly learned neural network dynamics surrogate must accurately reproduce such properties, which is often not the case for models trained to minimize the trajectory mean-squared-error. In this talk, I will discuss a new training method for neural surrogate dynamics that utilizes the time evolution of phase space neighborhoods. This method can be seen as a form of Sobolev training for learning dynamical systems, and I will present empirical results highlighting the improved statistical accuracy of the trained models.

Recent advances in Artificial Intelligence are reshaping the landscape of scientific research. In this talk, I will explore the role of Deep Generative Models in solving complex scientific problems, moving beyond traditional simulation methods. I will first discuss the mathematical mechanisms of modern generative models—such as diffusion models and flow-based models—and analyze their capabilities from a mathematical perspective. One of the key applications is Material Discovery. I will demonstrate how generative AI can accelerate the inverse design process, efficiently exploring vast chemical spaces to identify novel materials with desired properties. Finally, I will discuss the evolution from task-specific models to Scientific Foundation Models. By training on massive multi-modal datasets, these foundation models aim to generalize across scientific domains, offering a unified framework that transcends specific equations or material classes, ultimately paving the way for autonomous scientific discovery.

In-context learning (ICL) is one of the key capabilities contributing to the great success of Transformer-based LLMs. To understand how ICL emerges, we study a simple training dynamics of linear transformer under the in-context linear regression setting.

With the regulation of "AI Acts" or the "General Data Protection Regulation (GDPR)" in the European Union, developing trustworthy AI systems has become increasingly important. Data unlearning aims to remove the influence of specific training samples from a trained model without requiring full retraining. Data unlearning in diffusion models remains underexplored and often suffers from quality degradation or incomplete forgetting. Thus, we investigate that unlearning the samples at all diffusion time steps equally might lead to poor-quality generation. Furthermore, we also focus on removing not only the harmful target concept but also benign co-occurring concepts. If time allows, I will introduce the detection methods against membership inference attack in LLMs that proposes a privacy-preserving technique for retrieval datasets using extreme value theory.

How reliable are large language models (LLMs)? In this talk, we first examine the performance of current LLMs in expert domains, using examples from the legal field such as the Korean Bar Exam and tax penalty problems. We show that, while flagship models achieve near-human performance in terms of 'knowledge', their reasoning capabilities remain limited. In the second part, we shift the focus from performance to AI alignment with humans. We demonstrate that ethically aligned LLMs can be easily manipulated by fine-tuning with as small as 600 unethical examples. Finally we briefly discuss our on-going efforts to develop robust post-training algorithms for AI alignment.

Phenomena such as emergence and grokking in deep networks can already be seen in extremely simplified two-layer models whose dynamics are explicitly analyzable. I will present heuristic and experimental evidence that these toy models capture essential aspects of large-scale networks, making them valuable analogies and motivating their mathematical study. This talk is based on joint work with Yoonsoo Nam (Oxford) and others.

Batch normalization has stood the test of time, remaining a core component of modern deep neural networks. Yet, the reasons behind its effectiveness continue to be debated. In practice, it is often applied ubiquitously and somewhat indiscriminately. In this work, we revisit batch normalization through the lens of lazy and rich dynamics, showing that it effectively shifts the training regime from lazy to rich. Through analytical results on simple models and supporting empirical evidence, we demonstrate that batch normalization is beneficial primarily when the initial model lies in a relatively lazy regime. We further investigate its role in the presence of skip connections, providing insights into when and where batch normalization should be applied.

Over the past two decades, gradient flows in the optimal transport framework have received significant attention due to their wide-ranging applications. They also describe distributional dynamics arising as mean-field limits of interacting Langevin particle systems, leading to McKean–Vlasov (nonlinear Fokker–Planck) equations. Despite their ubiquity, a general asymptotic theory for nonconvex energy functionals remains largely open. We establish an asymptotic convergence theory for Wasserstein gradient flows driven by nonconvex functionals on $\mathcal{P}_2(M)$, the space of probability measures with finite second moment on a Riemannian manifold $M$, by developing a Łojasiewicz–Simon framework on the Wasserstein tangent space. This is joint work with Beomjun Choi (KAIST) and Seunghoon Jeong (POSTECH).

I will give an overview of recent results due to myself and Bob Gray (East Anglia) showing undecidability of certain algorithmic problems in Artin groups and braid groups. I will not assume much in the way of group theory, but instead focus on the algorithmic aspects of the results.

Traditional generative modeling relies on maximum likelihood estimation, equivalent to minimizing the Kullback–Leibler divergence between the true and generated distributions. In modern high-dimensional settings where data often lie on low-dimensional manifolds, the true distribution may not admit a density with respect to Lebesgue measure, making this approach problematic. To circumvent this, the original GAN replaced likelihood-based training with a minimax formulation using the Jensen–Shannon divergence, but training suffers from vanishing gradients; the Wasserstein GAN alleviated these issues to some extent but did not resolve the fundamental instability of minimax optimization. These challenges motivated the rise of diffusion models, which offer stable training but do not directly minimize a statistical distance and typically require many iterative steps for generation. This talk will present two recent results on direct minimization of statistical distances—one based on pure minimization and one in a minimax form—both pointing toward the potential for stably achieving fast one-step generation, in contrast to the many-step nature of diffusion models.

Transformers are the dominant architecture for modern LLMs. However they have some fundamental limitations when it comes to their efficiency and expressivity. This talk will discuss on what future architectures might look like for language modeling (and AI more broadly).

Transformers are the dominant architecture for modern LLMs. However they have some fundamental limitations when it comes to their efficiency and expressivity. This talk will discuss on what future architectures might look like for language modeling (and AI more broadly).

Molecular dynamics (MD) simulations have been widely employed to investigate the dynamical behavior of molecular systems. Despite continuous methodological advances, classical MD simulations suffer from an inherent limitation: nuclear motion is treated entirely within a classical framework, thereby neglecting essential nuclear quantum effects (NQEs) such as zero-point energy and nuclear tunneling. To address this limitation, path integral molecular dynamics (PIMD) has been developed. It represents each nucleus as an ensemble of fictitious particles connected in a ring polymer configuration, thereby enabling the anticipation of quantum nuclear statistics. Although PIMD is theoretically robust, its practical application is often limited by its high computational cost, particularly in generating sufficiently converged ring polymer ensembles which require extensive sampling. Here, we explore the use of an energy-based model (EBM) to learn the equilibrium distribution of ring polymer configurations. With a proposed model, the most time-consuming aspects of conventional PIMD sampling can be effectively bypassed, enabling efficient evaluation of nuclear quantum effects. The results demonstrate that EBM can be a promising strategy for accelerating quantum nuclear simulations while retaining statistical accuracy.

This work addresses the inverse problem in gauge/gravity duality—specifically the reconstruction of bulk spacetime geometry from boundary entanglement entropy—by leveraging generative AI to overcome the limitations of traditional analytical methods. We propose a novel Transformer-based approach that reframes this physical mapping as a sequence-to-sequence translation task, effectively decoding quantum information into geometric metric functions. To tackle the scarcity of analytical models, we utilize a large-scale synthetic dataset derived from random geometries, employing a crucial stochastic noise injection technique that forces the model to learn the precise local differential relationships hidden within the Ryu-Takayanagi formula. Our results demonstrate that the model successfully reconstructs unknown black hole geometries, suggesting that generative AI offers a powerful new framework for decoding the structure of spacetime in quantum gravity and strongly correlated systems.

Artificial intelligence is increasingly reshaping the way new materials are discovered and developed, with the potential to impact a wide range of applications that contribute to a better quality of life. In recent years, materials AI has progressed rapidly, demonstrating promising results in areas such as materials property prediction, molecular and materials design, and synthesis planning. Despite these advances, significant challenges remain in translating AI-driven research outcomes into practical tools that can be effectively used by human experts. In this talk, we begin with a brief introduction to LG AI Research, outlining our mission and vision for advancing AI technologies that deliver real-world value. We then review key research directions in materials AI and discuss emerging trends. Finally, we introduce how LG AI Research is advancing materials AI so that materials researchers can become experts and experts can achieve new breakthroughs with AI. We present several AI-driven materials platforms developed at LG AI Research and demonstrate how these platforms are designed to assist human experts and accelerate the materials development process. This talk aims to illustrate how materials AI can contribute to meaningful scientific and societal impact.

In recent years, Neural Operators have emerged as a powerful framework for learning mappings between infinite-dimensional function spaces, bridging data-driven learning and scientific computing. This talk presents a unified perspective on the evolution of Neural Operator learning from physics-informed formulations to numerically structured frameworks. First, we introduce extensions of Deep Operator Networks (DeepONet), including GraphDeepONet, which enables robust predictions for time-dependent PDEs on irregular grids, and Physics-Informed HyperDeepONet, which learns a generalized inverse mapping for tissue elasticity reconstruction. Finally, we turn to the numerical integration aspect, highlighting the Finite Element Operator Network (FEONet) and its extensions, UCONet, which embed variational structures and physics priors directly into the neural operator architecture. Overall, these studies demonstrate how Neural Operators can evolve into structured, physics-informed, and numerically consistent frameworks that seamlessly connect machine learning and numerical analysis for advancing scientific AI.

Scientific computing using deep learning has seen significant advancements in recent years. There has been growing interest in models that learn the operator from the parameters of a partial differential equation (PDE) to the corresponding solutions. Deep Operator Network (DeepONet) and Fourier Neural operator, among other models, have been designed with structures suitable for handling functions as inputs and outputs, enabling real-time predictions as surrogate models for solution operators. There has also been significant progress in the research on surrogate models based on graph neural networks (GNNs), specifically targeting the dynamics in time-dependent PDEs. In this paper, we propose GraphDeepONet, an autoregressive model based on GNNs, to effectively adaptDeepONet, which is well-known for successful operator learning. GraphDeepONet exhibits robust accuracy in predicting solutions compared to existing GNN-based PDE solver models. It maintains consistent performance even on irregular grids, leveraging the advantages inherited from DeepONet and enabling predictions on arbitrary grids. Additionally, unlike traditional DeepONet and its variants, GraphDeepONet enables time extrapolation for time-dependent PDE solutions. We also provide theoretical analysis of the universal approximation capability of GraphDeepONet in approximating continuous operators across arbitrary time intervals.

Recently, universal machine learning interatomic potentials (uMLIPs) have attracted increasing attention because they can provide reasonable accuracy for diverse applications without fine-tuning. As large, high-quality quantum mechanical datasets continue to be developed and released to the public, uMLIPs are rapidly evolving toward greater universality and higher fidelity. In this presentation, focusing on the SevenNet family, I will discuss the current status and future directions of uMLIPs. I will also introduce new research avenues enabled by uMLIPs.

Generating graphs that strictly satisfy specific topological constraints is a fundamental challenge in network science, where traditional MCMC methods often struggle with exactness and efficiency. In this work, we propose Deep Graph Generation (DGG), a reinforcement learning framework that formulates this task as a sequential Markov Decision Process. To efficiently navigate the vast combinatorial space, we design a hierarchical action space via conditional edge sampling and employ a Graph Neural Network policy for scalable generalization. Our results on degree-degree correlation (assortativity) tasks demonstrate that DGG achieves higher precision with an order of magnitude fewer steps compared to MCMC baselines, validating the effectiveness of our deep RL formulation for constrained graph generation.

While the stochasticity of gradient descent is known to be crucial for training neural networks, the impact of the noise structure itself remains underexplored. In this work, we investigate the role of anti-correlated noise in training dynamics. We theoretically derive the explicit form of the implicit regularizer induced by anti-correlated noise and verify through simulations that the average SGD dynamics follow our theoretical predictions. Moreover, we empirically demonstrate that this structured noise leads to lower generalization errors in both matrix sensing and classification tasks. Our findings provide a comprehensive understanding of how structured noise can be leveraged to improve neural network generalization.

Additive parameter updates, as used in gradient descent and its adaptive variants, form the backbone of modern machine-learning optimization. However, these additive schemes often require many iterations and carefully tuned learning-rate schedules to handle variations in scale and curvature of the loss landscape. In this talk, I will introduce Expectation Reflection (ER), a multiplicative learning paradigm that updates parameters according to the ratio of observed to predicted outputs rather than their differences. ER removes the need for ad hoc loss functions and learning-rate tuning while preserving internal consistency. I will then describe how ER extends naturally to multilayer networks. Finally, I will show that ER can be interpreted as a modified gradient-descent method incorporating an inverse target-propagation mapping. Together, these results position ER as a fast, stable, and scalable alternative to conventional optimization methods for training neural networks.

Activation functions critically influence trainability and expressivity, and recent work has therefore explored a broad range of nonlinearities. However, widely used Gaussian i.i.d. initializations are designed to preserve activation variance under wide or infinite width assumptions. In deep and relatively narrow networks with sigmoidal nonlinearities, these schemes often drive preactivations into saturation, and collapse gradients. To address this, we introduce an odd-sigmoid activations and propose an activation aware initialization tailored to any function in this class. Our method remains robust over a wide band of variance scales, preserving both forward signal variance and backpropagated gradient norms even in very deep and narrow networks. Empirically, across standard image benchmarks we find that the proposed initialization is substantially less sensitive to depth, width, and activation scale than Gaussian initializations. In physics informed neural networks (PINNs), scaled odd-sigmoid activations combined with our initialization achieve lower losses than Gaussian based setups, suggesting that diagonal-plus-noise weights provide a practical alternative when Gaussian initialization breaks down.

Hamilton–Jacobi (HJ) equations connect deterministic optimal control to PDEs: the value function satisfies a first-order HJ/HJB equation via dynamic programming, naturally interpreted in the viscosity sense. I briefly review this link and then move to stochastic differential games, where uncertainty and an adversary yield a viscous Hamilton–Jacobi–Isaacs (HJI) equation with a minimax Hamiltonian, leading to nonconvex/nonsmooth difficulties in high dimensions. I present a PINN-based policy iteration approach that alternates between policy evaluation (solving a linear viscous PDE for fixed feedback policies via a PINN residual loss) and policy improvement (pointwise minimax updates using $\nabla v$). Under convex–concave structure and uniform ellipticity, the method enjoys contraction-type convergence, with overall error controlled by the PINN residual and the iteration error. Numerical results in low and high dimensions demonstrate stability and scalability.

Recent advances in foundation models have revealed impressive reasoning abilities across modalities, yet their internal mechanisms often remain opaque and unstable. This talk explores our recent works on enhancing and understanding the reasoning capabilities of foundation models. First, we focus on the compositional reasoning problems in vision-language models (VLMs). Our proposed READ-CLIP improves compositional reasoning by fine-tuning the text encoder through token-level reconstruction and sentence-level alignment of paraphrased captions. This dual objective encourages the model to capture structured relationships within and across linguistic expressions, leading to stronger compositional generalization. Second, we focus on the reasoning capabilities of large language models (LLMs) guided by process reward models (PRMs). We introduce the notion of reward trajectory volatility, and empirically show that correct reasoning paths exhibit low volatility. We then propose Volatility-Scaled Guided Decoding (VSGD), which rescales step-level rewards with volatility, thus guiding LLM decoding towards more coherent and reliable reasoning outcomes.

Formalizing mathematics involves translating mathematical statements from natural language into a precise formal language that computers can interpret. As modern mathematics becomes deeper and more complex, the importance of formalization has grown significantly. In this talk, I will provide a brief introduction to the concept of formalization, discuss its significance, and explore how formalization and AI interact with each other.

Theorem proving with large language models (LLMs) has attracted significant attention as an important research direction toward building reliable and trustworthy AI systems. Alongside this, autoformalization—the task of translating mathematical statements expressed in natural language into formal language using LLMs—has also emerged as a central challenge and has seen growing research efforts. In this talk, I will provide an overview of recent trends in theorem proving with LLMs and reflect on what these trends imply for theorem proving and autoformalization.

I will present my research on a novel physics-informed operator learning network called Spectral Operator Network (SpecONet) for solving the 3D Navier-Stokes equations (NSE). Applications such as ensemble weather forecasting and the construction of digital twins have continued to demand ever-escalating amounts of high-fidelity fluid dynamics data. However, conventional numerical schemes are restricted to meet this escalating demand, as they require prohibitive computational resources to compute multidimensional NSE solutions. Accordingly, I will introduce SPeONet which efficiently overcomes this restriction and to offer several key advantages: 1) it generates a huge number of NSE solutions in near real-time; 2) by leveraging spectral methods, it achieves superior accuracy with fewer nodal points than existing operator networks; 3) it eliminates the need for reference solutions during training, thus reducing computational overhead; and 4) it is flexibile across various inputs, including initial conditions, boundary conditions, and forcing functions.

We investigate the fundamental differences between the statistical learning paradigm underlying Large Language Models(LLMs) and human language acquisition. We then develop a mathematical framework to explain why LLMs can achieve human-level linguistic competence despite these differences. Within this framework, we argue that LLMs can evolve from superficial pattern extraction to substantive language understanding without explicitly resolving the symbol grounding problem, although some limitations remain. We also offer a theoretical explanation for the emergence of scaling laws, a key empirical finding in modern AI research.

With the rapidly increasing energy demand in the AI ​​era, the need for nuclear fusion technology, a future energy source, is growing. For power generation using nuclear fusion energy, ultra-high-temperature hydrogen plasma must be maintained stably for extended periods. However, determining the temperature of this plasma (diagnosis), how it changes (prediction), and how to maintain the plasma at a desired state (control) remain active research topics. This talk will introduce my ongoing research utilizing machine learning techniques for plasma diagnosis, prediction, and control. This will cover data-driven time-series prediction using nuclear fusion data, approaches to inverse problems using physics-informed neural networks, and nuclear fusion control and optimization using reinforcement learning.

This article introduces Huber means on Riemannian manifolds, providing a robust alternative to the Fréchet mean by integrating elements of both $L_2$ and $L_1$ loss functions. The Huber means are designed to be highly resistant to outliers while maintaining efficiency, making it a valuable generalization of Huber's $M$-estimator for manifold-valued data. We comprehensively investigate the statistical and computational aspects of Huber means, demonstrating their utility in manifold-valued data analysis. Specifically, we establish nearly minimal conditions for ensuring the existence and uniqueness of the Huber mean and discuss regularity conditions for unbiasedness. The Huber means are consistent and enjoy the central limit theorem. Additionally, we propose a novel moment-based estimator for the limiting covariance matrix, which is used to construct a robust one-sample location test procedure and an approximate confidence region for location parameters. The Huber mean is shown to be highly robust and efficient in the presence of outliers or under heavy-tailed distributions. Specifically, it achieves a breakdown point of at least 0.5, the highest among all isometric equivariant estimators, and is more efficient than the Fréchet mean under heavy-tailed distributions. Numerical examples on spheres and the space of symmetric positive-definite matrices further illustrate the efficiency and reliability of the proposed Huber means on Riemannian manifolds. The Python library 'Geomstats' offers an implementation of this estimator.

Stationarity and ergodicity play a central role in time series analysis, forming the theoretical basis for consistent estimation, asymptotic normality, and the validity of statistical inference. While these properties have been well studied in classical models such as ARMA and GARCH, their theoretical understanding in neural network–based time series models remains limited. This study develops sufficient conditions for the stationarity and ergodicity of multivariate neural network autoregressive moving average (NN-ARMA) processes. The analysis builds upon the iterated random function (IRF) approach, providing a general theoretical foundation for the stability of nonlinear stochastic systems driven by neural architectures. Furthermore, the research investigates the generalization capability of stationary ergodic NN-ARMA models through Rademacher complexity bounds. By adapting the results of Mohri and Rostamizadeh (2010), we derive generalization error bounds under weak dependence, bridging statistical learning theory and time-series ergodic theory. Together, these results aim to establish a unified framework that links the theoretical stability of neural network time series models with their learning capacity, offering insights into both the probabilistic and algorithmic behavior of modern stochastic neural models.

Nearest neighbor search on the hypersphere is a core primitive in AI and information retrieval. This talk surveys key methodologies for spherical $k$-nearest neighbor ($k$-NN) search, and compare their guarantees, strengths, and limitations. We analyze the phenomenon of similarity concentration in high-dimensional spaces, where the gap between near and far neighbors diminishes. This effect degrades pruning efficiency and increases collision rates, thereby sharpening the trade-offs between time, space, and accuracy in spherical LSH schemes. Finally, we consider the impact of anisotropy and clustering commonly observed in real-world data. We discuss how such structures may be exploited to develop faster, data-aware spherical $k$-NN algorithms while retaining theoretical guarantees.