PDE and Applied Mathematics

Training accurate dynamical surrogates for chaotic systems is difficult as their trajectories are extremely sensitive to initial conditions and model parameters, making exact future forecasts fundamentally impossible. Yet chaotic systems possess ergodicity and time-invariant measures, which give rise to well-defined global statistical properties. A properly learned neural network dynamics surrogate must accurately reproduce such properties, something conventional mean squared error-based training struggles to achieve.

In this talk, I will first introduce neural ordinary differential equations as effective priors for modeling dynamical systems. I will then discuss the problem of properly training these models on chaotic data, highlighting two key issues in particular: the unstable training problem for long data, and making models properly reproduce the statistical properties of the ground truth systems. After presenting my solutions to each of these problems, namely homotopy-based training and neighborhood-based training, I will close with a discussion on future research directions.

The recent success of large language models (LLMs), such as ChatGPT, marks a remarkable advance in machine learning and artificial intelligence. However, it remains premature to assert that these models possess genuine reasoning capabilities. Although LLMs excel at generating coherent and contextually relevant text based on extensive training data, their performance in domains requiring precise logical reasoning—such as mathematics—remains limited. In particular, LLMs frequently struggle with exact mathematical operations, positioning mathematics as a critical benchmark for assessing an AI model's capacity for logical reasoning.

In this talk, we explore whether language models can learn and internalize the exact rules of mathematical operations purely from data. Our investigation is grounded in the phenomenon of grokking—a surprising behavior where a model, after extended training, abruptly transitions from poor to near-perfect generalization. By examining the inner workings of the training dynamics, we aim to uncover how language models develop implicit understanding of mathematical rules. We propose that this understanding may emerge through the formation of low-dimensional geometric representations, shedding light on how such models encode abstract structures during training.

Recently, the virtual element method has emerged as a flexible generalization of the finite element method to polygonal and polyhedral meshes. In this talk, we introduce the virtual element method, focusing on its basic principles, implementation, and applications. First, we discuss the key ideas for the construction of virtual element spaces and the resulting discrete formulation, together with a brief comparison with the finite element method. We then outline the practical aspects of an implementation of the virtual element method. Finally, we review several recent results and applications of the virtual element method.

In this work, we propose a novel neural-network-based framework for particle methods commonly used in gas dynamics and fluid mechanics. While classical particle methods track the time evolution of particles using explicit time-stepping schemes such as the Euler or fourth-order Runge–Kutta (RK4) methods, we demonstrate that particle trajectories can instead be approximated through Physics-Informed Neural Networks (PINNs). Moreover, motivated by the score-based formulation underlying kinetic equations, we approximate the score function by an additional neural network and couple it with the network that learns particle trajectories. This results in an interacting two-network architecture, where the learned score function guides the evolution of particles while the particle dynamics, in turn, inform the score approximation. Through this coupled framework, we aim to approximate solutions of the spatially homogeneous Landau equation at the particle level. The proposed approach combines the interpretability of particle-based numerical methods with the flexibility of neural network approximations, providing a scalable and flexible alternative to conventional time-integration schemes for kinetic equations.

The universal approximation theorem shows that neural networks can approximate any function, but the number of parameters required grows exponentially with dimension of the domain. This leaves open the question of why neural networks are so effective in practice, especially for high-dimensional data. A key advance came from the theory of Barron spaces: if the target function f lies in a Barron space, then a two-layered network with n parameters can approximate it within error of O(n^{-1/2}) order. However, there remains a gap between the regularity conditions of the Barron space B^{s+1} and the Sobolev space H^s. Recent study have proved that deep ReLU networks of depth L can bound L² error in terms of the B^s-norm rather than B¹-norm, under the condition sL ≤ 1/2. In this talk, we introduce a log-weighted Barron space B^{log}, which posses weaker regularity than B^s for any s>0. We prove that functions in B^{log} can be approximate in L² norm by deep ReLU networks. Moreover, we generalize this to define B^{s,log}, establish an approximation bound in terms of H¹ norm, and estimate depth to assure the bound.

Physics-Informed Neural Networks (PINNs) have emerged as a promising framework for solving partial differential equations (PDEs). The standard PINN approach relies on minimizing the L² norm of the PDE residual. However, this conventional method faces a challenge when applied to the BGK model. In fact, the convergence of the L² PDE residual to zero does not necessarily guarantee the convergence of the approximate solution to the exact solution. This discrepancy indicates that the standard L² PDE residual is insufficient for the network to learn the true solution of the BGK model. To overcome this limitation, we propose an weighted L² physics-informed loss, derived from the stability theory of the BGK model. We prove that minimizing these weighted PDE residuals ensures the convergence of the PINN solution to the true solution. We also conduct numerical experiments to validate the performance of the proposed loss functions. The results confirm that the proposed weighted loss enhances the accuracy of PINNs for solving the BGK model.

In recent years, Neural Operators have emerged as a powerful framework for learning mappings between infinite-dimensional function spaces, bridging data-driven learning and scientific computing. This talk presents a unified perspective on the evolution of Neural Operator learning from physics-informed formulations to numerically structured frameworks. First, we introduce extensions of Deep Operator Networks (DeepONet), including GraphDeepONet, which enables robust predictions for time-dependent PDEs on irregular grids, and Physics-Informed HyperDeepONet, which learns a generalized inverse mapping for tissue elasticity reconstruction. Finally, we turn to the numerical integration aspect, highlighting the Finite Element Operator Network (FEONet) and its extensions, UCONet, which embed variational structures and physics priors directly into the neural operator architecture. Overall, these studies demonstrate how Neural Operators can evolve into structured, physics-informed, and numerically consistent frameworks that seamlessly connect machine learning and numerical analysis for advancing scientific AI.

Activation functions critically influence trainability and expressivity, and recent work has therefore explored a broad range of nonlinearities. However, widely used Gaussian i.i.d. initializations are designed to preserve activation variance under wide or infinite width assumptions. In deep and relatively narrow networks with sigmoidal nonlinearities, these schemes often drive preactivations into saturation, and collapse gradients. To address this, we introduce an odd-sigmoid activations and propose an activation aware initialization tailored to any function in this class. Our method remains robust over a wide band of variance scales, preserving both forward signal variance and backpropagated gradient norms even in very deep and narrow networks. Empirically, across standard image benchmarks we find that the proposed initialization is substantially less sensitive to depth, width, and activation scale than Gaussian initializations. In physics informed neural networks (PINNs), scaled odd-sigmoid activations combined with our initialization achieve lower losses than Gaussian based setups, suggesting that diagonal-plus-noise weights provide a practical alternative when Gaussian initialization breaks down.

In this presentation, we introduce stochastic gradient Langevin dynamics (SGLD) as a principled approach to solving stochastic optimization problems. We begin by reviewing the foundational work of Raginsky et al. (2019, COLT), which established global convergence guarantees for SGLD under suitable assumptions. Building on this theoretical foundation, we then present recent advances that extend these results in several directions, including improved non-asymptotic convergence rates, relaxed regularity assumptions on the objective and stochastic gradients, and the development of new algorithmic variants tailored to modern optimization settings.

In solving fluid equations, the control of the pressure term is a central issue. For the two-dimensional incompressible, inviscid Euler equations, taking the curl eliminates the pressure term and yields the vorticity equation ∂tω + (u · ∇)ω = 0, which implies that the vorticity is conserved along the fluid particle trajectories. Using a streamfunction, the velocity field satisfies the Biot-Savart law. Consequently, the point vortex model arises as a singular solution of the 2D Euler equations. Furthermore, the vortex system admits a Hamiltonian structure and is integrable.

Viscoelastic materials such as polymers, soft biological tissues, and high-temperature metals exhibit stress-strain behavior with fading memory, which can be modeled by Volterra integral equations of the second kind. This study develops and analyzes numerical methods for viscoelastic models characterized by exponential and power-law stress relaxation. By introducing internal variables to transform the Volterra convolution integral into auxiliary ordinary differential equations, we employ the symmetric interior penalty Galerkin (SIPG) method for spatial discretization and Crank–Nicolson-type schemes in time. For weakly singular kernels in fractional-order viscoelasticity, a linear interpolation approach is used to treat the singularity while maintaining accuracy. We present a priori stability and error estimates independent of Grönwall's inequality, yielding bounds that grow linearly with time and ensuring suitability for long-time simulations. Theoretical results are verified through numerical experiments implemented in the FEniCS environment, including simulations using real material data, and supported by residual-based a posteriori error analysis. Furthermore, we explore physics-informed neural networks (PINNs) as an alternative solver for integro-differential equations, showing that they can accurately and efficiently capture memory effects and facilitate parameter identification. Together, these results establish a unified computational framework for accurate, stable, and efficient simulation of viscoelastic phenomena.

Barron spaces have gained significant attention in the machine learning community for their ability to provide dimension-free convergence rates in two-layer neural networks. This paper extends Barron norm estimates for the whole-space static Schrödinger equation to incorporate the nonlocal term. We show that if both the source term and the potential function of the equation belong to the Barron space, and the potential function has a nonnegative lower bound, the solution remains within the Barron space. As a consequence, we prove that the two-layer network can approximate solutions of the fractional Schrödinger equation with a rate determined by the constant quantified by the given data.

Operator learning offers a unified framework for PDE inverse problems and time-dependent dynamics from partial observations, yet two practical challenges remain. For inverse problems, labeled pairs of measurements and unknown fields are often unavailable. For dynamical prediction, performance can degrade when observations are defined on irregular grids. We address both issues within a DeepONet-based operator learning framework. First, we introduce Physics-Informed Deep Inverse Operator Networks (PI-DIONs), which incorporate physics-based losses to enable label-free training while retaining competitive performance in supervised settings. We further extend stability estimates from inverse problem theory to the operator learning setting, yielding bounds that characterize how accuracy depends on the number of samples and grid resolutions. Second, we propose GraphDeepONet, an autoregressive graph-based operator network that combines DeepONet with message passing to enable prediction on arbitrary irregular grids and improve stability across grid configurations. We provide approximation and confirm consistent performance on irregular grid dynamics, including shallow-water and Navier-Stokes benchmarks. Finally, we provide a universal approximation result for GraphDeepONet over arbitrary time intervals.

Optimal Transport (OT) provides a principled framework for transforming one probability distribution into another via a cost-minimizing map. Leveraging this framework to transport a simple prior distribution to a complex data distribution has emerged as a powerful approach for generative modeling, including OT maps, flow matching, and diffusion models. In this talk, I will introduce recent approaches to generative modeling built on OT and Wasserstein gradient flow, highlighting their theoretical insights and practical implications.

In this talk, I will share experiments where my machine learning method failed to infer PDE solutions. Currently, machine learning methods have become popular for explaining and predicting complex systems, such as CFD and weather forecasting simulations. Conventionally, these simulations have been conducted using numerical schemes, for example, FDM, FEM, and FVM to compute Navier-Stokes-based models, yet they require massive computational time and resources. Instead, machine learning methods, such as Physics-Informed Neural Networks (PINNs) and Deep Operator Networks (DeepONets), have recently emerged, as they are expected to provide accurate solutions in a fraction of time with fewer resources. Nevertheless, PINNs are limited to single instances, and DeepONets, being supervised learning methods, require solution data, which can be prohibitive to obtain. Therefore, I have developed unsupervised learning networks for solving Navier-Stokes equations. However, this is highly challenging due to the limited development of relevant theories and methodologies for designing and training effective neural networks. I will share the trials and errors I encountered during my research, and then discuss how to overcome them.

When oscillators interact each other, they exhibit variety of phenomenon such as synchronization, chaotic motion, and chimera states. Such a system, called coupled oscillator system, is prevalent from neuronal oscillators and circadian rhythms to quantum optics and engineering. In this talk, we study the forward and inverse problem of the coupled oscillator systems. For the forward problem, we investigate the existence of periodic measure-valued solutions for a class of nonlocal continuity equations, which include the mean-field equations arising from the coupled systems of oscillators. We use a fixed point theorem for geodesically convex spaces constructed by optimal mass transport. On the other hand, in many real-life situations, we often lack the information of the exact model, in particular the interaction force between oscillators. Thus, in the next section, we study the inverse problem to search for the coupling function among oscillators using physics-informed neural networks.