FASTMath Seminar Series

The purpose of the seminar series is to invite domain scientists to talk about their math challenges in order to motivate and encourage collaborations between science teams and FASTMath. 

November 8, 2023, 2:00pm-3:00pm Eastern

Moderators: John Jakeman, SNL; Victor Paludetto Magri, LLNL

Speakers:

• Pieterjan Robbe, SNL: "Multilevel Bayesian calibration with summary statistics for the prediction of xenon diffusion in UO2 nuclear fuel"
  Despite the recent shift towards renewable energy, the US is still the world's largest producer of nuclear power. Uranium dioxide (UO2) is the main fuel type used in these power plants.  A key performance indicator for nuclear fuel performance is the rate of fission gas release. In this work, we construct a Bayesian calibration framework for an atomistic model that explains the mechanisms behind this fission gas release. The mechanistic model contains a total of 185 uncertain parameters, that we would like to estimate based on available data from historic thermodynamic and gas release experiments. The calibrated model can then be used to make predictions about the fission gas diffusivity, and hence the performance, of new UO2-based nuclear fuel types. Significant challenges arise because of the high dimensionality of the problem, because of the expensive model evaluations, and because the original experimental data is often inaccessible, i.e., we only have access to so-called data summaries, such as measured nominal values and associated uncertainties. We address these challenges by using polynomial-based surrogates to replace the expensive model evaluations, and by using synthetic data sets to match the model predictions with the reported summary statistics. Our results capture and improve upon the established fit from Turnbull et. al., used in many nuclear fuel performance codes to date. Finally, we illustrate how multilevel approaches, that use a combination of samples from the surrogate as well as the actual model, can be used to further increase the accuracy of our estimate.

• Nathaniel Overton-Katz, LBNL: "Complex Higher-order Cut Cell Meshes using Geometric Conservation Laws"
  We present a rapid, adaptive mesh algorithm for generating cut-cell Cartesian meshes from general implicit functions. The approach uses a surface triangulation of points that is built up from 1D grid line intersections, to 2D planar polygonizations, and finally 3D triangulated surfaces. Intersection and subdivision algorithms are used to locate points on the implicit surface and use iterative methods and line searches given the normal from the implicit function. This makes the resulting STL suitable for use with signed distance functions, mesh refinement, and constructive solid geometries generated from CAD definitions. To make this approach compatible with finite-volume conservation laws, we use the discretized surface to produce the necessary higher-order area and volume moments that satisfy the corresponding geometric conservation laws, discretely, up to very high order. The resulting algorithm is suitable for high-accuracy cut-cell mesh generation and is GPU capable for faster computation.

August 16, 2023, 2:00pm-3:00pm Eastern

Moderators: Tyler Chang, ANL; Viktor Reshniak, ORNL

Speakers:

• Steven Roberts, LLNL: "Improved Time-Stepping for the BISICLES Ice Sheet Model"
  The BISICLES ice sheet model relies on accurate and stable numerical methods to predict long-term sea-level rise and ice sheet instability. The overwhelming computational cost in BISICLES comes from computing the ice velocity field via a coupled nonlinear tensor solve. The default explicit time integrator in BISICLES performs only one of these solves per timestep, but this imposes a restrictive timestep size for stability and is only first order accurate. In this talk, I will discuss how we upgraded the time-stepping capabilities of BISICLES by introducing efficient, high-order methods from SUNDIALS. The underlying mathematical model for the ice sheet can be viewed as either an ordinary differential equation or an index-1 differential-algebraic equation. The former interpretation allows us to apply explicit Runge-Kutta schemes, while the latter allows us to apply a half-explicit Runge-Kutta method with a similar per-step cost to the default integrator but one order more accurate. For these methods to operate on the adaptive ice sheet mesh, we developed interoperability between the Chombo and SUNDIALS FASTMath libraries. Through numerical tests, we have found that this work enables orders of magnitude improvements in temporal accuracy and faster solutions in stability-limited physical regimes.

• Jacob Merson, RPI: "Parallel Coupler for Multimodel Simulations (PCMS): A new approach for independent model coupling of Tokamak plasma simulations"
  Fusion power promises to be one of the most transformative technologies of our time, however, many fundamental questions about plasma physics and reactor operation remain. Answering these questions require exascale multiscale and multiphysics simulations where each component requires a broad range of domain and computational expertise. Over the past 50 years, large teams have invested thousands of man-hours into the development of purpose-built codes that can efficiently simulate the physics in specific portions of the reactor volume that use fundamentally different discretizations, time-stepping methods, mathematical models, and so forth. Given the code complexity, need for specialized numerics in each part of the reactor volume, and expertise required to develop these codes, it is not feasible to develop a new multiphysics code that encompasses the entire reactor volume using a homogenized framework. Currently, ad-hoc coupling methods are used, but do not scale to the needs of full-device, or full-plant modeling. This talk presents a new framework for tight-coupling of distinct codes at-scale on exascale supercomputers. It will describe the approach for data-transfer and parallel control, and interrogation-based field transfer operations. Including, recent results coupling a core/edge plasma simulation and integration with a high-level language for dynamic control of coupled simulations.

April 24, 2023, 2:30pm-3:30pm Eastern

Moderators: Victor Magri, LLNL; John Jakeman, SNL

Speakers:

• Cody Balos, LLNL: "An Exascale-Ready and Flexible Implementation of Symplectic Partitioned Runge-Kutta Methods in SUNDIALS"
  Symplectic Partitioned Runge-Kutta (SPRK) methods are not new, and there is plenty of literature that demonstrates their effectiveness when applied to separable Hamiltonian problems.  However, few available implementations of SPRK methods are suited for high-performance computing. In this talk, I will discuss new implementations of SPRK methods up to order 8 in the SUNDIALS time-integration library. These SPRK implementations are GPU-enabled and ready to run on Exascale class HPC systems.

• Jerry Watkins, SNL: "Ongoing challenges in improving and maintaining computational performance in the ice-sheet modeling code MALI"
  High-resolution simulations of polar ice sheets play a crucial role in the ongoing effort to develop more accurate and reliable Earth system models for probabilistic sea-level projections. Increasing the fidelity and resolution of these simulations on the latest DOE supercomputers poses significant computational challenges and demands the adoption of modern software techniques. For example, many of today's supercomputers contain a diverse set of computing architectures and require specific programming interfaces in order to obtain optimal efficiency. In an effort to avoid architecture-specific programming and maintain productivity across platforms, the ice-sheet modeling code known as MPAS-Albany Land Ice (MALI) uses high-level abstractions to integrate modular components for performance portable code across a variety of different architectures. This has led to rapid integration of new technologies but an increase in complexity when improving and maintaining computational performance. This talk focuses on current and future challenges that need to be addressed for high computational throughput of ice-sheet models on DOE machines.

March 20, 2023, 3:00pm-4:00pm Eastern

Moderators:  Julie Bessac, ANL; Pieter Ghysels, LBNL

Speakers:

• Marieme Ngom, ANL: "Neural Network Architectures for Periodic Data"
  In this talk, we present two neural networks architectures that are tailored for periodic data. The first architecture is a type of Fourier neural networks with a choice of activation and loss functions that yields results that mimic the Fourier decomposition. The second architecture is a specific instance of neural ordinary differential equations and features a novel regularization strategy that ensures its stability. This strategy is based on established ode theory on the stability of linear canonical systems with periodic coefficients. We demonstrate the effectiveness of the presented architectures on various tasks such as approximating periodic functions, solving differential equations and discovering unknown dynamics.

• Yang Liu, LBNL: "Babich ansatz: a geometrical-optics-like ansatz for Green's function of wave equations with variable coefficients"
  We present a fast and high-order accurate representation of the Hadamard-Babich (HB) ansatz for the Green's function of the high-frequency Helmholtz and Maxwell equations in smooth inhomogeneous media with arbitrary excitation sources. The proposed algorithm first solves the phase and HB coefficients via eikonal and transport equations using a coarse mesh, and then compresses the resulting HB interactions using several new butterfly algorithms developed through the FastMATH project, leading to optimal CPU and memory complexities for any bounded 3D domains.

February 21, 2023, 4:00pm-5:00pm Eastern

Moderators:  Tyler Chang, ANL; Viktor Reshniak, ORNL

Speakers:

• Christian Glusa, SNL: "Algebraic multigrid solvers for nonlocal equations"
  The naive discretization of nonlocal operators (such as fractional-order derivatives and boundary integral equations) leads to matrices with significant density, as compared to classical PDEs. This makes the efficient solution of nonlocal models a challenging task. In this presentation, we will discuss on-going research into hierarchical matrix assembly and algebraic multigrid solution techniques that are suitable for nonlocal models and allow to recover quasi-optimal scaling.

• Toby Issac, ANL: "Robust Expected Information Gain in Optimal Experimental Design"
  Expected information gain (EIG) is a useful measure of experiment optimality, but it can rarely be computed exactly, and there are drawbacks associated with different approximations. Nested Monte Carlo and related methods are popular approaches to estimating EIG. We note two issues with the use of these estimators. The first is the effect that outliers can have in the estimators when they are undersampled. The second is the sensitivity of experiments to perturbations in the prior: the sensitivity of an experiment's EIG to perturbations in the prior should affect its suitability in a risk averse situation. We propose a quantity called robust expected information gain (REIG) that maximizes the minimum EIG over perturbations in the prior which maps onto a fast post-processing of Monte Carlo estimators, which in some cases can also address the outlier affect.

January 24, 2023, 2:00pm-2:30pm Eastern

Moderators:  Victor Magri, LLNL; John Jakeman, SNL

Speakers:

• Ahmed Attia, ANL: "Optimal Experimental Design for Bayesian Inversion"
  Optimal data acquisition, for inverse problems, can be modeled as an optimal experimental design (OED) problem, which has gained wide popularity and attention from researchers in various fields in statistics, engineering, and applied math. Challenges in model-constrained OED include high-dimensionality of the underlying inverse problem, misrepresentation of uncertainties and experimental setup, among others. In this talk, we provide an overview of our recent developments in the area of model-constrained OED and optimal sensor placement. Specifically, we discuss an efficient stochastic learning approach for solving challenging binary optimization problems ideally suited for solving challenging binary OED problems for sensor placement. Additionally we discuss efficient approaches for robustifying the solution of such OED problems.

June 28, 2022, 2:00pm-3:00pm Eastern (recording)

Moderators:  Cody Balos, LLNL; Roel Van Beeumen, LBNL

Speakers:

• Bryan  M. Wong, University of California - Riverside: "Probing Electron Dynamics of Complex Chemical and Material Systems in Real Time"
  Electronic and light-harvesting materials continue to garner significant attention due to their importance in energy conversion and light-induced technologies. However, several challenges exist in improving their performance: these electronic multifunctional systems are complex, coupled, many-body systems with both structural and electronic interactions with their surrounding environments. All of these processes occur at different time and length scales, and span an immense multi-scale space (i.e., chemical interactions within their environment, all in tandem with external electromagnetic and/or electric fields). To this end, we have developed several computational approaches based on real-time, time-dependent density functional theory (RT-TDDFT) to directly probe and calculate these complex chemical systems. Our approach significantly differs from conventional DFT methods in that we can directly calculate real-time dynamical effects in the presence of strong, interactions with the surrounding environment. Using these RT-TDDFT capabilities, I will highlight our recent work in light-harvesting systems and complex materials. By treating these large systems at a quantum-mechanical level of detail, we show that the energy-transfer dynamics in these chemical systems are surprisingly rich and complex. Most importantly, these time-domain studies provide an intuitive approach to probe the microscopic details of real-time energy-transfer mechanisms to both understand and tailor these complex chemical systems for realistic engineering applications.

• Vojtech Vlcek, University of California - Santa Barbara: "Dynamical Correlations and Routes for Efficient Embedding and Downfolding"
  I will discuss the proposed multilayered approach to describing the electronic dynamics in quantum materials. The strongly correlated phenomena can be, in many typical cases, captured by energetically or spatially localized states embedded in a sea of weakly correlated electrons. Using the many-body perturbation theory based on Green's function formalism, we aim to capture the excited state phenomena by a successive embedding of deterministic perturbative and explicitly correlated methods in stochastic many-body approaches appropriate for treating a large number of weakly (and moderately) correlated quantum particles. I will illustrate the first steps toward the multiscale embedding approach involving the dynamical downfolding. I will highlight the critical steps involved and the actual and potential bottlenecks. I will discuss how combining improved algorithms and implementations with efficient low-scaling stochastic numerical techniques constitutes an ideal platform for simulating complex nanoscale systems with thousands of electrons at a minimal computational cost.

May 20, 2022, 4:00pm-5:00pm Eastern

Moderators:  Yang Liu, LBNL; Ben Whitney, ORNL

Speakers:

• Michael Zingale, Stony Brook University: "Algorithmic Improvements for Coupling Hydrodynamics and Reactions in Astrophysical Flows"
  Energy production in stellar environments is dominated by thermonuclear energy release, which can take place in quiet convective flows or explosive environments.  Multidimensional models of astrophysical reactive flows have many challenges: capturing the relevant length and timescales, and keeping the different physics inputs coupled together over the course of the simulation.  Reaction networks are stiff, requiring implicit time integrators, while the hydrodynamics is treated explicitly.  The standard approach in the field is operator splitting of the reactions, but this can break down when the reactions are vigorous.  I will discuss some new time integration methods we have been developing in the AMReX-Astrophysics Castro code (https://github.com/amrex-astro/Castro).  Two alternate time-integration methods are being developed.  The first follows the traditional spectral deferred corrections (SDC) method of using low order approximations to the time updates to build a high-order solution through iteration.  With this approach we can do fully fourth-order reactive hydrodynamics simulations.  We are also exploring a simpler approach that uses some of the ideas of SDC together with our existing hydrodynamics and reaction solvers to eliminate the operator splitting error.  Both methods will be shown and comparisons to the traditional operator splitting approach will be discussed.

• Yuxi Chen, Princeton University: "Magnetohydrodynamics with Embedded Particle-in-Cell Model and Its Application to Magnetic Reconnection"
  It is challenging to capture kinetic phenomena in global simulations due to significant differences between the kinetic scales and global scales. The magnetohydrodynamics with embedded particle-in-cell model (MHD-EPIC) is developed to incorporate local kinetic physics into global simulations. MHD-EPIC combines the physics capability of a particle-in-cell (PIC) code and the efficiency of an MHD model by coupling a semi-implicit PIC code with an MHD model. The PIC code is used to cover regions where kinetic effects are important, while the MHD model handles the rest part of the simulation domain. So far, the application of the MHD-EPIC model has been focused on magnetic reconnection, which is a ubiquitous phenomenon in many laboratory and astrophysical systems, including fusion devices, stellar atmospheres, and (exo-)planetary and pulsar magnetospheres. In this presentation, I will first discuss the model coupling approach and then show an application of the MHD-EPIC model to the magnetic island coalescence problem and its comparison with the full PIC simulation.

March 8, 2022, 11:00am-12:00pm Eastern (recording)

Moderators:  Ahmed Attia, ANL; Juliane Mueller, LBNL

Speaker:

• James Amundson, FNAL: "Computational Topics in Particle Accelerator Simulation"
  Particle accelerators are enabling technology for scientific research high energy physics, nuclear physics, and many other fields. The most computationally difficult aspect of simulation the dynamics of particle accelerator beams is the inclusion of collective effects. I will give an overview of the challenges presented in the simulation of collective effects in beam dynamics. A major application of beam dynamic simulations is in the design optimization of new and/or upgraded accelerators. I will also describe topics in accelerator design optimization and recent progress in addressing them.

February 16, 2022, 3:00pm-4:00pm Eastern

Moderators:  Cody Balos, LLNL; Cameron Smith, RPI

Speakers:

• David Green, ORNL: "ASCR-Relevant Challenges of the Fusion Energy RF-SciDAC Partnership"
  The RF-SciDAC Center has the goal of predicting how magnetically confined fusion plasmas respond to externally applied Radio Frequency (RF) power in the context of heating and current drive for fusion reactors. Amongst the present and near-term ASCR-relevant challenges the Center is facing are the following topics: (i) solving time-harmonic (indefinite) Maxwell's equations with non-linear boundary conditions in 3D domains with high geometric fidelity boundary shapes - this necessitates degrees-of-freedom reduction via high order (elements and meshing), adaptivity, preconditioning to enable robust scaling of iterative solvers, and simplification of the interaction with CAD representations of domain boundaries; (ii) solving fluid transport in highly anisotropic (magnetized plasma) mediums, again with high geometric fidelity domain boundaries which are not aligned with the anisotropy - our approach here, to enable robust meshing of variable simulation geometries, is a combination of an unstructured mesh with high order elements to resolve the anisotropy without introducing unwanted transport. This approach requires the development of preconditioners which reduce the condition number of the resulting system for high order / high anisotropy cases; (iii) high-dimensional PDEs describing kinetic transport - presently we are investigating adaptive sparse-grids; (iv) non-local kernels for the plasma conductivity in the wave equation, which have previously been handled via Fourier spectral methods but which are insufficient when incorporating high geometric fidelity in the domain boundary; and (v) robust frameworks for the acceleration of code-to-code Picard type iteration. In this presentation, we will give an overview of these challenges, the status of our progress on solving them and identify opportunities for collaboration.

• Jeff Candy, General Atomics: "Eulerian Gyrokinetics: Spectral and pseudospectral discretization in CGYRO"

January 31, 2022, 4:00pm-5:00pm Eastern

Moderators:  Yang Liu, LBNL; Ben Whitney, ORNL

Speakers:

• Thomas Maier, ORNL: "Math Needs for Materials Quantum Monte Carlo Applications"
  Quantum Monte Carlo (QMC) methods are widely used to study the finite temperature behavior of strongly interacting electron systems. Here I will use the DCA++ application that implements a dynamic cluster QMC algorithm as an example to discuss several challenges that could benefit from collaborations with applied mathematicians. These include acceleration of the simulations (AI/ML), extraction of real time dynamics from the imaginary time QMC data (inverse problems), and managing large tensors that describe the effective interaction between electrons and provide the deepest insight into the physics of the system (linear eigensolvers and data compression).

• Martin Head-Gordon, UC Berkeley/LBNL: "Can applied math help solve the electron correlation problem in computational quantum chemistry?"
  This talk will provide a discussion of some issues that the applied mathematics community may be able to address as part of the fundamental challenge of efficiently evaluating the correlation energy of electrons in molecules. Computational quantum chemistry seeks to approximate the many-electron Schrodinger equation by introducing (i) the algebraic approximation (a one-particle basis set to convert a partial differential equation into algebraic equations), and (ii) the correlation model (which truncates those equations in a rational and efficient way). These approximations together reduce the formal exponential compute complexity of the Schrodinger equation to polynomial. However, the development of fast algorithms for electron correlation offers the prospect of further lowering the non-linear polynomial scaling of compute costs, and is an area where applied math is poised to play an important role in our SciDAC scientific partnership. I will present some of the challenges and the possibilities using one of the simplest correlation models, second order Moller-Plesset (MP2) theory, which has compute costs scaling formally with the 5th power of molecule size.

October 26, 2021, 4:00pm-5:00pm Eastern

Moderators:  Cody Balos, LLNL; Roel Van Beeumen, LBNL

Speakers:

• Eirik Endeve, ORNL: "Considerations for time integration methods in nuclear astrophysics applications"
  Core-collapse supernovae and binary neutron star mergers are astrophysical events of considerable interest in nuclear physics. They are dominant sources of heavy elements in the Universe, and emit photons, neutrinos, and gravitational waves. Gaining insights into the physical processes driving these multi-messenger events relies heavily on modeling and simulation. The models solve a coupled system of equations for self-gravity, magneto-hydrodynamics for nuclear matter, nuclear reaction networks, and neutrino transport, and involve a wide range of spatial and temporal scales. In this talk, we will give a brief overview of the physical models and some of the numerical methods targeted by the Exascale Computing Project’s ExaStar team. To initiate discussions, we will place particular emphasis on considerations relevant to the design of time integration methods for deployment in nuclear astrophysics applications.

• Robert Edwards, Jefferson Lab: "Computing the Properties of Matter"
  I will review some of the numerical challenges faced in using lattice field theory techniques for computations in Nuclear Physics.

September 14, 2021, 4:00pm-5:00pm Eastern

Moderators:  Cameron Smith, RPI; Ben Whitney, ORNL

Speakers:

• Stephen Price, LANL: "Future numerical and computational challenges in DOE ice sheet modeling"
  In this talk, I will briefly summarize DOE progress on ice sheet model development and applications during the past 10 years, under joint BER and ASCR funding. I will then discuss remaining challenges and improvements that future FASTMath research efforts might contribute to, with a focus on four areas: 1) numerical and computational methods, 2) meshing, 3) optimization and uncertainty quantification approaches, and 4) model physics. Specific context will be provided through examples of past and ongoing efforts in the area of ice sheet modeling. 

• Peter Bosler, SNL: "Compact, Performance-Portable Semi-Lagrangian Methods for E3SM"
  The ASCR/BER partnership SciDAC Project, \emph{Non-hydrostatic dynamics with characteristic discontinuous Galerkin methods}, develops semi-Lagrangian (SL) algorithms and associated software for passive tracer transport in E3SM that are both (a) tailored for the advanced architectures of current and anticipated DOE LCF computing platforms and (b) provably successful at achieving the fundamental required traits of a climate model's transport scheme: conservation, accuracy, tracer consistency, shape preservation, and computational efficiency. Integrating SL transport into E3SM frequently requires algorithmic improvements in other parts of the code (time stepping, in particular) and exposes opportunities for investigating non-hydrostatic effects and radiative-convective equilibrium at high resolution. In this talk, we examine the impacts and applications of this work on the E3SM Atmosphere Model's (EAM) version 2 and discuss ongoing work with MPAS-Ocean targeted at the version 3 biogeochemistry science campaign.   We conclude with discussion questions associated with this work that may be relevant for future FastMath collaborations.

July 29, 2021, 4:00pm-5:00pm Eastern

Moderators:  Ahmed Attia, ANL; Yang Liu, LBNL

Speakers:

• S.C. Jardin, PPPL:  "The M3D-C1 code"
  We describe the M3D-C1 code, emphasizing the linear solvers and opportunities for uncertainty quantification.   M3D-C1 is an initial-value magnetohydrodynamic code based on 3D finite elements with C1 continuity and semi-implicit time-stepping.   The finite element mesh is unstructured in the (R,Z) plane, but structured in the toroidal angle φ.  We take advantage of the structure by utilizing a block-Jacobi preconditioner based on multiple instances of SuperLU_dist for the full 3D GMRES iterative sparse matrix solves.  The solution depends in a complex way on the initial conditions, boundary conditions, and the values of the dissipation coefficients (particle diffusivity, thermal conductivity, viscosity, electrical resistivity) which also determine resolution requirements.

• CS Chang, PPPL: "Needs for Advanced Linear Systems and UQ in the Nonequilibrium Fusion Kinetic Code XGC"
  XGC started as a partnership code between fusion application SciDAC Center and the ASCR SciDAC Institutes a decade and a half ago to solve the nonequilibrium kinetic physics problems in the boundary region of magnetic fusion reactors utilizing particle-in-cell method on leadership class computers.  XGC has been making difficult scientific discoveries that have not been possible by other methods.  XGC is now entering into new multiscale interaction regimes that solve together the small-scale/small-amplitude kinetic turbulence, the large-scale/large-amplitude fluid instabilities, and the background plasma profile dynamics.  The soon-to-arrive exascale computers will be powerful enough to enable such a multiscale simulation and prediction of ITER plasmas in realistic geometry.  New physics capabilities may require new mathematical tools that can optimize XGC’s performance on the new exascale hardware/software architectures.  Uncertainty quantification of the exascale computing results and reduction to digital-twins/surrogate-models are another serious class of topics that XGC is facing.  In this discussion, we will emphasize the needs for advanced linear and UQ systems.  One advantageous UQ property of XGC is that, even though the simulations are expensive, the input dimensionality is low (only on the order of a dozen) and a single simulation can provide many data points since the background plasma itself is time-evolving.  Successful digital-twins and surrogate-models can be highly valuable in experimental design.