FASTMath AI/ML Capabilities

Researchers from across the FASTMath fields are working on all aspects of machine learning for DOE applications.  At a high level, the research performed falls into the five categories, which are listed below.  Several links to videos that outline the machine learning work done in FASTMath are provided.

1. Robust accelerated learning.  Data analytics, Solvers, Optimization, UQ, and Time integration methods in FASTMath are being employed to improve and scale learning methods.

   • Tiernan Casey, 'Manifold learning in dynamical systems', play movie

   • Pieter Ghysels, Alice Gatti, & Esmond Ng, 'Deep learning for graph partitioning and other graph problems in scientific computing', play movie

   • Sven Leyffer, Dominic Yang, & Prasanna Balaprakash, 'Nonlinear optimization over machine-learning surrogates', play movie

2. Predicting with confidence and complex predictions.  Advances in optimization and UQ are providing machine learning the tools to work with uncertainty and make complex predictions requiring optimization of multiple loss functions simultaneously. 

   • Ahmed M. Attia: 'Innovative ML/AI Perspectives for UQ, DA, & ODE ', play movie

3. Augmented and reinforced learning.  Data analytics and UQ are being employed to augment data when measurements are limited and suggest new measurements that will maximize understanding of predictive systems.  

   • Roger Ghanem: 'PLoM: Probabilistic learning of manifolds the small data challenge', play movie

4. Embedding physical understanding into predictions.  FASTMath has extensive experience with solving ODE/PDEs.  Researchers have been using experimental design and reduced order modeling techniques to infuse physical understanding into machine learning methods.   

   • Rick Archibald & Feng Bao, 'Robust and efficient algorithm for training stochastic neural networks', play movie

   • Hong Zhang, 'PNODE A memory-efficient neural ODE framework based on high-level adjoint differentiation', play movie

   • Hong Zhang, 'Stiffness-aware neural network for learning Hamiltonian systems', play movie

5. Data assimilation and model calibration.  Our understanding of PDEs and machine learning is being combined to accelerate assimilation of complex data and calibration of models in high dimensional parameter space.

   • Julie Bessac, Johann Rudi, & Amanda Lenzi, 'Parameter inference with neural networks', play movie

   • Emil M. Constantinescu, Julie Bessac, Ahmed Attia, Paul Kent, & Anouar Benali, 'Learning from different simulations: Bayesian Optimization', play movie

   • Emil M. Constantinescu, 'Data Fusion with Multimodal Data', play movie

   • Xiaoye S. Li & Yang Liu, 'GPTune: Autotuning HPC Codes Using Bayesian Optimization with Gaussian Process Surrogates', play movie

   • Juliane Mueller: 'HYPPO: DL model hyperparameter optimization with uncertainty', play movie

FASTMath researchers are using the tools and theory designed in their fields of research to bring unique understanding and fundamental developments to the five categories mentioned above.  FASTMath is designing software that allows these mathematical developments to be used at scale on leadership computing facilities.