All Issue

2025 Vol.47

Article

A New Paradigm of Artificial Neural Network Learning for Atmospheric and Oceanic Predictions: Physics-Informed Neural Networks and Operator Learning

대기 및 해양 예측을 위한 인공신경망 학습의 새로운 패러다임: 물리정보 신경망과 연산자 학습

Dong-Hoon Kim1
,
Il-Ju Moon1*
김 동훈1
,
문 일주1*
1Typhoon Research Center, Jeju National University, Jeju, 63243 Korea
1제주대학교 태풍연구센터 (63243) 제주특별자치도 제주시 제주대학교 102
*Corresponding Author
27 February 2025. pp. 1-15
PDF XML Citation
Abstract

This study explores the potential of Physics-Informed Neural Networks (PINNs) and Operator Learning (OL) techniques in advancing scientific computation and prediction. PINNs integrate physical principles into neural network training, enabling accurate modeling and prediction of complex physical systems, even in data-scarce and noisy environments. They have been successfully applied to solve challenging nonlinear partial differential equations (PDEs), such as Euler and Navier-Stokes equations, and continue to evolve with variants. Meanwhile, OL methods, represented by Deep Operator Network (DeepONet) and Fourier Neural Operator (FNO), focus on learning mappings between function spaces. These methods excel in high-dimensional data processing and have demonstrated remarkable performance in applications such as global atmospheric modeling with NVIDIA’s FourCastNet. Hybrid approaches, such as Physics-Informed DeepONet (PIDON) and Physics-Informed Neural Operator (PINO), combine the strengths of PINNs and OL. These methods leverage data-driven learning and physical constraints, achieving superior generalization and prediction accuracy. Notably, PINO enables zero-shot super-resolution predictions by integrating multi-resolution data with PDE constraints. While PINNs and OL individually present powerful tools for modeling and prediction, their computational cost and sensitivity to noise pose challenges. Hybrid approaches offer a pathway to address these issues by optimizing their integration through quantitative analyses. Future research directions include accelerating training through high-performance computing, extending applications to multiscale problems, and designing innovative loss functions to enhance data efficiency. This work synthesizes the latest advancements in PINNs, OL, and hybrid methods, providing a new paradigm for precise and efficient scientific computation across diverse fields.

Keywords
Scientific Machine Learning (SciML)
Physics-Informed Neural Networks (PINNs)
Operator Learning (OL)
Fourier Neural Operator (FNO)
Deep Operator Network (DeepONet)
References
1

Amini Niaki S, Haghighat E, Campbell T, Poursartip A, Vaziri R (2021) Physics-informed neural network for modelling the thermochemical curing process of composite-tool systems during manufacture. Comput Method Appl M 384:113959. doi:10.1016/j.cma.2021.113959

10.1016/j.cma.2021.113959
2

Baker N, Alexander F, Bremer T, Hagberg A, Kevrekidis Y, Najm H, Parashar M, Patra A, Sethian J, Wild S, Willcox K, Lee S (2019) Workshop report on basic research needs for scientific machine learning: core technologies for artificial intelligence. USDOE Office of Science (SC), Washington DC, 109 p

10.2172/1478744
3

Bonev B, Kurth T, Hundt C, Pathak J, Baust M, Kashinath K, Anandkumar A (2023) Spherical fourier neural operators: learning stable dynamics on the sphere. In: Proceedings of the 40th International Conference on Machine Learning, ICML 2023, Hawaii, 23-29 July 2023, pp 2806-2823

4

Cai S, Wang Z, Wang S, Perdikaris P, Karniadakis GE (2021) Physics-informed neural networks for heat transfer problems. J Heat Transf 143(6):060801. doi:10.1115/1.4050542

10.1115/1.4050542
5

Chen F, Chen F, Sondak D, Protopapas P, Mattheakis M, Liu S, Agarwal D, Di Giovanni M (2020) NeuroDiffEq: a python package for solving differential equations with neural networks. J Open Source Softw 5:1931. doi:10.21105/joss.01931

10.21105/joss.01931
6

Chen T, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw Learn Syst 6(4):911-917

10.1109/72.39225318263379
7

Cheng C, Zhang GT (2021) Deep learning method based on physics informed neural network with Resnet block for solving fluid flow problems. Water 13(4):423. doi:10.3390/w13040423

10.3390/w13040423
8

Choi BJ, Jin HS, Lkhagvasuren B (2024) Applications of the Fourier neural operator in a regional ocean modeling and prediction. Front Mar Sci 11:1383997. doi:10.3389/fmars.2024.1383997

10.3389/fmars.2024.1383997
9

Cuomo S, Di Cola VS, Giampaolo F, Rozza G, Raissi M, Piccialli F (2022) Scientific machine learning through physics-informed neural networks: Where we are and what's next. J Sci Comput 92(3):88. doi:10.1007/s10915-022-01939-z

10.1007/s10915-022-01939-z
10

Cybenko G (1989) Approximation by superpositions of a sigmoidal function. Math Control Signals Syst 2(4):303-314

10.1007/BF02551274
11

Dwivedi V, Srinivasan B (2020) Physics Informed Extreme Learning Machine (PIELM) - a rapid method for the numerical solution of partial differential equations. Neurocomputing 391:96-118. doi:10.1016/j.neucom.2019.12.099

10.1016/j.neucom.2019.12.099
12

Fang Z (2021) A high-efficient hybrid physics-informed neural networks based on convolutional neural network. IEEE Trans Neural Netw Learn Syst 33(10):5514-5526. doi:10.1109/TNNLS.2021.3070878

10.1109/TNNLS.2021.307087833848251
13

Gao H, Sun L, Wang JX (2021) PhyGeoNet: physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain. J Comput Phys 428:110079. doi:10.1016/j.jcp.2020.110079

10.1016/j.jcp.2020.110079
14

Gardner JR, Pleiss G, Bindel D, Weinberger KQ, Wilson AG (2018) Gpytorch: blackbox matrix-matrix Gaussian process inference with GPU acceleration. Adv Neural Inf Process Syst 31:7587-7597

15

Geneva N, Zabaras N (2020) Modeling the dynamics of pde systems with physics-constrained deep auto-regressive networks. J Comput Phys 403:109056. doi:10.1016/j.jcp.2019.109056

10.1016/j.jcp.2019.109056
16

Guo S (2023) Operator learning via physics-informed DeepONet: Let's implement it from scratch. blog post. https://towardsdatascience.com/operator-learning-via-physics-informed-deeponet-lets-implement-it-from-scratch-6659f3179887 Accessed 1 Dec 2024

17

Haghighat E, Juanes R (2021) SciANN: A Keras/TensorFlow wrapper for scientific computations and physics-informed deep learning using artificial neural networks. Comput Methods Appl Mech Eng 373:113552. doi:10.48550/arXiv.2005.08803

10.1016/j.cma.2020.113552
18

He Q, Barajas-Solano D, Tartakovsky G, Tartakovsky AM (2020) Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport. Adv Water Resour 141:103610. doi:10.1016/j.advwatres.2020.103610

10.1016/j.advwatres.2020.103610
19

Hennigh O, Narasimhan S, Nabian MA, Subramaniam A, Tangsali K, Rietmann M, Ferrandis JdA, Byeon W, Fang Z, Choudhry S (2020) NVIDIA SimNet: an AI-accelerated multi-physics simulation framework. Preprint at arXiv:2012.07938. doi:10.48550/arXiv.2012.07938

10.1007/978-3-030-77977-1_36
20

Islam M, Thakur MSH, Mojumder S, Hasan MN (2021) Extraction of material properties through multi-fidelity deep learning from molecular dynamics simulation. Comput Mater Sci 188:110187. doi:10.1016/j.commatsci.2020.110187

10.1016/j.commatsci.2020.110187
21

Jin X, Cai S, Li H, Karniadakis GE (2021) NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations. J Comput Phys 426:109951. doi:10.1016/j.jcp.2020.109951

10.1016/j.jcp.2020.109951
22

Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L (2021) Physics-informed machine learning. Nat Rev Phys 3(6):422-440

10.1038/s42254-021-00314-5
23

Koryagin A, Khudorozkov R, Tsimfer S (2019) PyDEns: a Python framework for solving differential equations with neural networks. Preprint at arXiv:1909.11544 doi:10.48550/arXiv.1909.11544

10.3997/2214-4609.202012125
24

Kovachki N, Li Z, Liu B, Azizzadenesheli K, Bhattacharya K, Stuart A, Anandkumar A (2023) Neural operator: learning maps between function spaces with applications to PDEs. J Mach Learn Res 24(89):1-97

25

Li Z, Kovachki N, Azizzadenesheli K, Liu B, Bhattacharya K, Stuart A, Anandkumar A (2021) Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895. doi:10.48550/arXiv.2010.08895

26

Li Z, Zheng H, Kovachki N, Jin D, Chen H, Liu B, Anandkumar A (2023) Physics-informed neural operator for learning partial differential equations. arXiv preprint arXiv:2111.03794. doi:10.1145/3648506

10.1145/3648506
27

Lu L, Jin P, Pang G, Zhang Z, Karniadakis GE (2021a) Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat Mach Intell 3:218-229. doi:10.1038/s42256-021-00302-5

10.1038/s42256-021-00302-5
28

Lu L, Meng X, Cai S, Mao Z, Goswami S, Zhang Z, Karniadakis GE (2022) A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Comput Methods Appl Mech Engrg 393:114778

10.1016/j.cma.2022.114778
29

Lu L, Meng X, Mao Z, Karniadakis GE (2021b) DeepXDE: a deep learning library for solving differential equations. SIAM Rev 63:208-228

10.1137/19M1274067
30

Mao Z, Jagtap AD, Karniadakis GE (2020) Physics-informed neural networks for high-speed flows. Comput Methods Appl Mech Engrg 360:112789. doi:10.1016/j.cma.2019.112789

10.1016/j.cma.2019.112789
31

Moseley B (2021) So, what is a physics-informed neural network?. Blog post. https://benmoseley.blog/my-research/so-what-is-a-physics-informed-neural-network/ Accessed 1 Dec 2024

32

Novak R, Xiao L, Hron J, Lee J, Alemi AA, Sohl-Dickstein J, Schoenholz SS (2019) Neural tangents: fast and easy infinite neural networks in python. arXiv preprint arXiv:1912.02803. doi:doi.org/10.48550/arXiv.1912.02803

33

Pathak J, Subramanian S, Harrington P, Raja S, Chattopadhyay A, Mardani M, Kurth T, Hall D, Li Z, Azizzadenesheli K, Hassanzadeh P, Kashinath K, Anandkumar A (2022) Fourcastnet: a global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.11214. doi:10.48550/arXiv.2202.11214

34

Rackauckas C, Nie Q (2017) DifferentialEquations.jl - a performant and feature - rich ecosystem for solving differential equations in Julia. J Open Res Softw 5:15. doi:10.5334/jors.151

10.5334/jors.151
35

Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput phys 378:686-707

10.1016/j.jcp.2018.10.045
36

Ramabathiran AA, Ramachandran P (2021) SPINN: sparse, Physics-based, and partially Interpretable Neural Networks for PDEs. J Comput Phys 445:110600. doi:10.1016/j.jcp.2021.110600

10.1016/j.jcp.2021.110600
37

Schiassi E, Furfaro R, Leake C, Florio MD, Johnston H, Mortari D (2021) Extreme theory of functional connections: a fast physics- informed neural network method for solving ordinary and partial differential equations. Neurocomputing 457:334-356. doi:10.1016/j.neucom.2021.06.015

10.1016/j.neucom.2021.06.015
38

Serrano L, Le Boudec L, Kassaï Koupaï A, Wang TX, Yin Y, Vittaut JN, Gallinari P (2023) Operator learning with neural fields: tackling PDEs on general geometries. Adv Neural Inf Process Syst 36:70581-70611

39

Tartakovsky AM, Marrero CO, Perdikaris P, Tartakovsky GD, Barajas-Solano D (2020) Physics-informed deep neural networks for learning parameters and constitutive relationships in subsurface flow problems. Water Resour Res 56(5):e2019WR026731. doi:10.1029/2019WR026731

10.1029/2019WR026731
40

Viana FAC, Nascimento RG, Dourado A, Yucesan YA (2021) Estimating model inadequacy in ordinary differential equations with physics-informed neural networks. Comput Struct 245:106458. doi:10.1016/j.compstruc.2020.106458

10.1016/j.compstruc.2020.106458
41

Waheed UB, Haghighat E, Alkhalifah T, Song C, Hao Q (2021) PINNeik: eikonal solution using physics-informed neural networks. Comput Geosci 155:104833. doi:10.1016/j.cageo.2021.104833

10.1016/j.cageo.2021.104833
42

Wang N, Chang H, Zhang D (2021a) Theory-guided auto-encoder for surrogate construction and inverse modeling. Comput Methods Appl Mech Engrg 385:114037. doi:10.1016/j.cma.2021.114037

10.1016/j.cma.2021.114037
43

Wang S, Wang H, Perdikaris P (2021b) Learning the solution operator of parametric partial differential equations with physics-informed DeepONets. Science advances 7(40):eabi8605

10.1126/sciadv.abi860534586842PMC8480920
44

Wilkinson MD, Dumontier M, Aalbersberg IJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten J-W, Santos LBdS, Bourne PE (2016) The FAIR guiding principles for scientific data management and stewardship. Scientific Data 3(1):1-9

10.1038/sdata.2016.1826978244
45

Xu K, Darve E (2020) ADCME: Learning spatially-varying physical fields using deep neural networks. Preprint at arXiv:2011.11955. doi:10.48550/arXiv.2011.11955

46

Yang L, Meng X, Karniadakis GE (2021) B-PINNs: Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data. J Comput Phys 425:109913. doi:10.1016/j.jcp.2020.109913

10.1016/j.jcp.2020.109913
47

Yang L, Zhang D, Karniadakis GE (2020) Physics-informed generative adversarial networks for stochastic differential equations. SIAM J Sci Comput 42(1):A292-A317. doi:10.1137/18M1225409

10.1137/18M1225409
48

Yucesan YA, Viana FAC (2021) Hybrid physics-informed neural networks for main bearing fatigue prognosis with visual grease inspection. Comput Ind 125:103386. doi:10.1016/j.compind.2020.103386

10.1016/j.compind.2020.103386
49

Zhang R, Liu Y, Sun H (2020) Physics-informed multi-LSTM networks for metamodeling of nonlinear structures. Comput Methods Appl Mech Engrg 369:113226. doi:10.1016/j.cma.2020.113226

10.1016/j.cma.2020.113226
50

Zhu Q, Liu Z, Yan J (2021) Machine learning for metal additive manufacturing: predicting temperature and melt pool fluid dynamics using physics-informed neural networks. Comput Mech 67(2):619-635. doi:10.1007/s00466-020-01952-9

10.1007/s00466-020-01952-9
51

Zhu Y, Zabaras N, Koutsourelakis PS, Perdikaris P (2019) Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. J Comput Phys 394:56-81. doi:10.1016/j.jcp.2019.05.024

10.1016/j.jcp.2019.05.024
Information
  • Publisher :Korea Institute of Ocean Science and Technology
  • Publisher(Ko) :한국해양과학기술원
  • Journal Title :Ocean and Polar Research
  • Journal Title(Ko) :Ocean and Polar Research
  • Volume : 47
  • Pages :1-15
  • Received Date : 2025-01-02
  • Revised Date : 2025-02-02
  • Accepted Date : 2025-02-04
  • Published Date : 2025-02-27
  • DOI :https://doi.org/10.4217/OPR.2025001
Journal Informaiton Ocean and Polar Research Ocean and Polar Research
Online Submission
Archives
: 2019 ~ 이전
  • scopus_OPR
  • NRF
  • KISTI Current Status
  • KISTI Cited-by
  • KCI 문헌 유사도 검사
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close