Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Promising directions of machine learning for partial differential equations

Nature Computational Science (2024)Cite this article

Subjects

Abstract

Partial differential equations (PDEs) are among the most universal and parsimonious descriptions of natural physical laws, capturing a rich variety of phenomenology and multiscale physics in a compact and symbolic representation. Here, we examine several promising avenues of PDE research that are being advanced by machine learning, including (1) discovering new governing PDEs and coarse-grained approximations for complex natural and engineered systems, (2) learning effective coordinate systems and reduced-order models to make PDEs more amenable to analysis, and (3) representing solution operators and improving traditional numerical algorithms. In each of these fields, we summarize key advances, ongoing challenges, and opportunities for further development.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of three areas where machine learning is advancing PDE research.
Fig. 2: Sparse regression procedure to discover PDEs from data, demonstrated on the incompressible Navier–Stokes equations.
Fig. 3: Illustration of sparse Bayesian PDE discovery applied to LES closure modeling in large-scale geophysical fluid simulations.
Fig. 4: Schematic of a neural network architecture used to discover a Koopman linearizing coordinate transformation.
Fig. 5: Machine-learned interpolation from coarse-grained to high-resolution flow fields.

Similar content being viewed by others

References

  1. Brezis, H. & Browder, F. Partial differential equations in the 20th century. Adv. Math. 135, 76–144 (1998).

    Article  MathSciNet  Google Scholar 

  2. Dissanayake, M. & Phan-Thien, N. Neural-network-based approximations for solving partial differential equations. Commun. Numer. Methods Eng. 10, 195–201 (1994).

    Article  Google Scholar 

  3. Rico-Martinez, R. & Kevrekidis, I. G. Continuous time modeling of nonlinear systems: a neural network-based approach. In Proc. IEEE International Conference on Neural Networks 1522–1525 (IEEE, 1993).

  4. González-García, R., Rico-Martìnez, R. & Kevrekidis, I. G. Identification of distributed parameter systems: a neural net based approach. Comput. Chem. Eng. 22, S965–S968 (1998).

    Article  Google Scholar 

  5. Raissi, M., Perdikaris, P. & Karniadakis, G. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019).

    Article  MathSciNet  Google Scholar 

  6. Yu, B. et al. The Deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6, 1–12 (2018).

    Article  MathSciNet  Google Scholar 

  7. Müller, J. & Zeinhofer, M. Deep Ritz revisited. Preprint at https://arxiv.org/abs/1912.03937 (2019).

  8. Gao, H., Zahr, M. J. & Wang, J.-X. Physics-informed graph neural Galerkin networks: a unified framework for solving PDE-governed forward and inverse problems. Comput. Methods Appl. Mech. Eng. 390, 114502 (2022).

    Article  MathSciNet  Google Scholar 

  9. Bruna, J., Peherstorfer, B. & Vanden-Eijnden, E. Neural Galerkin schemes with active learning for high-dimensional evolution equations. J. Comput. Phys. 496, 112588 (2024).

    Article  MathSciNet  Google Scholar 

  10. Battaglia, P. W. et al. Relational inductive biases, deep learning and graph networks. Preprint at https://arxiv.org/abs/1806.01261 (2018).

  11. Sanchez-Gonzalez, A. et al. Learning to simulate complex physics with graph networks. In Proc. International Conference on Machine Learning 8459–8468 (PMLR, 2020).

  12. Burger, M. et al. Connections between deep learning and partial differential equations. Eur. J. Appl. Math. 32, 395–396 (2021).

    Article  Google Scholar 

  13. Loiseau, J.-C. & Brunton, S. L. Constrained sparse Galerkin regression. J. Fluid Mech. 838, 42–67 (2018).

    Article  MathSciNet  Google Scholar 

  14. Cranmer, M. et al. Lagrangian neural networks. Preprint at https://arxiv.org/abs/2003.04630 (2020).

  15. Brunton, S. L., Noack, B. R. & Koumoutsakos, P. Machine learning for fluid mechanics. Annu. Rev. Fluid Mech. 52, 477–508 (2020).

    Article  MathSciNet  Google Scholar 

  16. Wang, R., Walters, R. & Yu, R. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference on Learning Representations (ICLR, 2021).

  17. Wang, R., Kashinath, K., Mustafa, M., Albert, A. & Yu, R. Towards physics-informed deep learning for turbulent flow prediction. In Proc. 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 1457–1466 (ACM, 2020).

  18. Brandstetter, J., Berg, R. V. D., Welling, M. & Gupta, J. K. Clifford neural layers for PDE modeling. In Eleventh International Conference on Learning Representations (ICLR, 2023)

  19. De Haan, P., Weiler, M., Cohen, T. & Welling, M. Gauge equivariant mesh CNNS: anisotropic convolutions on geometric graphs. In International Conference on Learning Representations (ICLR, 2021).

  20. Brandstetter, J., Welling, M. & Worrall, D. E. Lie point symmetry data augmentation for neural PDE solvers. In Proc. International Conference on Machine Learning 2241–2256 (PMLR, 2022).

  21. Brandstetter, J., Worrall, D. & Welling, M. Message passing neural PDE solvers. Preprint at https://arxiv.org/abs/2202.03376 (2022).

  22. Karniadakis, G. E. et al. Physics-informed machine learning. Nat. Rev. Phys. 3, 422–440 (2021).

    Article  Google Scholar 

  23. Brunton, S. L. & Kutz, J. N. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems and Control 2nd edn (Cambridge Univ. Press, 2022).

  24. Bongard, J. & Lipson, H. Automated reverse engineering of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 104, 9943–9948 (2007).

    Article  Google Scholar 

  25. Schmidt, M. & Lipson, H. Distilling free-form natural laws from experimental data. Science 324, 81–85 (2009).

    Article  Google Scholar 

  26. Brunton, S. L., Proctor, J. L. & Kutz, J. N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 113, 3932–3937 (2016).

    Article  MathSciNet  Google Scholar 

  27. Cranmer, M. Interpretable machine learning for science with PySR and SymbolicRegression.jl. Preprint at https://arxiv.org/abs/2305.01582 (2023).

  28. Rudy, S. H., Brunton, S. L., Proctor, J. L. & Kutz, J. N. Data-driven discovery of partial differential equations. Sci. Adv 3, e1602614 (2017).

    Article  Google Scholar 

  29. Schaeffer, H. Learning partial differential equations via data discovery and sparse optimization. Proc. Math. Phys. Eng. Sci. 473, 20160446 (2017).

    MathSciNet  Google Scholar 

  30. Zanna, L. & Bolton, T. Data-driven equation discovery of ocean mesoscale closures. Geophys. Res. Lett. 47, e2020GL088376 (2020).

    Article  Google Scholar 

  31. Schmelzer, M., Dwight, R. P. & Cinnella, P. Discovery of algebraic Reynolds-stress models using sparse symbolic regression. Flow Turbulence Combustion 104, 579–603 (2020).

    Article  Google Scholar 

  32. Beetham, S. & Capecelatro, J. Formulating turbulence closures using sparse regression with embedded form invariance. Phys. Rev. Fluids 5, 084611 (2020).

    Article  Google Scholar 

  33. Beetham, S., Fox, R. O. & Capecelatro, J. Sparse identification of multiphase turbulence closures for coupled fluid-particle flows. J. Fluid Mech. 914, A11 (2021).

    Article  MathSciNet  Google Scholar 

  34. Bakarji, J. & Tartakovsky, D. M. Data-driven discovery of coarse-grained equations. J. Comput. Phys. 434, 110219 (2021).

    Article  MathSciNet  Google Scholar 

  35. Maslyaev, M., Hvatov, A. & Kalyuzhnaya, A. Data-driven partial derivative equations discovery with evolutionary approach. In Proc. Computational Science–ICCS 2019: 19th International Conference Part V 19, 635–641 (Springer, 2019).

  36. Xu, H., Zhang, D. & Wang, N. Deep-learning based discovery of partial differential equations in integral form from sparse and noisy data. J. Comput. Phys. 445, 110592 (2021).

    Article  MathSciNet  Google Scholar 

  37. Xu, H., Chang, H. & Zhang, D. DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm. J. Comput. Phys. 418, 109584 (2020).

    Article  MathSciNet  Google Scholar 

  38. Xu, H., Zhang, D. & Zeng, J. Deep-learning of parametric partial differential equations from sparse and noisy data. Phys. Fluids 33, 037132 (2021).

    Article  Google Scholar 

  39. Xu, H. & Zhang, D. Robust discovery of partial differential equations in complex situations. Phys. Rev. Res. 3, 033270 (2021).

    Article  Google Scholar 

  40. Chen, Y., Luo, Y., Liu, Q., Xu, H. & Zhang, D. Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE). Phys. Rev. Res. 4, 023174 (2022).

    Article  Google Scholar 

  41. Taira, K. & Colonius, T. The immersed boundary method: a projection approach. J. Comput. Phys. 225, 2118–2137 (2007).

    Article  MathSciNet  Google Scholar 

  42. Colonius, T. & Taira, K. A fast immersed boundary method using a nullspace approach and multi-domain far-field boundary conditions. Comput. Methods Appl. Mech. Eng. 197, 2131–2146 (2008).

    Article  MathSciNet  Google Scholar 

  43. Van Breugel, F., Kutz, J. N. & Brunton, B. W. Numerical differentiation of noisy data: a unifying multi-objective optimization framework. IEEE Access 8, 196865–196877 (2020).

    Article  Google Scholar 

  44. Messenger, D. A. & Bortz, D. M. Weak SINDy: Galerkin-based data-driven model selection. Multiscale Model. Simul. 19, 1474–1497 (2021).

    Article  MathSciNet  Google Scholar 

  45. Messenger, D. A. & Bortz, D. M. Weak SINDy for partial differential equations. J. Comput. Phys. 443, 110525 (2021).

    Article  MathSciNet  Google Scholar 

  46. Schaeffer, H. & McCalla, S. G. Sparse model selection via integral terms. Phys. Rev. E 96, 023302 (2017).

    Article  MathSciNet  Google Scholar 

  47. Fasel, U., Kutz, J. N., Brunton, B. W. & Brunton, S. L. Ensemble-SINDy: robust sparse model discovery in the low-data, high-noise limit, with active learning and control. Proc. R. Soc. A 478, 20210904 (2022).

    Article  MathSciNet  Google Scholar 

  48. Gurevich, D. R., Reinbold, P. A. & Grigoriev, R. O. Robust and optimal sparse regression for nonlinear PDE models. Chaos 29, 103113 (2019).

    Article  MathSciNet  Google Scholar 

  49. Alves, E. P. & Fiuza, F. Data-driven discovery of reduced plasma physics models from fully kinetic simulations. Phys. Rev. Res. 4, 033192 (2022).

    Article  Google Scholar 

  50. Reinbold, P. A., Gurevich, D. R. & Grigoriev, R. O. Using noisy or incomplete data to discover models of spatiotemporal dynamics. Phys. Rev. E 101, 010203 (2020).

    Article  Google Scholar 

  51. Suri, B., Kageorge, L., Grigoriev, R. O. & Schatz, M. F. Capturing turbulent dynamics and statistics in experiments with unstable periodic orbits. Phys. Rev. Lett. 125, 064501 (2020).

    Article  Google Scholar 

  52. Reinbold, P. A., Kageorge, L. M., Schatz, M. F. & Grigoriev, R. O. Robust learning from noisy, incomplete, high-dimensional experimental data via physically constrained symbolic regression. Nat. Commun. 12, 3219 (2021).

    Article  Google Scholar 

  53. Pope, S. A more general effective-viscosity hypothesis. J. Fluid Mech. 72, 331–340 (1975).

    Article  Google Scholar 

  54. Ling, J., Kurzawski, A. & Templeton, J. Reynolds averaged turbulence modelling using deep neural networks with embedded invariance. J. Fluid Mech. 807, 155–166 (2016).

    Article  MathSciNet  Google Scholar 

  55. Duraisamy, K., Iaccarino, G. & Xiao, H. Turbulence modeling in the age of data. Annu. Rev. Fluid Mech. 51, 357–377 (2019).

    Article  MathSciNet  Google Scholar 

  56. Ahmed, S. E. et al. On closures for reduced order models—a spectrum of first-principle to machine-learned avenues. Phys. Fluids 33, 091301 (2021).

    Article  Google Scholar 

  57. Supekar, R. et al. Learning hydrodynamic equations for active matter from particle simulations and experiments. Proc. Natl Acad. Sci. USA 120, e2206994120 (2023).

    Article  MathSciNet  Google Scholar 

  58. Kaptanoglu, A. A. et al. PySINDy: a comprehensive Python package for robust sparse system identification. J. Open Source Softw. 7, 3994 (2022).

    Article  Google Scholar 

  59. Long, Z., Lu, Y., Ma, X. & Dong, B. PDE-Net: learning PDEs from data. In Proc. International Conference on Machine Learning 3208–3216 (PMLR, 2018).

  60. Long, Z., Lu, Y. & Dong, B. PDE-Net 2.0: learning PDEs from data with a numeric-symbolic hybrid deep network. J. Comput. Phys. 399, 108925 (2019).

    Article  MathSciNet  Google Scholar 

  61. Atkinson, S. Bayesian hidden physics models: uncertainty quantification for discovery of nonlinear partial differential operators from data. Preprint at https://arxiv.org/abs/2006.04228 (2020).

  62. Cai, J.-F., Dong, B., Osher, S. & Shen, Z. Image restoration: total variation, wavelet frames and beyond. J. Am. Math. Soc. 25, 1033–1089 (2012).

    Article  MathSciNet  Google Scholar 

  63. Dong, B., Jiang, Q. & Shen, Z. Image restoration: wavelet frame shrinkage, nonlinear evolution PDEs and beyond. Multiscale Model. Simul. 15, 606–660 (2017).

    Article  MathSciNet  Google Scholar 

  64. Schaeffer, H., Caflisch, R., Hauck, C. D. & Osher, S. Sparse dynamics for partial differential equations. Proc. Natl Acad. Sci. USA 110, 6634–6639 (2013).

    Article  MathSciNet  Google Scholar 

  65. Cranmer, M. D., Xu, R., Battaglia, P. & Ho, S. Learning symbolic physics with graph networks. Preprint at https://arxiv.org/abs/1909.05862 (2019).

  66. Cranmer, M. et al. Discovering symbolic models from deep learning with inductive biases. In Advances in Neural Information Processing Systems (eds Larochelle, H. et al.) 17429–17442 (Curran Associates, Inc., 2020).

  67. Callaham, J. L., Koch, J. V., Brunton, B. W., Kutz, J. N. & Brunton, S. L. Learning dominant physical processes with data-driven balance models. Nat. Commun. 12, 1016 (2021).

    Article  Google Scholar 

  68. Cross, M. C. & Hohenberg, P. C. Pattern formation outside of equilibrium. Rev. Mod. Phys. 65, 851–1112 (1993).

    Article  Google Scholar 

  69. Bakarji, J., Callaham, J., Brunton, S. L. & Kutz, J. N. Dimensionally consistent learning with Buckingham Pi. Nat. Comput. Sci. 2, 834–844 (2022).

    Article  Google Scholar 

  70. Koopman, B. O. Hamiltonian systems and transformation in Hilbert space. Proc. Natl Acad. Sci. USA 17, 315–318 (1931).

    Article  Google Scholar 

  71. Mezić, I. & Banaszuk, A. Comparison of systems with complex behavior. Phys. D Nonlinear Phenomena 197, 101–133 (2004).

    Article  MathSciNet  Google Scholar 

  72. Mezić, I. Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dyn. 41, 309–325 (2005).

    Article  MathSciNet  Google Scholar 

  73. Mezić, I. Analysis of fluid flows via spectral properties of the Koopman operator. Ann. Rev. Fluid Mech. 45, 357–378 (2013).

    Article  MathSciNet  Google Scholar 

  74. Rowley, C. W., Mezić, I., Bagheri, S., Schlatter, P. & Henningson, D. Spectral analysis of nonlinear flows. J. Fluid Mech. 645, 115–127 (2009).

    Article  MathSciNet  Google Scholar 

  75. Kutz, J. N., Brunton, S. L., Brunton, B. W. & Proctor, J. L. Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems (SIAM, 2016).

  76. Budišić, M., Mohr, R. & Mezić, I. Applied Koomanism. Chaos 22, 047510 (2012).

    Article  MathSciNet  Google Scholar 

  77. Brunton, S. L., Budišić, M., Kaiser, E. & Kutz, J. N. Modern Koopman theory for dynamical systems. SIAM Rev. 64, 229–340 (2022).

    Article  MathSciNet  Google Scholar 

  78. Cover, T. M. Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans. Electron Comput. EC-14, 326–334 (1965).

    Article  Google Scholar 

  79. Hopf, E. The partial differential equation ut + uux = μuxx. Commun. Pure Appl. Math. 3, 201–230 (1950).

    Article  Google Scholar 

  80. Cole, J. D. On a quasi-linear parabolic equation occurring in aerodynamics. Q. Appl. Math. 9, 225–236 (1951).

    Article  MathSciNet  Google Scholar 

  81. Ablowitz, M. J., Kaup, D. J., Newell, A. C. & Segur, H. The inverse scattering transform-fourier analysis for nonlinear problems. Stud. Appl. Math. 53, 249–315 (1974).

    Article  MathSciNet  Google Scholar 

  82. Ablowitz, M. J. & Segur, H. Solitons and the Inverse Scattering Transform (SIAM, 1981).

  83. Lusch, B., Kutz, J. N. & Brunton, S. L. Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun. 9, 4950 (2018).

    Article  Google Scholar 

  84. Wehmeyer, C. & Noé, F. Time-lagged autoencoders: deep learning of slow collective variables for molecular kinetics. J. Chem. Phys 148, 241703 (2018).

    Article  Google Scholar 

  85. Mardt, A., Pasquali, L., Wu, H. & Noé, F. VAMPnets: deep learning of molecular kinetics. Nat. Commun. 9, 5 (2018).

    Article  Google Scholar 

  86. Takeishi, N., Kawahara, Y. & Yairi, T. Learning Koopman invariant subspaces for dynamic mode decomposition. In Advances in Neural Information Processing Systems 1130–1140 (NIPS, 2017).

  87. Yeung, E., Kundu, S. & Hodas, N. Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. Preprint at https://arxiv.org/abs/1708.06850 (2017).

  88. Otto, S. E. & Rowley, C. W. Linearly recurrent autoencoder networks for learning dynamics. SIAM J. Appl. Dyn. Syst. 18, 558–593 (2019).

    Article  MathSciNet  Google Scholar 

  89. Li, Q., Dietrich, F., Bollt, E. M. & Kevrekidis, I. G. Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator. Chaos 27, 103111 (2017).

    Article  MathSciNet  Google Scholar 

  90. Eivazi, H., Guastoni, L., Schlatter, P., Azizpour, H. & Vinuesa, R. Recurrent neural networks and Koopman-based frameworks for temporal predictions in a low-order model of turbulence. Int. J. Heat Fluid Flow 90, 108816 (2021).

    Article  Google Scholar 

  91. Gin, C., Lusch, B., Brunton, S. L. & Kutz, J. N. Deep learning models for global coordinate transformations that linearise PDEs. Eur. J. Appl. Math. 32, 515–539 (2021).

    Article  MathSciNet  Google Scholar 

  92. Noé, F. & Nuske, F. A variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Model. Simul. 11, 635–655 (2013).

    Article  MathSciNet  Google Scholar 

  93. Nüske, F., Keller, B. G., Pérez-Hernández, G., Mey, A. S. & Noé, F. Variational approach to molecular kinetics. J.Chem. Theory Comput. 10, 1739–1752 (2014).

    Article  Google Scholar 

  94. Williams, M. O., Kevrekidis, I. G. & Rowley, C. W. A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J. Nonlinear Sci. 6, 1307–1346 (2015).

    Article  MathSciNet  Google Scholar 

  95. Williams, M. O., Rowley, C. W. & Kevrekidis, I. G. A kernel approach to data-driven Koopman spectral analysis. J. Comput. Dyn. 2, 247–265 (2015).

    Google Scholar 

  96. Klus, S. Data-driven model reduction and transfer operator approximation. J. Nonlinear Sci. 28, 985–1010 (2018).

    Article  MathSciNet  Google Scholar 

  97. Kutz, J. N., Proctor, J. L. & Brunton, S. L. Applied Koopman theory for partial differential equations and data-driven modeling of spatio-temporal systems. Complexity 2018, 6010634 (2018).

    Google Scholar 

  98. Page, J. & Kerswell, R. R. Koopman analysis of burgers equation. Phys. Rev. Fluids 3, 071901 (2018).

    Article  Google Scholar 

  99. Lu, L., Jin, P. & Karniadakis, G. E. DeepONet: learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. Preprint at https://arxiv.org/abs/1910.03193 (2019).

  100. Lu, L., Jin, P., Pang, G., Zhang, Z. & Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat. Mach. Intell. 3, 218–229 (2021).

    Article  Google Scholar 

  101. Kovachki, N. et al. Neural operator: learning maps between function spaces with applications to PDEs. J. Mach. Learn. Res. 24, 1–97 (2023).

    MathSciNet  Google Scholar 

  102. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016).

  103. Colbrook, M. J., Ayton, L. J. & Szőke, M. Residual dynamic mode decomposition: robust and verified Koopmanism. J. Fluid Mech. 955, A21 (2023).

    Article  MathSciNet  Google Scholar 

  104. Colbrook, M. J. & Townsend, A. Rigorous data-driven computation of spectral properties of Koopman operators for dynamical systems. Commun. Pure Appl. Math. 77, 221–283 (2024).

    Article  MathSciNet  Google Scholar 

  105. Lumley, J. Toward a turbulent constitutive relation. J. Fluid Mech. 41, 413–434 (1970).

    Article  Google Scholar 

  106. Berkooz, G., Holmes, P. & Lumley, J. The proper orthogonal decomposition in the analysis of turbulent flows. Annu. Rev. Fluid Mech. 25, 539–575 (1993).

    Article  MathSciNet  Google Scholar 

  107. Holmes, P., Lumley, J. L., Berkooz, G. & Rowley, C. W. Turbulence, Coherent Structures, Dynamical Systems and Symmetry 2nd edn (Cambridge Univ. Press, 2012).

  108. Taira, K. et al. Modal analysis of fluid flows: an overview. AIAA J. 55, 4013–4041 (2017).

    Article  Google Scholar 

  109. Taira, K. et al. Modal analysis of fluid flows: applications and outlook. AIAA J. 58, 998–1022 (2020).

    Article  Google Scholar 

  110. Benner, P., Gugercin, S. & Willcox, K. A survey of projection-based model reduction methods for parametric dynamical systems. SIAM Rev. 57, 483–531 (2015).

    Article  MathSciNet  Google Scholar 

  111. Qian, E., Kramer, B., Peherstorfer, B. & Willcox, K. Lift & Learn: physics-informed machine learning for large-scale nonlinear dynamical systems. Phys. D Nonlinear Phenom. 406, 132401 (2020).

    Article  MathSciNet  Google Scholar 

  112. Peherstorfer, B. & Willcox, K. Data-driven operator inference for nonintrusive projection-based model reduction. Comput. Methods Appl. Mech. Eng. 306, 196–215 (2016).

    Article  MathSciNet  Google Scholar 

  113. Benner, P., Goyal, P., Kramer, B., Peherstorfer, B. & Willcox, K. Operator inference for non-intrusive model reduction of systems with non-polynomial nonlinear terms. Comput. Methods Appl. Mech. Eng. 372, 113433 (2020).

    Article  MathSciNet  Google Scholar 

  114. Peherstorfer, B., Drmac, Z. & Gugercin, S. Stability of discrete empirical interpolation and gappy proper orthogonal decomposition with randomized and deterministic sampling points. SIAM J. Sci. Comput. 42, A2837–A2864 (2020).

    Article  MathSciNet  Google Scholar 

  115. Holmes, P. & Guckenheimer, J. (eds) Nonlinear Oscillations, Dynamical Systems and Bifurcations of Vector Fields Vol. 42 (Springer, 1983).

  116. Rowley, C. W. & Marsden, J. E. Reconstruction equations and the Karhunen-Loève expansion for systems with symmetry. Phys. D Nonlinear Phenom. 142, 1–19 (2000).

    Article  Google Scholar 

  117. Abraham, R., Marsden, J. E. & Ratiu, T. Manifolds, Tensor Analysis and Applications Vol. 75 (Springer, 1988).

  118. Marsden, J. E. & Ratiu, T. S. Introduction to Mechanics and Symmetry 2nd edn (Springer, 1999).

  119. Lee, K. & Carlberg, K. T. Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. J. Comput. Phys. 404, 108973 (2020).

    Article  MathSciNet  Google Scholar 

  120. Schmid, P. J. Dynamic mode decomposition for numerical and experimental data. J. Fluid Mech. 656, 5–28 (2010).

    Article  MathSciNet  Google Scholar 

  121. Loiseau, J.-C., Noack, B. R. & Brunton, S. L. Sparse reduced-order modeling: sensor-based dynamics to full-state estimation. J. Fluid Mech. 844, 459–490 (2018).

    Article  Google Scholar 

  122. Deng, N., Noack, B. R., Morzynski, M. & Pastur, L. R. Low-order model for successive bifurcations of the fluidic pinball. J. Fluid Mech. 884, A37 (2020).

    Article  MathSciNet  Google Scholar 

  123. Loiseau, J.-C. Data-driven modeling of the chaotic thermal convection in an annular thermosyphon.Theor. Comput. Fluid Dyn 34, 339–365 (2020).

    Article  MathSciNet  Google Scholar 

  124. Guan, Y., Brunton, S. L. & Novosselov, I. Sparse nonlinear models of chaotic electroconvection. R. Soc. Open Sci. 8, 202367 (2021).

    Article  Google Scholar 

  125. Kaptanoglu, A. A., Morgan, K. D., Hansen, C. J. & Brunton, S. L. Physics-constrained, low-dimensional models for magnetohydrodynamics: first-principles and data-driven approaches. Phys. Rev. E 104, 015206 (2021).

    Article  MathSciNet  Google Scholar 

  126. Kaptanoglu, A. A., Callaham, J. L., Aravkin, A., Hansen, C. J. & Brunton, S. L. Promoting global stability in data-driven models of quadratic nonlinear dynamics. Phys. Rev. Fluids 6, 094401 (2021).

    Article  Google Scholar 

  127. Deng, N., Noack, B. R., Morzyński, M. & Pastur, L. R. Galerkin force model for transient and post-transient dynamics of the fluidic pinball. J. Fluid Mech. 918, A4 (2021).

    Article  MathSciNet  Google Scholar 

  128. Callaham, J. L., Loiseau, J.-C., Rigas, G. & Brunton, S. L. Nonlinear stochastic modelling with Langevin regression. Proc. R. Soc. A 477, 20210092 (2021).

    Article  MathSciNet  Google Scholar 

  129. Callaham, J. L., Brunton, S. L. & Loiseau, J.-C. On the role of nonlinear correlations in reduced-order modeling. J. Fluid Mech. 938, A1 (2022).

    Article  Google Scholar 

  130. Callaham, J. L., Rigas, G., Loiseau, J.-C. & Brunton, S. L. An empirical mean-field model of symmetry-breaking in a turbulent wake. Sci. Adv. 8, eabm4786 (2022).

    Article  Google Scholar 

  131. Pathak, J., Lu, Z., Hunt, B. R., Girvan, M. & Ott, E. Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data. Chaos 27, 121102 (2017).

    Article  MathSciNet  Google Scholar 

  132. Pathak, J., Hunt, B., Girvan, M., Lu, Z. & Ott, E. Model-free prediction of large spatiotemporally chaotic systems from data: a reservoir computing approach. Phys. Rev. Lett. 120, 024102 (2018).

    Article  Google Scholar 

  133. Champion, K., Lusch, B., Kutz, J. N. & Brunton, S. L. Data-driven discovery of coordinates and governing equations. Proc. Natl Acad. Sci. USA 116, 22445–22451 (2019).

    Article  MathSciNet  Google Scholar 

  134. Baddoo, P. J., Herrmann, B., McKeon, B. J. & Brunton, S. L. Kernel learning for robust dynamic mode decomposition: linear and nonlinear disambiguation optimization (LANDO). Proc. R. Soc. A 478, 20210830 (2022).

    Article  Google Scholar 

  135. Reed, M. & Simon, B. Methods of Modern Mathematical Physics. I 2nd edn (Academic, 1980).

  136. Courant, R. & Hilbert, D. Methods of Mathematical Physics: Partial Differential Equations (Wiley, 2008).

  137. Li, Z. et al. Neural operator: graph kernel network for partial differential equations. Preprint at https://arxiv.org/abs/2003.03485 (2020).

  138. Li, Z. et al. Multipole graph neural operator for parametric partial differential equations. Preprint at https://arxiv.org/abs/2006.09535 (2020).

  139. Li, Z. et al. Fourier neural operator for parametric partial differential equations. Preprint at https://arxiv.org/abs/2010.08895 (2020).

  140. Gin, C. R., Shea, D. E., Brunton, S. L. & Kutz, J. N. DeepGreen: deep learning of Green’s functions for nonlinear boundary value problems. Sci. Rep. 11, 21614 (2021).

    Article  Google Scholar 

  141. de Hoop, M. V., Kovachki, N. B., Nelsen, N. H. & Stuart, A. M. Convergence rates for learning linear operators from noisy data. SIAM-ASA J. Uncertain. Quantif. 11, 480–513 (2023).

    Article  MathSciNet  Google Scholar 

  142. De Hoop, M., Huang, D. Z., Qian, E. & Stuart, A. M. The cost-accuracy trade-off in operator learning with neural networks. Preprint at https://arxiv.org/abs/2203.13181 (2022).

  143. Mollenhauer, M., Mücke, N. & Sullivan, T. Learning linear operators: infinite-dimensional regression as a well-behaved non-compact inverse problem. Preprint at https://arxiv.org/abs/2211.08875 (2022).

  144. Lange, H., Brunton, S. L. & Kutz, J. N. From Fourier to Koopman: spectral methods for long-term time series prediction. J. Mach. Learn. Res. 22, 1–38 (2021).

    MathSciNet  Google Scholar 

  145. Lanthaler, S., Mishra, S. & Karniadakis, G. E. Error estimates for DeepONets: a deep learning framework in infinite dimensions. Trans. Math. Appl. 6, tnac001 (2022).

    MathSciNet  Google Scholar 

  146. Kovachki, N., Lanthaler, S. & Mishra, S. On universal approximation and error bounds for Fourier neural operators. J. Mach. Learn. Res. 22, 13237–13312 (2021).

    MathSciNet  Google Scholar 

  147. Lyu, Y., Zhao, X., Gong, Z., Kang, X. & Yao, W. Multi-fidelity prediction of fluid flow based on transfer learning using Fourier neural operator. Phys. Fluids 35, 077118 (2023).

    Article  Google Scholar 

  148. Gopakumar, V. et al. Plasma surrogate modelling using Fourier neural operators. Nucl. Fusion 64, 056025 (2024).

    Article  Google Scholar 

  149. Kurth, T. et al. FourCastNet: accelerating global high-resolution weather forecasting using adaptive Fourier neural operators. In Proc. Platform for Advanced Scientific Computing Conference 1–11 (ACM, 2023).

  150. Raonić, B. et al. Convolutional neural operators for robust and accurate learning of PDEs. In Advances in Neural Information Processing Systems (eds Oh, A. et al.) 77187–77200 (NeurIPS, 2023).

  151. Fanaskov, V. & Oseledets, I. Spectral neural operators. Preprint at https://arxiv.org/abs/2205.10573 (2022).

  152. Chen, T. & Chen, H. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans. Neural Networks 6, 911–917 (1995).

    Article  Google Scholar 

  153. Cao, S. Choose a transformer: Fourier or Galerkin. Adv. Neural Inf. Process. Syst. 34, 24924–24940 (2021).

    Google Scholar 

  154. Li, Z., Meidani, K. & Farimani, A. B. Transformer for partial differential equations’ operator learning. Transact. Mach. Learn. Res. https://openreview.net/pdf?id=EPPqt3uERT (2023).

  155. Vinuesa, R. & Brunton, S. L. Enhancing computational fluid dynamics with machine learning. Nat. Comput. Sci. 2, 358–366 (2022).

    Article  Google Scholar 

  156. Mishra, S. A machine learning framework for data driven acceleration of computations of differential equations. Math. Eng. 1, 118–146 (2019).

    Article  MathSciNet  Google Scholar 

  157. Bar-Sinai, Y., Hoyer, S., Hickey, J. & Brenner, M. P. Learning data-driven discretizations for partial differential equations. Proc. Natl Acad. Sci. USA 116, 15344–15349 (2019).

    Article  MathSciNet  Google Scholar 

  158. Stevens, B. & Colonius, T. Enhancement of shock-capturing methods via machine learning. Theor. Comput. Fluid Dyn. 34, 483–496 (2020).

    Article  MathSciNet  Google Scholar 

  159. Barrault, M., Maday, Y., Nguyen, N. C. & Patera, A. T. An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations. C. R. Math. 339, 667–672 (2004).

    Article  MathSciNet  Google Scholar 

  160. Chaturantabut, S. & Sorensen, D. C. Nonlinear model reduction via discrete empirical interpolation. SIAM J. Sci. Comput. 32, 2737–2764 (2010).

    Article  MathSciNet  Google Scholar 

  161. Stevens, B. & Colonius, T. FiniteNet: a fully convolutional LSTM network architecture for time-dependent partial differential equations. Preprint at https://arxiv.org/abs/2002.03014 (2020).

  162. Yousif, M. Z., Zhang, M., Yu, L., Vinuesa, R. & Lim, H. A transformer-based synthetic-inflow generator for spatially developing turbulent boundary layers. J. Fluid Mech. 957, A6 (2023).

    Article  MathSciNet  Google Scholar 

  163. Kochkov, D. et al. Machine learning-accelerated computational fluid dynamics. Proc. Natl Acad. Sci. USA 118, e2101784118 (2021).

    Article  MathSciNet  Google Scholar 

  164. Fukami, K., Fukagata, K. & Taira, K. Super-resolution reconstruction of turbulent flows with machine learning. J. Fluid Mech. 870, 106–120 (2019).

    Article  MathSciNet  Google Scholar 

  165. Fukami, K., Fukagata, K. & Taira, K. Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows. J. Fluid Mech. 909, A9 (2021).

    Article  MathSciNet  Google Scholar 

  166. Fukami, K. & Taira, K. Robust machine learning of turbulence through generalized Buckingham Pi-inspired pre-processing of training data. In APS Division of Fluid Dynamics Meeting Abstracts A31-004 (APS, 2021).

  167. Pan, S., Brunton, S. L. & Kutz, J. N. Neural implicit flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data. J. Mach. Learn. Res. 24, 1607–1666 (2023).

    MathSciNet  Google Scholar 

  168. Takamoto, M. et al. PDEBench: an extensive benchmark for scientific machine learning. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS, 2022).

Download references

Acknowledgements

We acknowledge support from the National Science Foundation AI Institute in Dynamic Systems (grant no. 2112085). S.L.B. acknowledges support from the Army Research Office (ARO W911NF-19-1-0045).

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Washington, Seattle, WA, USA

    Steven L. Brunton

  2. Department of Applied Mathematics, University of Washington, Seattle, WA, USA

    J. Nathan Kutz

Authors
  1. Steven L. Brunton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Nathan Kutz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.L.B. and J.N.K. contributed equally to this Review.

Corresponding author

Correspondence to Steven L. Brunton.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Computational Science thanks Lu Lu, Samuel Lanthaler and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Primary Handling Editor: Fernando Chirigati, in collaboration with the Nature Computational Science team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brunton, S.L., Kutz, J.N. Promising directions of machine learning for partial differential equations. Nat Comput Sci (2024). https://doi.org/10.1038/s43588-024-00643-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s43588-024-00643-2

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics