Physics-informed neural networks (PINNs) solver on Julia. GSoC 2020. Second evaluation.

Previous blog

NeuralPDE.jl: Scientific Machine Learning for Partial Differential Equations.

NeuralPDE.jl is a solver package that consists of neural network solvers for partial differential equations using scientific machine learning (SciML) techniques such as physics-informed neural networks (PINNs) and deep BSDE solvers. This package utilizes deep neural networks and neural stochastic differential equations to solve high dimensional PDEs at a greatly reduced cost and greatly increased generality compared to classical methods.

Features:

  • Physics-Informed Neural Networks for automated PDE solving

  • Forward-Backwards Stochastic Differential Equation (FBSDE) methods for parabolic PDEs

  • Deep learning based solvers for optimal stopping time and Kolmogorov backwards equations

The goals of my project on GSoC is development and improvement PINNs solver for NeuralPDE.jl.

What is done to the second evaluation?

  • Higher-order derivatives (more 2nd) for the numeric derivative.

  • All basic types of boundary conditions like Neumann, Robin, and periodic boundary conditions.

  • System of PDEs

  • Add Readme and Docs

  • Add new tests.

Dependencies

"$VERSION"
0.5s
"1.4.2"
using Pkg
Pkg.add("ModelingToolkit")
Pkg.add("DiffEqFlux")
Pkg.add("Flux")
Pkg.add("Plots")
Pkg.add("Test")
117.9s
]add NeuralPDE#master
110.3s
using ModelingToolkit
219.3s
using NeuralPDE
619.7s
using DiffEqFlux
0.2s
using Flux
0.2s
using Plots
0.2s
using Test
0.2s

Examples

Let's take a look at a couple of examples with new features that have been added.

ODE with high order derivative

Let's consider ODE with 3rd order derivative:

@parameters x θ
@variables u(..)
@derivatives Dxxx'''~x
@derivatives Dx'~x
#ODE
eq = Dxxx(u(x,θ)) ~ cos(pi*x)
# Initial and boundary conditions
bcs = [u(0.,θ) ~ 0.0,
       u(1.,θ) ~ cos(pi),
       Dx(u(1.,θ)) ~ 1.0]
# Space and time domains
domains = [x  IntervalDomain(0.0,1.0)]
# Discretization
dx = 0.1
discretization = NeuralPDE.PhysicsInformedNN(dx)
# Neural network and optimizer
opt = Flux.ADAM(0.01)
chain = FastChain(FastDense(1,8,Flux.σ),FastDense(8,1))
pde_system = PDESystem(eq,bcs,domains,[x],[u])
prob = NeuralPDE.discretize(pde_system,discretization)
alg = NeuralPDE.NNDE(chain,opt,autodiff=false)
phi,res = solve(prob,alg,verbose=true, maxiters=10000)
analytic_sol_func(x) = (π*x*(-x+(π^2)*(2*x-3)+1)-sin(π*x))/(π^3)
xs = [domain.domain.lower:dx/10:domain.domain.upper for domain in domains][1]
u_real  = [analytic_sol_func(x) for x in xs]
u_predict  = [first(phi(x,res.minimizer)) for x in xs]
@test u_predict  u_real atol = 10.0
x_plot = collect(xs)
plot(x_plot ,u_real)
plot!(x_plot ,u_predict)
281.1s

The linear PDE system

@parameters x, y, θ
@variables u1(..), u2(..)
@derivatives Dx'~x
@derivatives Dy'~y
# System of pde
eqs = [Dx(u1(x,y,θ)) + 4*Dy(u2(x,y,θ)) ~ 0,
      Dx(u2(x,y,θ)) + 9*Dy(u1(x,y,θ)) ~ 0]
# Initial and boundary conditions
bcs = [[u1(x,0,θ) ~ 2x, u2(x,0,θ) ~ 3x]]
# Space and time domains
domains = [x  IntervalDomain(0.0,1.0), y  IntervalDomain(0.0,1.0)]
# Discretization
dx = 0.1
discretization = NeuralPDE.PhysicsInformedNN(dx)
# Neural network and optimizer
opt = Flux.ADAM(0.1)
chain = FastChain(FastDense(2,8,Flux.σ),FastDense(8,2))
pde_system = PDESystem(eqs,bcs,domains,[x,y],[u1,u2])
prob = NeuralPDE.discretize(pde_system,discretization)
alg = NeuralPDE.NNDE(chain,opt,autodiff=false)
phi,res = solve(prob,alg,verbose=true, maxiters=500)
analytic_sol_func(x,y) = [1/3*(6x - y), 1/2*(6x - y)]
xs,ys = [domain.domain.lower:dx:domain.domain.upper for domain in domains]
u_real  = [analytic_sol_func(x,y) for x in xs  for y in ys]
u_predict  = [phi([x,y],res.minimizer) for x in xs  for y in ys]
@test u_predict  u_real atol = 10.0
62.4s
Test Passed

2D wave equation with Neumann boundary condition

Let's solve this 2d wave equation:

@parameters x, t, θ
@variables u(..)
@derivatives Dxx''~x
@derivatives Dtt''~t
@derivatives Dt'~t
#2D PDE
C=1
eq  = Dtt(u(x,t,θ)) ~ C^2*Dxx(u(x,t,θ))
# Initial and boundary conditions
bcs = [u(0,t,θ) ~ 0.,# for all t > 0
       u(1,t,θ) ~ 0.,# for all t > 0
       u(x,0,θ) ~ x*(1. - x), #for all 0 < x < 1
       Dt(u(x,0,θ)) ~ 0. ] #for all  0 < x < 1]
# Space and time domains
domains = [x  IntervalDomain(0.0,1.0),
           t  IntervalDomain(0.0,1.0)]
# Discretization
dx = 0.1
discretization = NeuralPDE.PhysicsInformedNN(dx)
# Neural network and optimizer
opt = Flux.ADAM(0.02)
chain = FastChain(FastDense(2,16,Flux.σ),FastDense(16,16,Flux.σ),FastDense(16,1))
pde_system = PDESystem(eq,bcs,domains,[x,t],[u])
prob = NeuralPDE.discretize(pde_system,discretization)
alg = NeuralPDE.NNDE(chain,opt,autodiff=false)
phi,res  = solve(prob,alg,verbose=true, maxiters=5000)
xs,ts = [domain.domain.lower:dx:domain.domain.upper for domain in domains]
analytic_sol_func(x,t) =  sum([(8/(k^3*pi^3)) * sin(k*pi*x)*cos(C*k*pi*t) for k in 1:2:50000])
u_predict = reshape([first(phi([x,t],res.minimizer)) for x in xs for t in ts],(length(xs),length(ts)))
u_real = reshape([analytic_sol_func(x,t) for x in xs for t in ts], (length(xs),length(ts)))
@test u_predict  u_real atol = 10.0
610.1s
Test Passed
diff_u = abs.(u_predict .- u_real)
p1 = plot(xs, ts, u_real, linetype=:contourf,title = "analytic");
p2 =plot(xs, ts, u_predict, linetype=:contourf,title = "predict");
p3 = plot(xs, ts, diff_u,linetype=:contourf,title = "error");
plot(p1,p2,p3)
9.1s

More examples you can find in docs or tests of NeuralPDE.

Future Goals

  • Setting boundary conditions of arbitrary geometry (in spherical and cartesian coordinates).

  • The ability to add additional conditions on the solution for the loss function (like the normalization condition)

  • Instead of using the entire set of points for training, in each optimization iteration, we could select randomly different residual points which will

    significantly reduce training time.

  • Support automatic differentiation(AD)

  • Investigate physics-informed neural network (PINN) training strategies

  • Expand the test suite.

  • High dimensionality

  • Combined NN solution with grid numerical methods

  • Estimation problem, data-driven solution

At the moment, high-order derivatives, systems of PDE with a large number of equations, and high-dimensional equations still remain a challenge.

Runtimes (1)