# Physics-informed neural networks (PINNs) solver on Julia. GSoC 2020. Second evaluation.

### NeuralPDE.jl: Scientific Machine Learning for Partial Differential Equations.

NeuralPDE.jl is a solver package that consists of neural network solvers for partial differential equations using scientific machine learning (SciML) techniques such as physics-informed neural networks (PINNs) and deep BSDE solvers. This package utilizes deep neural networks and neural stochastic differential equations to solve high dimensional PDEs at a greatly reduced cost and greatly increased generality compared to classical methods.

• Physics-Informed Neural Networks for automated PDE solving

• Forward-Backwards Stochastic Differential Equation (FBSDE) methods for parabolic PDEs

• Deep learning based solvers for optimal stopping time and Kolmogorov backwards equations

The goals of my project on GSoC is development and improvement PINNs solver for NeuralPDE.jl.

### What is done to the second evaluation?

• Higher-order derivatives (more 2nd) for the numeric derivative.

• All basic types of boundary conditions like Neumann, Robin, and periodic boundary conditions.

• System of PDEs

### Dependencies

"\$VERSION"
0.5s
"1.4.2"
using Pkg
Pkg.add("ModelingToolkit")
Pkg.add("DiffEqFlux")
Pkg.add("Flux")
Pkg.add("Plots")
Pkg.add("Test")
117.9s
]add NeuralPDE#master
110.3s
using ModelingToolkit
219.3s
using NeuralPDE
619.7s
using DiffEqFlux
0.2s
using Flux
0.2s
using Plots
0.2s
using Test
0.2s

### Examples

Let's take a look at a couple of examples with new features that have been added.

### ODE with high order derivative

Let's consider ODE with 3rd order derivative:

@parameters x θ
@variables u(..)
@derivatives Dxxx'''~x
@derivatives Dx'~x
#ODE
eq = Dxxx(u(x,θ)) ~ cos(pi*x)
# Initial and boundary conditions
bcs = [u(0.,θ) ~ 0.0,
       u(1.,θ) ~ cos(pi),
       Dx(u(1.,θ)) ~ 1.0]
# Space and time domains
domains = [x ∈ IntervalDomain(0.0,1.0)]
# Discretization
dx = 0.1
discretization = NeuralPDE.PhysicsInformedNN(dx)
# Neural network and optimizer
opt = Flux.ADAM(0.01)
chain = FastChain(FastDense(1,8,Flux.σ),FastDense(8,1))
pde_system = PDESystem(eq,bcs,domains,[x],[u])
prob = NeuralPDE.discretize(pde_system,discretization)
alg = NeuralPDE.NNDE(chain,opt,autodiff=false)
phi,res = solve(prob,alg,verbose=true, maxiters=10000)
analytic_sol_func(x) = (π*x*(-x+(π^2)*(2*x-3)+1)-sin(π*x))/(π^3)
xs = [domain.domain.lower:dx/10:domain.domain.upper for domain in domains]
u_real  = [analytic_sol_func(x) for x in xs]
u_predict  = [first(phi(x,res.minimizer)) for x in xs]
@test u_predict ≈ u_real atol = 10.0
x_plot = collect(xs)
plot(x_plot ,u_real)
plot!(x_plot ,u_predict)
281.1s

### The linear PDE system

@parameters x, y, θ
@variables u1(..), u2(..)
@derivatives Dx'~x
@derivatives Dy'~y
# System of pde
eqs = [Dx(u1(x,y,θ)) + 4*Dy(u2(x,y,θ)) ~ 0,
      Dx(u2(x,y,θ)) + 9*Dy(u1(x,y,θ)) ~ 0]
# Initial and boundary conditions
bcs = [[u1(x,0,θ) ~ 2x, u2(x,0,θ) ~ 3x]]
# Space and time domains
domains = [x ∈ IntervalDomain(0.0,1.0), y ∈ IntervalDomain(0.0,1.0)]
# Discretization
dx = 0.1
discretization = NeuralPDE.PhysicsInformedNN(dx)
# Neural network and optimizer
opt = Flux.ADAM(0.1)
chain = FastChain(FastDense(2,8,Flux.σ),FastDense(8,2))
pde_system = PDESystem(eqs,bcs,domains,[x,y],[u1,u2])
prob = NeuralPDE.discretize(pde_system,discretization)
alg = NeuralPDE.NNDE(chain,opt,autodiff=false)
phi,res = solve(prob,alg,verbose=true, maxiters=500)
analytic_sol_func(x,y) = [1/3*(6x - y), 1/2*(6x - y)]
xs,ys = [domain.domain.lower:dx:domain.domain.upper for domain in domains]
u_real  = [analytic_sol_func(x,y) for x in xs  for y in ys]
u_predict  = [phi([x,y],res.minimizer) for x in xs  for y in ys]
@test u_predict ≈ u_real atol = 10.0
62.4s
Test Passed

### 2D wave equation with Neumann boundary condition

Let's solve this 2d wave equation:

@parameters x, t, θ
@variables u(..)
@derivatives Dxx''~x
@derivatives Dtt''~t
@derivatives Dt'~t
#2D PDE
C=1
eq  = Dtt(u(x,t,θ)) ~ C^2*Dxx(u(x,t,θ))
# Initial and boundary conditions
bcs = [u(0,t,θ) ~ 0.,# for all t > 0
       u(1,t,θ) ~ 0.,# for all t > 0
       u(x,0,θ) ~ x*(1. - x), #for all 0 < x < 1
       Dt(u(x,0,θ)) ~ 0. ] #for all  0 < x < 1]
# Space and time domains
domains = [x ∈ IntervalDomain(0.0,1.0),
           t ∈ IntervalDomain(0.0,1.0)]
# Discretization
dx = 0.1
discretization = NeuralPDE.PhysicsInformedNN(dx)
# Neural network and optimizer
opt = Flux.ADAM(0.02)
chain = FastChain(FastDense(2,16,Flux.σ),FastDense(16,16,Flux.σ),FastDense(16,1))
pde_system = PDESystem(eq,bcs,domains,[x,t],[u])
prob = NeuralPDE.discretize(pde_system,discretization)
alg = NeuralPDE.NNDE(chain,opt,autodiff=false)
phi,res  = solve(prob,alg,verbose=true, maxiters=5000)
xs,ts = [domain.domain.lower:dx:domain.domain.upper for domain in domains]
analytic_sol_func(x,t) =  sum([(8/(k^3*pi^3)) * sin(k*pi*x)*cos(C*k*pi*t) for k in 1:2:50000])
u_predict = reshape([first(phi([x,t],res.minimizer)) for x in xs for t in ts],(length(xs),length(ts)))
u_real = reshape([analytic_sol_func(x,t) for x in xs for t in ts], (length(xs),length(ts)))
@test u_predict ≈ u_real atol = 10.0
610.1s
Test Passed
diff_u = abs.(u_predict .- u_real)
p1 = plot(xs, ts, u_real, linetype=:contourf,title = "analytic");
p2 =plot(xs, ts, u_predict, linetype=:contourf,title = "predict");
p3 = plot(xs, ts, diff_u,linetype=:contourf,title = "error");
plot(p1,p2,p3)
9.1s

More examples you can find in docs or tests of NeuralPDE.

### Future Goals

• Setting boundary conditions of arbitrary geometry (in spherical and cartesian coordinates).

• The ability to add additional conditions on the solution for the loss function (like the normalization condition)

• Instead of using the entire set of points for training, in each optimization iteration, we could select randomly different residual points which will

significantly reduce training time.

• Investigate physics-informed neural network (PINN) training strategies

• Expand the test suite.

• High dimensionality

• Combined NN solution with grid numerical methods

• Estimation problem, data-driven solution

At the moment, high-order derivatives, systems of PDE with a large number of equations, and high-dimensional equations still remain a challenge.