Physics-Informed Neural Networks

A PINN trains a neural network to satisfy a differential equation by penalizing the equation's residual at sampled points inside the domain.

The core idea

A physics-informed neural network (PINN) represents the solution of a differential equation as a neural network u(x,t) with trainable weights. Instead of fitting only measured data, it also enforces that the network obeys the governing equation. This is done by computing the equation's residual through automatic differentiation and adding it to the loss.

For a PDE written as N[u]=0 with boundary and initial conditions, the total loss combines several terms: a data term where measurements exist, a residual term evaluated at interior collocation points, and terms for the boundary and initial conditions. Minimizing this loss drives the network toward a function that both matches data and satisfies the physics.

Why automatic differentiation matters

The derivatives in the residual, such as du/dt or the Laplacian of u, are obtained exactly from the network by automatic differentiation, not by finite differences on a grid. This means a PINN is mesh-free: collocation points can be scattered anywhere in the domain, and the network is a continuous function you can query at any coordinate.

A minimal residual

import torch
# residual of the 1D heat equation u_t = alpha u_xx
def residual(net, x, t, alpha):
    xt = torch.stack([x, t], dim=1).requires_grad_(True)
    u = net(xt)
    g = torch.autograd.grad(u, xt, torch.ones_like(u), create_graph=True)[0]
    u_x, u_t = g[:,0:1], g[:,1:2]
    u_xx = torch.autograd.grad(u_x, xt, torch.ones_like(u_x), create_graph=True)[0][:,0:1]
    return u_t - alpha * u_xx

Strengths and limits

PINNs handle irregular geometries, high dimensions, and sparse data gracefully, and they solve forward and inverse problems with the same code. They can be slow to train and struggle with stiff, multi-scale, or sharply advective problems, where the loss landscape is hard to optimize. They are a flexible tool, not a universal replacement for finite-element or spectral solvers.