PINN Loss and Collocation Points
The PINN objective is a weighted sum of residual, boundary, initial, and data losses evaluated at sampled collocation points.
Anatomy of the loss
A physics-informed network is trained on a composite loss. The residual loss measures how far the network is from satisfying the PDE at interior points. The boundary loss enforces conditions on the domain edge. The initial loss fixes the state at t=0. An optional data loss ties the network to any real measurements. Each term is a mean squared quantity over its own set of sample points.
The full objective is L = w_r L_res + w_b L_bc + w_i L_ic + w_d L_data, where the weights w balance the terms. Choosing these weights is one of the central practical difficulties: if the residual weight dominates, boundary conditions drift; if it is too small, the network ignores the physics.
Collocation points
Collocation points are the coordinates where the residual is evaluated. Because a PINN is mesh-free, these can be drawn uniformly at random, on a grid, or from a low-discrepancy sequence such as Latin hypercube or Sobol sampling. More points give a finer enforcement of the equation but cost more per training step.
Adaptive sampling
Uniform sampling wastes effort where the solution is smooth and under-resolves where it is sharp. Adaptive schemes place more points where the residual is currently large, refining the network's attention on difficult regions such as boundary layers or shock fronts. Residual-based resampling can markedly improve accuracy on advection-dominated problems.
Weighting strategies
- Fixed manual weights, tuned by trial and error
- Gradient-normalization schemes that balance the magnitude of each term's gradient
- Neural tangent kernel analysis that predicts which term converges too slowly
- Treating the weights as learnable parameters in a min-max formulation
Getting the balance right often matters more than the network's size or depth. A well-weighted small network frequently beats a poorly balanced large one.