Tensors

Multidimensional generalizations of scalars, vectors, and matrices, indexed by any number of dimensions.

The generalization

A tensor extends the sequence scalar, vector, matrix to arbitrary dimension. A scalar is a rank-0 tensor, a vector rank-1, a matrix rank-2, and a stack of matrices rank-3. In computing, a tensor is simply a multidimensional array; the number of indices needed to address an element is its rank, or number of axes.

Two meanings of tensor

The word carries two related senses. In machine learning and array programming, a tensor is any n-dimensional array of numbers. In physics and differential geometry, a tensor is a geometric object whose components transform in a specific way under changes of coordinates, so that the physical quantity it represents is basis-independent. The stress tensor and the metric tensor are examples of the second sense.

Operations

Tensor operations generalize matrix operations. Contraction sums over a shared index, generalizing matrix multiplication and the dot product. The Einstein summation convention, which implies a sum over any repeated index, gives a compact notation for these contractions and is mirrored by the einsum function in numerical libraries.

Tensor decompositions

Just as matrices factor via SVD, higher tensors have decompositions such as the CP (canonical polyadic) and Tucker decompositions, which express a tensor as sums or products of simpler factors. These reveal low-rank structure in multidimensional data and compress it, though tensor rank is subtler than matrix rank.

import numpy as np
A = np.random.rand(2, 3)
B = np.random.rand(3, 4)
# einsum expresses contraction over the shared index j
C = np.einsum('ij,jk->ik', A, B)
print(C.shape)   # (2, 4), same as A @ B

Physical fields in a plasma, such as pressure anisotropy and the electromagnetic stress, are described by tensors, and the discretized simulation state is stored as high-rank arrays indexed by space, time, and field component.