Multilinear Algebra

Google DeepMind stores convolutional layer weights as rank-4 tensors: a layer with 512 filters of size 3x3 and 256 channels is a (512,256,3,3) tensor with 1,179,648 parameters.

• **TensorFlow/PyTorch:** all deep learning operations are tensor contractions -- convolution, attention, matmul implemented via einsum.
• **Physics:** inertia tensor, Cauchy stress tensor, Maxwell tensor are rank-2+ tensors in engineering computations.
• **PCA:** covariance matrix is a symmetric (0,2)-tensor; its spectral decomposition gives principal components.

Tensors and Tensor Products

Google DeepMind stores convolutional layer weights as rank-4 tensors: a layer with 512 filters of size 3x3 and 256 input channels is a (512,256,3,3) tensor -- 1,179,648 parameters in a single layer.

Exterior Algebra and Differential Forms

Maxwell's electromagnetic theory is compactly written via the 2-form F=dA: all 4 Maxwell equations reduce to dF=0 and d*F=J -- two equations instead of four.

Symmetric Tensors and Quadratic Forms

In Principal Component Analysis the data metric is the covariance matrix -- a symmetric (0,2)-tensor. Its eigen-decomposition finds the principal axes of scatter.

Key ideas

• **Tensor (p,q):** p contravariant, q covariant indices. Under a basis change each index transforms by a Jacobian factor.
• **Exterior product:** v^w=-w^v, dim Lambda^k(V)=C(n,k). Encodes volume and orientation -- basis of Maxwell's equations.
• **Symmetric tensors:** quadratic forms, PCA. Spectral theorem: Q(v)>0 for all v!=0 iff all eigenvalues > 0.