Core Identities

Every word GPT-4 reads is encoded through sine and cosine. Positional encoding is 8th-grade trigonometry in 1024-dimensional space. In 1965, Cooley and Tukey published the FFT - the algorithm that made MP3, JPEG, 5G, and modern astronomy possible. Its core: sum-of-angles formulas that are 400 years old. The entire chain starts from one equality: $\sin^2\theta + \cos^2\theta = 1$.

• **Transformers (GPT, LLaMA):** positional encoding uses $\sin(pos/10000^{2i/d})$ - trigonometry encodes word order in every LLM query
• **RoPE in LLaMA/Mistral:** rotation matrices built from sin/cos encode relative token positions; attention scores unfold via cos(α - β)
• **JPEG and MP3:** DCT (discrete cosine transform) uses power-reduction to compress - without it, photos and audio would be 10x larger
• **Signal processing:** sum formulas convert a product of sinusoids into a sum of frequencies - the foundation of AM/FM modulation, Wi-Fi, 5G

Предварительные знания

• Trigonometric Functions

The Pythagorean Identity

In 1965, Cooley and Tukey published the Fast Fourier Transform. It made MP3, JPEG, 5G, and modern astronomy possible - all of it resting on trigonometric identities that are 400 years old. At the heart of this mathematics sits one equation.

This is not "similar to the Pythagorean theorem" - it **is** the Pythagorean theorem, written on the unit circle. The point $(\cos\theta,\, \sin\theta)$ lies at distance 1 from the origin. Distance comes from $x^2 + y^2 = r^2$. Substituting gives the identity.

Dividing both sides by $\cos^2\theta$ and by $\sin^2\theta$ produces two more identities:

The three Pythagorean identities are a single formula written three ways. Only $\sin^2\theta + \cos^2\theta = 1$ needs to be remembered - the others follow by dividing in 5 seconds.

The identity holds in 512-dimensional space exactly as in two dimensions. Positional encoding in transformers - $PE(pos, 2i) = \sin(pos / 10000^{2i/d})$ - uses the same unit circle in each of 256 dimension pairs. Every word in GPT-4 carries a coordinate where $\sin^2 + \cos^2 = 1$ holds literally.

Sum and Difference Formulas

How does one find $\sin(75°)$ exactly - without a calculator, without tables? $75° = 45° + 30°$. Both are standard angles. A formula for $\sin(\alpha + \beta)$ turns known values into an exact answer.

In the sine formula the sign matches $\pm$; in the cosine formula the sign is **reversed** ($\mp$). Memory aid: 'sine is friendly (preserves the sign), cosine is stubborn (flips it).'

The geometric proof is elegant. Points $A = (\cos\alpha, \sin\alpha)$ and $B = (\cos\beta, \sin\beta)$ on the unit circle. The distance $|AB|^2$ computed two ways - via the distance formula and via the law of cosines - equates to give $\cos(\alpha - \beta)$ immediately.

Sum formulas are the engine of signal modulation. AM/FM radio, 5G, Wi-Fi - all rely on the product $\sin\alpha \cdot \sin\beta$. The sum formula converts it into a sum of sinusoids: $\sin\alpha \cdot \sin\beta = \tfrac{1}{2}[\cos(\alpha - \beta) - \cos(\alpha + \beta)]$. Multiplying frequencies is adding arguments.

Double-Angle Formulas

The double-angle formulas are not separate identities. They are the **special case** of the sum formulas with $\alpha = \beta$. One substitution step:

From the Pythagorean identity, $\cos(2\theta)$ can be written three ways - each convenient in a different setting:

Choose the form based on what is in the expression: only cos - use $2\cos^2\theta - 1$, only sin - use $1 - 2\sin^2\theta$, both - use $\cos^2\theta - \sin^2\theta$.

RoPE (Rotary Position Embeddings) - the positional encoding used in LLaMA, Mistral, and GPT-NeoX - literally multiplies each query/key component pair by a rotation matrix $\begin{pmatrix}\cos m\theta & -\sin m\theta \\ \sin m\theta & \cos m\theta\end{pmatrix}$. When the attention score $q \cdot k$ is computed, two rotations multiply - and the result contains $\cos((m-n)\theta)$, the relative distance between positions. This is not a metaphor: it is the literal application of $\cos(\alpha - \beta) = \cos\alpha\cos\beta + \sin\alpha\sin\beta$.

Half-Angle Formulas and Power Reduction

Half-angle formulas are the double-angle formulas read in reverse. From $\cos(2\theta) = 2\cos^2\theta - 1$, solving for $\cos^2\theta$:

Replacing $\theta$ with $\theta/2$ gives the half-angle formulas:

The sign $\pm$ is determined by the quadrant of $\theta/2$, not $\theta$. If $\theta = 300°$, then $\theta/2 = 150°$ - second quadrant, where sine is positive and cosine is negative.

For the tangent, formulas **without** $\pm$ exist:

The power-reduction forms - $\sin^2\theta = (1 - \cos 2\theta)/2$ and $\cos^2\theta = (1 + \cos 2\theta)/2$ - turn a squared trig function into a linear expression. DCT (discrete cosine transform) in JPEG and MP3 relies on this directly: squared cosines collapse via power reduction, and compression becomes computable.

Key Ideas

• **$\sin^2\theta + \cos^2\theta = 1$ is the Pythagorean theorem** on the unit circle, not a separate fact
• **Sum and difference formulas** are the second foundation: sin(α ± β) and cos(α ± β) - everything else derives from these
• **Double-angle formulas** are a special case of the sum formulas (β = α): sin(2θ) = 2sinθ cosθ; cos(2θ) has three forms
• **Power-reduction and half-angle** are the double-angle formulas read backwards; essential for integration and for DCT in JPEG/MP3

Related Topics

Trigonometric identities unite geometry, algebra, and analysis:

• Trigonometric Functions — Definitions of sin, cos, tan - the foundation for the identities
• Inverse Trigonometric Functions — Identities are used when simplifying expressions with arcsin, arccos, arctan
• Differentiation Rules — Derivatives of trig functions are derived using sum formulas
• Antiderivatives — Power-reduction formulas are the key technique for integrating sin^2, cos^2

Вопросы для размышления

• Try deriving sin(3θ) using sin(2θ + θ) and the sum formulas. How many steps does it take? What pattern emerges for sin(nθ)?
• Why does the projectile range formula contain sin(2α) rather than sin(α)? What does the double angle represent physically in this context?
• Positional encoding in transformers uses different frequencies per dimension: $pos / 10000^{2i/d}$. Why exactly 10000 as the base - how does this connect to the sum-of-angles formula and the need to distinguish distant positions?

Связанные уроки

• trig-01 — The unit circle is the geometric foundation of all identities
• trig-03 — Identities are needed to simplify arcsin, arccos expressions
• calc-07-derivative-rules — Derivatives of trig functions are derived via sum formulas
• calc-10-antiderivatives — Power-reduction formulas are the key to integrating sin^2, cos^2