Texturing

1974: Catmull's hand changes everything

1974. University of Utah. Ed Catmull places his hand on a digitizing tablet and models it as a 3D mesh of 350 polygons. Then he takes a photograph of that same hand and projects it onto the polygons. Renders. For the first time in history, a computer-generated 3D object looks real - not a synthetic form, but an actual hand. Catmull publishes the method, joins Lucasfilm, co-founds Pixar. Texture mapping becomes the foundation of all computer graphics for the next 50 years.

A flat quad of 2 triangles passes itself off as a rock face with a thousand details. A 50K-polygon character looks like 5 million. This is not magic - it is texturing.

• Every polygonal renderer: games, films, AR filters in Instagram and Snapchat
• Photogrammetry scanning of real actors for AAA games (Horizon, God of War)
• Neural textures in NeRF and Gaussian Splatting pipelines
• Virtual clothing try-on in e-commerce (IKEA AR, Zara AR)
• Medical visualization: textured 3D models of organs reconstructed from MRI data

UV mapping: flattening 3D surfaces into a 2D plane

1974. Ed Catmull - PhD student at the University of Utah, future president of Pixar - places his hand on a digitizing tablet. He builds a 3D model of the hand from 350 polygons. Then he takes a photograph of that hand and drapes it over the polygons like wallpaper on a wall. Renders it. For the first time in the history of computer graphics, a 3D object looks real. Not shaded, not colored - genuinely **real**. That was the world's first texture mapping.

The idea is elegantly simple: each vertex of a 3D mesh gets coordinates $(u, v) \in [0, 1]^2$ that point to a location in a texture image. During rasterization, the GPU interpolates $(u, v)$ across each triangle and samples the color from the texture. A three-dimensional surface unfolds into a flat sheet - like peeling an orange and pressing the skin flat.

UV is not an arbitrary choice of letters. X, Y, Z are taken - they name the 3D spatial axes. U and V are the next available letters in the alphabet. The $[0, 1]$ range is normalized: regardless of texture resolution (256x256 or 4096x4096), the coordinate 0.5 always means the center.

UV unwrapping is a hard problem with no perfect solution. Unfolding a sphere without distortion is impossible - which is exactly why every map projection of Earth lies somewhere. Game characters face the same challenge: arms, head, and body are unwrapped as separate "islands" that artists manually arrange on the UV sheet. A bad unwrap means the texture stretches like wallpaper hung on a curved wall.

Stable Diffusion generates images at 512x512 or 1024x1024 with no knowledge of 3D. But pipelines like Zero123 take the generated 2D art and automatically construct UV unwraps for 3D meshes. Catmull projected a photo by hand - diffusion models now do it in seconds.

**Tile and Wrap modes**: UV coordinates can exceed $[0, 1]$. `GL_REPEAT` tiles the texture (grass, brick, asphalt - everything tiles). `GL_CLAMP_TO_EDGE` stretches the border pixels. `GL_MIRRORED_REPEAT` flips on every repeat - this removes visible seams at tile boundaries.

Filtering and mipmaps: when pixels disagree

Nearest neighbor is the cheapest way to sample a texture. Take the pixel closest to $(u, v)$. The result looks like Minecraft. Sometimes that is an intentional aesthetic - but usually it is an unwanted artifact.

Bilinear filtering takes the 4 nearest pixels and interpolates linearly along both axes. The result is smooth when magnifying (enlarging). But when minifying (shrinking distant geometry) - the texture shimmers. On far-away objects each frame the GPU picks different pixels, and the image visibly "crawls".

**Aliasing during minification** is a fundamental sampling problem. The Nyquist theorem states: sampling frequency must be at least twice the highest frequency of the signal. A distant texture with fine details violates this limit - aliasing is the result.

Lance Williams proposed mipmaps in 1983. The idea: precompute filtered versions of the texture at each 2x reduction. A 1024x1024 texture spawns a pyramid: 512x512, 256x256 ..., 2x2, 1x1. Total storage is only 4/3 of the original (geometric series: $\sum_{k=0}^{\infty} (1/4)^k = 4/3$). The GPU picks the right level (LOD - Level of Detail) automatically from the UV derivative.

Trilinear filtering = bilinear within two adjacent mip levels + linear blend between those levels. This removes the visible "pop" when LOD transitions. Anisotropic filtering (AF) goes further: it accounts for the viewing angle of the surface. A floor viewed at a steep angle blurs with regular mipmaps - AF samples the texture at multiple points along the angle of inclination. AF x16 has been standard in games since 2005, costing about 1-2% of frame time.

**Texture compression**: BC7/BPTC (DX11+) and ASTC (mobile) store 4x4 pixel blocks in 128 bits instead of 512. Compression ratio 6:1-8:1 with minimal visual loss. The GPU decompresses blocks directly in cache - no CPU involvement. PlayStation 5 games use ASTC throughout: it saves VRAM and reduces memory bandwidth.

Normal maps and PBR: lying about geometry, honest about physics

A normal map is a textured lie. A lie that made the PS2 era look like PS4. Instead of adding millions of real polygons for scratches and rivets - the surface normal changes per pixel. The GPU believes the surface is rough in one spot and smooth in another. Lighting is computed using the fake normal. A flat quad of 2 triangles looks like a brick wall with every crack and chip.

Normals in a normal map are stored in tangent space - a local coordinate system attached to the surface. Three vectors: tangent ($T$), bitangent ($B$), and normal ($N$) form the TBN matrix. Without TBN, normals would only work on static objects - no rotation, no deformation. The bluish tint of normal maps (RGB near [128, 128, 255]) means "normal points straight up" - the neutral vector $(0, 0, 1)$ remapped to $[0, 1]$.

PBR (Physically Based Rendering) transformed the industry in 2012-2014. Disney released its principled shader to the game development community. Unreal Engine 4 adopted the metallic-roughness workflow as the default. Three parameters plus a color fully describe a physically plausible material.

**PBR metallic-roughness**: `albedo` (base color), `metallic` (0 = dielectric, 1 = metal), `roughness` (0 = mirror, 1 = fully matte). Metal surfaces have no diffuse component - they reflect light tinted by the surface color. Dielectrics reflect approximately 4% of incoming light as unsaturated white (Fresnel F0 = 0.04). The math underneath: Cook-Torrance BRDF with GGX distribution, Smith geometry term, and Schlick Fresnel.

The full PBR texture set in a modern AAA game: albedo, normal, metallic, roughness, AO, height/displacement, emissive. A single character may need 7-8 textures at 4K each - around 256 MB of VRAM for one hero. Texture streaming and virtual textures (sparse textures in Vulkan/DX12) handle this: the GPU keeps only visible mip levels in memory.

NeRF (Neural Radiance Fields, 2020) bypasses UV mapping entirely. A neural network encodes color and density as a function of position and viewing direction: $(x, y, z, \theta, \phi) \to (r, g, b, \sigma)$. No textures, no unwrapping, no artist labor. But view-dependent effects - highlights that shift with the viewing angle - are handled correctly, which ordinary textures cannot do. Gaussian Splatting (2023) achieves the same in real time.

**Baking**: for real-time rendering, a high-poly mesh (millions of polygons) is "baked" into a set of PBR textures for the low-poly game version. The normal map encodes high-poly geometric detail. A game character is typically low-poly (10-50K triangles) paired with textures baked from a version with 5-10 million polygons.

Итоги

• UV mapping unfolds a 3D surface into a flat [0,1]^2 square - like pressing orange peel flat
• Bilinear filtering removes pixelation on magnification; mipmaps remove shimmer on minification
• Mipmaps add only 33% memory overhead by precomputing the full LoD pyramid (geometric series)
• Normal maps change the surface normal per pixel - a flat quad pretends to be a rock face with a thousand details
• PBR metallic-roughness: three parameters (albedo, metallic, roughness) describe a physically correct material
• NeRF skips UV mapping entirely, encoding the scene as a neural 5D function

What comes next

Textures are the data that downstream rendering systems consume. Normal maps feed into lighting and shadows; the full PBR set powers the entire shader pipeline.

• Shadows and AO — Normal maps feed into shadow mapping and SSAO for physically accurate shading
• Shaders — GLSL sampler2D and texture() - hands-on UV sampling inside shader code
• NeRF and Neural Rendering — Neural alternative to UV textures with no manual artist unwrapping

Вопросы для размышления

• Catmull wrote texture mapping as a PhD thesis - with no idea he was laying the foundation for Pixar and an entire industry. Which of today's experimental ideas in NeRF and Gaussian Splatting might rewrite the rules the same way in 50 years?

Связанные уроки

• cg-05 — PBR textures feed data into Cook-Torrance BRDF
• cg-07 — Normal maps are used in shadow mapping and SSAO
• cg-09 — Shaders read textures via sampler2D uniforms
• cg-19 — NeRF learns a 5D texture without UV unwrapping
• la-02-dot-product — Dot product underlies UV normalization and TBN matrices
• arch-09-cache