Rotations and rigid-body motion — SE(3)

Lesson 01 ended on a quiet assumption: to place a camera, align a scan, or move an object, we can just "transform" it. Translation is trivial — vectors add. Rotation is where it gets hard, and getting it wrong silently wrecks real systems: optimizers that drift off the space of valid rotations, controllers that lock up at a pole, interpolations that swing wide. This lesson builds rigid-body motion from first principles and, crucially, the one trick — optimizing on the tangent space — that every pose estimator in this track (registration, bundle adjustment, NeRF pose refinement, SLAM) leans on.

1 · Why rotation is not a vector space

Translations are a vector space: any two add, they commute (a+b = b+a), scaling is free, and gradient descent can step in any direction and stay valid. Rotations fail every one of these, and the failures are not academic — they are the source of the bugs this lesson exists to prevent.

Rotations don't commute. Take a book. Rotate it 90° about the vertical axis, then 90° about the axis pointing right; note its pose. Start over and do the two in the opposite order. The final poses differ. Formally R₁R₂ ≠ R₂R₁ — order matters, so "add up the rotations" is meaningless. (Translations, by contrast, always commute.)

Valid rotations form a curved surface, not a flat space. A 3D rotation is a 3×3 matrix R with two properties: its columns are orthonormal (RᵀR = I) and it preserves handedness (\det R = +1). That is 9 numbers tied down by 6 constraints, leaving 3 degrees of freedom — but those 3 DOF live on a curved 3-dimensional surface (the group SO(3)) embedded in 9-dimensional matrix space. Step off it — as an optimizer nudging all 9 entries surely will — and you get a matrix that shears and scales, no longer a rotation. This single fact ("the valid set is a curved manifold you fall off of") is what makes the naïve approaches break and what section 4 fixes properly.

2 · Four ways to write a rotation

Rotation matrix — R ∈ SO(3). Nine numbers, orthonormal columns. It acts directly (v' = Rv) and composes by matrix multiply (R₁R₂). Perfect for computing; terrible for storing or optimizing, because 9 numbers with 6 constraints are hugely redundant, and any unconstrained update breaks orthonormality, forcing a re-orthonormalization (e.g. via SVD) every step.

Euler angles — (yaw, pitch, roll). Three numbers, one per intuitive axis, applied in sequence. Minimal and human-readable, which is why they show up in UIs and datasheets. But they carry a fatal singularity: gimbal lock. When the middle rotation reaches ±90°, the first and third rotation axes line up — they now do the same thing, and one degree of freedom vanishes. Near that pole, small pose changes demand huge angle changes, interpolation misbehaves, and optimizers stall. Euler angles also don't compose or interpolate cleanly (which order? which axes?). Use them to display a rotation, never to optimize one. The widget below lets you drive a frame into gimbal lock and watch a DOF disappear.

Axis-angle — (θ, \hat{k}). Euler's theorem: every rotation is a single rotation by angle θ about some unit axis \hat{k}. Pack it as the 3-vector ω = θ\hat{k} — exactly 3 numbers, no redundancy. This is the most physically honest parameterization and, we'll see, it is the coordinate on the Lie algebra. Its only awkwardness: composing two axis-angles isn't a simple formula (you convert to matrices or quaternions first), and θ=0 leaves the axis undefined (a removable singularity).

Quaternion — unit q = (w, x, y, z). The practitioner's default for storing and interpolating orientation. A unit quaternion encodes the axis-angle as

Four numbers, one constraint (unit norm) — far less redundant than a matrix, and no gimbal lock anywhere. Composition is the Hamilton product (cheaper than matmul), rotating a vector is v' = q\,v\,q^-1, and interpolation is SLERP (spherical linear interpolation), which walks the shortest arc on the sphere of orientations at constant angular speed. One subtlety to know cold: quaternions double-cover SO(3) — q and −q represent the same rotation (note the θ/2: rotating by 2π sends q → −q but returns the object to start). SLERP must pick the shorter of the two to avoid taking the long way around.

// Euler (intrinsic Z-Y-X) → rotation matrix, then draw R·e_x, R·e_y, R·e_z
function Rz(a){return [[cos(a),-sin(a),0],[sin(a),cos(a),0],[0,0,1]];}
function Ry(a){return [[cos(a),0,sin(a)],[0,1,0],[-sin(a),0,cos(a)]];}
function Rx(a){return [[1,0,0],[0,cos(a),-sin(a)],[0,sin(a),cos(a)]];}
const R = matmul(matmul(Rz(yaw), Ry(pitch)), Rx(roll));

// matrix → quaternion (no singularity); angle θ = 2·acos(w)
const w = 0.5*Math.sqrt(1 + R[0][0]+R[1][1]+R[2][2]);
// ... x,y,z from off-diagonal terms ...
const theta = 2*Math.acos(Math.min(1, Math.abs(w)));

// GIMBAL LOCK: at pitch → ±90°, the yaw (Z) and roll (X) axes align.
const gimbal = Math.abs(Math.abs(pitch) - Math.PI/2) < 0.12;
// health of R as a rotation: ‖RᵀR − I‖ should be ~0 (drifts if you optimise entries directly)

3 · Rigid-body motion — SE(3)

A camera or object pose is a rotation and a translation: a rigid transform, the group SE(3) (Special Euclidean, 6 DOF = 3 rotation + 3 translation). Write it as a pair (R, t) acting on a point by p' = Rp + t. The clean way to compose and invert these is the homogeneous 4×4 form (the same trick CV lesson 04 used to fold translation into a matrix):

Now chaining transforms is just matrix multiply, T_AC = T_AB\,T_BC, which is how you walk a point from one coordinate frame to another (object → world → camera → image is a chain of these). The inverse has a closed form worth memorizing — you invert the rotation and undo the translation in the rotated frame:

Note it is not just (−R, −t): the translation must be rotated back by Rᵀ. Getting this inverse wrong (forgetting the Rᵀ on t) is a classic frame-convention bug — the twin of the row/col and BGR bugs from CV lesson 01, and it puts your camera in the wrong place by exactly the rotated translation.

4 · The Lie-algebra trick — optimizing on the manifold

Here is the section the whole lesson was built for. Section 1 warned that SO(3) is a curved surface you fall off of. So how do you run gradient descent on a rotation — as bundle adjustment, ICP, and SLAM all must — without producing invalid matrices?

The tangent space. At the identity, SO(3) has a flat tangent space called so(3) (the Lie algebra), and it turns out to be exactly the skew-symmetric matrices. Any ω = (ω₁, ω₂, ω₃) ∈ ℝ³ maps to one via the hat operator (the same [·]_× cross-product matrix from CV lesson 05):

So the tangent space is an ordinary, unconstrained 3-vector ω — exactly the 3 DOF a rotation really has, with no manifold to fall off. (For full poses, se(3) uses a 6-vector twist ξ = (ρ, ω): translation-like part ρ and rotation part ω.)

The exp map: tangent → manifold. Exponentiating a skew matrix lands you back on SO(3), and the series collapses into a closed form — Rodrigues' formula — with θ = ‖ω‖ and \hat k = ω/θ:

This is precisely the axis-angle → matrix conversion from section 2, now revealed as the exponential map. Its inverse, the log map, reads the axis-angle back out of a rotation matrix. Together they give a lossless dictionary between the flat, optimizer-friendly ℝ³ (or ℝ⁶) and the curved group.

Worked number. A small rotation of θ = 0.02 rad about \hat k = (0,0,1) has ω = (0,0,0.02). Rodrigues gives R ≈ I + 0.02[\hat k]_× (since \sin θ ≈ θ, 1-\cos θ ≈ 0), i.e. the top-left block is [[1, -0.02],[0.02, 1]] — a hair of yaw. For small twists \exp([ω]_×) ≈ I + [ω]_×, which is why the tangent space is the natural place to linearize and why Jacobians in ξ are clean.

Where this points next

We can now represent, compose, invert, and — the part that matters — optimize the placement of any camera or object in 3D, staying on the manifold of valid motions the whole time. That machinery is inert without something to place. So the next question is the most concrete one in the track: where does 3D data actually come from? Lesson 03 covers the sensors — stereo, structured light, time-of-flight, LiDAR — and the operation that turns their output into the rawest 3D representation from lesson 01: back-projecting a depth map through K^-1 into a point cloud, expressed in world coordinates via exactly the SE(3) pose we just built.

Interview prompts

• Why can't you represent an optimizable rotation with a plain 3×3 matrix of free parameters? (§1, §4 — updates leave SO(3) (columns stop being orthonormal); you'd re-orthonormalize every step. Optimize a twist and exp back instead.)
• What is gimbal lock and when does it occur? (§2 — at the middle Euler angle ±90° two rotation axes align, losing a DOF; near it, interpolation and control degrade.)
• Why quaternions over Euler angles for storing/interpolating orientation? (§2 — no gimbal lock, cheap composition, stable SLERP along the shortest arc; watch the q ≡ -q double cover.)
• Write the inverse of a rigid transform (R,t). (§3 — (Rᵀ, -Rᵀt); the translation is rotated back — forgetting Rᵀ is a classic frame bug.)
• How does a SLAM/BA back-end optimize camera poses on a curved space? (§4 — parameterize updates as twists ξ∈ℝ⁶ in se(3), take Jacobians in ξ, retract with \exp; iterates stay valid.)
• What is Rodrigues' formula and how does it relate to the exp map? (§4 — the closed form of \exp([ω]_×); it is axis-angle → matrix, mapping the so(3) tangent to SO(3).)