The dot product takes two vectors and produces a single number:

$$\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n$$

Multiply corresponding entries, then add them all up. For example:

$$(3, 1, 4) \cdot (2, 5, 0) = 3 \times 2 + 1 \times 5 + 4 \times 0 = 6 + 5 + 0 = 11$$

What it measures

The dot product tells you how much two vectors point in the same direction.

• Positive: The vectors point roughly the same way
• Zero: The vectors are perpendicular (orthogonal) — they share no information
• Negative: The vectors point in roughly opposite directions

More precisely:

$$\mathbf{u} \cdot \mathbf{v} = |\mathbf{u}| \; |\mathbf{v}| \; \cos\theta$$

where $\theta$ is the angle between the vectors. When $\theta = 0°$ (same direction), $\cos\theta = 1$ and the dot product is maximised. When $\theta = 90°$ (perpendicular), $\cos\theta = 0$ and the dot product is zero.

Try it yourself

Drag the tips of u and v. The dot product, angle, and projection update in real time. The dashed orange line shows the projection of u onto v.

Try making the vectors perpendicular (dot product = 0). Then make them point the same direction (dot product maximised). Then opposite directions (dot product negative).

Orthogonality

Two vectors are orthogonal (perpendicular) when their dot product is zero. This is one of the most important concepts in linear algebra:

• In PCA, the principal components are orthogonal to each other
• In factor analysis, the dimensions are (usually) orthogonal
• Orthogonal directions carry independent information — knowing one tells you nothing about the other

A set of vectors that are all mutually orthogonal and all have length 1 is called orthonormal. Orthonormal vectors form a coordinate system — a clean set of axes with no redundancy.

Projection

The dot product also gives you projection: how much of one vector lies along another. The projection of $\mathbf{u}$ onto a unit vector $\hat{\mathbf{v}}$ is simply:

$$\text{proj} = \mathbf{u} \cdot \hat{\mathbf{v}}$$

This is the “shadow” of $\mathbf{u}$ cast onto the line defined by $\hat{\mathbf{v}}$. When PCA projects data onto a principal component, it’s computing dot products with an eigenvector — exactly this operation.