Vectors & Spaces

Vector Arithmetic

A vector in $\mathbb{R}^n$ is an ordered list of $n$ real numbers. In two dimensions we write $\mathbf{v} = \begin{pmatrix} v_1 \\ v_2 \end{pmatrix}$, which you can picture as an arrow from the origin to the point $(v_1, v_2)$. The numbers $v_1$ and $v_2$ are its components.

Two operations define everything: addition and scalar multiplication. Both work component-wise:

Geometrically, addition places vectors tip-to-tail: slide the tail of $\mathbf{v}$ to the tip of $\mathbf{u}$, and the sum $\mathbf{u} + \mathbf{v}$ is the arrow from the origin to where $\mathbf{v}$'s tip now lands. Scalar multiplication by $c > 1$ stretches the arrow; by $0 < c < 1$ it shrinks it; by $c < 0$ it flips its direction.

The magnitude (or length) of $\mathbf{v}$ is:

This is just the Pythagorean theorem applied to the components. A vector with magnitude $1$ is called a unit vector. Any nonzero vector can be normalized to a unit vector by dividing by its magnitude: $\hat{\mathbf{v}} = \frac{\mathbf{v}}{\|\mathbf{v}\|}$.

Dot Product

The dot product is a way to multiply two vectors that produces a single number (a scalar). For $\mathbf{u} = (u_1, u_2)$ and $\mathbf{v} = (v_1, v_2)$:

This has a beautiful geometric interpretation. If $\theta$ is the angle between $\mathbf{u}$ and $\mathbf{v}$, then:

The dot product measures how much the two vectors point in the same direction. When $\theta = 0°$ they're parallel and the dot product is maximized. When $\theta = 90°$, $\cos 90° = 0$, so $\mathbf{u} \cdot \mathbf{v} = 0$ — the vectors are orthogonal (perpendicular). When $\theta > 90°$ the dot product is negative, meaning the vectors point away from each other.

Notice that $\mathbf{v} \cdot \mathbf{v} = v_1^2 + v_2^2 = \|\mathbf{v}\|^2$, so the magnitude satisfies $\|\mathbf{v}\| = \sqrt{\mathbf{v} \cdot \mathbf{v}}$. The dot product gives us length for free.

Span & Basis

A linear combination of vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k$ is any expression of the form $c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k$ where the $c_i$ are real numbers. The set of all possible linear combinations is called the span:

Think of span as the question: "what is every destination I can reach using these vectors as building blocks, with any scaling I choose?" Two parallel vectors in $\mathbb{R}^2$ only span a line — no matter how you scale and add them, you stay on that line. Two non-parallel vectors span all of $\mathbb{R}^2$ — you can reach any point in the plane.

A set of vectors is linearly independent if no vector in the set can be written as a linear combination of the others — none is redundant. A basis is a linearly independent set that spans the whole space. It is the minimum information needed to describe every point.

• The standard basis for $\mathbb{R}^2$: $\mathbf{e}_1 = (1,0)^T$, $\mathbf{e}_2 = (0,1)^T$ — the coordinate axes.
• Any two non-parallel vectors in $\mathbb{R}^2$ form a valid basis.
• Three vectors in $\mathbb{R}^2$ are always linearly dependent — one can always be expressed using the other two.

The number of vectors in any basis for a space is always the same, no matter which basis you choose. This number is the dimension of the space. $\mathbb{R}^2$ has dimension $2$; $\mathbb{R}^3$ has dimension $3$.