4. Vector Spaces

Vector Spaces

Basis and Dimension

A basis of a vector space $V$ is a set of linearly independent vectors that span $V$. The dimension of $V$, denoted as $\text{dim}(V)$, is the number of vectors in any basis of $V$.

Orthogonality, Orthogonal Complements, and Projections

Two vectors $\mathbf{v}$ and $\mathbf{w}$ in a vector space are orthogonal if their dot product is zero, i.e., $\mathbf{v} \cdot \mathbf{w} = 0$.

The orthogonal complement of a subspace $W$ of a vector space $V$, denoted as $W^{\perp}$, is the set of all vectors in $V$ that are orthogonal to every vector in $W$.

The projection of a vector $\mathbf{v}$ onto a subspace $W$ is the closest vector in $W$ to $\mathbf{v}$. It is given by:

$$ \text{proj}_{\mathbf{w}}(\mathbf{v}) = \frac{\mathbf{v} \cdot \mathbf{w}}{\|\mathbf{w}\|^2} \mathbf{w} $$

Orthogonal Bases, Gram-Schmidt Process

An orthogonal basis of a vector space $V$ is a basis consisting of orthogonal vectors.

The Gram-Schmidt process is a method used to construct an orthogonal basis for a given set of linearly independent vectors. Given a set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n\}$, the process constructs an orthogonal set $\{\mathbf{u}_1, \mathbf{u}_2, \ldots, \mathbf{u}_n\}$ such that each $\mathbf{u}_i$ is orthogonal to all preceding vectors and has the same span as $\mathbf{v}_i$.