7. Inner Product Spaces

Inner Product Spaces

Inner Product, Norms, and Orthogonality in Euclidean Spaces

An inner product on a vector space $V$ is a function $\langle \cdot, \cdot \rangle: V \times V \rightarrow \mathbb{R}$ that satisfies the following properties for all vectors $\mathbf{u}, \mathbf{v}, \mathbf{w}$ and scalars $a, b$:

1. Linearity: $\langle a\mathbf{u} + b\mathbf{v}, \mathbf{w} \rangle = a\langle \mathbf{u}, \mathbf{w} \rangle + b\langle \mathbf{v}, \mathbf{w} \rangle$
2. Symmetry: $\langle \mathbf{u}, \mathbf{v} \rangle = \langle \mathbf{v}, \mathbf{u} \rangle$
3. Positive-definiteness: $\langle \mathbf{v}, \mathbf{v} \rangle \geq 0$, and $\langle \mathbf{v}, \mathbf{v} \rangle = 0$ if and only if $\mathbf{v} = \mathbf{0}$

The norm of a vector $\mathbf{v}$ in an inner product space is defined as $\|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}$.

Two vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal if their inner product is zero, i.e., $\langle \mathbf{u}, \mathbf{v} \rangle = 0$.

Orthogonal and Orthonormal Bases

An orthogonal basis of a vector space $V$ is a basis consisting of pairwise orthogonal vectors. An orthonormal basis is an orthogonal basis in which each vector has unit norm.

Gram-Schmidt Orthogonalization Process in Inner Product Spaces

The Gram-Schmidt process is a method used to construct an orthogonal (or orthonormal) basis for a given set of linearly independent vectors in an inner product space. Given a set of vectors $\{\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n\}$, the process constructs an orthogonal (or orthonormal) set $\{\mathbf{u}_1, \mathbf{u}_2, \ldots, \mathbf{u}_n\}$ such that each $\mathbf{u}_i$ is orthogonal (or orthonormal) to all preceding vectors and has the same span as $\mathbf{v}_i$.