2.2 Inner Product Spaces

In the previous section, we defined vector spaces. Now, we will define inner product spaces, which are vector spaces with an additional structure called an inner product.

Table of Contents

• Table of Contents
• Definition
• Summary and Next Steps

Definition

An inner product is a generalization of the familiar dot product in . It is a function that takes two vectors and returns a scalar, satisfying certain properties.

The dot product in is defined as a function such that for any two vectors and in ,

To generalize to complex vector spaces, we need to be careful with applying these properties. Notice that

meaning either or is negative, violating the positive-definiteness property. To account for this, we make it such that the inner product is linear in one argument and conjugate-linear in the other. The choice of which argument is linear and which is conjugate-linear is arbitrary, but we will follow the physics convention of making the second argument linear. Thus, the linearity property becomes

As a direct consequence of this, we have

As we will see later, the inner product is unique for finite-dimensional vector spaces. As such, we can drop the and opt for a different notation. Different texts use different notations, but we will use the Dirac notation, which is common in quantum mechanics:

(This notation has many advantages, including a built-in duality between vectors and linear functionals, which we will discuss later.) With all these definitions, we can now formally define an inner product.

Definition 2.2.1 (Inner Product, Inner Product Space)

An inner product on a complex vector space is a function such that for any vectors and any scalars ,

1. (conjugate symmetry),
2. (linearity in the second argument),
3. and (positive-definiteness).

A vector space equipped with an inner product is called an inner product space.

If you have studied any relativity, you might recognize that these properties are somewhat problematic. Null vectors, which are non-zero vectors with zero norm, exist in relativity. Moreover, the metric in relativity can be negative, which violates the positive-definiteness property. These issues arise because the metric in relativity is not positive-definite. Instead, it is a pseudo-inner product, which relaxes the positive-definiteness property to allow for null vectors and negative norms.

As the complex inner product is not actually bilinear, we need to be careful when manipulating inner products. It is more accurate to say that the inner product is sesquilinear (meaning "one and a half linear"), or Hermitian.

Box 2.2.2 (Shorthand Notations)

We can use the following shorthand notations for inner products:

Example 2.2.3 (Examples of Inner Products)

Here are some examples of inner products:

• The standard inner product on is defined as follows. For any two vectors and in ,

• For , the space of infinite sequences of complex numbers, we can define the inner product as follows. For any two sequences and in ,

  provided that the series converges.

• Suppose and are two functions in (the space of complex polynomials in ). We can define the inner product as follows:

  Here, is a weight function that is positive and integrable on the interval .

• Suppose and are two functions in (the space of continuous functions on the interval ). We can define the inner product as follows:

Definition 2.2.4 (Orthogonal, Normal, Orthonormal)

Definition 2.2.4 (orthonormal) Two vectors and in an inner product space are said to be orthogonal if their inner product is zero, i.e., . A vector is said to be a normal vector if its norm is one, i.e., . A basis of an inner product space is said to be orthonormal if

where is the Kronecker delta, which is if and otherwise.

Example 2.2.5 (Inner Product of Direct Sums)

Suppose and are inner product spaces with inner products and , respectively. Let be the direct sum of and . We can define an inner product on as follows:

Also, if we define the identification and , then and are orthogonal subspaces of .

Example 2.2.6 (Examples of Orthonormal Bases)

Here are some examples of orthonormal bases:

• In or , the standard basis defined as

  where the is in the -th position, is an orthonormal basis with respect to the standard inner product.

• In , the set of functions is an orthonormal basis with respect to the inner product defined as

As one can see, orthonormal bases are very useful in inner product spaces. They allow us to easily compute inner products and norms of vectors by simply taking the coefficients of the vectors in the basis. If we have another basis that is not orthonormal, we can always use the Gram-Schmidt process to convert it into an orthonormal basis.

The Gram-Schmidt process is a method for orthonormalizing a set of vectors in an inner product space. Suppose we have a basis of an inner product space .

Consider the first two vectors and . We can define the first orthonormal vector as the normalization of :

Next, if we decompose into a component parallel to and a component orthogonal to , we have

and we can find by projecting onto :

Thus, we have

We can then normalize to get the second orthonormal vector :

For the third, we can repeat the process. We decompose into components parallel and orthogonal to the subspace spanned by and :

where

Thus, we have

We can then normalize to get the third orthonormal vector :

More generally, for the -th vector , we can decompose it into components parallel and orthogonal to the subspace spanned by :

where

Thus, we have

so

Repeating this process for all vectors in the basis , we obtain an orthonormal basis of .

Also, important to the discussion of inner products is the Schwarz inequality.

Theorem 2.2.7 (Schwarz Inequality)

For any vectors and in an inner product space ,

Equality holds if and only if and are linearly dependent (i.e., one is a scalar multiple of the other).

---

Proof. Consider the vector , where is a scalar to be determined later. By the positive-definiteness property of the inner product, we have

Expanding the left-hand side, we get

Choosing (assuming ), we have

which implies

If and are linearly dependent, then for some scalar , and we have equality. Conversely, if we have equality, then , which implies , and thus and are linearly dependent.

---

It is certainly possible to visualize the Schwarz inequality in or . Essentially, it states that if you project one vector onto another, the length of the projection cannot exceed the length of the original vector. However, the power of our approach is that it works in any inner product space, even those that are infinite-dimensional and/or not easily visualizable.

Definition 2.2.8 (Norm, Normed Vector Space)

The norm of a vector in an inner product space is defined as

The norm satisfies the following properties for any vectors and any scalar :

1. and (positive-definiteness),
2. (absolute homogeneity),
3. (triangle inequality).

Also, we write the shorthand to mean .

A vector space equipped with a norm is called a normed vector space. Normed vector spaces are more general than inner product spaces, as not all norms can be derived from an inner product. However, every inner product space is a normed vector space, as the norm can be derived from the inner product.

Also, a norm induces a metric on the vector space, defined as

Theorem 2.2.9 (Inner Product Characterization via Parallelogram Law)

If is a normed vector space with norm , then is an inner product space if and only if the norm satisfies the parallelogram law

for all vectors .

---

Proof. () Suppose is an inner product space with inner product . Then, for any vectors , we have

() Suppose is a normed vector space with norm satisfying the parallelogram law. To skip some tedious algebra, we can use the polarization identity to define an inner product as follows:

It can be verified that this inner product satisfies all the properties of an inner product.

---

By the way, why is it called the "parallelogram law"? Because in , if you have two vectors and originating from the same point, they form a parallelogram. If you take the sum of the squares of the lengths of the diagonals of the parallelogram, it equals twice the sum of the squares of the lengths of the sides. This is a direct consequence of the law of cosines.

Theorem 2.2.10 (Inner Product Existence)

Every finite-dimensional vector space can be equipped with an inner product.

---

Proof. Let be a finite-dimensional vector space with basis . We can define an inner product on by specifying the inner products of the basis vectors. Specifically, given two vectors and , their inner product is obviously

Then, we just need to define for all . This is arbitrary; any choice will do, as long as the resulting inner product satisfies the properties of an inner product. Let's label the matrix such that . Since , we have , so (i.e., is Hermitian).

Also, as the inner product is positive-definite, we have if and only if . This makes our matrix have a determinant that is non-zero, as if , then there exists a non-zero vector such that , which implies .

In summary, we can define an inner product on by choosing any Hermitian matrix with a non-zero determinant and defining . Then, the inner product of any two vectors in is given by

Thus, every finite-dimensional vector space can be equipped with an inner product.

---

The matrix is called the metric tensor matrix.

Example 2.2.11 (Norms on Cⁿ)

Consider . The usual inner product and its associated norm are given by

where . Additionally, the metric function induced by this norm is

Another norm on is

for any positive real number . This is called the -norm.

To verify that this is indeed a norm, we need to check the three properties of a norm:

1. Positive-definiteness: and . This is clearly true, as the sum of non-negative numbers is non-negative, and the sum is zero if and only if all are zero.

2. Absolute homogeneity: for any scalar . This is also true, as

3. Triangle inequality: . This is a bit more involved, but it can be proven using Minkowski's inequality, which states that for any sequences of real numbers and ,

   We can apply this inequality to the sequences and to obtain the triangle inequality for the -norm. Thus, the -norm is indeed a norm on .

This norm is not induced by an inner product unless , for it only satisfies the parallelogram law when .

Summary and Next Steps

In this section, we defined inner product spaces, which are vector spaces equipped with an inner product. We discussed the properties of inner products, norms, and metrics, and we saw that every finite-dimensional vector space can be equipped with an inner product.

Here are the key points to remember:

• An inner product is a function that takes two vectors and returns a scalar, satisfying conjugate symmetry, linearity in one argument, and positive-definiteness.
• An inner product space is a vector space equipped with an inner product.
• The norm of a vector is derived from the inner product and satisfies positive-definiteness, absolute homogeneity, and the triangle inequality.
• A norm induces a metric on the vector space.
• Every finite-dimensional vector space can be equipped with an inner product.

Now, our main obstacle is the fact that we are restricted to one vector space. How does one traverse between different vector spaces? This is where the concept of linear maps comes in, which we will discuss in the next section.