Skip to main content

Vectors: Part 2

This part covers more of the fundamental operations on vectors, introducing the different ways to multiply vectors together. Additionally, we'll explore one application of these operations, which is to formulate the equations of planes in 3D space.

Table of Contents​

Vector Multiplication: The Dot Product/Scalar Product​

Until now, we have introduced these operations on vectors:

  • Addition:
  • Subtraction:
  • Scalar Multiplication:

We will now introduce a new operation: the dot product, a way to multiply two vectors together. The dot product is sort of the linear algebra equivalent of multiplication. It's a scalar quantity, so sometimes it's called the "scalar product."

The usual way courses introduce the dot product is by defining it as:

However this definition is not very intuitive, and it doesn't give us much insight into what the dot product actually does.

One way to think of the dot product is to think about the "projection" of one vector onto the other.

Hold a pencil, and observe its shadow on the table/ground. As you rotate the pencil, the shadow changes in length. The length of the shadow is the projection of the pencil onto the table.

We can also show this graphically:

The dot product is then defined as:

The length of the projection can be calculated through trigonometry:

And therefore, the dot product can also be calculated as:

The dot product, due to this projection property, is helpful to measure the "similarity" between two vectors.

When and are pointing in the same direction, the dot product is positive.

When they are perpendicular, the dot product is zero, since the projection of one vector onto the other is zero. It's like holding the pencil perpendicular to the table, you get no shadow.

When they are pointing in opposite directions, the dot product is negative.

Magnitude from the Dot Product​

The dot product can also be used to calculate the magnitude of a vector:

Algebraic Proof of Magnitude from the Dot Product​

Geometric Proof of Magnitude from the Dot Product​

Recall that the dot product of with is the projection of onto times the length of . If both are instead, then the dot product is the projection of onto itself times the length of .

Since the projection of onto itself is just... itself, the dot product is the length of times the length of which is .

Commutativity of the Dot Product​

The dot product is commutative, meaning that .

Let and .

Algebraic Proof of Commutativity of the Dot Product​

Since multiplication is commutative, for all .

Therefore, .

Geometric Proof of Commutativity of the Dot Product​

(Coming soon.)

Distributivity of the Dot Product​

The dot product is distributive over addition, meaning that .

Let and .

Algebraic Proof of Distributivity of the Dot Product​

The sum of the two vectors is:

Hence, the dot product of with the sum is:

Thus proving the distributivity of the dot product.

Geometric Proof of Distributivity of the Dot Product​

(Coming soon.)

Associativity of the Dot Product with Scalar Multiplication​

The dot product is associative with scalar multiplication, meaning that .

Let and .

Algebraic Proof of Associativity of the Dot Product with Scalar Multiplication​

The scalar multiple of is:

Hence, the dot product of the scalar multiple with is:

Thus proving the associativity of the dot product with scalar multiplication.

Geometric Proof of Associativity of the Dot Product with Scalar Multiplication​

(Coming soon.)

Cauchy-Schwarz Inequality​

The Cauchy-Schwarz Inequality states that for any two vectors and in :

This is a very important inequality in mathematics.

Algebraic Proof of the Cauchy-Schwarz Inequality​

Consider a function .

The function is the square of the distance between the vectors and . Why is there? Because we want to find the value of that minimizes the distance between the vectors.

For a vector , the magnitude is .

Therefore, for all .

Recall the magnitude from the dot product in Equation and substitute into :

Because of all the properties of the dot product, we can basically treat this as a regular binomial expansion:

Oh look, a quadratic! Since can be any real number, we can plug in (the vertex):

Substituting the values of , , and from Equation into Equation :

Note that we used various properties of the dot product to simplify the quadratic, like the commutativity and associativity with scalar multiplication.

Additionally, note that in the last step, we took the principal square root of both sides, hence the absolute value.

Which proves . To prove , we need to show that the equality holds when .

This aligns with the second part of the Cauchy-Schwarz Inequality.

Geometric Proof of the Cauchy-Schwarz Inequality​

This is actually really easy to prove geometrically, and is an excellent example of the various approaches one can take to interpret a mathematical statement.

When you project one vector onto another, the length of the projection is always less than or equal to the length of the vector.

Therefore, the projection of onto is less than or equal to the length of , proving the inequality.

In addition, when the vectors are parallel, the projection is equal to the length of the vector. This is when the equality holds.

Vector Triangle Inequality​

The Triangle Inequality states that for any two vectors and in :

Algebraic Proof of the Triangle Inequality​

Let's build on top of the Cauchy-Schwarz Inequality to prove the Triangle Inequality.

Recall that the Cauchy-Schwarz Inequality states that for any two non-zero vectors and in :

Geometric Proof of the Triangle Inequality​

Once again, the Triangle Inequality is trivial to prove geometrically.

Consider two vectors and :

Consider the triangle formed by the vectors , placed at the tip of , as well as their sum .

Now consider the regular triangle inequality, which states that the sum of the lengths of two sides of a triangle is greater than or equal to the length of the third side.

The sum of the lengths of and is greater than or equal to the length of .

Therefore, the Vector Triangle Inequality holds, given the regular triangle inequality.

A More Rigorous Geometric Proof of the Triangle Inequality

To prove the regular triangle inequality, consider the angle between and , labeled . Splitting the triangle into two right triangles:

The new line we introduced is the height of the triangle, which we'll call . is always positive. It splits the triangle into two right triangles.

The vector is split into two vectors: and .

By the Pythagorean Theorem:

Subtracting from both sides of both equations yields:

Since is positive by definition:

The same applies for . Hence we have a new system of inequalities:

Adding the inequalities in Equation :

Thus proving the Vector Triangle Inequality.

Equivalence of Definitions: Angle Between Vectors​

The two definitions of the dot product are equivalent. To show this, we first need to define the angle between two vectors.

Given two non-zero vectors and in , we construct a triangle with the vectors as sides:

Is the triangle always possible to construct?

We need to show that a triangle like this is always possible to construct for any two vectors.

If the sum of two sides is less than the third side, then the triangle is not possible. So:

  1. If , then the triangle is not possible.
  2. If , then the triangle is not possible.
  3. If , then the triangle is not possible.

We can show that the triangle is always possible to construct by showing that none of these conditions are met.

Recall the triangle inequality:

Therefore:

Both and are equivalent since they both measure the distance between the same vectors.

This means that the second and third conditions are never met.

For the first condition, consider manipulating and using the triangle inequality:

Therefore, the triangle is always possible to construct.

Define the angle between the vectors as . Recall the cosine rule:

Substitute , , and :

Recall that the square of the magnitude of a vector is the dot product of the vector with itself in Equation :

Expanding the left side:

Subtracting from both sides:

Therefore, the dot product can be defined as the product of the magnitudes of the vectors and the cosine of the angle between them.

When , , and the dot product is zero. They are perpendicular and are hence called orthogonal.

When , , and the dot product is the product of the magnitudes of the vectors.

When , , and the dot product is the negative of the product of the magnitudes of the vectors.

Alternate Derivation of the Dot Product​

Here's an alternative derivation of the dot product based on its properties, that I found in a MIT Physics textbook.

Consider two vectors and :

We can express both vectors as linear combinations of the unit vectors , , and :

The dot product of the vectors is:

Let's consider how the dot product acts on the unit vectors:

  1. On the same unit vector, both are parallel, and the dot product is the product of the magnitudes, i.e. .
  2. On different unit vectors, they are perpendicular, and the dot product is .

By the distributive property, we can expand the dot product of the two vectors. All combinations of the unit vectors will yield except for the same unit vectors, which will yield :

Aligning with the definition of the dot product.

While we have used as an example, this derivation holds for any .

However, this derivation does not provide much geometric insight, nor does it generalize well to other vector spaces.

Cross Product/Vector Product​

When we introduced the dot product, we mentioned that it was only one of two ways to multiply vectors. The other way is through the "cross product".

The main difference is that unlike the dot product, the cross product yields a vector, not a scalar. This is why it's sometimes called the "vector product".

However, the cross product is much more limited;

  • A dot product works for any .
  • A cross product only works for .

The cross product of two vectors is something that's orthogonal (perpendicular) to both vectors.

The definition of the cross product is below, but it's best not to just memorize it:

For each row in the cross product, we ignore the corresponding component in the original vectors and multiply the other two components. So for example, in the first row, we ignore and and multiply and .

Now that we have the definition, let's consider some properties of the cross product.

What Does the Cross Product Represent?​

The cross product of two vectors is a vector that's orthogonal to both vectors.

That means, given two vectors, if you construct a plane with the two vectors as sides, the cross product is perpendicular to that plane, it "sticks out" of the plane.

This is why the cross product is only defined in ; in , there's no third dimension for the cross product to "stick out" into.

However, one might ask, for any plane, there's two vectors perpendicular to it (one above and one below). Which one is it?

To answer this, the orientation of the cross product is determined by the right-hand rule.

If you curl your right hand from the first vector to the second vector, your thumb points in the direction of the cross product.

The Magnitude of the Cross Product​

We've shown the direction of the cross product, but since the cross product is a vector, it also has a magnitude.

To find the magnitude of the cross product, essentially we create a parallelogram with the two vectors as sides. The area of this parallelogram is the magnitude of the cross product.

We can make use of some geometry to find the magnitude of the cross product. Recall that the area of a parallelogram is the base times the height (you can show this by cutting off a triangle and moving it to the other side).

We can set the base to one of the vectors (we'll use ) and calculate the height using the angle:

We can use trigonometry to find the height:

The area of the parallelogram is the base times the height. Therefore:

Relationship Between the Cross Product and its Geometry​

We've shown the different ways to interpret the cross product. However, we have not yet proven the equivalence of the different interpretations.

Our goal is to show that:

First, let's consider the square of the magnitude of the cross product. Recall from Equation that the square of the magnitude of a vector is the dot product of the vector with itself:

According to the dot product definition in Equation , the dot product of two vectors is the sum of the products of their corresponding components. As such, the dot product of with itself is:

That's a lot of terms, but we can simplify it by grouping the terms:

Keep note of that for now - we will shift our focus to another equation:

We are going to do something that no sane person would do; we are going to square both sides:

Miraculously, the right side of the equation is very similar to the right side of Equation .

Finally, let's add the two equations. The red terms (fortunately) cancel out, so we are left with:

Recall that the magnitude of a vector is the square root of the sum of the squares of its components, which is exactly what we have here. Hence:

Rearranging, we get:

Taking the square root of both sides, we get:

And we're done.

Distributive Property of the Cross Product​

The cross product also satisfies the distributive property:

This can be proven by expanding both sides of the equation and showing that they are equal, which for now, I'm not going to do.

Compatibility with Scalar Multiplication​

(Anti)Commutative Property of the Cross Product​

Recall how we described the direction of the cross product: if you curl your right hand from the first vector to the second vector, your thumb points in the direction of the cross product.

However, this means that if you curl your right hand from the second vector to the first vector, your thumb points in the opposite direction. Hence, the direction depends on the order of the vectors. In other words:

When you switch the order of the vectors, the cross product changes sign. This is known as the "anticommutative property".

Cross Product of the Basis Vectors​

Let , , and be the basis vectors in .

Recall that the cross product of two vectors is a vector that's orthogonal to both vectors. If we draw the basis vectors:

The magnitude of the cross product of two vectors is the area of the parallelogram formed by the two vectors. Since they all have length and are orthogonal to each other, the area of the parallelogram is .

Hence, using that, along with the right-hand rule, we can find the cross product of the basis vectors:

One way to remember this is to just say "ijkijk", and in that order, you get the cross product of the vectors.

Alternate Derivation for Cross Product​

There's another way to derive the algebraic expression for the cross product, which I found in the same MIT Physics textbook.

We can make some definitions:

Then, the cross product of and is, based on the distributive property of the cross product:

We can find the first term; let :

We can expand this using the distributive property of the cross product:

Similarly, we can find the other terms:

Adding them all up, we get:

Or, alternatively:

More on the Angle Between Vectors​

We've shown the two different ways to multiply vectors, the dot product and the cross product. Both of them have definitions that involve the angle between the vectors. As such, we can use them to find the angle between two vectors.

Angle Between Vectors from the Dot Product​

Recall the definition of the dot product:

Rearranging, we get:

Therefore, the angle between two vectors is:

Angle Between Vectors from the Cross Product​

Recall the definition of the (magnitude of the) cross product:

Rearranging, we get:

Therefore, the angle between two vectors is:

Comparison of the Dot and Cross Products​

We already know that the dot product is a scalar and the cross product is a vector. Let's explore some other differences between the two.

When the Vectors are Collinear​

Recall that the cross product essentially measures the area of the parallelogram formed by the two vectors. Well, if they're collinear, then the parallelogram gets "squished" into a line, and the area becomes zero. Therefore, the cross product of two collinear vectors is zero.

However, the dot product is not affected by this. Since the dot product is the product of a "projection" of one vector onto the other, it is actually the highest when the vectors are collinear. Using the pencil analogy, if you put the pencil flat on the table, the shadow is the longest.

When the Vectors are Perpendicular​

When the vectors are perpendicular, the dot product is zero, as the projection of one vector onto the other is zero.

However, the cross product is not zero. When the vectors are perpendicular, the cross product is the highest, since the height of the parallelogram is exactly equal to the magnitude of the vector.

Graphical Analysis of the Products​

One other way to compare the dot and cross products is to consider their graphs.

Recall the definitions of the products based on the angles:

One of them has a term, and the other has a term. As such, we can make use of the properties of the sine and cosine functions to compare the two.

We can graph the two functions for :

The red graph is the cosine function, and the blue graph is the sine function.

From the graph, we can see that the cosine function is at its highest when the angle is zero, and the sine function is at its highest when the angle is . This is in line with our previous observations.

Summary Table of the Dot and Cross Products Based on the Angle​

Equation of a Plane in R3​

Let's consider a plane in .

A plane is a two-dimensional surface that extends infinitely in all directions. But before that, let's consider the equation of a line in , then later, we can extend it to .

One way to describe a line in is to use the general form: . Before proceeding, it's worth playing around with the equation to understand what it represents.

is the coefficient of . It affects the slope, as well as the -intercept of the line.

Similarly, is the coefficient of . It affects the slope, as well as the -intercept of the line.

Changing does not affect the -intercept, and changing does not affect the -intercept.

Options

Consider two points on the line, one at and the other at .

We can calculate the intercepts of the line using the general form:

Using two points, we can calculate the slope of the line:

Next, let's consider the normal vector to this line. You can think of it as rotating the line by . To rotate by , we can swap the and components and negate one of them. A visual representation of this is:

Since the normal vector is perpendicular, the slope of the normal vector can be calculated as:

Recall that the slope is the ratio of the change in to the change in . Hence, given a , then . As such, the normal vector is:

Extension to R3​

First, let's consider how to describe a plane in . We've used the general form of the equation of a line in to describe a line. Given the equation , we will need another dimension to describe a plane;

We can interpret this equation geometrically as follows:

  • At , and .
  • At , and .
  • At , and .

We can use the same logic used in to interpret and analyze the equation of a plane in .

Firstly, we can extend the equation of a line to as . By adding a term, we add a new "degree of freedom" to the equation, allowing us to describe a plane.

We will now look at the equation of a plane in in a different way.

Choose a vector on the plane and a normal vector to the plane. So is perpendicular to the plane, and any vector parallel to the plane is perpendicular to .

When two vectors are perpendicular, their dot product is zero. So, for any vector, , parallel to the plane (think of it as drawing on top of the plane):

Define an arbitrary vector on the plane.

Then, if we get the vector between and , i.e. (the purple vector in the diagram), it's parallel to the plane. Therefore, is perpendicular to , and as such,

We can expand into and and into their components:

This closely resembles the original equation of a plane in , .

Hence, with a point on the plane and a normal vector to the plane, the equation of the plane is:

For example, consider the plane with a point and a normal vector . Then, for a point on the plane:

Alternatively, we can plug in the values of and directly into the equation of a plane in :

This is very useful for anything involving three-dimensional geometry.

Normal Vector from a Plane Equation​

We know how to find the equation of a plane given a point on the plane and a normal vector to the plane. We can also do this in reverse; given the equation of a plane, we find the normal vector to the plane.

Recall the point-normal equation of a plane in :

Let's set and . We can then expand the dot product:

Recall that the equation of a plane in is . In that sense:

Recall that the normal vector to the plane is . Therefore, the normal vector to the plane is . This is similar to the normal vector to a line in ; .

Example Problem: Distance from a Point to a Plane​

Let a plane in be defined by the equation .

Let a point be an arbitrary point in the same space.

Find the shortest distance, , from the point to the plane.

The shortest distance is when the line from the point to the plane is perpendicular to the plane:

Therefore, the vector from the point to the plane is parallel to the normal vector of the plane, i.e. .

We shall first break the vectors into their components:

We are given the equation of the plane as . Let . Then,

Let be the magnitude of .

Remember that is the defining point of the plane, so it naturally lies on the plane. We can use some geometry to find the shortest distance from the point to the plane.

First, let's draw a diagram. Although it's in 2D, the concept is identical.

We can go about this in two ways. Let's first consider the trigonometric approach.

By SOHCAHTOA:

There's also another way to think about this. Recall that the dot product of two vectors and is the projection of onto multiplied by the magnitude of . In our context, to get , we can project onto . Since the dot product is the product of the projection and magnitude, we need to divide by the magnitude of .

As such,

Plugging in Equations and :

Recall from Equation that . Therefore:

This is the shortest distance from the point to the plane. To demonstrate this, let's consider an example.

Let a plane in be defined by the equation .

To find the distance between a point and the plane:

Summary​

We have covered a lot of ground in this chapter, primarily relating to the dot and cross products of vectors. We have proved the properties of the dot and cross products both algebraically and geometrically.

The dot product is a scalar product with two interpretations:

  • Geometrically, it is the product of the projection of one vector onto the other and the magnitude of the other vector.
  • Algebraically, it is the sum of the products of the corresponding components of the two vectors.

The dot product is:

  • Commutative:
  • Distributive:
  • Linear:

On the other hand, the cross product is a vector product with two interpretations:

  • Geometrically, it has the magnitude as the area of the parallelogram formed by the two vectors. The direction is perpendicular to the plane formed by the two vectors.
  • Algebraically, it is a certain combination of the components of the two vectors. This will be useful in the next chapter when we discuss matrices.

The cross product is:

  • Anticommutative:
  • Distributive:
  • Linear:

We also discussed the angle between two vectors and how it relates to the dot and cross products.

Finally, we applied the concepts of the dot and cross products to find the equation of a plane in and the shortest distance from a point to a plane.

The equation for a plane in is:

In vector form, the equation of a plane is:

Where is any point on the plane, is a point on the plane, and is the normal vector to the plane.

The equation for the shortest distance from a point to a plane is: