Skip to main content

Gradient

The gradient of a function has multiple interpretations and uses in mathematics. We will explore the gradient in the context of functions of multiple variables.

Table of Contents​

Introduction​

As previously stated, there are a few ways to think about the gradient of a function.

Purely computationally, the gradient is essentially a collection of all the partial derivatives of a function.

So for a function , the gradient is:

Consider the function . The gradient of is then:

The gradient is often denoted as , which is pronounced "nabla f" or "del f".

Hence, to create a more general definition, we can define the gradient of a function as:

Notice that the gradient is a vector, and its dimensions match the number of inputs to the function. In terms of basis vectors, the gradient can be written as:

Recall that the partial derivative is an "incomplete" way to measure the rate of change. In this sense, the gradient can be thought of as the "full" derivative of a function of multiple variables.

The Nabla​

One convenient way to think about the gradient is to consider the symbol as a vector of partial derivative operators. It's easier to understand this with an example. For a function , is:

Then, the gradient of is simply a vector multiplication:

Gradients in the Context of Graphs​

Outside of the computational context, there's a graphical way to think about the gradient. Consider this simple function , and its gradient:

We can plot this as a vector field.

In this graph, the gradient is represented as a vector field, more commonly known as a "gradient field".

One thing to note is that the gradient points in the "direction of the steepest ascent" of the function. So if you were to walk along the surface of the function, you would be walking in the direction where the function increases the fastest. This is not immediately obvious, but will become more apparent in light of directional derivatives.

Gradient in Contour Plots​

It's important to understand how the gradient relates to contour plots.

Consider the function . The contour plot of this function is:

The gradient of is:

Like before, we can plot this as a gradient field along with the contour plot:

The important thing to notice is that the vector appears to point perpendicular to the contour lines. To see why this is the case, zoom in on 2 contour lines:

Recall that the gradient points in the direction of the steepest ascent. Instead of thinking of the steepest ascent, consider which direction the function increases from to in the shortest distance. This is essentially considering the shortest path between the two contour lines. Since the contour lines are almost parallel to each other, the gradient vector will be perpendicular to the contour lines.

Summary and Next Steps​

In this section, we introduced the concept of the gradient of a function.

Here are the key points to remember:

  • The gradient of a function is a vector of partial derivatives.
  • The gradient points in the direction of the steepest ascent of the function.
  • The gradient is perpendicular to the contour lines of the function.
  • The gradient can be thought of as the "full" derivative of a function of multiple variables.

Next, we will explore the directional derivative, which is a generalization of the derivative in a specific direction.