Differentiation

Differentiation is the process of finding the derivative of a function. Essentially, while we know how to find the slope of a straight line, the derivative allows us to find the slope of a curve at any point.

In other words, what is the speed right now? What is the rate of change of a function at a certain point?

Table of Contents

• Table of Contents
• Conceptual Introduction
  • Derivative from Limits
  • Delta and d
  • Is the Derivative a Fraction?
  • Example Problem: Inferring the Derivative
• Tangent Line Equations
• Differentiation of Sums of Functions
• Differentiation of Products of Functions
• Chain Rule
  • Proof
• Powers
• Derivative of Trigonometric Functions
  • Proof of Derivative of Sine
• Differentiability and Continuity
• Higher Order Derivatives
• Implicit Differentiation and the Natural Log
• Quotient Rule
• Derivatives of Exponential Functions
  • Example: Derivative of x^x
• Derivatives of Logarithmic Functions
• Derivatives of Inverse Trigonometric Functions
• Derivatives of Hyperbolic Functions
• Summary
  • Derivative Cheat Sheet

Conceptual Introduction

Imagine a car moving along a road. The car's position at time is given by a function . Let's say we graph the car's position as a function of time:

To find the car's velocity, , at time , we can find the slope of the line tangent to the graph of at . In this case, since it is a straight line, we can simply take two points and find :

The slope of the line is , so the car's velocity at any given time is .

However, in most cases the function will not be a straight line, so we need a more general way to find the slope, which is where the derivative comes in.

Derivative from Limits

Instead of thinking about the slope of a function as a whole, a derivative deals with the slope at a singular point. Let's say we have a function :

Suppose we want to find the slope of the function at . What this means is that if we construct line that's tangent to the graph of at , what is the slope of that line?

To find the slope of the tangent line at , we can take two points on the graph of that are very close to . Let's say we take and :

From the above, we can see that the slope of the tangent line at can be expressed as follows:

As approaches , the slope of the tangent line approaches a certain value. This value is the derivative of the function at .

As such, the derivative of a function at a point is defined as:

Another way to write this is to have the second point as some other variable , so that the derivative at is:

Symbols

• Some textbooks use instead of for some reason.
• The derivative of a function is often denoted as , pronounced "f prime of x".
• is known as Leibniz notation, while is known as Lagrange notation.

Delta and d

In the above, we used to represent a small change in , and likewise for . As the change approaches , another symbol is used to represent this infinitesimally small change: and .

When using , the concept of it approaching is "built-in" to the symbol, so we can write the derivative as:

If we want to differentiate, let's say, , we can write it as:

The fraction on the left is the differential operator.

Is the Derivative a Fraction?

Back in Newton's and Leibniz's time, the derivative was defined in terms of infinitesimals, which are quantities that are smaller than any finite quantity but not zero. This led to the idea that the derivative was a fraction, with and being the numerator and denominator. It is common in introductory calculus courses to treat the derivative as a ratio, especially due to its intuitive nature. In the future, we will see that treating the derivative as a fraction is very beneficial in many cases, like implicit differentiation, related rates, substitution, separation of variables, and more.

However, this notion of the derivative as a fraction is not entirely accurate. In modern mathematics, calculus is built on Analysis, which is the rigorous study of limits and continuity. The derivative, as we've shown, is defined as a limit, not a fraction. The concept of an infinitesimal also fell out of favor due to its lack of rigor, and is replaced by the Epsilon-Delta definition of limits (except in non-standard analysis and smooth infinitesimal analysis).

When we manipulate them like fractions, what we're really doing "under the hood" is applying the rules of differentiation, like its linearity, chain rule, and product rule. Limits often play nicely with these rules, which is why treating the derivative as a fraction works a lot of the time.

A really simple example of where this heuristic fails is in partial derivatives, but we're getting ahead of ourselves.

Example Problem: Inferring the Derivative

is a function that is differentiable throughout its entire domain.

It has these properties:

1. for any and .
2. .

Using the above information, as well as the definition of the derivative, find .

(Source)

This is a nice problem because it requires thinking outside the box.

Let's apply the definition of the derivative:

From the first property, we can rewrite as:

Substituting this into the derivative:

We can utilize the linearity of the limit to split it into two parts:

Using the second property, we know that the second limit is .

Tangent Line Equations

As we've seen, the derivative of a function at a point is the slope of the tangent line at that point. Using that slope, as well as a point, we can find the equation of the tangent line.

Recall the point-slope form of a line:

where is the slope of the line, and is a point on the line.

Derivation of the Point-Slope Form

A geometric way to think about this is to consider any point on the line, and then measure the distance from that point to the specific point .

First, consider the point-slope form of a line:

Next, consider adding an arbitrary point on the line:

• The vertical distance between the two points is .
• The horizontal distance between the two points is .

Recall that the slope of a line is the ratio of the vertical change to the horizontal change:

Hence:

The (more boring) algebraic way to find the equation of the line is to use the slope formula:

Given a point and a slope , the equation of the line is:

We need to find the -intercept . Since the line passes through :

Therefore:

Thus, the equation of the line is:

And rearranging gives:

We can set the point to be some and , and the slope to be :

Differentiation of Sums of Functions

The derivative of a sum of functions is the sum of the derivatives of the functions. To demonstrate this, let's find the derivative of the function .

Once again, let's find the derivative at . Look at the figure below:

The straight line is the part, and the curve is the whole function. The two arrows represent that at that point, the total value of the function is the sum of the two terms, with each arrow representing one term.

Recall that to take the slope, you imagine another point some distance away and measure how much changes:

The key idea is that this green can be "split" into two parts: one from the term and one from the term.

Therefore:

Or more generally,

Differentiation of Products of Functions

The derivative of a product of functions is a bit more complicated.

However there's still a neat visual way to understand it. Let's say we have two functions and :

Generally, to visualize products, we can think of the area of a rectangle.

Options

Now imagine that increases by a small amount . The change in the product, , can be thought of as the change in the area of the rectangle.

Options

We can split the change in area into three parts:

Options

Therefore, the change in the product can be expressed as:

The last term, , will approach because it is the product of two infinitesimally small changes. Therefore, the derivative of the product of two functions is:

Another way to write this is:

Chain Rule

The chain rule is a way to differentiate composite functions.

Let's say we have two functions and , and we want to find the derivative of .

To do this, we can think of as a function of , and then differentiate that function with respect to .

Conceptually, the chain rule can be thought of as a way to "break down" the function into smaller parts.

Imagine a knob that you can turn to change , and that turns another knob that changes , which in turn changes .

To measure how much changes when you turn the first knob, you can measure how much changes, and then how much changes when you change .

Another way to think of it is through algebraic manipulation, assuming :

Proof

To prove the chain rule, let's say we have a function and , assuming that and are differentiable at some point .

The derivative of with respect to is:

We can rewrite this as:

Since is differentiable at , as , . Thus:

Powers

The derivative of a power function is:

To show this, let's make a general power function and apply the limit to it:

The term is a binomial. In an expanded binomial, you're essentially choosing every possible combination of terms from the two binomials. For example, expands to because:

• There's one way to choose from and :
• There are three ways to choose from and : , , and
• There are three ways to choose from and : , , and
• There's one way to choose from and :

The formalization of this is the binomial theorem:

Where is the binomial coefficient, which is what you get when you choose items from items.

Applying this to the limit:

The key part is that after canceling out the terms, the only term that remains is the one where . This is because the other terms have a in them, which will go to as approaches .

Therefore, the derivative of is:

What about non-integers?

The power rule works for all real powers, not just integers. Here's a proof.

Let's say we have a power function:

To find the derivative, we can use the chain rule:

A few things to note:

• This relies on the fact that the derivative of is .
• This also relies on the fact that the derivative of is itself .

Derivative of Trigonometric Functions

The derivatives of the trigonometric functions are as follows:

We'll just look at the first one, , to show how it's derived.

(TODO)

The derivative of can be derived in a similar way.

Derivatives using Quotient Rule

The quotient rule is discussed later here.

The derivatives of , , , and can be derived using the quotient rule. For example, to find the derivative of :

Plugging in and :

Proof of Derivative of Sine

The proof involves two assumed limits:

The first one is proved using squeeze theorem, and the second one can be proved simply like this:

To prove that the derivative of is , we can use the limit definition of the derivative:

Next, we can use the angle sum formula for sine:

Substituting this into the limit definition:

Differentiability and Continuity

The differentiability of a function at a point refers to whether the function has a derivative at that point.

The important thing to note is this: if a function is differentiable at , then it is continuous at .

However, the converse is not necessarily true: a function can be continuous at a point but not differentiable at that point.

As previously mentioned, a function is continuous at a point if:

A function is differentiable at a point if:

To prove that differentiability implies continuity, we'll use a slightly different form of the limit definition of the derivative:

Let's suppose we want to find this limit below, assuming is differentiable at :

Therefore:

Hence, if is differentiable at , then it is continuous at .

Higher Order Derivatives

The derivative of a function is itself a function, so it can be differentiated again.

The second derivative of a function is denoted as or , the latter being sort of a notation hack.

The second derivative is the derivative of the first derivative, and the third derivative is the derivative of the second derivative, and so on.

Higher order derivatives can be denoted as , where is the order of the derivative.

Implicit Differentiation and the Natural Log

Implicit differentiation is a way to differentiate functions that are not explicitly defined.

For example, consider the equation of a circle:

To differentiate this, we can treat as some constant and as some function . Just like how we took small changes in to find the derivative of , we can take small changes in and to find the derivative of .

Remember that should be constant, so .

We can also use implicit differentiation to find the derivative of the natural logarithm.

Given , we can rewrite this as . Then, we can once again think of moving along both the and axes and find the derivative of with respect to .

Quotient Rule

The quotient rule is a way to differentiate functions that are the ratio of two functions. It can be derived using information from the above sections.

Given , the derivative of is:

Therefore, the derivative of is:

Derivatives of Exponential Functions

is a special function because its derivative is itself.

This can be shown via implicit differentiation:

Alternatively, you can use the limit definition of the derivative:

To find the derivative of , where is some constant, you can use the chain rule:

Example: Derivative of

To find the derivative of , you can rewrite it as and then differentiate it:

Derivatives of Logarithmic Functions

The derivative of is , as shown previously.

To find the derivative of , you can use the change of base formula.

Start with , then rewrite it as .

Crucially, can be of any base.

Using this, you can find the derivative of :

Derivatives of Inverse Trigonometric Functions

The derivatives of the inverse trigonometric functions are as follows:

To find the derivative of , you can use implicit differentiation.

Start with , then rewrite it as .

To find , you can use the Pythagorean identity .

Thus:

The derivative of can be found similarly.

Another example; to find the derivative of , you can rewrite it as .

We used a trigonometric identity, that is . Here's how it's derived:

The derivatives of the other inverse trigonometric functions can be found similarly.

Derivatives of Hyperbolic Functions

Hyperbolic functions are analogs of the trigonometric functions, defined in terms of exponentials.

For example, and .

Instead of tracing a unit circle, hyperbolic functions trace a unit hyperbola, which is why they're called "hyperbolic".

A circle is defined by , while a hyperbola is defined by .

Both the hyperbolic and trigonometric functions have similar properties, including derivatives.

The unit hyperbola and unit circle are shown below:

The derivatives of the hyperbolic functions are as follows:

Summary

• The derivative of a function at a point is the slope of the tangent line at that point.
• The derivative of a function is denoted as or . Formally it is defined as the limit of the difference quotient.
• A function is differentiable at a point if the limit of the difference quotient exists at that point.
• Differentiability implies continuity, but continuity does not necessarily imply differentiability.
• The product rule, chain rule, and quotient rule are ways to differentiate functions.
• Implicit differentiation is a way to differentiate functions that are not explicitly defined.

Derivative Cheat Sheet

I use Lagrange's notation here, where the derivative of is denoted as , simply for brevity.

Function	Derivative