# Differentiation from first principles
The _difference quotient_ of $f(x)$ at value $x$ is given by $\frac{f(x+h)-f(x)}{h}. \tag{1}$
The limit of the difference quotient as $h$ goes to zero is the derivative of $f(x)$ at $x$, which is to say $f'(x) = \lim_{h\to 0}\frac{f(x+h)-f(x)}{h}.\tag{2}$
Instead of $h$, people often use $\delta_x$.
Geometrically, this corresponds with the idea that if we draw a line through some point $(x,f(x))$ on a function and another nearby point $(x+h, f(x+h))$, as the two points get close together, $h$ gets smaller and smaller and the line gets closer and closer to being a tangent to $f(x)$. In the limit as $h$ tends to zero they are infinitesimally close and the line is a tangent.
The process for differentiating from first principles is to evaluate the difference quotient, and then take this limit. If we use a specific value for $x$ then we obtain the gradient of the function at $x$, but if we do it symbolically we obtain the _derivative_, a function which we can evaluate to obtain the gradient of any point of the original function.
# Differentiability
If we say a function $f$ is differentiable at a value of $x$, we are saying that the difference quotient of $f$ tends to a limit at $x$ as $h$ (or $\delta x$) tends to zero. Or, formally, since we are dealing only with real functions for now, $\exists~l \in \mathbf{R}~\text{s.t.} \lim \limits_{h \to 0}\frac{f(x+h)-f(x)}{h} = l$
Saying that it tends to the limit means it must tend to the same limit from the right as from the left, if it tends to a different limit when approached from one direction than from the other, then no limit exists at $x$. A simple example of this would be $\tan x$ which is not differentiable at $\frac{\pi}{2} +n\pi \quad n \in \mathbf{Z}.$ To understand the issue, consider $\lim \limits_{x \to \frac{\pi}{2}} \tan x.$
As can be seen from the diagram below, as $x$ approaches $\frac{\pi}{2}$ from the left, the function tends towards positive infinity and as we approach from the right, it approaches negative infinity.
![[Tan limit example.png]]
This corresponds geometrically with the points on the unit circle where the line is vertical and therefore $\tan$ (as the gradient of the line) is undefined. As $\tan x$ has period $\pi$, it is not differentiable any of these points.
For this reason, people tend to talk about a function as being differentiable on an interval, meaning that the limit of the difference quotient exists at all points in that interval, and therefore it is differentiable at all points in that interval.
# Notation for derivatives
If we have $f(x)=x^2$ and say $f'(x) = 2x$ this is "[Lagrange's notation](https://en.wikipedia.org/wiki/Notation_for_differentiation#Lagrange's_notation)"(So named because it was invented by Leonhard Euler). The equivalent is $\frac{\mathop{\mathrm{d} y}}{\mathop{\mathrm{d}x}} = 2x$ in "[Leibniz's notation](https://en.wikipedia.org/wiki/Leibniz%27s_notation)"(which I think he did invent. Euler can't have invented everything). Leibniz was actually thinking about this as a fraction with an "infinitessimal" denominator, but this is very problematic as this denominator doesn't obey normal rules of arithmetic (eg you can't add them up etc). Since Weierstrass and Cauchy, people have retconned this notation into meaning the limit of the difference quotient, which allows it to work the way Leibniz wanted, while becoming fully rigorous and consistent with normal expectations.
We can also use this notation without first defining $f(x)$ by writing things like $\frac{\mathop{\mathrm{d}}}{\mathop{\mathrm{d}x}}x^2 = 2x,$
which is read as "d by dx of $x^2$ is $2x
quot;, or "the derivative with respect to $x$ of $x^2$ is $2xquot;. This is something Lagrange's notation can't do and is very convenient for evaluating more complex derivatives where you can go some of the way and drop in $\frac{\mathop{\mathrm{d}}}{\mathop{\mathrm{d}x}}$ of some subexpression you're going to tackle in a later step (this is going to be useful when applying the [[Rules and Techniques for Derivatives#Product and quotient rules|quotient rule]] for example.
Most written maths seem to use whatever notation is most convenient and in particular will use this form liberally even when generally using Lagrange notation.
## Rules and Techniques for differentiation
Note there are various [[Rules and Techniques for Derivatives|rules and techniques]] that need to be mastered by practise
1. The [[Rules and Techniques for Derivatives#Sum and Difference rule|sum and difference rule]]
2. The [[Rules and Techniques for Derivatives#Product and quotient rules|product and quotient rules]]
3. The [[Rules and Techniques for Derivatives#Chain rule and $u$-substitution|chain rule]] and the associated technique of $u$-substitution
## Uses of differentiation
### Stationary points
One obvious use of differentiation is to find stationary points, which occur where the first derivative is zero.
At a local maximum the graph of a function is increasing on the left and decreasing on the right, and at a local minimum it is decreasing on the left and increasing on the right. This allows us to use something known as the [[First derivative test]] to determine the nature of the stationary points. If a function is twice differentiable, they can also be investigated by the [[Second Derivative Test]].
## Smoothness and continuity
This is not relevant in my studies yet, but I found it interesting, and is discussed in this video:
If a function is continuous at all points and its derivative is continuous on all points, we say it is in $\mathcal{C}^1$, the set of continuously differentiable functions. If its second derivative is continuous then it is in $\mathcal{C}^2$, the set of continuously _second_ differentiable functions, and so on. If it is continuously differentiable infinitely many times then it lives in $\mathcal{C}^\infty$ and we say that the function is smooth.
# Typesetting derivatives in latex
There is no particular challenge to typesetting derivatives in Lagrange notation. The simplest way to typeset a derivative in Leibniz notation in latex is something like
```latex
\frac{dy}{dx}
```
to produce
$
\frac{dy}{dx}
$
This is actually (I seem to recall) how Knuth used to write derivatives so you'd be in good company. However the standard now (see [here](https://tex.stackexchange.com/questions/14821/whats-the-proper-way-to-typeset-a-differential-operator) and [here](https://tug.org/tugboat/tb18-1/tb54becc.pdf)) is to write them with the "d" in roman (ie not italics). We also want to ensure there is some space before and after the "dx" like an operator. This doesn't matter as much for derivatives actually but is important for integrals. To do this better there are a plethora of possible macros etc to help.
I generally use the [esdiff](https://ctan.org/pkg/esdiff?lang=en) package, which would allow you to write just
```latex
\diff{y}{x}
```
...and get a nice looking derivative. To get a second derivative using esdiff you would do
```latex
\diff[2]{y}{x}
```
It also has an analogous `\diffp` macro which does partial derivatives in the way you might expect.
These nice options are not available in a mathjaxx situation like in obsidian or jupyter notebook or if I'm typing equations into a forum or whatever. So there you can get a slightly better format by doing something like
```latex
\frac{\mathop{\mathrm{d}y}}{\mathop{\mathrm{d}x}}
```
$
\frac{\mathop{\mathrm{d}y}}{\mathop{\mathrm{d}x}}
$
I frequently don't bother with the `mathop` in these notes as it really just adjusts the spacing to make it look a bit more like an operator and I know what I mean.
One of my other packages (maybe `amsmath` but I'm really not sure) adds the `\d` macro which is basically equivalent to the above so makes writing integrals a bit easier although again is generally not available in mathjaxx.