“Since Newton, mankind has come to realise that the laws of physics are always expressed in the language of differential equations” – Steven Strogatz # Definition A differential equation is an equation that relates a function to one or more of its derivatives. This is useful because many processes are much easier or more intuitive to describe in terms of how they change rather than their state under any given input conditions. The following are some simple differential equations, showcasing different notations for writing the derivative: $ \begin{align*} \frac{\mathop{\mathrm{d}} y}{\mathop{\mathrm{d}}x} &= 3x +y \tag{1}\\ y' &= y \sin x \tag{2}\\ \dot{\mathbf{x}}&=\mathbf{a}t + \mathbf{v}_0 \tag{3} \end{align*} $ Since a differential equation specifies how some unknown function changes, _solving_ a differential equation involves finding the function or functions which change in such a way that they satisfy the equation. Typically these form a family of functions, in which case there will be a _general solution_ involving one or more constants of integration, or if some initial conditions are given (typically specific points the desired solution must travel through) or a carefully-selected ("_well-posed_") set of boundary conditions then a _particular solution_ can be found. When initial conditions are provided, the problem is sometimes called an _initial value problem_(IVP), and when boundary conditions are provided it is a _boundary value problem_. Boundary value problems are typically more challenging to solve. For example, the general solution to equation (2) is $ y=c\mathrm{e}^{- \cos x} $ ...where $c$ is an arbitrary constant. If we also have an initial condition say that $y(0)=1$, then we can solve for a particular value of $c$ that makes this so. $ \begin{align*} y &= c\mathrm{e}^{-\cos x}\\ 1 &= c\mathrm{e}^{-\cos 0}\\ 1 &= c\mathrm{e}^{-1}\\ c &= \mathrm{e}\\ \text{Thus}~y &= (\mathrm{e})\mathrm{e}^{-\cos x}\\ &= \mathrm{e}^{1-\cos x}\\ \end{align*} $ So $y=\mathrm{e}^{1-\cos x}$ is a particular solution that satisfies the condition that $y(0)=1$.  # Basic classification ## Type If the equation specifies only regular derivatives as in examples 1-3 then it is an _ordinary differential equation_ (ODE). Don't be fooled by the word "ordinary", these equations are generally challenging to solve, and many ODEs which seem at first glance to look benign have no solutions with ordinary analytic functions, and it can be hard to tell which type is which. For example $y' = x^2 - y$ is simple to solve and is the sort of thing a person might encounter early in an introductory course of differential equations. Consider now $y' = x^2 - y^2$ This doesn't look very different but in fact there are no solutions of this equation which involve just the standard analytic functions and [Bessel functions](https://en.wikipedia.org/wiki/Bessel_function) (so named because they were invented by [Daniel Bernoulli](https://en.wikipedia.org/wiki/Daniel_Bernoulli)) have to be used to express the solution. If the equation involves multiple independent variables it will necessarily use partial derivatives, and it is therefore a _partial differential equation_ (PDE). Here are some PDEs $ \frac{\partial u}{\partial t} = \kappa \frac{\partial^2 u}{\partial x^2} $ ... the one-dimensional heat equation. Solving this simple-looking equation required the invention of the field of [Fourier Analysis](https://en.wikipedia.org/wiki/Fourier_analysis), which might give you some insight into the fact that PDEs are generally significantly more difficult to solve than ODEs. $\displaystyle {\frac {\partial V}{\partial t}}+{\frac {1}{2}}\sigma ^{2}S^{2}{\frac {\partial ^{2}V}{\partial S^{2}}}=rV-rS{\frac {\partial V}{\partial S}}$ ... the [Black-Scholes equation](https://en.wikipedia.org/wiki/Black%E2%80%93Scholes_equation) which is the most basic pricing model for financial derivatives (and is in fact a form of the heat equation). It is also common to write partial derivatives using a subscript notation, and here is a Lagrange's equation (which is a PDE) written in this way $(1+f_y^2)f_{xx}-2f_xf_yf_{xy}+(1+f_x^2)f_{yy}=0$ If you have a $\mathop{\mathrm{d}}B$, $\mathop{\mathrm{d}}W$, or $\mathop{\mathrm{d}}Z$ term then it might be you are extra lucky and have a _stochastic differential equation_ (SDE), which involve a stochastic process (in this case a Brownian motion) and therefore require special techniques to solve, and are generally even more troublesome than PDEs. An example of an SDE is $\displaystyle \mathop{\mathrm{d}}X_{t}=\mu X_{t}\,\mathop{\mathrm{d}} t+\sigma X_{t}\,\mathop{\mathrm{d}} W_{t},$ ...which is the price process for a stock in the Black-Scholes model. Here, it's saying that a stock price is a Brownian motion with constant drift $\mu$ and volatility $\sigma$. An example of a fancier SDE is the [Heston model](https://en.wikipedia.org/wiki/Heston_model), which is a more up-to-date derivatives pricing model than Black-Scholes and is in use by financial firms today. It says $\displaystyle dS_{t}=\mu S_{t}\,dt+{\sqrt {\nu _{t}}}S_{t}\,dW_{t}^{S},$ Where $d{\sqrt {\nu _{t}}}=-\theta {\sqrt {\nu _{t}}}\,dt+\delta \,dW_{t}^{\nu} $ So instead of volatility just being a constant, $\sigma$ as in Black-Scholes, it is a stochastic process. You can also have stochastic models which combine some kind of Brownian term with a jump term to allow discontinuous changes. ## Order The _order_ of a differential equation is the order of the highest derivative in the equation, not the power of the coefficient terms. So for example $x^3 y'' + 3 y' + y = 2x^4 + 3$ is a second-order ordinary differential equation, because it involves up to the second derivative of $y$. ## Homogeneity A _homogenous_ differential equation does not involve any non-differential terms other than the dependent variable. So $3y'' - 6y' + 6y = 0$ is homogeneous whereas $3y'' - 6y' + 6y = x-3$ is _heterogenous_ or _inhomogeneous_ (sometimes called "non-homogeneous" by people who know more about maths than language). If your function is in $y$ and you can arrange it so all the $y$ and $y'$ terms are on the left and the right is zero, it his homogeneous. ## Linearity An ODE is _linear_ if it can be written in the general form $a_n(x)y^{(n)}+a_{n-1}(x)y^{(n-1)}+\cdots+a_2(x)y''+a_1(x)y'+a_0(x)y=g(x)$ where: - $a_n, a_{n-1}, \cdots, a_1, a_0$ are constants or functions which do not depend on $y$ or its derivatives - $g(x)$ is the inhomogeneous term (or 0). Say we are looking at a differential equation involving some $y=f(t)$. For it to be _linear_, it must satisfy two conditions 1. Powers: The power $y$ and each (partial) derivative of $y$ must be no higher than one 2. Coefficients: The only terms must be $y$ and (partial) derivatives of $y$ multiplied by coefficients which do not depend on $y$. It's worth saying both for clarity, but it can be seen that actually the "powers" condition derives from the "coefficients" condition given that if you have $y^2$ you have a term in $y$ which has a coefficient ($y$) that depends on $y$ and similarly for powers of derivatives or products of derivatives and $y$. The coefficients condition is more general because it also prevents terms like $y\frac{\mathrm{d}y}{\mathrm{d}t}$ where you have a derivative multiplied by the dependent variable/unknown function itself. So: - $y'' + y^2 = 0$ is non-linear due to the first rule (it has a term in $y$ with a power greater than 1), - $y y'=4$ is non-linear due to the second rule (the term in $y'$ has a coefficient which depends on $y$, or the converse-either way works), - $\frac{y}{y'}=0$ is non-linear due to the second rule (the term in $y$ has a coefficient which depends on $y$), and $\frac{y'}{y}=0$ is also non-linear but because of the coefficient of the term in $y'$ this time. - $\cos(x) y' + \sin(x) \mathrm{e}^y=0$y is non-linear because although both coefficients do not depend on $y$, $\mathrm{e}^y$ is non-linear. - $\cos(x) y' + \sin(x) \mathrm{e}^xy''=0$ is linear because the coefficients do not depend on $y$, and there are no powers of $y$ or derivatives which are higher than 1. - $x^2 y'' + 4 y' + \frac{k}{m} y = 0$ is linear (provided $k$ and $m$ are constants and $x$ is not a function of $y$) because the powers of $y$ and its derivatives are no higher than one and it satisfies the coefficients rule also Linear equations (and linear systems) are _significantly_ easier to solve, and to solve non-linear systems frequently involves finding a suitable linear approximation near a particular point. # Resources - https://nu-cem.github.io/Computational_Physics/markdown/ODE_introduction.html - A course involving practical numerical ODE and PDE solving in python. - https://mathcentre.ac.uk/students/topics/differential-equations/differentialequations2/ - https://www.youtube.com/playlist?list=PLwIFHT1FWIUJYuP5y6YEM4WWrY4kEmIuS