#### Formatting — Scalars, Vectors, Matrices, Spaces | Object | Convention | Examples | | ------------------- | ----------------------------------- | --------------------------------------------------------------- | | Scalar | Lowercase, non-bold | $x, t, \gamma, \lambda, \theta$ | | Vector | Lowercase, bold | $\mathbf{x}, \mathbf{q}, \boldsymbol{\theta}, \boldsymbol{\mu}$ | | Matrix | Uppercase, bold | $\mathbf{A}, \mathbf{M}, \mathbf{R}, \boldsymbol{\Sigma}$ | | Space / Set | Calligraphic uppercase | $\mathcal{X}, \mathcal{S}, \mathcal{A}, \mathcal{M}$ | | Number Set | Blackboard bold | $\mathbb{R}, \mathbb{C}, \mathbb{Z}, \mathbb{H}$ | | Lie Group | $\mathrm{Name}(n)$ or $\mathcal{G}$ | $\mathrm{SO}(3), \mathrm{SE}(3), \mathcal{G}$ | | Lie Algebra | Fraktur | $\mathfrak{g}, \mathfrak{so}(3), \mathfrak{se}(3)$ | | Scalar parameter | Non-bold Greek | $\gamma, \lambda, \beta, \epsilon$ | | Vector parameter | Bold Greek | $\boldsymbol{\theta}, \boldsymbol{\phi}, \boldsymbol{\lambda}$ | | Function / map | Non-bold | $f, g, h, \Psi, \phi$ | | Functional / energy | Calligraphic or explicit subscript | $\mathcal{L}, \mathcal{K}, \mathcal{P}, F[\cdot]$ | >[!warning] Boldness Rule of Thumb >Bold indicates **dimensionality > 1**. A parameter $\theta \in \mathbb{R}$ is non-bold; the same letter as a parameter vector $\boldsymbol{\theta} \in \mathbb{R}^n$ is bold. | Name | Symbols | Remarks | | ----------------------------------------------- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | [[Set]] | $A,B,X$ | | | [[Space]] | $\mathcal{X},\mathcal{Y}$ | Set endowed with additional structure | | [[Vector Space]] | $X,V$ | Set endowed with linear algebra structure | | [[Metric Space and Completeness\|Metric Space]] | $(\mathcal{X},d_{\mathcal{X}})$ | Set endowed with metric structure | | [[Field]] | $F$ | Set with addition, subtraction, multiplication and division, in almost any case $\mathbb{R}^d$ or $\mathbb{C}^d$ | | | | | | Name | Symbols | Remarks | | ------------------------------------ | :-----: | ------------------------ | | [[Tensors\|Covariant Component]] | $v^{i}$ | Transforms like $1$-Form | | [[Tensors\|Contravariant Component]] | $w_i$ | Transforms like vector | --- #### Linear Algebra | Operation | Canonical | Alternatives | Remarks | | ---------------------------------------------- | :------------------------------: | ------------------------------------------ | ------------------------------------------------ | | Transpose | $\mathbf{A}^\top$ | | | | Conjugate Transpose | $\mathbf{A}^{H}$ | | Reserve $*$ for optimality | | Inverse | $\mathbf{A}^{-1}$ | | | | [[Moore-Penrose Pseudoinverse\|Pseudoinverse]] | $\mathbf{A}^{\dagger}$ | | | | Identity Matrix | $\mathbf{I}$ or $\mathbf{I}_n$ | $\mathbb{1}^{n\times n}$, $\boldsymbol{1}$ | Use $\mathbf{I}_n$ when dimension needs emphasis | | Zero Matrix | $\mathbf{0}$ | | Subscript $_{m \times n}$ if needed | | Zero Vector | $\mathbf{0}$ or $\boldsymbol{0}$ | | | | All-Ones Vector | $\mathbb{1}_n$ | $\boldsymbol{1}_n$ | As used in OT notes for marginal constraints | | Basis | $\mathbb{I}_X$ | | | | Scalar absolute value | $\lvert x \rvert$ | | Use `\lvert`, `\rvert` | | Vector / matrix norm | $\lVert \mathbf{x} \rVert$ | | Use `\lVert`, `\rVert` | | Specific norm | $\lVert \cdot \rVert_p$ | | Subscript for type | --- #### Time and Indexing Conventions | Setting | Index | Example | Where used | | ---------- | ------ | ---------------------------------------------- | ------------------------------- | | Continuous | $t$ | $\mathbf{x}(t)$, $X_t$, $dX_t$ | SDEs, Control, Dynamics | | Discrete | $k, n$ | $\mathbf{x}_k$, $X_n$, $\Delta X_k$ | MPC, Markov chains, Filtering | | Sequence | $t$ | $\mathbf{h}_t$, $\mathbf{x}_t$ | RNNs, Transformers | | Components | $i,j$ | $[\mathbf{x}]_i$, $x_i$ | Brackets or subscript | --- #### Differential Equations This notation is used in pure mathematics, where no special application is considered. | Name | Symbols | Remarks | | ---------- | ---------------------------------------------------- | --------------------------------------------- | | Function | $x=f(t)$ | Physics notation, $t$ is independent variable | | | $y= f(x)$ | Math notation, $x$ is independent variable | | Functional | $F\left (x,y,y',\ldots, y^{(n-1)} \right )=y^{(n)}$ | Explicit form | | | $F\left(x, y, y', y'',\ \ldots,\ y^{(n)}\right) = 0$ | Implicit form | | | | | >[!note] >In contrast, the study of [[Static, Dynamic and Stochastic Systems|dynamical systems]] in physics and engineering usually replaces the independent variable $x$ by the time $t$. --- #### (Convex) Optimization | Name | Symbols | Remarks | | ------------------------------------------------------------ | :---------------------------------------------------: | ---------------------------------------------------------------------------- | | [[Optimization Problem - Standard and Convex\|Optimization Problem Standard Form]] | $f_0,f$ | Objective function, outputs scalar value | | | $\mathbf{f}(\mathbf{x})\preceq \boldsymbol{0}$ | Inequality constraints ($m$) | | | $\mathbf{h}(\mathbf{x})=\boldsymbol{0}$ | Equality constraints ($p$) | | | $\mathcal{C}$ | [[Convexity\|Convex]] feasible set | | | $\mathcal{X}$ | General feasible set | | Problem Domain | $\mathcal{D}$ | | | Feasible Point | $\mathbf{x}^{*}$ | Point in domain that fulfills constraints, makes problem feasible | | Optimal Function Value | $p^{*}$ | Problem is infeasible, if $p^*=\infty$ and unbounded below, if $p^*=-\infty$ | | [[Lagrange Dual Problem]] | $L(\mathbf{x},\boldsymbol{\lambda},\boldsymbol{\nu})$ | Linear approximation of indicator functions for constraints | | | $\boldsymbol{\lambda}$ | Lagrange multipliers for inequality constraints | | | $\boldsymbol{\nu}$ | Lagrange multipliers for equality constraints | | [[Lagrange Dual Problem\|Lagrange Dual Function]] | $g(\boldsymbol{\lambda},\boldsymbol{\nu})$ | [[Set\|Infimum]] of dual problem over $\mathbf{x}\in \mathcal{D}$ | | Optimal [[Lagrange Dual Problem\|dual value]] | $d^*$ | | | | | | --- ### Overloaded Symbols Cheat Sheet | Symbol | Meaning 1 | Meaning 2 | Meaning 3 | | --------------------- | -------------------------------- | ----------------------------------- | ----------------------------- | | $\pi$ | Policy (RL) | Transportation plan (OT) | Representation (Group Theory) | | $\mathcal{L}$ | Loss function (ML) | Lagrangian (Mechanics) | | | $\mathbf{C}$ | Coriolis matrix (Robotics) | Output matrix (LTI) | | | $\mathbf{q}$ | Joint configuration (Robotics) | Unit quaternion | | | $\mathbf{H}$ | Homogeneous transform (Robotics) | Channel matrix (Comms) | | | $J$ | RL return / objective | Jacobian matrix | | | $T$ | Final time / horizon | Temperature | | | $p$ | Density / probability | Optimal primal value (Optimization) | | | $\mu$ | Measure | Mean / drift | | | $\boldsymbol{\theta}$ | Joint angles (Robotics) | Parameters (ML) | |