# Optimization Methods
Optimization methods used in Active Inference for minimizing free energy, including gradient-based, coordinate descent, and variational optimization approaches.
## Methods for Free Energy Minimization
### Variational Bayes (Coordinate Ascent)
```math
q^*(s_k) = \exp\left(\mathbb{E}_{q(s_{\backslash k})}[\ln p(o, s)]\right) / Z_k
```
### Fixed-Point Iteration
```math
\mu^{(n+1)} = f(\mu^{(n)}) = \mu^{(n)} - \kappa \nabla_\mu F(\mu^{(n)})
```
### Newton's Method
```math
\mu^{(n+1)} = \mu^{(n)} - [H_F(\mu^{(n)})]^{-1} \nabla_\mu F(\mu^{(n)})
```
## Comparison
| Method | Convergence Rate | Cost per Step | Robustness |
| --- | --- | --- | --- |
| Gradient descent | Linear | $O(d)$ | High |
| Newton's method | Quadratic | $O(d^3)$ | Low |
| Natural gradient | Superlinear | $O(d^2)$ | Medium |
| Coordinate ascent | Linear | $O(d)$ | High |
## Implementation
```python
class FreeEnergyOptimizer:
def __init__(self, method='gradient', learning_rate=0.1):
self.method = method
self.lr = learning_rate
def minimize(self, F_fn, grad_fn, mu_init, max_iter=100, tol=1e-6):
mu = mu_init.copy()
for i in range(max_iter):
grad = grad_fn(mu)
if np.linalg.norm(grad) < tol:
break
mu -= self.lr * grad
return mu
```
## Related Topics
- [[gradient_descent]] — Gradient descent methods
- [[convergence_analysis]] — Convergence properties
- [[knowledge_base/mathematics/optimization_theory]] — Optimization theory
- [[knowledge_base/mathematics/variational_methods]] — Variational methods