# Basic [[Transformer]] - ![[Pasted image 20220307183126.webp]] - ![[Pasted image 20220621164717.webp]] - Feed forward blocks, are two [[Dense]] MLPs with [[Relu]]. Residual connections in between - Uses [[Attention]] - [[Embedding]] [[Layers]] transform between 1 hot and vector rep - [[Position Encoding]] + [[Token Embedding]] - [[Position Wise Feed Forward]]