# TinyBERT - [TinyBERT: Distilling BERT for Natural Language Understanding](https://arxiv.org/abs/1909.10351) - novel Transformer distillation method to accelerate inference and reduce model size while maintaining accuracy - specially designed for [Knowledge Distillation](Knowledge%20Distillation.md) (KD) of the Transformer-based models - plenty of knowledge encoded in a large teacher BERT can be effectively transferred to a small student Tiny-BERT - [GLUE](GLUE.md)