Transformers

Essence of Linear Algebra (16): Linear Algebra in Deep Learning

Deep learning is large-scale matrix computation. From backpropagation as the chain rule in matrix form, to im2col turning convolutions into GEMM, to attention as soft retrieval via dot products -- see every core DL …