Interactive KL Divergence Visualisation
An interactive visualisation of Kullback–Leibler divergence. Shape two distributions and watch forward vs reverse KL, the pointwise integrand, and the effects of asymmetry, support mismatch and discretisation.
Robot Chinwag
Gradients of Matrix Multiplication in Deep Learning
Deriving the gradients for the backward pass for matrix multiplication using tensor calculus
Demystifying Tensor Parallelism
How does tensor parallelism work?
The Tensor Calculus You Need for Deep Learning
Deriving the gradient for the backward pass using tensor calculus and index notation
Einsum, Deriving the Gradient for the Backward Pass
Obtaining the gradient of the matrix inverse
Matrix Inverse, Deriving the Gradient for the Backward Pass
Obtaining the gradient of the matrix inverse
Cross-Entropy Loss (Softmax) Gradient Used In Deep Learning
Obtaining the gradient of the Cross-entropy loss (softmax and negative log-likelihood loss function
Linear Layer, Deriving the Gradient for the Backward Pass
Deriving the gradient for the backward pass for the linear layer using tensor calculus
Layer Normalization, Deriving the Gradient for the Backward Pass
Obtaining the gradient of the layer normalization layer
Backpropagation and Multivariable Calculus
A quick intro on backpropagation and multivariable calculus for deep learning