Demystifying Tensor Parallelism
How does tensor parallelism work?
Robot Chinwag
Linear Layer, Deriving the Gradient for the Backward Pass
Deriving the gradient for the backward pass for the linear layer using tensor calculus
Layer Normalization, Deriving the Gradient for the Backward Pass
Obtaining the gradient of the layer normalization layer
The Tensor Calculus You Need for Deep Learning
Deriving the gradient for the backward pass using tensor calculus and index notation
Backpropagation and Multivariable Calculus
A quick intro on backpropagation and multivariable calculus for deep learning