Matrix Calculus
1 minute read

Matrix Derivatives

1. Scalar-by-scalar

Gradient is also a scalar.

2. Scalar-by-vector

Gradient

Also a vector, which has the same size with input $x$.

Hessian

A matrix of the size $m \times m$. (m : dimension of $x$)

3. Vector-by-vector

Gradient

Recent Posts

Inverted Indexing
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Deep Contextualized Word Representations
Pretraining-Based Natural Language Generation for Text Summarization
Style Transfer from Non-Parallel Text by Cross-Alignment