It subtracts a mean and divides by a standard deviation computed over selected feature dimensions, then applies learned scale and offset parameters. Unlike batch normalization, it does not depend on statistics from other examples in the batch.
Layer normalization stabilizes neural network activations by normalizing features within an individual example.
It subtracts a mean and divides by a standard deviation computed over selected feature dimensions, then applies learned scale and offset parameters. Unlike batch normalization, it does not depend on statistics from other examples in the batch.