mygrad.nnet.layers.batchnorm#

mygrad.nnet.layers.batchnorm(x: ArrayLike, *, gamma: Optional[ArrayLike] = None, beta: Optional[ArrayLike] = None, eps: float, constant: Optional[bool] = None) → Tensor[source]#

Performs batch normalization on x:

y(x) = (x - E[x]) / sqrt(Var[x] + eps)
batchnorm(x) = gamma * y(x) + beta

Where \(E[x]\) and \(Var[x]\) represent the mean and variance, respectively, over axis-1 of x. The subsequent affine transformation on y is optional.

Parameters

xarray_like, shape=(N, C, …): The batch to be normalized within each entry of C
gammaOptional[array_like], shape=(C,): Optional per-channel scaling factors to be applied after the normalization step.
betaOptional[array_like], shape=(C,): Optional per-channel scaling bias factors to be applied after the normalization step.
epsReal: A small non-negative number.
constantbool, optional (default=False): If True, the resulting Tensor is a constant.

Returns

mygrad.Tensor: The batch-normalized data.

Examples

>>> import mygrad as mg
>>> from mygrad.nnet import batchnorm
>>> x = mg.Tensor([1., 4., 1.]).reshape(3, 1)
>>> batchnorm(x, eps=0)
Tensor([[-0.70710678],
        [ 1.41421356],
        [-0.70710678]])