batchnorm: update docs
This commit is contained in:
parent
ce46843459
commit
5253841acc
@ -36,5 +36,6 @@ swish
|
|||||||
These layers don't affect the structure of the network but may improve training times or reduce overfitting.
|
These layers don't affect the structure of the network but may improve training times or reduce overfitting.
|
||||||
|
|
||||||
```@docs
|
```@docs
|
||||||
|
BatchNorm
|
||||||
Dropout
|
Dropout
|
||||||
```
|
```
|
||||||
|
@ -2,8 +2,8 @@
|
|||||||
testmode!(m)
|
testmode!(m)
|
||||||
testmode!(m, false)
|
testmode!(m, false)
|
||||||
|
|
||||||
Put layers like [`Dropout`](@ref) and `BatchNorm` into testing mode (or back to
|
Put layers like [`Dropout`](@ref) and [`BatchNorm`](@ref) into testing mode
|
||||||
training mode with `false`).
|
(or back to training mode with `false`).
|
||||||
"""
|
"""
|
||||||
function testmode!(m, val::Bool=true)
|
function testmode!(m, val::Bool=true)
|
||||||
prefor(x -> _testmode!(x, val), m)
|
prefor(x -> _testmode!(x, val), m)
|
||||||
@ -48,7 +48,7 @@ _testmode!(a::Dropout, test) = (a.active = !test)
|
|||||||
BatchNorm(dims...; λ = identity,
|
BatchNorm(dims...; λ = identity,
|
||||||
initβ = zeros, initγ = ones, ϵ = 1e-8, momentum = .1)
|
initβ = zeros, initγ = ones, ϵ = 1e-8, momentum = .1)
|
||||||
|
|
||||||
Batch Normalization Layer
|
Batch Normalization Layer for [`Dense`](@ref) layer.
|
||||||
|
|
||||||
See [Batch Normalization: Accelerating Deep Network Training by Reducing
|
See [Batch Normalization: Accelerating Deep Network Training by Reducing
|
||||||
Internal Covariate Shift](https://arxiv.org/pdf/1502.03167.pdf)
|
Internal Covariate Shift](https://arxiv.org/pdf/1502.03167.pdf)
|
||||||
@ -65,6 +65,8 @@ julia> m = Chain(
|
|||||||
BatchNorm(10),
|
BatchNorm(10),
|
||||||
softmax)
|
softmax)
|
||||||
Chain(Dense(784, 64), BatchNorm(64, λ = NNlib.relu), Dense(64, 10), BatchNorm(10), NNlib.softmax)
|
Chain(Dense(784, 64), BatchNorm(64, λ = NNlib.relu), Dense(64, 10), BatchNorm(10), NNlib.softmax)
|
||||||
|
|
||||||
|
julia> opt = SGD(params(m), 10) # a crazy learning rate
|
||||||
```
|
```
|
||||||
"""
|
"""
|
||||||
mutable struct BatchNorm{F,V,N}
|
mutable struct BatchNorm{F,V,N}
|
||||||
|
Loading…
Reference in New Issue
Block a user