fix vecnorm in docs
This commit is contained in:
parent
abcefb8ae3
commit
a012d0bd51
|
@ -1,7 +1,7 @@
|
|||
# Regularisation
|
||||
|
||||
Applying regularisation to model parameters is straightforward. We just need to
|
||||
apply an appropriate regulariser, such as `vecnorm`, to each model parameter and
|
||||
apply an appropriate regulariser, such as `norm`, to each model parameter and
|
||||
add the result to the overall loss.
|
||||
|
||||
For example, say we have a simple regression.
|
||||
|
@ -15,12 +15,12 @@ loss(x, y) = crossentropy(softmax(m(x)), y)
|
|||
We can regularise this by taking the (L2) norm of the parameters, `m.W` and `m.b`.
|
||||
|
||||
```julia
|
||||
penalty() = vecnorm(m.W) + vecnorm(m.b)
|
||||
penalty() = norm(m.W) + norm(m.b)
|
||||
loss(x, y) = crossentropy(softmax(m(x)), y) + penalty()
|
||||
```
|
||||
|
||||
When working with layers, Flux provides the `params` function to grab all
|
||||
parameters at once. We can easily penalise everything with `sum(vecnorm, params)`.
|
||||
parameters at once. We can easily penalise everything with `sum(norm, params)`.
|
||||
|
||||
```julia
|
||||
julia> params(m)
|
||||
|
@ -28,7 +28,7 @@ julia> params(m)
|
|||
param([0.355408 0.533092; … 0.430459 0.171498])
|
||||
param([0.0, 0.0, 0.0, 0.0, 0.0])
|
||||
|
||||
julia> sum(vecnorm, params(m))
|
||||
julia> sum(norm, params(m))
|
||||
26.01749952921026 (tracked)
|
||||
```
|
||||
|
||||
|
@ -40,7 +40,7 @@ m = Chain(
|
|||
Dense(128, 32, relu),
|
||||
Dense(32, 10), softmax)
|
||||
|
||||
loss(x, y) = crossentropy(m(x), y) + sum(vecnorm, params(m))
|
||||
loss(x, y) = crossentropy(m(x), y) + sum(norm, params(m))
|
||||
|
||||
loss(rand(28^2), rand(10))
|
||||
```
|
||||
|
@ -57,6 +57,6 @@ julia> activations(c, rand(10))
|
|||
param([0.0330606, -0.456104])
|
||||
param([0.61991, 0.38009])
|
||||
|
||||
julia> sum(vecnorm, ans)
|
||||
julia> sum(norm, ans)
|
||||
2.639678767773633 (tracked)
|
||||
```
|
||||
|
|
Loading…
Reference in New Issue