TastyPancakes/Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #381 from piever/pv/docs

Browse Source 
fix julia 1 changes in tutorial
This commit is contained in:
Mike J Innes 2018-09-04 16:00:58 +01:00 committed by GitHub
commit 41cf1f2a84
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 7 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/src/models/basics.md
View File

@ -172,7 +172,7 @@ using Flux
layers = [Dense(10, 5, σ), Dense(5, 2), softmax] layers = [Dense(10, 5, σ), Dense(5, 2), softmax]
model(x) = foldl((x, m) -> m(x), x, layers) model(x) = foldl((x, m) -> m(x), layers, init = x)
model(rand(10)) # => 2-element vector model(rand(10)) # => 2-element vector
``` ```

12
docs/src/models/regularisation.md
View File

@ -1,7 +1,7 @@
# Regularisation # Regularisation
Applying regularisation to model parameters is straightforward. We just need to Applying regularisation to model parameters is straightforward. We just need to
apply an appropriate regulariser, such as `vecnorm`, to each model parameter and apply an appropriate regulariser, such as `norm`, to each model parameter and
add the result to the overall loss. add the result to the overall loss.
For example, say we have a simple regression. For example, say we have a simple regression.
@ -15,12 +15,12 @@ loss(x, y) = crossentropy(softmax(m(x)), y)
We can regularise this by taking the (L2) norm of the parameters, `m.W` and `m.b`. We can regularise this by taking the (L2) norm of the parameters, `m.W` and `m.b`.
```julia ```julia
penalty() = vecnorm(m.W) + vecnorm(m.b) penalty() = norm(m.W) + norm(m.b)
loss(x, y) = crossentropy(softmax(m(x)), y) + penalty() loss(x, y) = crossentropy(softmax(m(x)), y) + penalty()
``` ```
When working with layers, Flux provides the `params` function to grab all When working with layers, Flux provides the `params` function to grab all
parameters at once. We can easily penalise everything with `sum(vecnorm, params)`. parameters at once. We can easily penalise everything with `sum(norm, params)`.
```julia ```julia
julia> params(m) julia> params(m)
@ -28,7 +28,7 @@ julia> params(m)
param([0.355408 0.533092; … 0.430459 0.171498]) param([0.355408 0.533092; … 0.430459 0.171498])
param([0.0, 0.0, 0.0, 0.0, 0.0]) param([0.0, 0.0, 0.0, 0.0, 0.0])
julia> sum(vecnorm, params(m)) julia> sum(norm, params(m))
26.01749952921026 (tracked) 26.01749952921026 (tracked)
``` ```
@ -40,7 +40,7 @@ m = Chain(
Dense(128, 32, relu), Dense(128, 32, relu),
Dense(32, 10), softmax) Dense(32, 10), softmax)
loss(x, y) = crossentropy(m(x), y) + sum(vecnorm, params(m)) loss(x, y) = crossentropy(m(x), y) + sum(norm, params(m))
loss(rand(28^2), rand(10)) loss(rand(28^2), rand(10))
``` ```
@ -57,6 +57,6 @@ julia> activations(c, rand(10))
param([0.0330606, -0.456104]) param([0.0330606, -0.456104])
param([0.61991, 0.38009]) param([0.61991, 0.38009])
julia> sum(vecnorm, ans) julia> sum(norm, ans)
2.639678767773633 (tracked) 2.639678767773633 (tracked)
``` ```