TastyPancakes/Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

optimiser clarity

Browse Source
This commit is contained in:
Mike J Innes 2017-10-18 12:22:45 +01:00
parent 7426faf37d
commit c4166fd725
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/src/training/optimisers.md
View File

@ -48,13 +48,15 @@ For the update step, there's nothing whatsoever wrong with writing the loop abov
```julia ```julia
opt = SGD([W, b], 0.1) # Gradient descent with learning rate 0.1 opt = SGD([W, b], 0.1) # Gradient descent with learning rate 0.1
opt() opt() # Carry out the update, modifying `W` and `b`.
``` ```
An optimiser takes a parameter list and returns a function that does the same thing as `update` above. We can pass either `opt` or `update` to our [training loop](training.md), which will then run the optimiser after every mini-batch of data. An optimiser takes a parameter list and returns a function that does the same thing as `update` above. We can pass either `opt` or `update` to our [training loop](training.md), which will then run the optimiser after every mini-batch of data.
## Optimiser Reference ## Optimiser Reference
All optimisers return a function that, when called, will update the parameters passed to it.
```@docs ```@docs
SGD SGD
Momentum Momentum