TastyPancakes/Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[WIP] add links to sgd

Browse Source
This commit is contained in:
Dhairya Gandhi 2018-11-12 17:53:53 +05:30
parent 4562682528
commit 07397bc950
1 changed files with 11 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
docs/src/training/optimisers.md
View File

@ -23,7 +23,7 @@ We want to update each parameter, using the gradient, in order to improve (reduc
```julia ```julia
using Flux.Tracker: grad, update! using Flux.Tracker: grad, update!
function sgd() struct sgd()
η = 0.1 # Learning Rate η = 0.1 # Learning Rate
for p in (W, b) for p in (W, b)
update!(p, -η * grads[p]) update!(p, -η * grads[p])
@ -60,11 +60,16 @@ An optimiser takes a parameter list and returns a function that does the same th
All optimisers return a `struct` that, when called with their `update!`, will update the parameters passed to it. All optimisers return a `struct` that, when called with their `update!`, will update the parameters passed to it.
```@docs ```@docs
SGD - [Descent](https://en.wikipedia.org/wiki/Stochastic_gradient_descent)
Descent - Momentum
Momentum - Nesterov
Nesterov - RMSProp
ADAM - ADAM
- AdaMax
- ADAGrad
- ADADelta
- AMSGrad
- NADAM
``` ```
## Optimiser API ## Optimiser API