TastyPancakes
/
Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #954 from FluxML/decaydocs 

[WIP] Decaydocs
This commit is contained in:
Dhairya Gandhi 2019-12-10 12:11:23 +05:30 committed by GitHub
parent f46b5243db e67f09c06d
commit ac4c49b63e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
src/optimise/optimisers.jl
View File

@ -444,7 +444,8 @@ end
""" """
InvDecay(γ) InvDecay(γ)
Applies inverse time decay to an optimiser Applies inverse time decay to an optimiser, i.e., the effective step size at iteration `n` is `eta / (1 + γ * n)` where `eta` is the initial step size. The wrapped optimiser's step size is not modified.
```
## Parameters ## Parameters
- gamma (γ): Defaults to `0.001` - gamma (γ): Defaults to `0.001`
@ -472,7 +473,7 @@ end
""" """
ExpDecay(eta, decay, decay_step, clip) ExpDecay(eta, decay, decay_step, clip)
Discount the learning rate `eta` by `decay` every `decay_step` till a minimum of `clip`. Discount the learning rate `eta` by a multiplicative factor `decay` every `decay_step` till a minimum of `clip`.
## Parameters ## Parameters
- Learning Rate (eta): Defaults to `0.001`. - Learning Rate (eta): Defaults to `0.001`.