Improve docs for decay optimisers

This commit is contained in:
Fredrik Bagge Carlson 2019-12-03 15:27:44 +08:00
parent f46b5243db
commit 6e94e59afd
1 changed files with 3 additions and 2 deletions

View File

@ -444,7 +444,8 @@ end
"""
InvDecay(γ)
Applies inverse time decay to an optimiser
Applies inverse time decay to an optimiser, i.e., the step effective step size at iteration `n` is `eta / (1 + γ * n)` where `eta` is the initial step size. The wrapped optimisers step size is not modified.
```
## Parameters
- gamma (γ): Defaults to `0.001`
@ -472,7 +473,7 @@ end
"""
ExpDecay(eta, decay, decay_step, clip)
Discount the learning rate `eta` by `decay` every `decay_step` till a minimum of `clip`.
Discount the learning rate `eta` by `decay` every `decay_step` till a minimum of `clip`. The wrapped optimisers step size is being modified by the outer optimiser.
## Parameters
- Learning Rate (eta): Defaults to `0.001`.