Correct some comments in decay docs
This commit is contained in:
parent
6e94e59afd
commit
e67f09c06d
|
@ -444,7 +444,7 @@ end
|
||||||
"""
|
"""
|
||||||
InvDecay(γ)
|
InvDecay(γ)
|
||||||
|
|
||||||
Applies inverse time decay to an optimiser, i.e., the step effective step size at iteration `n` is `eta / (1 + γ * n)` where `eta` is the initial step size. The wrapped optimisers step size is not modified.
|
Applies inverse time decay to an optimiser, i.e., the effective step size at iteration `n` is `eta / (1 + γ * n)` where `eta` is the initial step size. The wrapped optimiser's step size is not modified.
|
||||||
```
|
```
|
||||||
|
|
||||||
## Parameters
|
## Parameters
|
||||||
|
@ -473,7 +473,7 @@ end
|
||||||
"""
|
"""
|
||||||
ExpDecay(eta, decay, decay_step, clip)
|
ExpDecay(eta, decay, decay_step, clip)
|
||||||
|
|
||||||
Discount the learning rate `eta` by `decay` every `decay_step` till a minimum of `clip`. The wrapped optimisers step size is being modified by the outer optimiser.
|
Discount the learning rate `eta` by a multiplicative factor `decay` every `decay_step` till a minimum of `clip`.
|
||||||
|
|
||||||
## Parameters
|
## Parameters
|
||||||
- Learning Rate (eta): Defaults to `0.001`.
|
- Learning Rate (eta): Defaults to `0.001`.
|
||||||
|
|
Loading…
Reference in New Issue