Add note on reset! usage in recurrence docs

This commit is contained in:
Julian P Samaroo 2019-03-06 00:00:00 -06:00
parent 22cb732657
commit 1930f40dec
1 changed files with 10 additions and 0 deletions

View File

@ -114,3 +114,13 @@ truncate!(m)
Calling `truncate!` wipes the slate clean, so we can call the model with more inputs without building up an expensive gradient computation.
`truncate!` makes sense when you are working with multiple chunks of a large sequence, but we may also want to work with a set of independent sequences. In this case the hidden state should be completely reset to its original value, throwing away any accumulated information. `reset!` does this for you.
In general, when training with recurrent layers in your model, you'll want to call `reset!` or `truncate!` for each loss calculation:
```julia
function loss(x,y)
l = Flux.mse(m(x), y)
Flux.reset!(m)
return l
end
```