diff --git a/docs/src/models/recurrence.md b/docs/src/models/recurrence.md index 7c20165d..8030f4c4 100644 --- a/docs/src/models/recurrence.md +++ b/docs/src/models/recurrence.md @@ -114,3 +114,13 @@ truncate!(m) Calling `truncate!` wipes the slate clean, so we can call the model with more inputs without building up an expensive gradient computation. `truncate!` makes sense when you are working with multiple chunks of a large sequence, but we may also want to work with a set of independent sequences. In this case the hidden state should be completely reset to its original value, throwing away any accumulated information. `reset!` does this for you. + +In general, when training with recurrent layers in your model, you'll want to call `reset!` or `truncate!` for each loss calculation: + +```julia +function loss(x,y) + l = Flux.mse(m(x), y) + Flux.reset!(m) + return l +end +```