</script><linkhref="https://fonts.googleapis.com/css?family=Lato|Roboto+Mono"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/fontawesome.min.css"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/solid.min.css"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/brands.min.css"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.css"rel="stylesheet"type="text/css"/><script>documenterBaseURL=".."</script><scriptsrc="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js"data-main="../assets/documenter.js"></script><scriptsrc="../siteinfo.js"></script><scriptsrc="../../versions.js"></script><linkhref="../assets/flux.css"rel="stylesheet"type="text/css"/><linkclass="docs-theme-link"rel="stylesheet"type="text/css"href="../assets/themes/documenter-dark.css"data-theme-name="documenter-dark"/><linkclass="docs-theme-link"rel="stylesheet"type="text/css"href="../assets/themes/documenter-light.css"data-theme-name="documenter-light"data-theme-primary/><scriptsrc="../assets/themeswap.js"></script></head><body><divid="documenter"><navclass="docs-sidebar"><divclass="docs-package-name"><spanclass="docs-autofit">Flux</span></div><formclass="docs-search"action="../search/"><inputclass="docs-search-query"id="documenter-search-query"name="q"type="text"placeholder="Search docs"/></form><ulclass="docs-menu"><li><aclass="tocitem"href="../">Home</a></li><li><spanclass="tocitem">Building Models</span><ul><li><aclass="tocitem"href="../models/basics/">Basics</a></li><li><aclass="tocitem"href="../models/recurrence/">Recurrence</a></li><li><aclass="tocitem"href="../models/regularisation/">Regularisation</a></li><li><aclass="tocitem"href="../models/layers/">Model Reference</a></li><li><aclass="tocitem"href="../models/advanced/">Advanced Model Building</a></li><li><aclass="tocitem"href="../models/nnlib/">NNlib</a></li></ul></li><li><spanclass="tocitem">Handling Data</span><ul><li><aclass="tocitem"href="../data/onehot/">One-Hot Encoding</a></li><li><aclass="tocitem"href="../data/dataloader/">DataLoader</a></li></ul></li><li><spanclass="tocitem">Training Models</span><ul><li><aclass="tocitem"href="../training/optimisers/">Optimisers</a></li><li><aclass="tocitem"href="../training/training/">Training</a></li></ul></li><li><aclass="tocitem"href="../gpu/">GPU Support</a></li><liclass="is-active"><aclass="tocitem"href>Saving & Loading</a><ulclass="internal"><li><aclass="tocitem"href="#Saving-Model-Weights-1"><span>Saving Model Weights</span></a></li><li><aclass="tocitem"href="#Checkpointing-1"><span>Checkpointing</span></a></li></ul></li><li><aclass="tocitem"href="../ecosystem/">The Julia Ecosystem</a></li><li><aclass="tocitem"href="../performance/">Performance Tips</a></li><li><aclass="tocitem"href="../community/">Community</a></li></ul><divclass="docs-version-selector field has-addons"><divclass="control"><spanclass="docs-label button is-static is-size-7">Version</span></div><divclass="docs-selector control is-expanded"><divclass="select is-fullwidth is-size-7"><selectid="documenter-version-selector"></select></div></div></div></nav><divclass="docs-main"><headerclass="docs-navbar"><navclass="breadcrumb"><ulclass="is-hidden-mobile"><liclass="is-active"><ahref>Saving & Loading</a></li></ul><ulclass="is-hidden-tablet"><liclass="is-active"><ahref>Saving & Loading</a></li></ul></nav><divclass="docs-right"><aclass="docs-edit-link"href="https://github.com/FluxML/Flux.jl/blob/master/docs/src/saving.md"title="Edit on GitHub"><spanclass="docs-icon fab"></span><spanclass="docs-label is-hidden-touch">Edit on GitHub</span></a><aclass="docs-settings-button fas fa-cog"id="documenter-settings-button"href="#"title="Settings"></a><aclass="docs-sidebar-button fa fa-bars is-hidden-desktop"id="documenter-sidebar-butt
Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)</code></pre><p>Models are just normal Julia structs, so it's fine to use any Julia storage format for this purpose. BSON.jl is particularly well supported and most likely to be forwards compatible (that is, models saved now will load in future versions of Flux).</p><divclass="admonition is-info"><headerclass="admonition-header">Note</header><divclass="admonition-body"><p>If a saved model's weights are stored on the GPU, the model will not load later on if there is no GPU support available. It's best to <ahref="../gpu/">move your model to the CPU</a> with <code>cpu(model)</code> before saving it.</p></div></div><h2id="Saving-Model-Weights-1"><aclass="docs-heading-anchor"href="#Saving-Model-Weights-1">Saving Model Weights</a><aclass="docs-heading-anchor-permalink"href="#Saving-Model-Weights-1"title="Permalink"></a></h2><p>In some cases it may be useful to save only the model parameters themselves, and rebuild the model architecture in your code. You can use <code>params(model)</code> to get model parameters. You can also use <code>data.(params)</code> to remove tracking.</p><pre><codeclass="language-Julia">julia> using Flux
julia> @save "mymodel.bson" weights</code></pre><p>You can easily load parameters back into a model with <code>Flux.loadparams!</code>.</p><pre><codeclass="language-julia">julia> using Flux
julia> model = Chain(Dense(10,5,relu),Dense(5,2),softmax)
julia> Flux.loadparams!(model, weights)</code></pre><p>The new <code>model</code> we created will now be identical to the one we saved parameters for.</p><h2id="Checkpointing-1"><aclass="docs-heading-anchor"href="#Checkpointing-1">Checkpointing</a><aclass="docs-heading-anchor-permalink"href="#Checkpointing-1"title="Permalink"></a></h2><p>In longer training runs it's a good idea to periodically save your model, so that you can resume if training is interrupted (for example, if there's a power cut). You can do this by saving the model in the <ahref="../training/training/">callback provided to <code>train!</code></a>.</p><pre><codeclass="language-julia">using Flux: throttle
end</code></pre><p>This will update the <code>"model-checkpoint.bson"</code> file every thirty seconds.</p><p>You can get more advanced by saving a series of models throughout training, for example</p><pre><codeclass="language-julia">@save "model-$(now()).bson" model</code></pre><p>will produce a series of models like <code>"model-2018-03-06T02:57:10.41.bson"</code>. You could also store the current test set loss, so that it's easy to (for example) revert to an older copy of the model if it starts to overfit.</p><pre><codeclass="language-julia">@save "model-$(now()).bson" model loss = testloss()</code></pre><p>You can even store optimiser state alongside the model, to resume training exactly where you left off.</p><pre><codeclass="language-julia">opt = ADAM()
@save "model-$(now()).bson" model opt</code></pre></article><navclass="docs-footer"><aclass="docs-footer-prevpage"href="../gpu/">« GPU Support</a><aclass="docs-footer-nextpage"href="../ecosystem/">The Julia Ecosystem »</a></nav></div><divclass="modal"id="documenter-settings"><divclass="modal-background"></div><divclass="modal-card"><headerclass="modal-card-head"><pclass="modal-card-title">Settings</p><buttonclass="delete"></button></header><sectionclass="modal-card-body"><p><labelclass="label">Theme</label><divclass="select"><selectid="documenter-themepicker"><optionvalue="documenter-light">documenter-light</option><optionvalue="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <ahref="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <spanclass="colophon-date"title="Thursday 26 March 2020 10:07">Thursday 26 March 2020</span>. Using Julia version 1.3.1.</p></section><footerclass="modal-card-foot"></footer></div></div></div></body></html>
