51 lines
10 KiB
HTML
51 lines
10 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en"><head><meta charset="UTF-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><title>Saving & Loading · Flux</title><script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
|
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
|
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
|
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
|
|
|
ga('create', 'UA-36890222-9', 'auto');
|
|
ga('send', 'pageview', {'page': location.pathname + location.search + location.hash});
|
|
</script><link href="https://fonts.googleapis.com/css?family=Lato|Roboto+Mono" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/fontawesome.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/solid.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/brands.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.css" rel="stylesheet" type="text/css"/><script>documenterBaseURL=".."</script><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js" data-main="../assets/documenter.js"></script><script src="../siteinfo.js"></script><script src="../../versions.js"></script><link href="../assets/flux.css" rel="stylesheet" type="text/css"/><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../assets/themes/documenter-dark.css" data-theme-name="documenter-dark"/><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../assets/themes/documenter-light.css" data-theme-name="documenter-light" data-theme-primary/><script src="../assets/themeswap.js"></script></head><body><div id="documenter"><nav class="docs-sidebar"><div class="docs-package-name"><span class="docs-autofit">Flux</span></div><form class="docs-search" action="../search/"><input class="docs-search-query" id="documenter-search-query" name="q" type="text" placeholder="Search docs"/></form><ul class="docs-menu"><li><a class="tocitem" href="../">Home</a></li><li><span class="tocitem">Building Models</span><ul><li><a class="tocitem" href="../models/basics/">Basics</a></li><li><a class="tocitem" href="../models/recurrence/">Recurrence</a></li><li><a class="tocitem" href="../models/regularisation/">Regularisation</a></li><li><a class="tocitem" href="../models/layers/">Model Reference</a></li><li><a class="tocitem" href="../models/advanced/">Advanced Model Building</a></li><li><a class="tocitem" href="../models/nnlib/">NNlib</a></li></ul></li><li><span class="tocitem">Handling Data</span><ul><li><a class="tocitem" href="../data/onehot/">One-Hot Encoding</a></li><li><a class="tocitem" href="../data/dataloader/">DataLoader</a></li></ul></li><li><span class="tocitem">Training Models</span><ul><li><a class="tocitem" href="../training/optimisers/">Optimisers</a></li><li><a class="tocitem" href="../training/training/">Training</a></li></ul></li><li><a class="tocitem" href="../gpu/">GPU Support</a></li><li class="is-active"><a class="tocitem" href>Saving & Loading</a><ul class="internal"><li><a class="tocitem" href="#Saving-Model-Weights-1"><span>Saving Model Weights</span></a></li><li><a class="tocitem" href="#Checkpointing-1"><span>Checkpointing</span></a></li></ul></li><li><a class="tocitem" href="../ecosystem/">The Julia Ecosystem</a></li><li><a class="tocitem" href="../utilities/">Utility Functions</a></li><li><a class="tocitem" href="../performance/">Performance Tips</a></li><li><a class="tocitem" href="../datasets/">Datasets</a></li><li><a class="tocitem" href="../community/">Community</a></li></ul><div class="docs-version-selector field has-addons"><div class="control"><span class="docs-label button is-static is-size-7">Version</span></div><div class="docs-selector control is-expanded"><div class="select is-fullwidth is-size-7"><select id="documenter-version-selector"></select></div></div></div></nav><div class="docs-main"><header class="docs-navbar"><nav class="breadcrumb"><ul class="is-hidden-mobile"><li class="is-active"><a href>Saving & Loading</a></li></ul><ul class="is-hidden-tablet"><li class="is-active"><a href>Saving & Loading</a></li></ul></nav><div class="docs-right"><a class="docs-edit-link" href="https://github.com/FluxML/Flux.jl/blob/master/docs/src/saving.md" title="Edit on GitHub"><span class="docs-icon fab"></span><span class="docs-label is-hidden-touch">Edit on GitHub</span></a><a class="docs-settings-button fas fa-cog" id="documenter-settings-button" href="#" title="Settings"></a><a class="docs-sidebar-button fa fa-bars is-hidden-desktop" id="documenter-sidebar-button" href="#"></a></div></header><article class="content" id="documenter-page"><h1 id="Saving-and-Loading-Models-1"><a class="docs-heading-anchor" href="#Saving-and-Loading-Models-1">Saving and Loading Models</a><a class="docs-heading-anchor-permalink" href="#Saving-and-Loading-Models-1" title="Permalink"></a></h1><p>You may wish to save models so that they can be loaded and run in a later session. The easiest way to do this is via <a href="https://github.com/MikeInnes/BSON.jl">BSON.jl</a>.</p><p>Save a model:</p><pre><code class="language-julia">julia> using Flux
|
|
|
|
julia> model = Chain(Dense(10,5,relu),Dense(5,2),softmax)
|
|
Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
|
|
|
|
julia> using BSON: @save
|
|
|
|
julia> @save "mymodel.bson" model</code></pre><p>Load it again:</p><pre><code class="language-julia">julia> using Flux
|
|
|
|
julia> using BSON: @load
|
|
|
|
julia> @load "mymodel.bson" model
|
|
|
|
julia> model
|
|
Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)</code></pre><p>Models are just normal Julia structs, so it's fine to use any Julia storage format for this purpose. BSON.jl is particularly well supported and most likely to be forwards compatible (that is, models saved now will load in future versions of Flux).</p><div class="admonition is-info"><header class="admonition-header">Note</header><div class="admonition-body"><p>If a saved model's weights are stored on the GPU, the model will not load later on if there is no GPU support available. It's best to <a href="../gpu/">move your model to the CPU</a> with <code>cpu(model)</code> before saving it.</p></div></div><h2 id="Saving-Model-Weights-1"><a class="docs-heading-anchor" href="#Saving-Model-Weights-1">Saving Model Weights</a><a class="docs-heading-anchor-permalink" href="#Saving-Model-Weights-1" title="Permalink"></a></h2><p>In some cases it may be useful to save only the model parameters themselves, and rebuild the model architecture in your code. You can use <code>params(model)</code> to get model parameters. You can also use <code>data.(params)</code> to remove tracking.</p><pre><code class="language-Julia">julia> using Flux
|
|
|
|
julia> model = Chain(Dense(10,5,relu),Dense(5,2),softmax)
|
|
Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
|
|
|
|
julia> weights = params(model);
|
|
|
|
julia> using BSON: @save
|
|
|
|
julia> @save "mymodel.bson" weights</code></pre><p>You can easily load parameters back into a model with <code>Flux.loadparams!</code>.</p><pre><code class="language-julia">julia> using Flux
|
|
|
|
julia> model = Chain(Dense(10,5,relu),Dense(5,2),softmax)
|
|
Chain(Dense(10, 5, NNlib.relu), Dense(5, 2), NNlib.softmax)
|
|
|
|
julia> using BSON: @load
|
|
|
|
julia> @load "mymodel.bson" weights
|
|
|
|
julia> Flux.loadparams!(model, weights)</code></pre><p>The new <code>model</code> we created will now be identical to the one we saved parameters for.</p><h2 id="Checkpointing-1"><a class="docs-heading-anchor" href="#Checkpointing-1">Checkpointing</a><a class="docs-heading-anchor-permalink" href="#Checkpointing-1" title="Permalink"></a></h2><p>In longer training runs it's a good idea to periodically save your model, so that you can resume if training is interrupted (for example, if there's a power cut). You can do this by saving the model in the <a href="../training/training/">callback provided to <code>train!</code></a>.</p><pre><code class="language-julia">using Flux: throttle
|
|
using BSON: @save
|
|
|
|
m = Chain(Dense(10,5,relu),Dense(5,2),softmax)
|
|
|
|
evalcb = throttle(30) do
|
|
# Show loss
|
|
@save "model-checkpoint.bson" model
|
|
end</code></pre><p>This will update the <code>"model-checkpoint.bson"</code> file every thirty seconds.</p><p>You can get more advanced by saving a series of models throughout training, for example</p><pre><code class="language-julia">@save "model-$(now()).bson" model</code></pre><p>will produce a series of models like <code>"model-2018-03-06T02:57:10.41.bson"</code>. You could also store the current test set loss, so that it's easy to (for example) revert to an older copy of the model if it starts to overfit.</p><pre><code class="language-julia">@save "model-$(now()).bson" model loss = testloss()</code></pre><p>You can even store optimiser state alongside the model, to resume training exactly where you left off.</p><pre><code class="language-julia">opt = ADAM()
|
|
@save "model-$(now()).bson" model opt</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../gpu/">« GPU Support</a><a class="docs-footer-nextpage" href="../ecosystem/">The Julia Ecosystem »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Wednesday 27 May 2020 11:52">Wednesday 27 May 2020</span>. Using Julia version 1.3.1.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
|