<htmllang="en"><head><metacharset="UTF-8"/><metaname="viewport"content="width=device-width, initial-scale=1.0"/><title>Advanced Model Building · Flux</title><script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
</script><linkhref="https://fonts.googleapis.com/css?family=Lato|Roboto+Mono"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/fontawesome.min.css"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/solid.min.css"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/brands.min.css"rel="stylesheet"type="text/css"/><linkhref="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.css"rel="stylesheet"type="text/css"/><script>documenterBaseURL="../.."</script><scriptsrc="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js"data-main="../../assets/documenter.js"></script><scriptsrc="../../siteinfo.js"></script><scriptsrc="../../../versions.js"></script><linkhref="../../assets/flux.css"rel="stylesheet"type="text/css"/><linkclass="docs-theme-link"rel="stylesheet"type="text/css"href="../../assets/themes/documenter-dark.css"data-theme-name="documenter-dark"/><linkclass="docs-theme-link"rel="stylesheet"type="text/css"href="../../assets/themes/documenter-light.css"data-theme-name="documenter-light"data-theme-primary/><scriptsrc="../../assets/themeswap.js"></script></head><body><divid="documenter"><navclass="docs-sidebar"><divclass="docs-package-name"><spanclass="docs-autofit">Flux</span></div><formclass="docs-search"action="../../search/"><inputclass="docs-search-query"id="documenter-search-query"name="q"type="text"placeholder="Search docs"/></form><ulclass="docs-menu"><li><aclass="tocitem"href="../../">Home</a></li><li><spanclass="tocitem">Building Models</span><ul><li><aclass="tocitem"href="../basics/">Basics</a></li><li><aclass="tocitem"href="../recurrence/">Recurrence</a></li><li><aclass="tocitem"href="../regularisation/">Regularisation</a></li><li><aclass="tocitem"href="../layers/">Model Reference</a></li><liclass="is-active"><aclass="tocitem"href>Advanced Model Building</a><ulclass="internal"><li><aclass="tocitem"href="#Customising-Parameter-Collection-for-a-Model-1"><span>Customising Parameter Collection for a Model</span></a></li><li><aclass="tocitem"href="#Freezing-Layer-Parameters-1"><span>Freezing Layer Parameters</span></a></li></ul></li><li><aclass="tocitem"href="../nnlib/">NNlib</a></li></ul></li><li><spanclass="tocitem">Handling Data</span><ul><li><aclass="tocitem"href="../../data/onehot/">One-Hot Encoding</a></li><li><aclass="tocitem"href="../../data/dataloader/">DataLoader</a></li></ul></li><li><spanclass="tocitem">Training Models</span><ul><li><aclass="tocitem"href="../../training/optimisers/">Optimisers</a></li><li><aclass="tocitem"href="../../training/training/">Training</a></li></ul></li><li><aclass="tocitem"href="../../gpu/">GPU Support</a></li><li><aclass="tocitem"href="../../saving/">Saving & Loading</a></li><li><aclass="tocitem"href="../../ecosystem/">The Julia Ecosystem</a></li><li><aclass="tocitem"href="../../utilities/">Utility Functions</a></li><li><aclass="tocitem"href="../../performance/">Performance Tips</a></li><li><aclass="tocitem"href="../../datasets/">Datasets</a></li><li><aclass="tocitem"href="../../community/">Community</a></li></ul><divclass="docs-version-selector field has-addons"><divclass="control"><spanclass="docs-label button is-static is-size-7">Version</span></div><divclass="docs-selector control is-expanded"><divclass="select is-fullwidth is-size-7"><selectid="documenter-version-selector"></select></div></div></div></nav><divclass="docs-main"><headerclass="docs-navbar"><navclass="breadcrumb"><ulclass="is-hidden-mobile"><li><aclass="is-disabled">Building Models</a></li><liclass="is-active"><ahref>Advanced Model Building</a></li></ul><ulclass="is-hidden-tablet"><liclass="is-active"><ahref>Advanced Model Building</a></li></ul></nav><divclass="docs-right"><aclass="docs-edit-link"href="https://github.com/FluxML/Flux.jl/blob/master/docs/src/models/advanced.md"title="Edit on GitHub"><spancla
Params([[0.66722 0.774872 0.249809; 0.843321 0.403843 0.429232; 0.683525 0.662455 0.065297]])</code></pre><p>Only the fields returned by <code>trainable</code> will be collected as trainable parameters of the layer when calling <code>Flux.params</code>.</p><p>Another way of achieving this is through the <code>@functor</code> macro directly. Here, we can mark the fields we are interested in by grouping them in the second argument:</p><pre><codeclass="language-julia">Flux.@functor Affine (W,)</code></pre><p>However, doing this requires the <code>struct</code> to have a corresponding constructor that accepts those parameters.</p><h2id="Freezing-Layer-Parameters-1"><aclass="docs-heading-anchor"href="#Freezing-Layer-Parameters-1">Freezing Layer Parameters</a><aclass="docs-heading-anchor-permalink"href="#Freezing-Layer-Parameters-1"title="Permalink"></a></h2><p>When it is desired to not include all the model parameters (for e.g. transfer learning), we can simply not pass in those layers into our call to <code>params</code>.</p><p>Consider a simple multi-layer perceptron model where we want to avoid optimising the first two <code>Dense</code> layers. We can obtain this using the slicing features <code>Chain</code> provides:</p><pre><codeclass="language-julia">m = Chain(
ps = Flux.params(m[3:end])</code></pre><p>The <code>Zygote.Params</code> object <code>ps</code> now holds a reference to only the parameters of the layers passed to it.</p><p>During training, the gradients will only be computed for (and applied to) the last <code>Dense</code> layer, therefore only that would have its parameters changed.</p><p><code>Flux.params</code> also takes multiple inputs to make it easy to collect parameters from heterogenous models with a single call. A simple demonstration would be if we wanted to omit optimising the second <code>Dense</code> layer in the previous example. It would look something like this:</p><pre><codeclass="language-julia">Flux.params(m[1], m[3:end])</code></pre><p>Sometimes, a more fine-tuned control is needed. We can freeze a specific parameter of a specific layer which already entered a <code>Params</code> object <code>ps</code>, by simply deleting it from <code>ps</code>:</p><pre><codeclass="language-julia">ps = params(m)
delete!(ps, m[2].b) </code></pre></article><navclass="docs-footer"><aclass="docs-footer-prevpage"href="../layers/">« Model Reference</a><aclass="docs-footer-nextpage"href="../nnlib/">NNlib »</a></nav></div><divclass="modal"id="documenter-settings"><divclass="modal-background"></div><divclass="modal-card"><headerclass="modal-card-head"><pclass="modal-card-title">Settings</p><buttonclass="delete"></button></header><sectionclass="modal-card-body"><p><labelclass="label">Theme</label><divclass="select"><selectid="documenter-themepicker"><optionvalue="documenter-light">documenter-light</option><optionvalue="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <ahref="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <spanclass="colophon-date"title="Thursday 21 May 2020 13:20">Thursday 21 May 2020</span>. Using Julia version 1.3.1.</p></section><footerclass="modal-card-foot"></footer></div></div></div></body></html>
