63 lines
11 KiB
HTML
63 lines
11 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en"><head><meta charset="UTF-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><title>GPU Support · Flux</title><script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||
|
||
ga('create', 'UA-36890222-9', 'auto');
|
||
ga('send', 'pageview', {'page': location.pathname + location.search + location.hash});
|
||
</script><link href="https://fonts.googleapis.com/css?family=Lato|Roboto+Mono" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/fontawesome.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/solid.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.11.2/css/brands.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.11.1/katex.min.css" rel="stylesheet" type="text/css"/><script>documenterBaseURL=".."</script><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js" data-main="../assets/documenter.js"></script><script src="../siteinfo.js"></script><script src="../../versions.js"></script><link href="../assets/flux.css" rel="stylesheet" type="text/css"/><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../assets/themes/documenter-dark.css" data-theme-name="documenter-dark"/><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../assets/themes/documenter-light.css" data-theme-name="documenter-light" data-theme-primary/><script src="../assets/themeswap.js"></script></head><body><div id="documenter"><nav class="docs-sidebar"><div class="docs-package-name"><span class="docs-autofit">Flux</span></div><form class="docs-search" action="../search/"><input class="docs-search-query" id="documenter-search-query" name="q" type="text" placeholder="Search docs"/></form><ul class="docs-menu"><li><a class="tocitem" href="../">Home</a></li><li><span class="tocitem">Building Models</span><ul><li><a class="tocitem" href="../models/basics/">Basics</a></li><li><a class="tocitem" href="../models/recurrence/">Recurrence</a></li><li><a class="tocitem" href="../models/regularisation/">Regularisation</a></li><li><a class="tocitem" href="../models/layers/">Model Reference</a></li><li><a class="tocitem" href="../models/nnlib/">NNlib</a></li></ul></li><li><span class="tocitem">Handling Data</span><ul><li><a class="tocitem" href="../data/onehot/">One-Hot Encoding</a></li><li><a class="tocitem" href="../data/dataloader/">DataLoader</a></li></ul></li><li><span class="tocitem">Training Models</span><ul><li><a class="tocitem" href="../training/optimisers/">Optimisers</a></li><li><a class="tocitem" href="../training/training/">Training</a></li></ul></li><li class="is-active"><a class="tocitem" href>GPU Support</a><ul class="internal"><li><a class="tocitem" href="#GPU-Usage-1"><span>GPU Usage</span></a></li></ul></li><li><a class="tocitem" href="../saving/">Saving & Loading</a></li><li><a class="tocitem" href="../ecosystem/">The Julia Ecosystem</a></li><li><a class="tocitem" href="../performance/">Performance Tips</a></li><li><a class="tocitem" href="../community/">Community</a></li></ul><div class="docs-version-selector field has-addons"><div class="control"><span class="docs-label button is-static is-size-7">Version</span></div><div class="docs-selector control is-expanded"><div class="select is-fullwidth is-size-7"><select id="documenter-version-selector"></select></div></div></div></nav><div class="docs-main"><header class="docs-navbar"><nav class="breadcrumb"><ul class="is-hidden-mobile"><li class="is-active"><a href>GPU Support</a></li></ul><ul class="is-hidden-tablet"><li class="is-active"><a href>GPU Support</a></li></ul></nav><div class="docs-right"><a class="docs-edit-link" href="https://github.com/FluxML/Flux.jl/blob/master/docs/src/gpu.md" title="Edit on GitHub"><span class="docs-icon fab"></span><span class="docs-label is-hidden-touch">Edit on GitHub</span></a><a class="docs-settings-button fas fa-cog" id="documenter-settings-button" href="#" title="Settings"></a><a class="docs-sidebar-button fa fa-bars is-hidden-desktop" id="documenter-sidebar-button" href="#"></a></div></header><article class="content" id="documenter-page"><h1 id="GPU-Support-1"><a class="docs-heading-anchor" href="#GPU-Support-1">GPU Support</a><a class="docs-heading-anchor-permalink" href="#GPU-Support-1" title="Permalink"></a></h1><p>NVIDIA GPU support should work out of the box on systems with CUDA and CUDNN installed. For more details see the <a href="https://github.com/JuliaGPU/CuArrays.jl">CuArrays</a> readme.</p><h2 id="GPU-Usage-1"><a class="docs-heading-anchor" href="#GPU-Usage-1">GPU Usage</a><a class="docs-heading-anchor-permalink" href="#GPU-Usage-1" title="Permalink"></a></h2><p>Support for array operations on other hardware backends, like GPUs, is provided by external packages like <a href="https://github.com/JuliaGPU/CuArrays.jl">CuArrays</a>. Flux is agnostic to array types, so we simply need to move model weights and data to the GPU and Flux will handle it.</p><p>For example, we can use <code>CuArrays</code> (with the <code>cu</code> converter) to run our <a href="../models/basics/">basic example</a> on an NVIDIA GPU.</p><p>(Note that you need to have CUDA available to use CuArrays – please see the <a href="https://github.com/JuliaGPU/CuArrays.jl">CuArrays.jl</a> instructions for more details.)</p><pre><code class="language-julia">using CuArrays
|
||
|
||
W = cu(rand(2, 5)) # a 2×5 CuArray
|
||
b = cu(rand(2))
|
||
|
||
predict(x) = W*x .+ b
|
||
loss(x, y) = sum((predict(x) .- y).^2)
|
||
|
||
x, y = cu(rand(5)), cu(rand(2)) # Dummy data
|
||
loss(x, y) # ~ 3</code></pre><p>Note that we convert both the parameters (<code>W</code>, <code>b</code>) and the data set (<code>x</code>, <code>y</code>) to cuda arrays. Taking derivatives and training works exactly as before.</p><p>If you define a structured model, like a <code>Dense</code> layer or <code>Chain</code>, you just need to convert the internal parameters. Flux provides <code>fmap</code>, which allows you to alter all parameters of a model at once.</p><pre><code class="language-julia">d = Dense(10, 5, σ)
|
||
d = fmap(cu, d)
|
||
d.W # CuArray
|
||
d(cu(rand(10))) # CuArray output
|
||
|
||
m = Chain(Dense(10, 5, σ), Dense(5, 2), softmax)
|
||
m = fmap(cu, m)
|
||
d(cu(rand(10)))</code></pre><p>However, if you create a customized model, <code>fmap</code> may not work out of the box.</p><pre><code class="language-julia">julia> struct ActorCritic{A, C}
|
||
actor::A
|
||
critic::C
|
||
end
|
||
|
||
julia> m = ActorCritic(ones(2,2), ones(2))
|
||
ActorCritic{Array{Float64,2},Array{Float64,1}}([1.0 1.0; 1.0 1.0], [1.0, 1.0])
|
||
|
||
julia> fmap(cu, m)
|
||
ActorCritic{Array{Float64,2},Array{Float64,1}}([1.0 1.0; 1.0 1.0], [1.0, 1.0])</code></pre><p>As you can see, nothing changed after <code>fmap(cu, m)</code>. The reason is that <code>Flux</code> doesn't know your customized model structure. To make it work as expected, you need the <code>@functor</code> macro.</p><pre><code class="language-julia">julia> Flux.@functor ActorCritic
|
||
|
||
julia> fmap(cu, m)
|
||
ActorCritic{CuArray{Float32,2,Nothing},CuArray{Float32,1,Nothing}}(Float32[1.0 1.0; 1.0 1.0], Float32[1.0, 1.0])</code></pre><p>Now you can see that the inner fields of <code>actor</code> and <code>critic</code> are transformed into <code>CuArray</code>. So what does the <code>@functor</code> macro do here? Basically, it will create a function like this:</p><pre><code class="language-julia">Flux.functor(m::ActorCritic) = (actor = m.actor, critic=m.critic), fields -> ActorCritic(fields...)</code></pre><p>And the <code>functor</code> will be called recursively in <code>fmap</code>. As you can see, the result of <code>functor</code> contains two parts, a <em>destructure</em> part and a <em>reconstrucutre</em> part. The first part is to make the customized model structure into <code>trainable</code> data structure known to <code>Flux</code> (here is a <code>NamedTuple</code>). The goal is to turn <code>m</code> into <code>(actor=cu(ones(2,2)), critic=cu(ones(2)))</code>. The second part is to turn the result back into a <code>ActorCritic</code>, so that we can get <code>ActorCritic(cu(ones(2,2)),cu(ones(2)))</code>.</p><p>By default, the <code>@functor</code> macro will transform all the fields in your customized structure. In some cases, you may only want to transform several fields. Then you just specify those fields manually like <code>Flux.@functor ActorCritic (actor,)</code> (note that the fields part must be a tuple). And make sure the <code>ActorCritic(actor)</code> constructor is also implemented.</p><p>As a convenience, Flux provides the <code>gpu</code> function to convert models and data to the GPU if one is available. By default, it'll do nothing, but loading <code>CuArrays</code> will cause it to move data to the GPU instead.</p><pre><code class="language-julia">julia> using Flux, CuArrays
|
||
|
||
julia> m = Dense(10,5) |> gpu
|
||
Dense(10, 5)
|
||
|
||
julia> x = rand(10) |> gpu
|
||
10-element CuArray{Float32,1}:
|
||
0.800225
|
||
⋮
|
||
0.511655
|
||
|
||
julia> m(x)
|
||
5-element CuArray{Float32,1}:
|
||
-0.30535
|
||
⋮
|
||
-0.618002</code></pre><p>The analogue <code>cpu</code> is also available for moving models and data back off of the GPU.</p><pre><code class="language-julia">julia> x = rand(10) |> gpu
|
||
10-element CuArray{Float32,1}:
|
||
0.235164
|
||
⋮
|
||
0.192538
|
||
|
||
julia> x |> cpu
|
||
10-element Array{Float32,1}:
|
||
0.235164
|
||
⋮
|
||
0.192538</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../training/training/">« Training</a><a class="docs-footer-nextpage" href="../saving/">Saving & Loading »</a></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> on <span class="colophon-date" title="Tuesday 3 March 2020 17:50">Tuesday 3 March 2020</span>. Using Julia version 1.3.1.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>
|