TastyPancakes/Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

build based on de0edf9

Browse Source
This commit is contained in:
autodocs 2017-02-18 15:11:53 +00:00
parent 9b7fc12310
commit 8e3d4682f5
12 changed files with 471 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

160
latest/backends.html Normal file
View File

@ -0,0 +1,160 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>
Backends · Flux
</title>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-36890222-9', 'auto');
ga('send', 'pageview');
</script>
<link href="https://cdnjs.cloudflare.com/ajax/libs/normalize/4.2.0/normalize.min.css" rel="stylesheet" type="text/css"/>
<link href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.5.0/styles/default.min.css" rel="stylesheet" type="text/css"/>
<link href="https://fonts.googleapis.com/css?family=Lato|Ubuntu+Mono" rel="stylesheet" type="text/css"/>
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css" rel="stylesheet" type="text/css"/>
<link href="assets/documenter.css" rel="stylesheet" type="text/css"/>
<script>
documenterBaseURL="."
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.2.0/require.min.js" data-main="assets/documenter.js"></script>
<script src="../versions.js"></script>
<link href="../flux.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<nav class="toc">
<h1>
Flux
</h1>
<form class="search" action="search.html">
<select id="version-selector" onChange="window.location.href=this.value">
<option value="#" selected="selected" disabled="disabled">
Version
</option>
</select>
<input id="search-query" name="q" type="text" placeholder="Search docs"/>
</form>
<ul>
<li>
<a class="toctext" href="index.html">
Home
</a>
</li>
<li>
<span class="toctext">
Building Models
</span>
<ul>
<li>
<a class="toctext" href="models/basics.html">
Model Building Basics
</a>
</li>
<li>
<a class="toctext" href="models/templates.html">
Model Templates
</a>
</li>
<li>
<a class="toctext" href="models/recurrent.html">
Recurrence
</a>
</li>
<li>
<a class="toctext" href="models/debugging.html">
Debugging
</a>
</li>
</ul>
</li>
<li>
<a class="toctext" href="batching.html">
Batching
</a>
</li>
<li class="current">
<a class="toctext" href="backends.html">
Backends
</a>
<ul class="internal"></ul>
</li>
<li>
<span class="toctext">
In Action
</span>
<ul>
<li>
<a class="toctext" href="examples/logreg.html">
Logistic Regression
</a>
</li>
</ul>
</li>
<li>
<a class="toctext" href="contributing.html">
Contributing &amp; Help
</a>
</li>
<li>
<a class="toctext" href="internals.html">
Internals
</a>
</li>
</ul>
</nav>
<article id="docs">
<header>
<nav>
<ul>
<li>
<a href="backends.html">
Backends
</a>
</li>
</ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/backends.md">
<span class="fa">
</span>
Edit on GitHub
</a>
</nav>
<hr/>
</header>
<h1>
<a class="nav-anchor" id="Batching-1" href="#Batching-1">
Batching
</a>
</h1>
<p>
[WIP]
</p>
<footer>
<hr/>
<a class="previous" href="batching.html">
<span class="direction">
Previous
</span>
<span class="title">
Batching
</span>
</a>
<a class="next" href="examples/logreg.html">
<span class="direction">
Next
</span>
<span class="title">
Logistic Regression
</span>
</a>
</footer>
</article>
</body>
</html>

160
latest/batching.html Normal file
View File

@ -0,0 +1,160 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>
Batching · Flux
</title>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-36890222-9', 'auto');
ga('send', 'pageview');
</script>
<link href="https://cdnjs.cloudflare.com/ajax/libs/normalize/4.2.0/normalize.min.css" rel="stylesheet" type="text/css"/>
<link href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.5.0/styles/default.min.css" rel="stylesheet" type="text/css"/>
<link href="https://fonts.googleapis.com/css?family=Lato|Ubuntu+Mono" rel="stylesheet" type="text/css"/>
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css" rel="stylesheet" type="text/css"/>
<link href="assets/documenter.css" rel="stylesheet" type="text/css"/>
<script>
documenterBaseURL="."
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.2.0/require.min.js" data-main="assets/documenter.js"></script>
<script src="../versions.js"></script>
<link href="../flux.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<nav class="toc">
<h1>
Flux
</h1>
<form class="search" action="search.html">
<select id="version-selector" onChange="window.location.href=this.value">
<option value="#" selected="selected" disabled="disabled">
Version
</option>
</select>
<input id="search-query" name="q" type="text" placeholder="Search docs"/>
</form>
<ul>
<li>
<a class="toctext" href="index.html">
Home
</a>
</li>
<li>
<span class="toctext">
Building Models
</span>
<ul>
<li>
<a class="toctext" href="models/basics.html">
Model Building Basics
</a>
</li>
<li>
<a class="toctext" href="models/templates.html">
Model Templates
</a>
</li>
<li>
<a class="toctext" href="models/recurrent.html">
Recurrence
</a>
</li>
<li>
<a class="toctext" href="models/debugging.html">
Debugging
</a>
</li>
</ul>
</li>
<li class="current">
<a class="toctext" href="batching.html">
Batching
</a>
<ul class="internal"></ul>
</li>
<li>
<a class="toctext" href="backends.html">
Backends
</a>
</li>
<li>
<span class="toctext">
In Action
</span>
<ul>
<li>
<a class="toctext" href="examples/logreg.html">
Logistic Regression
</a>
</li>
</ul>
</li>
<li>
<a class="toctext" href="contributing.html">
Contributing &amp; Help
</a>
</li>
<li>
<a class="toctext" href="internals.html">
Internals
</a>
</li>
</ul>
</nav>
<article id="docs">
<header>
<nav>
<ul>
<li>
<a href="batching.html">
Batching
</a>
</li>
</ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/batching.md">
<span class="fa">
</span>
Edit on GitHub
</a>
</nav>
<hr/>
</header>
<h1>
<a class="nav-anchor" id="Batching-1" href="#Batching-1">
Batching
</a>
</h1>
<p>
[WIP]
</p>
<footer>
<hr/>
<a class="previous" href="models/debugging.html">
<span class="direction">
Previous
</span>
<span class="title">
Debugging
</span>
</a>
<a class="next" href="backends.html">
<span class="direction">
Next
</span>
<span class="title">
Backends
</span>
</a>
</footer>
</article>
</body>
</html>

12
latest/contributing.html
View File

@ -74,6 +74,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -109,7 +119,7 @@ Contributing &amp; Help
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/contributing.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/contributing.md">
<span class="fa"> <span class="fa">
</span> </span>

16
latest/examples/logreg.html
View File

@ -74,6 +74,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="../batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="../backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -112,7 +122,7 @@ Logistic Regression
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/examples/logreg.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/examples/logreg.md">
<span class="fa"> <span class="fa">
</span> </span>
@ -131,12 +141,12 @@ Logistic Regression with MNIST
</p> </p>
<footer> <footer>
<hr/> <hr/>
<a class="previous" href="../models/debugging.html"> <a class="previous" href="../backends.html">
<span class="direction"> <span class="direction">
Previous Previous
</span> </span>
<span class="title"> <span class="title">
Debugging Backends
</span> </span>
</a> </a>
<a class="next" href="../contributing.html"> <a class="next" href="../contributing.html">

12
latest/index.html
View File

@ -81,6 +81,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -115,7 +125,7 @@ Home
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/index.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/index.md">
<span class="fa"> <span class="fa">
</span> </span>

12
latest/internals.html
View File

@ -74,6 +74,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -109,7 +119,7 @@ Internals
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/internals.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/internals.md">
<span class="fa"> <span class="fa">
</span> </span>

12
latest/models/basics.html
View File

@ -91,6 +91,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="../batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="../backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -128,7 +138,7 @@ Model Building Basics
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/models/basics.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/models/basics.md">
<span class="fa"> <span class="fa">
</span> </span>

16
latest/models/debugging.html
View File

@ -75,6 +75,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="../batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="../backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -112,7 +122,7 @@ Debugging
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/models/debugging.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/models/debugging.md">
<span class="fa"> <span class="fa">
</span> </span>
@ -218,12 +228,12 @@ Previous
Recurrence Recurrence
</span> </span>
</a> </a>
<a class="next" href="../examples/logreg.html"> <a class="next" href="../batching.html">
<span class="direction"> <span class="direction">
Next Next
</span> </span>
<span class="title"> <span class="title">
Logistic Regression Batching
</span> </span>
</a> </a>
</footer> </footer>

12
latest/models/recurrent.html
View File

@ -75,6 +75,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="../batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="../backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -112,7 +122,7 @@ Recurrence
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/models/recurrent.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/models/recurrent.md">
<span class="fa"> <span class="fa">
</span> </span>

33
latest/models/templates.html
View File

@ -91,6 +91,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="../batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="../backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action
@ -128,7 +138,7 @@ Model Templates
</a> </a>
</li> </li>
</ul> </ul>
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/ba22a5cedce39c2f7d38b4468f15e7eb45cc1f47/docs/src/models/templates.md"> <a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/de0edf94456d9d826fc8543e1178eccf5ab035f7/docs/src/models/templates.md">
<span class="fa"> <span class="fa">
</span> </span>
@ -183,16 +193,20 @@ This is much better: we can now make as many affine layers as we want. This is a
<pre><code class="language-julia">@net type MyAffine <pre><code class="language-julia">@net type MyAffine
W W
b b
x -&gt; W * x + b x -&gt; x * W + b
end</code></pre> end</code></pre>
<p> <p>
The function provided, The function provided,
<code>x -&gt; W * x + b</code> <code>x -&gt; x * W + b</code>
, will be used when , will be used when
<code>MyAffine</code> <code>MyAffine</code>
is used as a model; it&#39;s just a shorter way of defining the is used as a model; it&#39;s just a shorter way of defining the
<code>(::MyAffine)(x)</code> <code>(::MyAffine)(x)</code>
method above. method above. (You may notice that
<code>W</code>
and
<code>x</code>
have swapped order in the model; this is due to the way batching works, which will be covered in more detail later on.)
</p> </p>
<p> <p>
However, However,
@ -244,7 +258,7 @@ Clearly, the
<code>x</code> <code>x</code>
. The . The
<code>Affine</code> <code>Affine</code>
layer fits the bill so we can instantiate layer fits the bill, so we can instantiate
<code>TLP</code> <code>TLP</code>
with two of them: with two of them:
</p> </p>
@ -290,12 +304,13 @@ However, for convenience and to avoid errors, we&#39;d probably rather specify t
<p> <p>
This is easy to implement using the usual Julia syntax for constructors: This is easy to implement using the usual Julia syntax for constructors:
</p> </p>
<pre><code class="language-julia">Affine(in::Integer, out::Integer) = Affine(randn(in, out), randn(1, out))</code></pre> <pre><code class="language-julia">Affine(in::Integer, out::Integer) =
Affine(randn(in, out), randn(1, out))</code></pre>
<p> <p>
In practice, these constructors tend to take the parameter initialisation function as an argument so that it&#39;s more easily customisable, and use In practice, these constructors tend to take the parameter initialisation function as an argument so that it&#39;s more easily customisable, and use
<code>Flux.initn</code> <code>Flux.initn</code>
by default (which is equivalent to by default (which is equivalent to
<code>randn()/100</code> <code>randn(...)/100</code>
). So ). So
<code>Affine</code> <code>Affine</code>
&#39;s constructor really looks like this: &#39;s constructor really looks like this:
@ -309,10 +324,10 @@ Supported syntax
</h2> </h2>
<p> <p>
The syntax used to define a forward pass like The syntax used to define a forward pass like
<code>x -&gt; W*x + b</code> <code>x -&gt; x*W + b</code>
behaves exactly like Julia code for the most part. However, it&#39;s important to remember that it&#39;s defining a dataflow graph, not a general Julia expression. In practice this means that anything side-effectful, or things like control flow and behaves exactly like Julia code for the most part. However, it&#39;s important to remember that it&#39;s defining a dataflow graph, not a general Julia expression. In practice this means that anything side-effectful, or things like control flow and
<code>println</code> <code>println</code>
s, won&#39;t work as expected. In future we&#39;ll continue expand support for Julia syntax and features. s, won&#39;t work as expected. In future we&#39;ll continue to expand support for Julia syntax and features.
</p> </p>
<footer> <footer>
<hr/> <hr/>

10
latest/search.html
View File

@ -74,6 +74,16 @@ Debugging
</li> </li>
</ul> </ul>
</li> </li>
<li>
<a class="toctext" href="batching.html">
Batching
</a>
</li>
<li>
<a class="toctext" href="backends.html">
Backends
</a>
</li>
<li> <li>
<span class="toctext"> <span class="toctext">
In Action In Action

40
latest/search_index.js
View File

@ -77,7 +77,7 @@ var documenterSearchIndex = {"docs": [
"page": "Model Templates", "page": "Model Templates",
"title": "Model Templates", "title": "Model Templates",
"category": "section", "category": "section",
"text": "... Calculating Tax Expenses ...So how does the Affine template work? We don't want to duplicate the code above whenever we need more than one affine layer:W₁, b₁ = randn(...)\naffine₁(x) = W₁*x + b₁\nW₂, b₂ = randn(...)\naffine₂(x) = W₂*x + b₂\nmodel = Chain(affine₁, affine₂)Here's one way we could solve this: just keep the parameters in a Julia type, and define how that type acts as a function:type MyAffine\n W\n b\nend\n\n# Use the `MyAffine` layer as a model\n(l::MyAffine)(x) = l.W * x + l.b\n\n# Convenience constructor\nMyAffine(in::Integer, out::Integer) =\n MyAffine(randn(out, in), randn(out))\n\nmodel = Chain(MyAffine(5, 5), MyAffine(5, 5))\n\nmodel(x1) # [-1.54458,0.492025,0.88687,1.93834,-4.70062]This is much better: we can now make as many affine layers as we want. This is a very common pattern, so to make it more convenient we can use the @net macro:@net type MyAffine\n W\n b\n x -> W * x + b\nendThe function provided, x -> W * x + b, will be used when MyAffine is used as a model; it's just a shorter way of defining the (::MyAffine)(x) method above.However, @net does not simply save us some keystrokes; it's the secret sauce that makes everything else in Flux go. For example, it analyses the code for the forward function so that it can differentiate it or convert it to a TensorFlow graph.The above code is almost exactly how Affine is defined in Flux itself! There's no difference between \"library-level\" and \"user-level\" models, so making your code reusable doesn't involve a lot of extra complexity. Moreover, much more complex models than Affine are equally simple to define." "text": "... Calculating Tax Expenses ...So how does the Affine template work? We don't want to duplicate the code above whenever we need more than one affine layer:W₁, b₁ = randn(...)\naffine₁(x) = W₁*x + b₁\nW₂, b₂ = randn(...)\naffine₂(x) = W₂*x + b₂\nmodel = Chain(affine₁, affine₂)Here's one way we could solve this: just keep the parameters in a Julia type, and define how that type acts as a function:type MyAffine\n W\n b\nend\n\n# Use the `MyAffine` layer as a model\n(l::MyAffine)(x) = l.W * x + l.b\n\n# Convenience constructor\nMyAffine(in::Integer, out::Integer) =\n MyAffine(randn(out, in), randn(out))\n\nmodel = Chain(MyAffine(5, 5), MyAffine(5, 5))\n\nmodel(x1) # [-1.54458,0.492025,0.88687,1.93834,-4.70062]This is much better: we can now make as many affine layers as we want. This is a very common pattern, so to make it more convenient we can use the @net macro:@net type MyAffine\n W\n b\n x -> x * W + b\nendThe function provided, x -> x * W + b, will be used when MyAffine is used as a model; it's just a shorter way of defining the (::MyAffine)(x) method above. (You may notice that W and x have swapped order in the model; this is due to the way batching works, which will be covered in more detail later on.)However, @net does not simply save us some keystrokes; it's the secret sauce that makes everything else in Flux go. For example, it analyses the code for the forward function so that it can differentiate it or convert it to a TensorFlow graph.The above code is almost exactly how Affine is defined in Flux itself! There's no difference between \"library-level\" and \"user-level\" models, so making your code reusable doesn't involve a lot of extra complexity. Moreover, much more complex models than Affine are equally simple to define."
}, },
{ {
@ -85,7 +85,7 @@ var documenterSearchIndex = {"docs": [
"page": "Model Templates", "page": "Model Templates",
"title": "Models in templates", "title": "Models in templates",
"category": "section", "category": "section",
"text": "@net models can contain sub-models as well as just array parameters:@net type TLP\n first\n second\n function (x)\n l1 = σ(first(x))\n l2 = softmax(second(l1))\n end\nendJust as above, this is roughly equivalent to writing:type TLP\n first\n second\nend\n\nfunction (self::TLP)(x)\n l1 = σ(self.first(x))\n l2 = softmax(self.second(l1))\nendClearly, the first and second parameters are not arrays here, but should be models themselves, and produce a result when called with an input array x. The Affine layer fits the bill so we can instantiate TLP with two of them:model = TLP(Affine(10, 20),\n Affine(20, 15))\nx1 = rand(20)\nmodel(x1) # [0.057852,0.0409741,0.0609625,0.0575354 ...You may recognise this as being equivalent toChain(\n Affine(10, 20), σ\n Affine(20, 15), softmax)given that it's just a sequence of calls. For simple networks Chain is completely fine, although the @net version is more powerful as we can (for example) reuse the output l1 more than once." "text": "@net models can contain sub-models as well as just array parameters:@net type TLP\n first\n second\n function (x)\n l1 = σ(first(x))\n l2 = softmax(second(l1))\n end\nendJust as above, this is roughly equivalent to writing:type TLP\n first\n second\nend\n\nfunction (self::TLP)(x)\n l1 = σ(self.first(x))\n l2 = softmax(self.second(l1))\nendClearly, the first and second parameters are not arrays here, but should be models themselves, and produce a result when called with an input array x. The Affine layer fits the bill, so we can instantiate TLP with two of them:model = TLP(Affine(10, 20),\n Affine(20, 15))\nx1 = rand(20)\nmodel(x1) # [0.057852,0.0409741,0.0609625,0.0575354 ...You may recognise this as being equivalent toChain(\n Affine(10, 20), σ\n Affine(20, 15), softmax)given that it's just a sequence of calls. For simple networks Chain is completely fine, although the @net version is more powerful as we can (for example) reuse the output l1 more than once."
}, },
{ {
@ -93,7 +93,7 @@ var documenterSearchIndex = {"docs": [
"page": "Model Templates", "page": "Model Templates",
"title": "Constructors", "title": "Constructors",
"category": "section", "category": "section",
"text": "Affine has two array parameters, W and b. Just like any other Julia type, it's easy to instantiate an Affine layer with parameters of our choosing:a = Affine(rand(10, 20), rand(20))However, for convenience and to avoid errors, we'd probably rather specify the input and output dimension instead:a = Affine(10, 20)This is easy to implement using the usual Julia syntax for constructors:Affine(in::Integer, out::Integer) = Affine(randn(in, out), randn(1, out))In practice, these constructors tend to take the parameter initialisation function as an argument so that it's more easily customisable, and use Flux.initn by default (which is equivalent to randn()/100). So Affine's constructor really looks like this:Affine(in::Integer, out::Integer; init = initn) =\n Affine(init(in, out), init(1, out))" "text": "Affine has two array parameters, W and b. Just like any other Julia type, it's easy to instantiate an Affine layer with parameters of our choosing:a = Affine(rand(10, 20), rand(20))However, for convenience and to avoid errors, we'd probably rather specify the input and output dimension instead:a = Affine(10, 20)This is easy to implement using the usual Julia syntax for constructors:Affine(in::Integer, out::Integer) =\n Affine(randn(in, out), randn(1, out))In practice, these constructors tend to take the parameter initialisation function as an argument so that it's more easily customisable, and use Flux.initn by default (which is equivalent to randn(...)/100). So Affine's constructor really looks like this:Affine(in::Integer, out::Integer; init = initn) =\n Affine(init(in, out), init(1, out))"
}, },
{ {
@ -101,7 +101,7 @@ var documenterSearchIndex = {"docs": [
"page": "Model Templates", "page": "Model Templates",
"title": "Supported syntax", "title": "Supported syntax",
"category": "section", "category": "section",
"text": "The syntax used to define a forward pass like x -> W*x + b behaves exactly like Julia code for the most part. However, it's important to remember that it's defining a dataflow graph, not a general Julia expression. In practice this means that anything side-effectful, or things like control flow and printlns, won't work as expected. In future we'll continue expand support for Julia syntax and features." "text": "The syntax used to define a forward pass like x -> x*W + b behaves exactly like Julia code for the most part. However, it's important to remember that it's defining a dataflow graph, not a general Julia expression. In practice this means that anything side-effectful, or things like control flow and printlns, won't work as expected. In future we'll continue to expand support for Julia syntax and features."
}, },
{ {
@ -136,6 +136,38 @@ var documenterSearchIndex = {"docs": [
"text": "Let's take our two-layer perceptron as an example again, running on MXNet:@net type TLP\n first\n second\n function (x)\n l1 = σ(first(x))\n l2 = softmax(second(l1))\n end\nend\n\nmodel = TLP(Affine(10, 20), Affine(21, 15))\n\nmxmodel = mxnet(model, (1, 20))Unfortunately, this model has a (fairly obvious) typo, which means that the code above won't run. Instead we get an error message:InferShape Error in dot5: [20:37:39] src/operator/./matrix_op-inl.h:271: Check failed: (lshape[1]) == (rshape[0]) dot shape error: (15,21) X (20,1)\n in Flux.Affine at affine.jl:8\n in TLP at test.jl:6\n in mxnet(::TLP, ::Tuple{Int64,Int64}) at model.jl:40\n in mxnet(::TLP, ::Vararg{Any,N} where N) at backend.jl:20Most frameworks would only give the error message here – not so helpful if you have thousands of nodes in your computational graph. However, Flux is able to give good error reports even when no Julia code has been run, e.g. when running on a backend like MXNet. This enables us to pinpoint the source of the error very quickly even in a large model.In this case, we can immediately see that the error occurred within an Affine layer. There are two such layers, but this one was called from the second line of TLP, so it must be the second Affine layer we defined. The layer expected an input of length 21 but got 20 instead.Of course, often a stack trace isn't enough to figure out the source of an error. Another option is to simply step through the execution of the model using Gallium. While handy, however, stepping isn't always the best way to get a \"bird's eye view\" of the code. For that, Flux provides a macro called @shapes:julia> @shapes model(rand(5,10))\n\n# /Users/mike/test.jl, line 18:\ngull = σ(Affine(10, 20)(Input()[1]::(5,10))::(5,20))::(5,20)\n# /Users/mike/.julia/v0.6/Flux/src/layers/affine.jl, line 8:\nlobster = gull * _::(21,15) + _::(1,15)\n# /Users/mike/test.jl, line 19:\nraven = softmax(lobster)This is a lot like Julia's own code_warntype; but instead of annotating expressions with types, we display their shapes. As a lowered form it has some quirks; input arguments are represented by Input()[N] and parameters by an underscore.This makes the problem fairly obvious. We tried to multiply the output of the first layer (5, 20) by a parameter (21, 15); the inner dimensions should have been equal.Notice that while the first Affine layer is displayed as-is, the second was inlined and we see a reference to where the W * x + b line was defined in Flux's source code. In this way Flux makes it easy to drill down into problem areas, without showing you the full graph of thousands of nodes at once.With the typo fixed, the output of @shapes looks as follows:# /Users/mike/test.jl, line 18:\nopossum = σ(Affine(10, 20)(Input()[1]::(5,10))::(5,20))::(5,20)\n# /Users/mike/test.jl, line 19:\nwren = softmax(Affine(20, 15)(opossum)::(5,15))::(5,15)" "text": "Let's take our two-layer perceptron as an example again, running on MXNet:@net type TLP\n first\n second\n function (x)\n l1 = σ(first(x))\n l2 = softmax(second(l1))\n end\nend\n\nmodel = TLP(Affine(10, 20), Affine(21, 15))\n\nmxmodel = mxnet(model, (1, 20))Unfortunately, this model has a (fairly obvious) typo, which means that the code above won't run. Instead we get an error message:InferShape Error in dot5: [20:37:39] src/operator/./matrix_op-inl.h:271: Check failed: (lshape[1]) == (rshape[0]) dot shape error: (15,21) X (20,1)\n in Flux.Affine at affine.jl:8\n in TLP at test.jl:6\n in mxnet(::TLP, ::Tuple{Int64,Int64}) at model.jl:40\n in mxnet(::TLP, ::Vararg{Any,N} where N) at backend.jl:20Most frameworks would only give the error message here – not so helpful if you have thousands of nodes in your computational graph. However, Flux is able to give good error reports even when no Julia code has been run, e.g. when running on a backend like MXNet. This enables us to pinpoint the source of the error very quickly even in a large model.In this case, we can immediately see that the error occurred within an Affine layer. There are two such layers, but this one was called from the second line of TLP, so it must be the second Affine layer we defined. The layer expected an input of length 21 but got 20 instead.Of course, often a stack trace isn't enough to figure out the source of an error. Another option is to simply step through the execution of the model using Gallium. While handy, however, stepping isn't always the best way to get a \"bird's eye view\" of the code. For that, Flux provides a macro called @shapes:julia> @shapes model(rand(5,10))\n\n# /Users/mike/test.jl, line 18:\ngull = σ(Affine(10, 20)(Input()[1]::(5,10))::(5,20))::(5,20)\n# /Users/mike/.julia/v0.6/Flux/src/layers/affine.jl, line 8:\nlobster = gull * _::(21,15) + _::(1,15)\n# /Users/mike/test.jl, line 19:\nraven = softmax(lobster)This is a lot like Julia's own code_warntype; but instead of annotating expressions with types, we display their shapes. As a lowered form it has some quirks; input arguments are represented by Input()[N] and parameters by an underscore.This makes the problem fairly obvious. We tried to multiply the output of the first layer (5, 20) by a parameter (21, 15); the inner dimensions should have been equal.Notice that while the first Affine layer is displayed as-is, the second was inlined and we see a reference to where the W * x + b line was defined in Flux's source code. In this way Flux makes it easy to drill down into problem areas, without showing you the full graph of thousands of nodes at once.With the typo fixed, the output of @shapes looks as follows:# /Users/mike/test.jl, line 18:\nopossum = σ(Affine(10, 20)(Input()[1]::(5,10))::(5,20))::(5,20)\n# /Users/mike/test.jl, line 19:\nwren = softmax(Affine(20, 15)(opossum)::(5,15))::(5,15)"
}, },
{
"location": "batching.html#",
"page": "Batching",
"title": "Batching",
"category": "page",
"text": ""
},
{
"location": "batching.html#Batching-1",
"page": "Batching",
"title": "Batching",
"category": "section",
"text": "[WIP]"
},
{
"location": "backends.html#",
"page": "Backends",
"title": "Backends",
"category": "page",
"text": ""
},
{
"location": "backends.html#Batching-1",
"page": "Backends",
"title": "Batching",
"category": "section",
"text": "[WIP]"
},
{ {
"location": "examples/logreg.html#", "location": "examples/logreg.html#",
"page": "Logistic Regression", "page": "Logistic Regression",