build based on e00ea2e
This commit is contained in:
autodocs
parent
635fa7cddf
commit
d626def76d
@ -57,6 +57,11 @@ Building Models
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/recurrent.html">
|
||||
Recurrence
|
||||
@ -104,7 +109,7 @@ Contributing & Help
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/7387ec20dacb0bb017df885b4b5fad3f5a816449/docs/src/contributing.md">
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/contributing.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
|
||||
@ -57,6 +57,11 @@ Building Models
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="../models/templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="../models/recurrent.html">
|
||||
Recurrence
|
||||
@ -107,7 +112,7 @@ Logistic Regression
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/7387ec20dacb0bb017df885b4b5fad3f5a816449/docs/src/examples/logreg.md">
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/examples/logreg.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
|
||||
@ -64,6 +64,11 @@ Building Models
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/recurrent.html">
|
||||
Recurrence
|
||||
@ -110,7 +115,7 @@ Home
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/7387ec20dacb0bb017df885b4b5fad3f5a816449/docs/src/index.md">
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/index.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
|
||||
@ -57,6 +57,11 @@ Building Models
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/recurrent.html">
|
||||
Recurrence
|
||||
@ -104,7 +109,7 @@ Internals
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/7387ec20dacb0bb017df885b4b5fad3f5a816449/docs/src/internals.md">
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/internals.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
|
||||
@ -74,6 +74,11 @@ A Function in Model's Clothing
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="recurrent.html">
|
||||
Recurrence
|
||||
@ -123,7 +128,7 @@ Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/7387ec20dacb0bb017df885b4b5fad3f5a816449/docs/src/models/basics.md">
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/models/basics.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
@ -278,12 +283,12 @@ Previous
|
||||
Home
|
||||
</span>
|
||||
</a>
|
||||
<a class="next" href="recurrent.html">
|
||||
<a class="next" href="templates.html">
|
||||
<span class="direction">
|
||||
Next
|
||||
</span>
|
||||
<span class="title">
|
||||
Recurrence
|
||||
Model Templates
|
||||
</span>
|
||||
</a>
|
||||
</footer>
|
||||
|
||||
@ -57,6 +57,11 @@ Building Models
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="recurrent.html">
|
||||
Recurrence
|
||||
@ -107,7 +112,7 @@ Debugging
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/7387ec20dacb0bb017df885b4b5fad3f5a816449/docs/src/models/debugging.md">
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/models/debugging.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
|
||||
@ -57,6 +57,11 @@ Building Models
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li class="current">
|
||||
<a class="toctext" href="recurrent.html">
|
||||
Recurrence
|
||||
@ -107,7 +112,7 @@ Recurrence
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/7387ec20dacb0bb017df885b4b5fad3f5a816449/docs/src/models/recurrent.md">
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/models/recurrent.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
@ -126,12 +131,12 @@ Recurrent Models
|
||||
</p>
|
||||
<footer>
|
||||
<hr/>
|
||||
<a class="previous" href="basics.html">
|
||||
<a class="previous" href="templates.html">
|
||||
<span class="direction">
|
||||
Previous
|
||||
</span>
|
||||
<span class="title">
|
||||
Model Building Basics
|
||||
Model Templates
|
||||
</span>
|
||||
</a>
|
||||
<a class="next" href="debugging.html">
|
||||
|
||||
338
latest/models/templates.html
Normal file
338
latest/models/templates.html
Normal file
@ -0,0 +1,338 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
|
||||
<title>
|
||||
Model Templates · Flux
|
||||
</title>
|
||||
<script>
|
||||
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
|
||||
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
|
||||
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
|
||||
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
|
||||
|
||||
ga('create', 'UA-36890222-9', 'auto');
|
||||
ga('send', 'pageview');
|
||||
|
||||
</script>
|
||||
<link href="https://cdnjs.cloudflare.com/ajax/libs/normalize/4.2.0/normalize.min.css" rel="stylesheet" type="text/css"/>
|
||||
<link href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.5.0/styles/default.min.css" rel="stylesheet" type="text/css"/>
|
||||
<link href="https://fonts.googleapis.com/css?family=Lato|Ubuntu+Mono" rel="stylesheet" type="text/css"/>
|
||||
<link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css" rel="stylesheet" type="text/css"/>
|
||||
<link href="../assets/documenter.css" rel="stylesheet" type="text/css"/>
|
||||
<script>
|
||||
documenterBaseURL=".."
|
||||
</script>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.2.0/require.min.js" data-main="../assets/documenter.js"></script>
|
||||
<script src="../../versions.js"></script>
|
||||
<link href="../../flux.css" rel="stylesheet" type="text/css"/>
|
||||
</head>
|
||||
<body>
|
||||
<nav class="toc">
|
||||
<h1>
|
||||
Flux
|
||||
</h1>
|
||||
<form class="search" action="../search.html">
|
||||
<select id="version-selector" onChange="window.location.href=this.value">
|
||||
<option value="#" selected="selected" disabled="disabled">
|
||||
Version
|
||||
</option>
|
||||
</select>
|
||||
<input id="search-query" name="q" type="text" placeholder="Search docs"/>
|
||||
</form>
|
||||
<ul>
|
||||
<li>
|
||||
<a class="toctext" href="../index.html">
|
||||
Home
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<span class="toctext">
|
||||
Building Models
|
||||
</span>
|
||||
<ul>
|
||||
<li>
|
||||
<a class="toctext" href="basics.html">
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li class="current">
|
||||
<a class="toctext" href="templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
<ul class="internal">
|
||||
<li>
|
||||
<a class="toctext" href="#Models-in-templates-1">
|
||||
Models in templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="#Constructors-1">
|
||||
Constructors
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="#Supported-syntax-1">
|
||||
Supported syntax
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="recurrent.html">
|
||||
Recurrence
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="debugging.html">
|
||||
Debugging
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<span class="toctext">
|
||||
In Action
|
||||
</span>
|
||||
<ul>
|
||||
<li>
|
||||
<a class="toctext" href="../examples/logreg.html">
|
||||
Logistic Regression
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="../contributing.html">
|
||||
Contributing & Help
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="../internals.html">
|
||||
Internals
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
<article id="docs">
|
||||
<header>
|
||||
<nav>
|
||||
<ul>
|
||||
<li>
|
||||
Building Models
|
||||
</li>
|
||||
<li>
|
||||
<a href="templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<a class="edit-page" href="https://github.com/MikeInnes/Flux.jl/tree/e00ea2e944c0950abce70ee9d4bebfb8bb3ab41a/docs/src/models/templates.md">
|
||||
<span class="fa">
|
||||
|
||||
</span>
|
||||
Edit on GitHub
|
||||
</a>
|
||||
</nav>
|
||||
<hr/>
|
||||
</header>
|
||||
<h1>
|
||||
<a class="nav-anchor" id="Model-Templates-1" href="#Model-Templates-1">
|
||||
Model Templates
|
||||
</a>
|
||||
</h1>
|
||||
<p>
|
||||
<em>
|
||||
... Calculating Tax Expenses ...
|
||||
</em>
|
||||
</p>
|
||||
<p>
|
||||
So how does the
|
||||
<code>Affine</code>
|
||||
template work? We don't want to duplicate the code above whenever we need more than one affine layer:
|
||||
</p>
|
||||
<pre><code class="language-julia">W₁, b₁ = randn(...)
|
||||
affine₁(x) = W₁*x + b₁
|
||||
W₂, b₂ = randn(...)
|
||||
affine₂(x) = W₂*x + b₂
|
||||
model = Chain(affine₁, affine₂)</code></pre>
|
||||
<p>
|
||||
Here's one way we could solve this: just keep the parameters in a Julia type, and define how that type acts as a function:
|
||||
</p>
|
||||
<pre><code class="language-julia">type MyAffine
|
||||
W
|
||||
b
|
||||
end
|
||||
|
||||
# Use the `MyAffine` layer as a model
|
||||
(l::MyAffine)(x) = l.W * x + l.b
|
||||
|
||||
# Convenience constructor
|
||||
MyAffine(in::Integer, out::Integer) =
|
||||
MyAffine(randn(out, in), randn(out))
|
||||
|
||||
model = Chain(MyAffine(5, 5), MyAffine(5, 5))
|
||||
|
||||
model(x1) # [-1.54458,0.492025,0.88687,1.93834,-4.70062]</code></pre>
|
||||
<p>
|
||||
This is much better: we can now make as many affine layers as we want. This is a very common pattern, so to make it more convenient we can use the
|
||||
<code>@net</code>
|
||||
macro:
|
||||
</p>
|
||||
<pre><code class="language-julia">@net type MyAffine
|
||||
W
|
||||
b
|
||||
x -> W * x + b
|
||||
end</code></pre>
|
||||
<p>
|
||||
The function provided,
|
||||
<code>x -> W * x + b</code>
|
||||
, will be used when
|
||||
<code>MyAffine</code>
|
||||
is used as a model; it's just a shorter way of defining the
|
||||
<code>(::MyAffine)(x)</code>
|
||||
method above.
|
||||
</p>
|
||||
<p>
|
||||
However,
|
||||
<code>@net</code>
|
||||
does not simply save us some keystrokes; it's the secret sauce that makes everything else in Flux go. For example, it analyses the code for the forward function so that it can differentiate it or convert it to a TensorFlow graph.
|
||||
</p>
|
||||
<p>
|
||||
The above code is almost exactly how
|
||||
<code>Affine</code>
|
||||
is defined in Flux itself! There's no difference between "library-level" and "user-level" models, so making your code reusable doesn't involve a lot of extra complexity. Moreover, much more complex models than
|
||||
<code>Affine</code>
|
||||
are equally simple to define.
|
||||
</p>
|
||||
<h2>
|
||||
<a class="nav-anchor" id="Models-in-templates-1" href="#Models-in-templates-1">
|
||||
Models in templates
|
||||
</a>
|
||||
</h2>
|
||||
<p>
|
||||
<code>@net</code>
|
||||
models can contain sub-models as well as just array parameters:
|
||||
</p>
|
||||
<pre><code class="language-julia">@net type TLP
|
||||
first
|
||||
second
|
||||
function (x)
|
||||
l1 = σ(first(x))
|
||||
l2 = softmax(second(l1))
|
||||
end
|
||||
end</code></pre>
|
||||
<p>
|
||||
Just as above, this is roughly equivalent to writing:
|
||||
</p>
|
||||
<pre><code class="language-julia">type TLP
|
||||
first
|
||||
second
|
||||
end
|
||||
|
||||
function (self::TLP)(x)
|
||||
l1 = σ(self.first(x))
|
||||
l2 = softmax(self.second(l1))
|
||||
end</code></pre>
|
||||
<p>
|
||||
Clearly, the
|
||||
<code>first</code>
|
||||
and
|
||||
<code>second</code>
|
||||
parameters are not arrays here, but should be models themselves, and produce a result when called with an input array
|
||||
<code>x</code>
|
||||
. The
|
||||
<code>Affine</code>
|
||||
layer fits the bill so we can instantiate
|
||||
<code>TLP</code>
|
||||
with two of them:
|
||||
</p>
|
||||
<pre><code class="language-julia">model = TLP(Affine(10, 20),
|
||||
Affine(20, 15))
|
||||
x1 = rand(20)
|
||||
model(x1) # [0.057852,0.0409741,0.0609625,0.0575354 ...</code></pre>
|
||||
<p>
|
||||
You may recognise this as being equivalent to
|
||||
</p>
|
||||
<pre><code class="language-julia">Chain(
|
||||
Affine(10, 20), σ
|
||||
Affine(20, 15), softmax)</code></pre>
|
||||
<p>
|
||||
given that it's just a sequence of calls. For simple networks
|
||||
<code>Chain</code>
|
||||
is completely fine, although the
|
||||
<code>@net</code>
|
||||
version is more powerful as we can (for example) reuse the output
|
||||
<code>l1</code>
|
||||
more than once.
|
||||
</p>
|
||||
<h2>
|
||||
<a class="nav-anchor" id="Constructors-1" href="#Constructors-1">
|
||||
Constructors
|
||||
</a>
|
||||
</h2>
|
||||
<p>
|
||||
<code>Affine</code>
|
||||
has two array parameters,
|
||||
<code>W</code>
|
||||
and
|
||||
<code>b</code>
|
||||
. Just like any other Julia type, it's easy to instantiate an
|
||||
<code>Affine</code>
|
||||
layer with parameters of our choosing:
|
||||
</p>
|
||||
<pre><code class="language-julia">a = Affine(rand(10, 20), rand(20))</code></pre>
|
||||
<p>
|
||||
However, for convenience and to avoid errors, we'd probably rather specify the input and output dimension instead:
|
||||
</p>
|
||||
<pre><code class="language-julia">a = Affine(10, 20)</code></pre>
|
||||
<p>
|
||||
This is easy to implement using the usual Julia syntax for constructors:
|
||||
</p>
|
||||
<pre><code class="language-julia">Affine(in::Integer, out::Integer) = Affine(randn(in, out), randn(1, out))</code></pre>
|
||||
<p>
|
||||
In practice, these constructors tend to take the parameter initialisation function as an argument so that it's more easily customisable, and use
|
||||
<code>Flux.initn</code>
|
||||
by default (which is equivalent to
|
||||
<code>randn()/100</code>
|
||||
). So
|
||||
<code>Affine</code>
|
||||
's constructor really looks like this:
|
||||
</p>
|
||||
<pre><code class="language-julia">Affine(in::Integer, out::Integer; init = initn) =
|
||||
Affine(init(in, out), init(1, out))</code></pre>
|
||||
<h2>
|
||||
<a class="nav-anchor" id="Supported-syntax-1" href="#Supported-syntax-1">
|
||||
Supported syntax
|
||||
</a>
|
||||
</h2>
|
||||
<p>
|
||||
The syntax used to define a forward pass like
|
||||
<code>x -> W*x + b</code>
|
||||
behaves exactly like Julia code for the most part. However, it's important to remember that it's defining a dataflow graph, not a general Julia expression. In practice this means that anything side-effectful, or things like control flow and
|
||||
<code>println</code>
|
||||
s, won't work as expected. In future we'll continue expand support for Julia syntax and features.
|
||||
</p>
|
||||
<footer>
|
||||
<hr/>
|
||||
<a class="previous" href="basics.html">
|
||||
<span class="direction">
|
||||
Previous
|
||||
</span>
|
||||
<span class="title">
|
||||
Model Building Basics
|
||||
</span>
|
||||
</a>
|
||||
<a class="next" href="recurrent.html">
|
||||
<span class="direction">
|
||||
Next
|
||||
</span>
|
||||
<span class="title">
|
||||
Recurrence
|
||||
</span>
|
||||
</a>
|
||||
</footer>
|
||||
</article>
|
||||
</body>
|
||||
</html>
|
||||
@ -57,6 +57,11 @@ Building Models
|
||||
Model Building Basics
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/templates.html">
|
||||
Model Templates
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="toctext" href="models/recurrent.html">
|
||||
Recurrence
|
||||
|
||||
@ -64,6 +64,46 @@ var documenterSearchIndex = {"docs": [
|
||||
"text": "... Booting Dark Matter Transmogrifiers ...We noted above that a \"model\" is a function with some number of trainable parameters. This goes both ways; a normal Julia function like exp is effectively a model with 0 parameters. Flux doesn't care, and anywhere that you use one, you can use the other. For example, Chain will happily work with regular functions:foo = Chain(exp, sum, log)\nfoo([1,2,3]) == 3.408 == log(sum(exp([1,2,3])))"
|
||||
},
|
||||
|
||||
{
|
||||
"location": "models/templates.html#",
|
||||
"page": "Model Templates",
|
||||
"title": "Model Templates",
|
||||
"category": "page",
|
||||
"text": ""
|
||||
},
|
||||
|
||||
{
|
||||
"location": "models/templates.html#Model-Templates-1",
|
||||
"page": "Model Templates",
|
||||
"title": "Model Templates",
|
||||
"category": "section",
|
||||
"text": "... Calculating Tax Expenses ...So how does the Affine template work? We don't want to duplicate the code above whenever we need more than one affine layer:W₁, b₁ = randn(...)\naffine₁(x) = W₁*x + b₁\nW₂, b₂ = randn(...)\naffine₂(x) = W₂*x + b₂\nmodel = Chain(affine₁, affine₂)Here's one way we could solve this: just keep the parameters in a Julia type, and define how that type acts as a function:type MyAffine\n W\n b\nend\n\n# Use the `MyAffine` layer as a model\n(l::MyAffine)(x) = l.W * x + l.b\n\n# Convenience constructor\nMyAffine(in::Integer, out::Integer) =\n MyAffine(randn(out, in), randn(out))\n\nmodel = Chain(MyAffine(5, 5), MyAffine(5, 5))\n\nmodel(x1) # [-1.54458,0.492025,0.88687,1.93834,-4.70062]This is much better: we can now make as many affine layers as we want. This is a very common pattern, so to make it more convenient we can use the @net macro:@net type MyAffine\n W\n b\n x -> W * x + b\nendThe function provided, x -> W * x + b, will be used when MyAffine is used as a model; it's just a shorter way of defining the (::MyAffine)(x) method above.However, @net does not simply save us some keystrokes; it's the secret sauce that makes everything else in Flux go. For example, it analyses the code for the forward function so that it can differentiate it or convert it to a TensorFlow graph.The above code is almost exactly how Affine is defined in Flux itself! There's no difference between \"library-level\" and \"user-level\" models, so making your code reusable doesn't involve a lot of extra complexity. Moreover, much more complex models than Affine are equally simple to define."
|
||||
},
|
||||
|
||||
{
|
||||
"location": "models/templates.html#Models-in-templates-1",
|
||||
"page": "Model Templates",
|
||||
"title": "Models in templates",
|
||||
"category": "section",
|
||||
"text": "@net models can contain sub-models as well as just array parameters:@net type TLP\n first\n second\n function (x)\n l1 = σ(first(x))\n l2 = softmax(second(l1))\n end\nendJust as above, this is roughly equivalent to writing:type TLP\n first\n second\nend\n\nfunction (self::TLP)(x)\n l1 = σ(self.first(x))\n l2 = softmax(self.second(l1))\nendClearly, the first and second parameters are not arrays here, but should be models themselves, and produce a result when called with an input array x. The Affine layer fits the bill so we can instantiate TLP with two of them:model = TLP(Affine(10, 20),\n Affine(20, 15))\nx1 = rand(20)\nmodel(x1) # [0.057852,0.0409741,0.0609625,0.0575354 ...You may recognise this as being equivalent toChain(\n Affine(10, 20), σ\n Affine(20, 15), softmax)given that it's just a sequence of calls. For simple networks Chain is completely fine, although the @net version is more powerful as we can (for example) reuse the output l1 more than once."
|
||||
},
|
||||
|
||||
{
|
||||
"location": "models/templates.html#Constructors-1",
|
||||
"page": "Model Templates",
|
||||
"title": "Constructors",
|
||||
"category": "section",
|
||||
"text": "Affine has two array parameters, W and b. Just like any other Julia type, it's easy to instantiate an Affine layer with parameters of our choosing:a = Affine(rand(10, 20), rand(20))However, for convenience and to avoid errors, we'd probably rather specify the input and output dimension instead:a = Affine(10, 20)This is easy to implement using the usual Julia syntax for constructors:Affine(in::Integer, out::Integer) = Affine(randn(in, out), randn(1, out))In practice, these constructors tend to take the parameter initialisation function as an argument so that it's more easily customisable, and use Flux.initn by default (which is equivalent to randn()/100). So Affine's constructor really looks like this:Affine(in::Integer, out::Integer; init = initn) =\n Affine(init(in, out), init(1, out))"
|
||||
},
|
||||
|
||||
{
|
||||
"location": "models/templates.html#Supported-syntax-1",
|
||||
"page": "Model Templates",
|
||||
"title": "Supported syntax",
|
||||
"category": "section",
|
||||
"text": "The syntax used to define a forward pass like x -> W*x + b behaves exactly like Julia code for the most part. However, it's important to remember that it's defining a dataflow graph, not a general Julia expression. In practice this means that anything side-effectful, or things like control flow and printlns, won't work as expected. In future we'll continue expand support for Julia syntax and features."
|
||||
},
|
||||
|
||||
{
|
||||
"location": "models/recurrent.html#",
|
||||
"page": "Recurrence",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user