2017-01-16 16:51:09 +00:00
<!DOCTYPE html>
2017-01-17 20:06:28 +00:00
< html lang = "en" >
< head >
< meta charset = "UTF-8" / >
< meta name = "viewport" content = "width=device-width, initial-scale=1.0" / >
< title >
2017-01-18 01:18:15 +00:00
Debugging · Flux
2017-01-17 20:06:28 +00:00
< / title >
< script >
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
2017-01-16 16:51:09 +00:00
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-36890222-9', 'auto');
ga('send', 'pageview');
2017-01-17 20:06:28 +00:00
< / script >
< link href = "https://cdnjs.cloudflare.com/ajax/libs/normalize/4.2.0/normalize.min.css" rel = "stylesheet" type = "text/css" / >
< link href = "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/9.5.0/styles/default.min.css" rel = "stylesheet" type = "text/css" / >
< link href = "https://fonts.googleapis.com/css?family=Lato|Ubuntu+Mono" rel = "stylesheet" type = "text/css" / >
< link href = "https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.6.3/css/font-awesome.min.css" rel = "stylesheet" type = "text/css" / >
2017-01-18 01:18:15 +00:00
< link href = "../assets/documenter.css" rel = "stylesheet" type = "text/css" / >
2017-01-17 20:06:28 +00:00
< script >
2017-01-18 01:18:15 +00:00
documenterBaseURL=".."
2017-01-17 20:06:28 +00:00
< / script >
2017-01-18 01:18:15 +00:00
< script src = "https://cdnjs.cloudflare.com/ajax/libs/require.js/2.2.0/require.min.js" data-main = "../assets/documenter.js" > < / script >
< script src = "../../versions.js" > < / script >
< link href = "../../flux.css" rel = "stylesheet" type = "text/css" / >
2017-01-17 20:06:28 +00:00
< / head >
< body >
< nav class = "toc" >
< h1 >
Flux
< / h1 >
2017-01-18 01:18:15 +00:00
< form class = "search" action = "../search.html" >
2017-01-17 20:06:28 +00:00
< select id = "version-selector" onChange = "window.location.href=this.value" >
< option value = "#" selected = "selected" disabled = "disabled" >
Version
< / option >
< / select >
< input id = "search-query" name = "q" type = "text" placeholder = "Search docs" / >
< / form >
< ul >
< li >
2017-01-18 01:18:15 +00:00
< a class = "toctext" href = "../index.html" >
2017-01-17 20:06:28 +00:00
Home
< / a >
< / li >
2017-01-18 12:45:25 +00:00
< li >
2017-01-18 23:22:30 +00:00
< span class = "toctext" >
Building Models
< / span >
< ul >
< li >
< a class = "toctext" href = "basics.html" >
2017-02-01 13:48:25 +00:00
Model Building Basics
2017-01-18 23:22:30 +00:00
< / a >
< / li >
2017-02-02 07:48:56 +00:00
< li >
< a class = "toctext" href = "templates.html" >
Model Templates
< / a >
< / li >
2017-01-18 23:22:30 +00:00
< li >
< a class = "toctext" href = "recurrent.html" >
2017-01-18 12:45:25 +00:00
Recurrence
2017-01-18 23:22:30 +00:00
< / a >
< / li >
< li class = "current" >
< a class = "toctext" href = "debugging.html" >
2017-01-18 12:45:25 +00:00
Debugging
2017-01-18 23:22:30 +00:00
< / a >
< ul class = "internal" > < / ul >
< / li >
< / ul >
2017-01-18 12:45:25 +00:00
< / li >
2017-02-18 15:11:53 +00:00
< li >
2017-02-20 10:53:09 +00:00
< span class = "toctext" >
Other APIs
< / span >
< ul >
< li >
2017-02-20 11:05:06 +00:00
< a class = "toctext" href = "../apis/batching.html" >
2017-02-18 15:11:53 +00:00
Batching
2017-02-20 10:53:09 +00:00
< / a >
< / li >
< li >
2017-02-20 11:05:06 +00:00
< a class = "toctext" href = "../apis/backends.html" >
2017-02-18 15:11:53 +00:00
Backends
2017-02-20 10:53:09 +00:00
< / a >
< / li >
2017-02-28 16:50:27 +00:00
< li >
< a class = "toctext" href = "../apis/storage.html" >
Storing Models
< / a >
< / li >
2017-02-20 10:53:09 +00:00
< / ul >
2017-02-18 15:11:53 +00:00
< / li >
2017-01-17 20:06:28 +00:00
< li >
< span class = "toctext" >
2017-01-18 12:45:25 +00:00
In Action
2017-01-17 20:06:28 +00:00
< / span >
< ul >
2017-01-18 01:18:15 +00:00
< li >
2017-01-18 12:45:25 +00:00
< a class = "toctext" href = "../examples/logreg.html" >
2017-03-09 00:26:06 +00:00
Simple MNIST
2017-01-18 01:18:15 +00:00
< / a >
< / li >
2017-02-28 16:21:45 +00:00
< li >
< a class = "toctext" href = "../examples/char-rnn.html" >
Char RNN
< / a >
< / li >
2017-01-17 20:06:28 +00:00
< / ul >
< / li >
2017-01-18 01:18:15 +00:00
< li >
< a class = "toctext" href = "../contributing.html" >
Contributing & Help
< / a >
< / li >
2017-01-18 12:45:25 +00:00
< li >
< a class = "toctext" href = "../internals.html" >
Internals
< / a >
< / li >
2017-01-17 20:06:28 +00:00
< / ul >
< / nav >
< article id = "docs" >
< header >
< nav >
< ul >
< li >
2017-01-18 23:22:30 +00:00
Building Models
< / li >
< li >
2017-01-18 01:18:15 +00:00
< a href = "debugging.html" >
Debugging
2017-01-17 20:06:28 +00:00
< / a >
< / li >
< / ul >
2017-07-27 19:47:35 +00:00
< a class = "edit-page" href = "https://github.com/MikeInnes/Flux.jl/tree/b4a0841e9df6da38b19abcbdc8b5c1105a329dd2/docs/src/models/debugging.md" >
2017-01-17 20:06:28 +00:00
< span class = "fa" >
< / span >
Edit on GitHub
< / a >
< / nav >
< hr / >
< / header >
< h1 >
2017-01-18 01:18:15 +00:00
< a class = "nav-anchor" id = "Debugging-Models-1" href = "#Debugging-Models-1" >
Debugging Models
2017-01-17 20:06:28 +00:00
< / a >
< / h1 >
2017-01-18 01:18:15 +00:00
< p >
2017-02-01 15:39:42 +00:00
Let' s take our two-layer perceptron as an example again, running on MXNet:
< / p >
< pre > < code class = "language-julia" > @net type TLP
first
second
function (x)
l1 = σ
l2 = softmax(second(l1))
end
end
model = TLP(Affine(10, 20), Affine(21, 15))
2017-03-08 21:44:04 +00:00
mxmodel = mxnet(model)
mxmodel(rand(10))< / code > < / pre >
2017-02-01 15:39:42 +00:00
< p >
Unfortunately, this model has a (fairly obvious) typo, which means that the code above won' t run. Instead we get an error message:
< / p >
2017-03-08 21:44:04 +00:00
< pre > < code class = "language-julia" > Error in operator dot2: [21:28:21] src/operator/tensor/./matrix_op-inl.h:460:
Check failed: lshape[1] == rshape[0] (20 vs. 21) dot shape error: (1,20) X (21,15)
Flux.Affine at affine.jl:8
TLP at basic.jl:6
(::Flux.MX.Model)(::Flux.Batch{Array{Float64,1},Array{Float64,2}}) at model.jl:105
(::Flux.MX.Model)(::Array{Float64,1}) at model.jl:107< / code > < / pre >
2017-02-01 15:39:42 +00:00
< p >
Most frameworks would only give the error message here –
< em >
even when no Julia code has been run
< / em >
, e.g. when running on a backend like MXNet. This enables us to pinpoint the source of the error very quickly even in a large model.
< / p >
< p >
In this case, we can immediately see that the error occurred within an
< code > Affine< / code >
layer. There are two such layers, but this one was called from the second line of
< code > TLP< / code >
, so it must be the second
< code > Affine< / code >
layer we defined. The layer expected an input of length 21 but got 20 instead.
< / p >
< p >
Of course, often a stack trace isn' t enough to figure out the source of an error. Another option is to simply step through the execution of the model using Gallium. While handy, however, stepping isn' t always the best way to get a " bird' s eye view" of the code. For that, Flux provides a macro called
< code > @shapes< / code >
:
< / p >
< pre > < code class = "language-julia" > julia> @shapes model(rand(5,10))
# /Users/mike/test.jl, line 18:
gull = σ
# /Users/mike/.julia/v0.6/Flux/src/layers/affine.jl, line 8:
lobster = gull * _::(21,15) + _::(1,15)
# /Users/mike/test.jl, line 19:
raven = softmax(lobster)< / code > < / pre >
< p >
This is a lot like Julia' s own
< code > code_warntype< / code >
; but instead of annotating expressions with types, we display their shapes. As a lowered form it has some quirks; input arguments are represented by
< code > Input()[N]< / code >
and parameters by an underscore.
< / p >
< p >
This makes the problem fairly obvious. We tried to multiply the output of the first layer
< code > (5, 20)< / code >
by a parameter
< code > (21, 15)< / code >
; the inner dimensions should have been equal.
< / p >
< p >
Notice that while the first
< code > Affine< / code >
layer is displayed as-is, the second was inlined and we see a reference to where the
< code > W * x + b< / code >
line was defined in Flux' s source code. In this way Flux makes it easy to drill down into problem areas, without showing you the full graph of thousands of nodes at once.
2017-01-18 01:18:15 +00:00
< / p >
2017-02-01 16:02:16 +00:00
< p >
With the typo fixed, the output of
< code > @shapes< / code >
looks as follows:
< / p >
< pre > < code class = "language-julia" > # /Users/mike/test.jl, line 18:
opossum = σ
# /Users/mike/test.jl, line 19:
wren = softmax(Affine(20, 15)(opossum)::(5,15))::(5,15)< / code > < / pre >
2017-01-17 20:06:28 +00:00
< footer >
< hr / >
2017-01-18 01:18:15 +00:00
< a class = "previous" href = "recurrent.html" >
2017-01-17 20:06:28 +00:00
< span class = "direction" >
Previous
< / span >
< span class = "title" >
2017-01-18 01:18:15 +00:00
Recurrence
< / span >
< / a >
2017-02-20 11:05:06 +00:00
< a class = "next" href = "../apis/batching.html" >
2017-01-18 01:18:15 +00:00
< span class = "direction" >
Next
< / span >
< span class = "title" >
2017-02-18 15:11:53 +00:00
Batching
2017-01-17 20:06:28 +00:00
< / span >
< / a >
< / footer >
< / article >
< / body >
< / html >
