TastyPancakes/Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,829 Commits 104 Branches 42 Tags 5.8 MiB
1d310d4532
Commit Graph

305 Commits

Author SHA1 Message Date
Mike J Innes
7726a5b605 inferrable 2018-06-26 14:12:57 +01:00
Mike J Innes
3b575930ca Merge branch 'master' into scalar_pad_stride 2018-06-26 14:05:07 +01:00
Avik Pal
3339ad5181 Integrate cudnn BatchNorm with Flux 2018-06-20 15:50:30 +05:30
Avik Pal
714ca23aba Change default value of epsilon to prevent CuDNN BatchNorm warnings 2018-06-20 12:11:22 +05:30
Avik Pal
65f2c33991
Merge pull request #2 from FluxML/master 
rebase
 2018-06-11 15:40:57 +05:30
Avik Pal
7f3d11cae0
Merge branch 'master' into depthwiseconv 2018-06-09 11:06:07 +05:30
Avik Pal
1d93fb8e59 Add new constructor and fix a typo in display 2018-06-09 11:02:15 +05:30
Tejan Karmali
d20771d6be
Default value of dilation 
dilation should be 1 by default
 2018-06-09 02:29:46 +05:30
Mike J Innes
4915b0c8dd
Merge pull request #268 from staticfloat/patch-2 
Add `dilation` kwarg to `Conv`
 2018-06-07 13:49:02 +01:00
Avik Pal
cd6a0856d5 Adds support for Depthwise Convolutions 2018-05-30 15:53:57 +05:30
Elliot Saba
e6efca4bf4 Add dilation kwarg to Conv 
Now that we have dilated convolution support in `NNlib`, this is enables support in Flux's `Conv` layer.
 2018-05-21 13:44:13 -07:00
James Bradbury
af12f006f2
Use broadcast for dropout 
Should be fast enough on GPU now that it's not going to be an optimization target again for a while. Hopefully isn't meaningfully slower on CPU?
 2018-05-20 04:04:33 -07:00
Mike J Innes
7d7d89569c rm this deprecation for 0.6 2018-05-01 12:20:36 +01:00
Mike J Innes
baff20514d gpu broadcast fix 2018-04-17 18:05:58 +01:00
Mike J Innes
8f73dc6e14 fix gpu cross entropy 2018-04-17 17:56:47 +01:00
tejank10
2ef25775c6 removed extra expand and fixed bug 2018-04-16 01:18:26 +05:30
Mike Innes
d12fb98f2a nicer batchnorm shape error 2018-04-15 20:29:25 +01:00
tejank10
2f5473d435 added expand in conv constructor 2018-04-16 00:59:11 +05:30
Mike J Innes
5fd240f525 interface tweaks 2018-04-15 20:04:42 +01:00
Mike J Innes
73a0be3e04 Merge branch 'master' into pull-request/07b0f95d 2018-04-15 17:10:29 +01:00
tejank10
b080f5c82e Scalar pad and stride 2018-04-15 20:32:40 +05:30
Mike J Innes
cb3ae8df6a rename normalise.jl 2018-04-15 15:45:46 +01:00
tejank10
f6097d58d6 Scalar pad/stride for Conv constructor 2018-04-15 12:15:41 +05:30
Brad Safnuk
b9a66c679d Fix error in initialization of σ. 2018-03-22 22:20:21 -04:00
Brad Safnuk
35299d4621 Fix type instability when loading onto a gpu. 
Also fixes Issue #216.
 2018-03-22 21:32:32 -04:00
Brad Safnuk
db2d9efb72 Update BatchNorm documentation 2018-03-15 21:59:38 -04:00
Brad Safnuk
6653ec86d9 Allow multidimensional inputs to batchnorm. 
Can be used in conjunction with convolutional layers, in addition
to dense layers, with the same api.
 2018-03-15 21:48:59 -04:00
Mike J Innes
8019f789f8 use normal log 2018-03-01 16:35:49 +00:00
Mike J Innes
ac57fc3c26 use @ fix in a few places 2018-03-01 16:31:20 +00:00
Mike J Innes
c2fea2acf6 revert this 2018-02-28 23:06:53 +00:00
Mike J Innes
7606b1a399 single-batch convolution 2018-02-28 14:25:32 +00:00
Mike J Innes
15d1d3256b conv api updates 2018-02-26 22:43:07 +00:00
Mike J Innes
491785a681 ignore state in mapleaves 2018-02-22 00:22:51 +00:00
Mike J Innes
ec65e2cec7 fix printing 2018-02-22 00:21:48 +00:00
Mike J Innes
989adcdc7d gpu fix 2018-02-17 12:41:53 +00:00
Mike J Innes
7aa6854c64 more correct 2018-02-16 00:06:15 +00:00
Mike J Innes
63862c2324 easier initialisation with weights 2018-02-15 20:52:29 +00:00
Mike J Innes
01c31e7fcc conv bias 2018-02-15 20:15:41 +00:00
Mike J Innes
2f29733888 Merge branch 'master' into HEAD 2018-02-13 14:45:37 +00:00
Mike J Innes
8432d8db06 batchnorm fix 2018-02-13 14:02:35 +00:00
Mike J Innes
fcbdc49d6b fix reserve usage 2018-02-08 10:27:26 +00:00
boathit
6e65789828 Register back! for logsigmoid and implement (logit)binarycrossentropy 2018-02-06 19:32:46 +08:00
Mike J Innes
9a6fcf057b hook up interface 2018-02-02 16:42:18 +00:00
Mike J Innes
0f1e7b5578 update rnn structure 2018-02-01 20:57:39 +00:00
Mike J Innes
4bfb603da6 gru forward 2018-01-31 13:46:55 +00:00
Mike J Innes
4207fb98f2 basic GPU tests 2018-01-16 17:58:14 +00:00
Mike J Innes
8f8589a7f4 fix initialisation 2018-01-10 14:11:52 +00:00
Mike J Innes
b44237468e Merge branch 'master' into gru 2018-01-10 13:59:33 +00:00
Mehul Tikekar
2fef799109 fix typo in conv.jl (fixes #133) 2018-01-08 16:46:58 -05:00
Mike J Innes
e3577d759c conv docs 2017-12-18 18:05:48 +00:00
Mike J Innes
269d8f36b9 conv padding 2017-12-18 18:05:38 +00:00
Mike J Innes
51f93d9f0e conv polish 2017-12-15 16:24:45 +00:00
Mike J Innes
9d0dd9fb7e layer wip 2017-12-15 13:22:57 +00:00
Mike J Innes
27d896943e
Merge pull request #120 from staticfloat/sf/dense_initialization 
Better default initialization for Dense layers
 2017-12-13 16:18:02 +00:00
Mike J Innes
e3a688e706 use kwarg 2017-12-13 15:27:15 +00:00
Mike J Innes
128725cefd Merge branch 'master' into sf/weighted_crossentropy 2017-12-13 15:14:47 +00:00
Mike J Innes
403cc26327 Merge branch 'master' into gru 2017-12-12 16:54:00 +00:00
Mike J Innes
86097e76fd tweak batchnorm example 2017-12-08 19:34:34 +00:00
Mike J Innes
6f997e798a Merge branch 'master' into batchnorm 2017-12-08 19:31:50 +00:00
Mike J Innes
1d916c81b5 Merge branch 'master' into HEAD 2017-12-08 18:31:55 +00:00
Elliot Saba
41446d547f Add weighted_crossentropy for imbalanced classification problems 2017-12-05 17:09:05 -08:00
Elliot Saba
c59b820bed Add glorot (Xavier) initialization 
Set default `Dense` and `RNN` inits to `glorot_uniform()` for `W`, `zeros` for `b`.
 2017-12-05 14:24:48 -08:00
baggepinnen
fa718c7475 Implement Gated Recurrent Unit 2017-11-24 14:33:06 +01:00
Mike J Innes
dc1f08a709
Merge pull request #98 from FluxML/log 
GPU-ready log function
 2017-11-23 17:17:39 +00:00
Mike J Innes
351d3d4771 std derivative 2017-11-21 17:04:04 +01:00
Mike J Innes
b06884b912 LayerNorm tweaks 2017-11-21 16:32:36 +01:00
skariel
11d53781b2 adding layer normalization 2017-11-21 16:30:24 +01:00
Mike J Innes
e0657d93ec mv numeric.jl to nnlib 2017-11-09 15:06:29 +00:00
Mike J Innes
2cb94981a0 gpu-ready log 2017-11-09 15:04:01 +00:00
Iblis Lin
6c7613e02b batchnorm: leverage TrackedArray mean 2017-11-02 14:20:34 +08:00
Iblis Lin
88bd8a8fbd batchnorm: make CuArrays happy 2017-11-02 14:02:41 +08:00
Iblis Lin
477da75428 batchnorm: fix mapchildren 2017-11-02 13:32:12 +08:00
Iblis Lin
5253841acc batchnorm: update docs 2017-11-02 13:32:12 +08:00
Iblis Lin
b3356cc6bb batchnorm: batch σ correct coefficient 2017-11-02 13:32:12 +08:00
Iblis Lin
e0201be770 batchnorm: parameterize momentum and epsilon 2017-11-02 13:32:12 +08:00
Iblis Lin
669273b008 layer: implement BatchNorm layer 
See [Batch Normalization: Accelerating Deep Network Training by Reducing
Internal Covariate Shift](https://arxiv.org/pdf/1502.03167.pdf)
 2017-11-02 13:32:12 +08:00
Mike J Innes
e943a39ee7 combine special cases 2017-10-31 16:37:33 +00:00
Mike J Innes
cf6b930f63 reorganise 2017-10-26 11:46:12 +01:00
CarloLucibello
711ea09d99 address comments 2017-10-25 02:35:27 +02:00
CarloLucibello
536ab3861d setmode! -> testmode! 2017-10-23 16:23:29 +02:00
CarloLucibello
86c7c9246e add == and < for tracked arrays 2017-10-23 11:41:08 +02:00
CarloLucibello
2e1ed4c3fc add dropout 2017-10-23 10:12:53 +02:00
Mike J Innes
2a66545ef8 rnn state reset 2017-10-19 17:21:08 +01:00
Mike J Innes
b817ce632c syntax highlighting 2017-10-18 15:44:06 +01:00
Mike J Innes
fd249b773e rnn docs 2017-10-18 15:30:05 +01:00
Mike J Innes
190f48a709 nnlib docs 2017-10-18 14:40:58 +01:00
Mike J Innes
0fbc8dff61 typoe 2017-10-18 12:48:58 +01:00
Mike J Innes
d6dd27dae5 dense layer example 2017-10-18 12:47:45 +01:00
Mike J Innes
23674b2555 logitcrossentropy tweaks 2017-10-17 17:58:32 +01:00
pevnak
4aa7741ba9 logit cross entropy 2017-10-17 17:57:46 +01:00
Mike J Innes
6dff8ca8d3 rename crossentropy loss 2017-10-17 17:36:18 +01:00
Mike J Innes
949fd9ba97 loss function tweaks 2017-10-17 17:30:11 +01:00
Mike J Innes
d3db051ca0 flip 2017-10-16 08:53:39 +01:00
Dave Kleinschmidt
2b95aff158 actually use init argument in LSTMCell 2017-10-03 19:26:42 +01:00
Mike J Innes
a60a754d68 beginnings of gpu support 2017-09-27 21:58:34 +01:00
Mike J Innes
4bafa2b374 generic tree functions 2017-09-27 21:11:21 +01:00
Mike J Innes
2ec8401d2c remove compiler 2017-09-27 20:48:39 +01:00
Mike J Innes
f2052739c1 tweaks 2017-09-12 14:11:03 +01:00
Mike J Innes
a3fe89e348 rnn tanh by default 2017-09-12 13:12:25 +01:00
Mike J Innes
6728295355 Merge pull request #63 from JobJob/rnncell-args 
Enables passing an activation function to RNN/RNNCell
 2017-09-12 13:10:43 +01:00
Mike J Innes
28bbef81b9 f 2017-09-12 13:06:32 +01:00
Mike J Innes
972ecab9f9 rm Over Seq 2017-09-12 13:03:16 +01:00
Joel Mason
00439555d1 Enables passing an activation function to RNN/RNNCell 
Also, fixes it not using the `init` function provided
 2017-09-12 20:54:56 +10:00
Mike J Innes
b9652f1812 typo, fixes #61 2017-09-12 10:45:07 +01:00
Mike J Innes
5f24d61ba3 important 2017-09-11 14:10:12 +01:00
Mike J Innes
7041ab9960 rm chainseq 2017-09-11 14:02:43 +01:00
Mike J Innes
33a5d26e57 chain utility note 2017-09-09 20:02:48 -04:00
Mike J Innes
fedee95b14 docs updates 2017-09-09 19:58:32 -04:00
Mike J Innes
a36d6d2af3 layer docs 2017-09-08 17:52:41 -04:00
Mike J Innes
f55b8cd20e track -> param 2017-09-07 15:13:04 -04:00
Mike J Innes
e837bb0745 rnn stuff 2017-09-07 00:27:04 -04:00
Mike J Innes
cca4d25a10 efficient traversal 2017-09-06 23:09:32 -04:00
Mike J Innes
4083c34547 seq stuff 2017-09-06 18:59:36 -04:00
Mike J Innes
1946c46e29 basic seq functionality 2017-09-06 14:03:25 -04:00
Mike J Innes
c95e9376a5 constructors 2017-09-05 19:25:34 -04:00
Mike J Innes
b023da1b7d lstm initialisation 2017-09-05 02:42:32 -04:00
Mike J Innes
61de692b50 lstm nonzero hidden state 2017-09-05 02:37:48 -04:00
Mike J Innes
ec02f1fabd batching in rnns 2017-09-05 02:29:31 -04:00
Mike J Innes
bd5822fd71 cleaner lstm 2017-09-03 02:24:47 -04:00
Mike J Innes
9642ae8cd6 basic recurrence 2017-09-03 02:12:44 -04:00
Mike J Innes
f6771b98cd clearer name for dense 2017-09-02 16:50:11 -04:00
Mike J Innes
9ce0439943 better mse 2017-08-24 11:40:51 +01:00
Mike J Innes
e4e9794f5e loss function gradients 2017-08-23 17:50:43 +01:00
Mike J Innes
0ce8c0cee4 param collection 2017-08-22 17:13:03 +01:00
Mike J Innes
ef681f16ea use nnlib for activations 2017-08-21 17:53:04 +01:00
Mike J Innes
3e0f45046c nicer show 2017-08-21 17:20:09 +01:00
Mike J Innes
09974caba0 extend affine with activation 2017-08-20 13:35:35 +01:00
Mike J Innes
18e69b33c9 forwarddiff does these 2017-08-19 22:05:50 +01:00
Mike J Innes
ad0e0ea5a7 explicitly broadcast sigmoid 2017-08-19 22:04:47 +01:00
Mike J Innes
4a9dc40e7c simplify organisation 2017-08-19 20:52:29 +01:00
Mike J Innes
318e503d9b remove rnns 2017-08-19 20:52:17 +01:00
Mike J Innes
b18d8cd08f rename chain.jl 2017-08-19 20:26:07 +01:00
Mike J Innes
0222103c7f fix these later 2017-08-19 20:05:22 +01:00
Mike J Innes
bd6bffde48 silo the compiler 2017-08-19 20:04:21 +01:00
Mike J Innes
f8482ff80c remove back!, update! 2017-08-18 10:18:45 +01:00
Mike J Innes
21089fea9c remove some explicit back! 2017-08-18 01:19:10 +01:00
Mike J Innes
052cc52ada remove inference, for now 2017-08-18 01:19:10 +01:00
Mike J Innes
8869f1f953 these are now useless 2017-08-18 01:05:46 +01:00
ylxdzsw
c9ae219613 simplify back! of Chain 2017-06-14 21:58:43 +08:00
ylxdzsw
cca21a617c training julia models 2017-06-12 19:39:34 +08:00
Mike J Innes
099b9fbeb7 collect exports 2017-06-05 16:37:06 +01:00
Mike J Innes
837173d65b model abstract is pretty useless 2017-06-05 16:08:23 +01:00
Mike J Innes
4685d2e672 strip down non-obvious exports 2017-06-05 15:47:26 +01:00
Mike J Innes
d1f370a2f1 move stateful 2017-06-02 16:02:47 +01:00
Mike J Innes
2b85c76785 preserve original param states 2017-06-02 15:46:24 +01:00
Mike J Innes
07c523a13d fix flatten/softmax batching behaviour 2017-06-01 19:28:02 +01:00
Elliot Saba
dfa4dcccb5 Fix activation functions 2017-05-25 18:12:58 -07:00
Mike J Innes
e04dcbd460 reorganise recurrent stuff 2017-05-04 10:45:44 +01:00
Mike J Innes
36f4103d7d cost is really a layer 2017-05-01 12:30:28 +01:00
Mike J Innes
14afe54143 fixes for recurrent networks 2017-04-19 17:17:37 +01:00