TastyPancakes/Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,939 Commits 104 Branches 42 Tags 5.8 MiB
66ce8d8066
Commit Graph

1356 Commits

Author SHA1 Message Date
pevnak
926411a449 removed most error, the only one in Fallbacks test persits 2018-08-03 15:14:25 +01:00
pevnak
c657d4e47f fixed the sum as suggested by mike 2018-08-03 15:14:25 +01:00
Simon Mandlik
02f343d44d fixed more dep warns, also in tests, but maximum, minimum and size in array.jl still need to be updated. As a result, some more tests may not pass for the time being 2018-08-03 15:14:25 +01:00
Simon Mandlik
0471c489e6 depwarns 2018-08-03 15:14:25 +01:00
pevnak
3510c837a8 zeros replaced by zero 2018-08-03 15:14:25 +01:00
pevnak
ea38c7dbea some more changes 2018-08-03 15:14:25 +01:00
pevnak
d6f5baee39 fixed fixes proposed by Carlo 2018-08-03 15:14:25 +01:00
pevnak
8ab209126d removed zeros fix 2018-08-03 15:14:25 +01:00
pevnak
e98538673a updated sum to be compliant with latest beta. Removed some depwarns 2018-08-03 15:14:25 +01:00
Mike J Innes
e5b3d27016 track_kw should be unnecessary 2018-08-03 15:14:10 +01:00
Avik Pal
4d17a1a809
Merge branch 'master' into depthwiseconv 2018-08-03 19:41:50 +05:30
Avik Pal
6a41f823c8 Update track function 2018-08-03 19:06:05 +05:30
Avik Pal
b4ba7df03a Merge branch 'master' of https://github.com/FluxML/Flux.jl into cudnn_batchnorm 2018-08-03 18:55:46 +05:30
Mike Innes
f5c9361617 matmul fix 2018-08-03 13:02:47 +01:00
Mike Innes
4cf6bac0c1 fix hook 2018-08-03 13:02:47 +01:00
Mike J Innes
70718e7a64 update treelike 2018-08-03 13:02:47 +01:00
Mike J Innes
d782b33701 syntax 2018-08-03 13:02:47 +01:00
Mike J Innes
85fd77d70a linalg deprecations 2018-08-03 13:02:47 +01:00
Mike J Innes
89872c5a8b val deprecations 2018-08-03 13:02:47 +01:00
Mike J Innes
474f578517 ObjectIdDict -> IdDict 2018-08-03 13:02:47 +01:00
Mike J Innes
aa209ee137 no longer needed 2018-08-03 13:02:47 +01:00
Mike J Innes
00cfe24d66 fix cat 2018-08-03 13:02:47 +01:00
Mike J Innes
adc216f182 fix broadcasting 2018-08-03 12:56:32 +01:00
Mike J Innes
e486c50610 fix data 2018-08-03 12:56:31 +01:00
Mike J Innes
fb8a220659 fix matmul 2018-08-03 12:56:31 +01:00
Mike J Innes
7057ca739e fix std usage 2018-08-03 12:56:27 +01:00
Mike J Innes
88a265154c deprecations 2018-08-03 12:54:31 +01:00
Mike J Innes
b18b51656c requires update 2018-08-03 12:54:24 +01:00
Mike J Innes
a49e2eae41 deprecated Void 2018-08-03 12:53:52 +01:00
Mike J Innes
1fd49c2a90 fix array show 2018-08-03 12:53:52 +01:00
Yueh-Hua Tu
5b37319289 Add Maxpool and Meanpool 2018-08-01 00:10:53 +08:00
Mike J Innes
a8ccc79f61 perf hacks 2018-07-30 20:08:44 +01:00
Avik Pal
2cc0f112f1 Updates 2018-07-27 20:12:49 +05:30
Avik Pal
7dd5ec16c9 Fix 2018-07-17 11:22:12 +05:30
Avik Pal
531ecccd38 Error statement 2018-07-17 10:14:23 +05:30
Avik Pal
4035641f00 Remove imports 2018-07-17 10:06:26 +05:30
Avik Pal
0bb3eaa1f6 Update CUDNN Batchnorm with new Flux AD 2018-07-17 09:40:20 +05:30
Avik Pal
646db81f94 Pull BatchNorm CPU updates 2018-07-17 09:24:38 +05:30
CarloLucibello
071dcdda87 update docs 2018-07-16 07:32:13 +02:00
CarloLucibello
185e9148b6 fix cpu batchnorm 2018-07-16 07:11:33 +02:00
Avik Pal
2664a16556 Update as per new AD 2018-07-13 14:12:46 +05:30
Avik Pal
0aabf9d86b
Merge branch 'master' into depthwiseconv 2018-07-13 14:04:19 +05:30
Mike J Innes
a0fd91b866
Merge pull request #307 from jarvist/master 
Add ADAMW "Fixing Weight Decay Regularization in Adam"
 2018-07-11 19:12:58 +01:00
Mike J Innes
dda51a0140 update docs 2018-07-11 15:31:22 +01:00
Mike Innes
10a169bb77 update cudnn rnn 2018-07-10 18:16:37 +01:00
Mike J Innes
70b5efeb4e basic nested AD 2018-07-10 09:03:09 +01:00
Mike J Innes
80af9a3830 broadcast efficiency 2018-07-09 23:40:07 +01:00
Mike J Innes
e763c342ee shave some memory 2018-07-09 19:44:14 +01:00
Mike J Innes
1430053b69 checkpoints 2018-07-09 17:52:34 +01:00
Mike J Innes
7778d17884 functional API 2018-07-09 16:57:44 +01:00
Mike J Innes
5e319c7395 fix gradient definitions 2018-07-09 13:39:10 +01:00
Mike J Innes
41b9412439 new grad api 2018-07-09 13:36:46 +01:00
Jarvist Moore Frost
344a750770 Merge branch 'master' of github.com:jarvist/Flux.jl into HEAD 2018-07-03 11:15:43 +01:00
Jarvist Moore Frost
aee4a83c55 Add ADAMW weight-decay. 
See http://www.fast.ai/2018/07/02/adam-weight-decay/ and the original
paper https://arxiv.org/abs/1711.05101.pdf for context.

I don't know what I'm doing, and this is quite possibly wrong - but on
a simple Char-RNN I have lying around on my harddisk, this seems to
improve the rate of learning consistently for different hyperparameters
vs. standard ADAM with the same decay constant.
 2018-07-03 11:11:32 +01:00
Mike J Innes
ce88273880 gradient hook 2018-07-02 13:19:13 +01:00
Mike Innes
5d8b63dc65 avoid implementation details in docs 2018-06-29 13:53:50 +01:00
Avik Pal
e3b10691d2 make cache optional param 2018-06-28 15:27:59 +05:30
Avik Pal
bcf094451c Fix typo 2018-06-28 14:45:35 +05:30
Avik Pal
d0b79e71e2 fix load error 2018-06-28 14:27:50 +05:30
Avik Pal
7ac9e191cb Revert 1 change 2018-06-28 14:25:22 +05:30
Avik Pal
5ccde88ce6 Minor fix for 5D support 2018-06-28 14:21:17 +05:30
Avik Pal
681d8c4dfc Remove cache 2018-06-28 12:11:32 +05:30
Avik Pal
8f43258ab7 Get the batchnorm working without cache 2018-06-28 12:04:25 +05:30
Avik Pal
4916c8e6da Add treelike for now 2018-06-27 14:54:49 +05:30
Matthew Kelley
864d72eef5 Overload Base.eps() for TrackedReal 2018-06-26 23:55:43 -06:00
Matthew Kelley
0e95be3326 Call Flux.Tracker.data() on ŷ for bce 2018-06-26 14:48:51 -06:00
Matthew Kelley
ed032cdb1e Change epsilon value to eps(ŷ) 2018-06-26 12:29:06 -06:00
Matthew Kelley
e08fd7a6d2 Added epsilon term to binarycrossentropy 2018-06-26 11:43:16 -06:00
Mike J Innes
88c16e62dd fixes #284 2018-06-26 15:09:26 +01:00
Mike J Innes
836e3872b6 style 2018-06-26 15:09:21 +01:00
Mike J Innes
2723c9ee04
Merge pull request #257 from staticfloat/sf/back_inf_nan 
Check for `Inf` and `NaN` within `back!(::TrackedReal)`
 2018-06-26 14:42:33 +01:00
Mike J Innes
0a04e3ba61 Chain activations 2018-06-26 14:30:46 +01:00
Mike J Innes
7726a5b605 inferrable 2018-06-26 14:12:57 +01:00
Mike J Innes
3b575930ca Merge branch 'master' into scalar_pad_stride 2018-06-26 14:05:07 +01:00
Mike Innes
7e3cf45ee4 better error 2018-06-25 11:36:52 +01:00
Avik Pal
24ba1c4e6c Make changes as per the review 2018-06-23 11:02:41 +05:30
Mike J Innes
aea1e73cde scalar gradients 2018-06-21 13:12:42 +01:00
Avik Pal
91850a8baf Add missing path to curnn.jl 2018-06-20 18:46:42 +05:30
Avik Pal
deb4950261 Make cuDNN take only 4D arrays 2018-06-20 15:54:38 +05:30
Avik Pal
3339ad5181 Integrate cudnn BatchNorm with Flux 2018-06-20 15:50:30 +05:30
Avik Pal
714ca23aba Change default value of epsilon to prevent CuDNN BatchNorm warnings 2018-06-20 12:11:22 +05:30
Avik Pal
185f34d9fe Add working backward pass 2018-06-20 12:09:54 +05:30
Avik Pal
bc47d02b3f Remove uncessary imports 2018-06-17 12:40:01 +05:30
Avik Pal
af5ab7f9ef Fix Tensor Descriptor Bug 2018-06-17 12:28:02 +05:30
Avik Pal
c6dcf079ce Update file structure and make function calls correct 2018-06-17 11:47:49 +05:30
Avik Pal
24d13ac326 Fix missing parenthesis 2018-06-12 21:32:56 +05:30
Avik Pal
f12e367cab Adding untested backward pass code 2018-06-12 18:26:09 +05:30
Avik Pal
a83e5d696d Typo 2018-06-12 17:51:52 +05:30
Avik Pal
d4b066fdf9 Forward Pass for BatchNorm Added 2018-06-12 17:49:21 +05:30
Avik Pal
65f2c33991
Merge pull request #2 from FluxML/master 
rebase
 2018-06-11 15:40:57 +05:30
Avik Pal
b59da95786 Merge branch 'depthwiseconv' of https://github.com/avik-pal/Flux.jl into depthwiseconv 2018-06-09 13:11:42 +05:30
Avik Pal
5d7ee884b8 Fix error while backpropagatio 2018-06-09 13:04:49 +05:30
Avik Pal
7f3d11cae0
Merge branch 'master' into depthwiseconv 2018-06-09 11:06:07 +05:30
Avik Pal
1d93fb8e59 Add new constructor and fix a typo in display 2018-06-09 11:02:15 +05:30
Tejan Karmali
d20771d6be
Default value of dilation 
dilation should be 1 by default
 2018-06-09 02:29:46 +05:30
Tejan Karmali
4a24b69976
Merge branch 'master' into nadam-opt 2018-06-08 16:54:41 +05:30
Mike J Innes
4915b0c8dd
Merge pull request #268 from staticfloat/patch-2 
Add `dilation` kwarg to `Conv`
 2018-06-07 13:49:02 +01:00
Mike J Innes
af8f3348eb
Merge pull request #270 from staticfloat/sf/tracked_repeat 
Add `TrackedArray` support for `repeat(x; inner, outer)`
 2018-06-06 17:34:58 +01:00
Mike Innes
2370bdbe91 see #205 2018-06-06 17:01:28 +01:00
Avik Pal
33a7f545b7
Merge branch 'master' into depthwiseconv 2018-05-30 15:58:35 +05:30
Avik Pal
cd6a0856d5 Adds support for Depthwise Convolutions 2018-05-30 15:53:57 +05:30
staticfloat@gmail.com
f390a39d77 Add TrackedArray support for repeat(x; inner, outer) 2018-05-22 17:41:05 -07:00
Elliot Saba
e6efca4bf4 Add dilation kwarg to Conv 
Now that we have dilated convolution support in `NNlib`, this is enables support in Flux's `Conv` layer.
 2018-05-21 13:44:13 -07:00
James Bradbury
af12f006f2
Use broadcast for dropout 
Should be fast enough on GPU now that it's not going to be an optimization target again for a while. Hopefully isn't meaningfully slower on CPU?
 2018-05-20 04:04:33 -07:00
staticfloat@gmail.com
9fdbe843ef Check for Inf and NaN within back!(::TrackedReal) 
This is often checked for within user code, no reason to do that, let's
do it for them within `back!(::TrackedReal)`
 2018-05-07 15:30:44 -07:00
Mike J Innes
24ad384a38
Merge pull request #243 from gustafsson/catdim 
Support for hcat and cat
 2018-05-07 13:04:31 +01:00
Mike Innes
ef9077d9fa style 2018-05-07 13:03:52 +01:00
Mike Innes
b59161a41e export Tracker again 2018-05-05 17:15:18 +01:00
Johan Gustafsson
5fc6190956 RowVector tests 2018-05-02 16:10:39 +02:00
Johan Gustafsson
94bb064a0f more tests of array promotion for concatenation 
# Conflicts:
#	test/tracker.jl
 2018-05-02 16:00:29 +02:00
Johan Gustafsson
1c189c62ed cat with multiple dims #156 
Co-authored-by: americast <sayan.sinha@iitkgp.ac.in>
 2018-05-02 15:59:46 +02:00
Johan Gustafsson
fb68529169 define back function right after forward function 2018-05-02 15:59:46 +02:00
Johan Gustafsson
509a2e59f6 cat promotions and mixed ranks 2018-05-02 15:59:46 +02:00
Johan Gustafsson
eaaf5fd34c vcat arrays with ndims>2 2018-05-02 15:59:46 +02:00
Johan Gustafsson
bcef5c4ab5 Support hcat and cat 2018-05-02 15:59:46 +02:00
Mike J Innes
7d7d89569c rm this deprecation for 0.6 2018-05-01 12:20:36 +01:00
Mike J Innes
9a7e6e9c5c hold off on some things 2018-05-01 12:18:56 +01:00
CarloLucibello
e186b958dd more exports 2018-05-01 12:13:14 +01:00
Mike J Innes
ee89a7797e
Merge pull request #245 from freeboson/adamax 
Add AdaMax optimizer
 2018-05-01 11:28:07 +01:00
Mike J Innes
5efbaddb97
Merge pull request #249 from ninjin/nin/minimum 
[RFC] Backpropagation for `maximum` and `minimum`
 2018-04-30 18:40:42 +01:00
Mike J Innes
73a51400b6 better error message 2018-04-30 12:09:15 +01:00
Pontus Stenetorp
cfd29b9c76 Backpropagation for maximum and minimum 2018-04-29 13:52:54 +01:00
Sujeet Akula
8c042bd522
element wise max() 2018-04-26 21:12:31 +10:00
Sujeet Akula
5e5f255f81
export typo 2018-04-26 17:42:04 +10:00
Sujeet Akula
4586bda5ab
export/test adamax 2018-04-26 17:40:11 +10:00
Sujeet Akula
b6508e2416
add adamax 2018-04-26 17:37:24 +10:00
Mike J Innes
baff20514d gpu broadcast fix 2018-04-17 18:05:58 +01:00
Mike J Innes
8f73dc6e14 fix gpu cross entropy 2018-04-17 17:56:47 +01:00
tejank10
2ef25775c6 removed extra expand and fixed bug 2018-04-16 01:18:26 +05:30
Mike Innes
d12fb98f2a nicer batchnorm shape error 2018-04-15 20:29:25 +01:00
tejank10
2f5473d435 added expand in conv constructor 2018-04-16 00:59:11 +05:30
Mike J Innes
8f29968c32
Merge pull request #207 from safnuk/pull-request/07b0f95d 
BatchNorm for convolutions
 2018-04-15 20:10:33 +01:00
Mike J Innes
683a73fed3 download info 2018-04-15 20:09:30 +01:00
Mike J Innes
5fd240f525 interface tweaks 2018-04-15 20:04:42 +01:00
Mike J Innes
73a0be3e04 Merge branch 'master' into pull-request/07b0f95d 2018-04-15 17:10:29 +01:00
Mike J Innes
642543808e
Merge pull request #226 from CarloLucibello/reshape 
fix reshape
 2018-04-15 16:53:21 +01:00
tejank10
b080f5c82e Scalar pad and stride 2018-04-15 20:32:40 +05:30
Mike J Innes
cb3ae8df6a rename normalise.jl 2018-04-15 15:45:46 +01:00
Mike J Innes
b05e755068 rm jit from cuda 2018-04-15 15:08:58 +01:00
tejank10
5cc681317a added stride for pooling in tracker 2018-04-15 15:07:04 +01:00
tejank10
f6097d58d6 Scalar pad/stride for Conv constructor 2018-04-15 12:15:41 +05:30
Mike Innes
9d7164f15f we'll do this differently 2018-04-14 02:09:35 +01:00
tejank10
65847bb745 moved epsilon into sqrt 2018-04-04 15:25:20 +05:30
tejank10
3ead662987 Update rule fixed 2018-04-04 15:18:44 +05:30
CarloLucibello
b415333233 fix reshape 2018-04-02 16:09:57 -04:00
tejank10
ea9b5471fa NADAM optimizer 2018-04-03 01:27:22 +05:30
Brad Safnuk
b9a66c679d Fix error in initialization of σ. 2018-03-22 22:20:21 -04:00
Brad Safnuk
35299d4621 Fix type instability when loading onto a gpu. 
Also fixes Issue #216.
 2018-03-22 21:32:32 -04:00
Mike J Innes
4320738d87 fix 2018-03-21 11:25:47 +00:00
Mike Innes
1c5f8e3534 ndims for shapes 2018-03-16 14:42:08 +00:00
Brad Safnuk
db2d9efb72 Update BatchNorm documentation 2018-03-15 21:59:38 -04:00
Brad Safnuk
6653ec86d9 Allow multidimensional inputs to batchnorm. 
Can be used in conjunction with convolutional layers, in addition
to dense layers, with the same api.
 2018-03-15 21:48:59 -04:00
Mike J Innes
e931552f7d
Merge pull request #200 from chengchingwen/repmat 
implement `back` of `repmat`
 2018-03-15 15:18:48 +00:00
Mike J Innes
5d7edb5aaa
Merge pull request #197 from chengchingwen/master 
Implement `prod` for `TrackedArray`
 2018-03-15 15:17:24 +00:00
boathit
2ec37790be eliminate ambiguity 2018-03-13 10:50:56 +08:00
boathit
ff2caf032c eliminate ambiguous 2018-03-12 22:48:16 +08:00
Mike J Innes
9ccbac8b80 jit gpu support 2018-03-07 19:18:27 +00:00
chengchingwen
43af3895b0 change prod implementation to avoid small xs 2018-03-07 21:03:13 +08:00
chengchingwen
c00f7f850f implement back of repmat 2018-03-07 20:43:59 +08:00
chengchingwen
7c721475c6 add gradient check for prod and fix dims in back(::typeof(prod),...) 2018-03-07 16:24:44 +08:00
Mike Innes
d21c313ea7 tweaks 2018-03-06 19:58:47 +00:00
Mike Innes
36baa7ec2c convnet primitives 2018-03-06 19:58:05 +00:00
Mike Innes
0802b4d5cf closes #198 2018-03-06 16:56:01 +00:00
Elliot Saba
6445295318 Better download detection 2018-03-06 08:45:45 -08:00
Elliot Saba
19f691d342 Use cache.julialang.org to store ML models 
It's annoying that when third party servers go down our tests break.
Let's at least make sure that if our tests break due to server outages
it's our fault.
 2018-03-06 08:03:21 -08:00
Mike Innes
3babeeb440 scalar hashing 2018-03-06 13:49:05 +00:00
chengchingwen
86d782a5ce implement prod for TrackedArray 2018-03-06 18:01:19 +08:00
Mike Innes
c95a97f6ae make epochs available 2018-03-06 03:01:56 +00:00
Mike Innes
432b9c3222 loadparams! 2018-03-06 02:45:31 +00:00
Mike Innes
65ed95190a fix 2018-03-05 23:44:25 +00:00
Mike Innes
bfd6a4c0ec cleaner interrupts 2018-03-05 23:05:45 +00:00
Mike Innes
5153cde847 move epochs 2018-03-05 22:56:22 +00:00
Mike J Innes
662439c164 closes #177 2018-03-05 17:24:46 +00:00
Elliot Saba
36295799ee Add permutedims() for tracked arrays 2018-03-02 10:22:28 -08:00
Mike J Innes
8019f789f8 use normal log 2018-03-01 16:35:49 +00:00
Mike J Innes
ac57fc3c26 use @ fix in a few places 2018-03-01 16:31:20 +00:00
Mike J Innes
c2fea2acf6 revert this 2018-02-28 23:06:53 +00:00
Mike J Innes
2eb38eedbf update gpu api 2018-02-28 22:51:08 +00:00
Mike J Innes
ccef9f4dd4 jit softmax 2018-02-28 22:07:35 +00:00
Mike J Innes
7606b1a399 single-batch convolution 2018-02-28 14:25:32 +00:00
Mike J Innes
6bdc2b37a9 inline call 2018-02-28 13:47:14 +00:00
Mike J Innes
a401f08cda compile layers 2018-02-27 22:40:51 +00:00
Mike J Innes
5a32976cbf basic compile step 2018-02-27 21:43:41 +00:00
Mike J Innes
bdb8aae107 move cache logic 2018-02-27 21:41:03 +00:00
Mike J Innes
2c74976602 more general 2018-02-27 01:25:40 +00:00
Mike J Innes
466b5c501a cpu/gpu conveniences 2018-02-26 23:10:59 +00:00
Mike J Innes
15d1d3256b conv api updates 2018-02-26 22:43:07 +00:00
Mike J Innes
54919b8dca rm deprecation 2018-02-22 00:23:02 +00:00
Mike J Innes
491785a681 ignore state in mapleaves 2018-02-22 00:22:51 +00:00
Mike J Innes
ec65e2cec7 fix printing 2018-02-22 00:21:48 +00:00
Mike J Innes
af2e6b7e1d fix 2018-02-22 00:15:38 +00:00
Mike J Innes
99b739cf00 fixes #176 2018-02-21 23:21:20 +00:00
Mike J Innes
e3b4b16e01
Merge pull request #178 from schmrlng/pull-request/e6f55641 
Convert OneHot CuArrays to dense CuArrays before passing to CUDNN methods
 2018-02-21 22:34:11 +00:00
Mike J Innes
6bdd283fbd no longer necessary 2018-02-21 22:29:31 +00:00
Iblis Lin
043fedde3c
introduce Reexport 
- Reexporting NNlib

fix #180
 2018-02-21 16:55:20 +08:00
Ed Schmerling
e6f556411a Convert OneHot CuArrays to dense CuArrays before passing to CUDNN methods 2018-02-19 17:32:15 -08:00
Mike J Innes
4035745f6e may help numerical tests 2018-02-19 12:51:02 +00:00
Mike J Innes
989adcdc7d gpu fix 2018-02-17 12:41:53 +00:00
Mike J Innes
11511982a4 numerical stability 2018-02-17 11:56:03 +00:00
Mike J Innes
e5791bc5f6 frequencies utility 2018-02-17 11:19:51 +00:00
Mike J Innes
e3b31b9b87
Merge pull request #169 from jessebett/jessechanges 
Reshape with Tuple Dimensions and Kronecker Product
 2018-02-16 14:16:42 +00:00
Mike J Innes
60f21d3ff2 don't override base method 2018-02-16 14:15:40 +00:00
Mike J Innes
5e861101f3 epochs util 2018-02-16 11:17:57 +00:00
Mike J Innes
7aa6854c64 more correct 2018-02-16 00:06:15 +00:00
Mike J Innes
ee3784964e fix for external modules 2018-02-15 22:27:00 +00:00
Mike J Innes
63862c2324 easier initialisation with weights 2018-02-15 20:52:29 +00:00
Mike J Innes
01c31e7fcc conv bias 2018-02-15 20:15:41 +00:00
Mike J Innes
bdd07a8bc6 fix 2018-02-14 22:34:11 +00:00
Mike J Innes
1b8b1cd7b1 check params by identity 2018-02-14 21:00:50 +00:00
Mike J Innes
5ea0ef6764 tracker fix 2018-02-13 16:15:36 +00:00
Mike J Innes
1baa7227e3 reorganise batches 2018-02-13 16:05:07 +00:00
Mike J Innes
34217b1fa2 Merge branch 'treebank' 2018-02-13 15:44:27 +00:00
Mike J Innes
49584fb72b rm logsigmoid 2018-02-13 14:52:29 +00:00
Mike J Innes
2f29733888 Merge branch 'master' into HEAD 2018-02-13 14:45:37 +00:00
Mike J Innes
8432d8db06 batchnorm fix 2018-02-13 14:02:35 +00:00
Mike J Innes
820cd3ae42 fixes #164 2018-02-13 13:31:35 +00:00
Mike J Innes
066cb45a38 remove old accuracy fn 2018-02-13 11:12:21 +00:00
Mike J Innes
236edbffec fixes #111 2018-02-13 10:20:38 +00:00
Mike J Innes
f22cfb5b43 re-enable printf 2018-02-12 15:05:09 +00:00
Mike J Innes
334ae9e1cb fixes #171 2018-02-12 12:31:15 +00:00
Mike J Innes
0b3c02fe8d document regularisation, fixes #160 2018-02-09 19:00:26 +00:00
Mike J Innes
0e0057b0c4 basics 2018-02-09 13:51:07 +00:00
jessebett
f84ee8eab0 reshape with tupled dimensions and kronecker product 2018-02-08 14:27:57 -05:00
Mike J Innes
70fbbf48fa humble beginnings of compiler 2018-02-08 18:11:26 +00:00
Mike J Innes
fc157a8c59 TrackedNumber -> TrackedReal 2018-02-08 17:18:40 +00:00
Mike J Innes
d1c56ca768 number fix 2018-02-08 17:04:48 +00:00
Mike J Innes
0f7a1ec022 test params funct 2018-02-08 16:13:20 +00:00
Mike J Innes
961de2ba44
Merge pull request #161 from FluxML/curnn 
WIP: CUDNN RNNs
 2018-02-08 13:06:52 +00:00
Iblis Lin
f7fdfbe3a9 fix params 2018-02-08 12:56:10 +00:00
Mike J Innes
fcbdc49d6b fix reserve usage 2018-02-08 10:27:26 +00:00
Mike J Innes
bc452fcd81 rewrite tests 2018-02-08 02:37:55 +00:00
Mike J Innes
d592f4e327 batch support 2018-02-08 01:45:48 +00:00
Mike J Innes
b8f148b012 hook up backward passes 2018-02-08 00:49:39 +00:00
Mike J Innes
a1d1930097 Merge branch 'master' into curnn 2018-02-07 23:23:02 +00:00
Mike J Innes
4511936a87 fixes #116 2018-02-07 23:21:04 +00:00
Mike J Innes
0ac924e8e1 fixups 2018-02-07 22:52:46 +00:00
Mike J Innes
39f7f8fdf3 tracked tuples 2018-02-07 22:21:42 +00:00
Mike J Innes
79e4e25fea seperate number type 2018-02-07 20:39:36 +00:00
Mike J Innes
282889970d seperate tracking infrastructure from array wrapper 2018-02-07 17:43:25 +00:00
Mike J Innes
30b3437c56 backward passes 2018-02-06 18:56:17 +00:00
Mike J Innes
f866fbe575 nullable c refactor 2018-02-06 15:01:48 +00:00
Mike J Innes
07e1b1e0a9 avoid val 2018-02-06 12:44:18 +00:00
boathit
7e37a96c6f Register back! for logsigmoid and implement (logit)binarycrossentropy 2018-02-06 19:36:16 +08:00
boathit
6e65789828 Register back! for logsigmoid and implement (logit)binarycrossentropy 2018-02-06 19:32:46 +08:00
Mike J Innes
a4bf5936b0 diagm 2018-02-05 18:29:35 +00:00
Mike J Innes
2fec75005d
Merge pull request #123 from GenaBitu/cat-fix 
Added vcat for multiple TrackedVectors
 2018-02-05 18:10:48 +00:00
Mike J Innes
47cebab26e test multiple inputs/dims 2018-02-05 18:09:54 +00:00
Mike J Innes
2a2475a9c2 get tracker graph 2018-02-05 17:40:07 +00:00
Mike J Innes
14086b8c2d train forward pass 2018-02-02 17:48:08 +00:00
Mike J Innes
9a6fcf057b hook up interface 2018-02-02 16:42:18 +00:00
Mike J Innes
b1c5786012 Merge branch 'master' into curnn 2018-02-02 15:56:44 +00:00
Mike J Innes
49e1e78f67 make data/value available 2018-02-02 15:56:04 +00:00
Mike J Innes
0f1e7b5578 update rnn structure 2018-02-01 20:57:39 +00:00
Mike J Innes
106502a75d typo 2018-01-31 21:57:04 +00:00
Mike J Innes
af3ccf85ff coagulate gates 2018-01-31 16:56:27 +00:00
Mike J Innes
4bfb603da6 gru forward 2018-01-31 13:46:55 +00:00
Mike J Innes
b1bb05403c basic forward pass 2018-01-30 18:18:37 +00:00
Mike J Innes
0b886507dc param offsets 2018-01-30 14:43:39 +00:00
Mike J Innes
af0c5523ff rnnTrainingReserveSize 2018-01-30 14:43:39 +00:00
Mike J Innes
3fb83d642d rnnWorkspaceSize 2018-01-30 14:43:39 +00:00
Mike J Innes
6b4e114d5d rnnParamSize 2018-01-30 14:43:39 +00:00
Mike J Innes
ee6c3e18a9 basic RNNDesc 2018-01-30 14:43:39 +00:00
Mike J Innes
842bf03051 typo 2018-01-30 14:43:05 +00:00
Mike J Innes
0c9549c469 rm lazy 2018-01-24 13:28:52 +00:00
Mike J Innes
5118ef9163 remove batching work for now 2018-01-24 13:12:38 +00:00
boathit
374d7a5f1e Registering backward function for logsoftmax 2018-01-21 15:20:59 +08:00
Mike J Innes
72eabde373 load data 2018-01-17 16:39:55 +00:00
Mike J Innes
bd57359535 docstrings 2018-01-17 16:12:12 +00:00
Mike J Innes
8cca7accf2 mnist 2018-01-17 15:55:37 +00:00
Mike J Innes
4207fb98f2 basic GPU tests 2018-01-16 17:58:14 +00:00
GenaBitu
bc8a32bc56
Merge branch 'master' into cat-fix 2018-01-16 11:01:31 +01:00
Mike J Innes
1beb30e19a closes #118 2018-01-15 17:00:47 +00:00
Mike J Innes
8f8589a7f4 fix initialisation 2018-01-10 14:11:52 +00:00
Mike J Innes
b44237468e Merge branch 'master' into gru 2018-01-10 13:59:33 +00:00
Mike J Innes
805cb9178f fixes #146 2018-01-10 12:48:50 +00:00
Mehul Tikekar
2fef799109 fix typo in conv.jl (fixes #133) 2018-01-08 16:46:58 -05:00
Mike J Innes
468f641f66 use Adapt 2018-01-08 16:34:22 +00:00
Mike J Innes
98b362729d pool padding 2017-12-18 18:18:14 +00:00
Mike J Innes
e3577d759c conv docs 2017-12-18 18:05:48 +00:00
Mike J Innes
269d8f36b9 conv padding 2017-12-18 18:05:38 +00:00
Mike J Innes
51f93d9f0e conv polish 2017-12-15 16:24:45 +00:00
Mike J Innes
386eafc443 reshape 2017-12-15 16:18:16 +00:00
Mike J Innes
73ae25289d remove old util 2017-12-15 16:18:01 +00:00
Mike J Innes
6890a61587 todo 2017-12-15 16:17:45 +00:00
Mike J Innes
9b833a4345 more onehot indexing 2017-12-15 16:17:39 +00:00
Mike J Innes
9d0dd9fb7e layer wip 2017-12-15 13:22:57 +00:00
Mike J Innes
0bf22dfb8e pool gradients 2017-12-15 02:29:14 +00:00
Mike J Innes
d949b31aa5 conv gradient 2017-12-15 02:24:32 +00:00
Mike J Innes
5b97d2ba04 closes #127 2017-12-13 18:24:56 +00:00
Mike J Innes
95d1287455 Merge branch 'master' into jacobian 2017-12-13 17:06:23 +00:00
Mike J Innes
27d896943e
Merge pull request #120 from staticfloat/sf/dense_initialization 
Better default initialization for Dense layers
 2017-12-13 16:18:02 +00:00
Mike J Innes
e3a688e706 use kwarg 2017-12-13 15:27:15 +00:00
Mike J Innes
128725cefd Merge branch 'master' into sf/weighted_crossentropy 2017-12-13 15:14:47 +00:00
Mike J Innes
29787eba45 fixes #114 2017-12-12 17:23:15 +00:00
Mike J Innes
b7b6c975bc fixes #110 2017-12-12 17:07:39 +00:00
Mike J Innes
403cc26327 Merge branch 'master' into gru 2017-12-12 16:54:00 +00:00
Mike J Innes
86097e76fd tweak batchnorm example 2017-12-08 19:34:34 +00:00
Mike J Innes
6f997e798a Merge branch 'master' into batchnorm 2017-12-08 19:31:50 +00:00
Mike J Innes
1d916c81b5 Merge branch 'master' into HEAD 2017-12-08 18:31:55 +00:00
Mike J Innes
24a6569589 Merge branch 'master' into amsgrad 2017-12-08 18:20:53 +00:00
Mike J Innes
f82dbf4798 Merge branch 'master' into HEAD 2017-12-08 17:00:31 +00:00
Mike J Innes
951c21366a fix regex 2017-12-08 16:42:30 +00:00
GenaBitu
7e51418679
Added back for multi-parameter vcat 2017-12-08 16:10:09 +01:00
baggepinnen
385dee9d16 Add jacobian function 2017-12-08 14:46:12 +01:00
GenaBitu
41f3eedc39
Proper multi-variable vcat 2017-12-07 17:50:18 +01:00
Elliot Saba
41446d547f Add weighted_crossentropy for imbalanced classification problems 2017-12-05 17:09:05 -08:00
Elliot Saba
c59b820bed Add glorot (Xavier) initialization 
Set default `Dense` and `RNN` inits to `glorot_uniform()` for `W`, `zeros` for `b`.
 2017-12-05 14:24:48 -08:00
GenaBitu
62b3600eca
Merge branch 'master' into cat-fix 2017-12-05 11:13:29 +01:00
baggepinnen
41febee9c1 Export and indent 2017-12-04 09:34:27 +01:00
baggepinnen
36001d085a Implement AMSGrad optimiser 2017-12-04 09:17:05 +01:00
Mike J Innes
cab235a578 gpu compat 2017-11-30 13:51:31 +00:00
Mike J Innes
19039f4881 export sigmoid 2017-11-30 13:37:38 +00:00
Mike J Innes
2d33f19346 onehot unk arg 2017-11-29 16:45:50 +00:00
baggepinnen
fa718c7475 Implement Gated Recurrent Unit 2017-11-24 14:33:06 +01:00
CarloLucibello
13b934c250 improve optimizers 2017-11-24 12:12:20 +01:00
Mike J Innes
dc1f08a709
Merge pull request #98 from FluxML/log 
GPU-ready log function
 2017-11-23 17:17:39 +00:00
Mike J Innes
9f5c4dd3e9
Merge pull request #104 from baggepinnen/patch-1 
Allow array of optimisers to train!
 2017-11-21 17:16:35 +01:00
Mike J Innes
351d3d4771 std derivative 2017-11-21 17:04:04 +01:00
Mike J Innes
b06884b912 LayerNorm tweaks 2017-11-21 16:32:36 +01:00
skariel
11d53781b2 adding layer normalization 2017-11-21 16:30:24 +01:00
Mike J Innes
979949d01a style 2017-11-21 15:25:09 +01:00
Fredrik Bagge Carlson
8991ce028c
Fix bug in rmsprop and adadelta 
`@. p.Δ = η * p.Δ / √acc` parses correctly while `@. p.Δ /= √acc*η` seems to parse like `@. p.Δ /= (√acc*η)`, hence the step size was de facto interpreted as `1/η`
 2017-11-14 17:32:16 +01:00
Mike J Innes
e0657d93ec mv numeric.jl to nnlib 2017-11-09 15:06:29 +00:00
Mike J Innes
2cb94981a0 gpu-ready log 2017-11-09 15:04:01 +00:00
Mike J Innes
e5d99d784e fixes #79 2017-11-09 14:53:26 +00:00
Mike J Innes
ccdc046546 fixes #79 2017-11-09 14:52:28 +00:00
Mike J Innes
752a9e2808 tree utilities 2017-11-08 22:19:01 +00:00
Mike J Innes
6eb2ec154b sentiment treebank loader 2017-11-08 22:19:01 +00:00
Mike J Innes
8777362eee exports 2017-11-08 22:19:01 +00:00
Mike J Innes
8b05317895 basic tree 2017-11-08 22:19:01 +00:00
Mike J Innes
7e9468d8f8 treebank skeleton 2017-11-08 22:19:01 +00:00
Mike J Innes
fcd091e8f0 Ac_mul_B derivatives 2017-11-08 22:18:45 +00:00
Mike J Innes
d4229c4815 useful params method 2017-11-08 22:18:45 +00:00
Mike J Innes
d6423eefe5 matrix-vector fast path 2017-11-08 22:18:45 +00:00
Fredrik Bagge Carlson
97244e0a68
Allow array of optimisers to train! 
This allows an array of optimisers to be sent to `train!`
 2017-11-04 13:27:32 +01:00
Mike J Innes
efa51f02e7 basic batch type 2017-11-02 11:49:42 +00:00
Mike J Innes
21ea93ffcd rename treelike 2017-11-02 11:47:34 +00:00
Iblis Lin
6c7613e02b batchnorm: leverage TrackedArray mean 2017-11-02 14:20:34 +08:00
Iblis Lin
88bd8a8fbd batchnorm: make CuArrays happy 2017-11-02 14:02:41 +08:00
Iblis Lin
477da75428 batchnorm: fix mapchildren 2017-11-02 13:32:12 +08:00
Iblis Lin
5253841acc batchnorm: update docs 2017-11-02 13:32:12 +08:00
Iblis Lin
b3356cc6bb batchnorm: batch σ correct coefficient 2017-11-02 13:32:12 +08:00
Iblis Lin
e0201be770 batchnorm: parameterize momentum and epsilon 2017-11-02 13:32:12 +08:00
Iblis Lin
669273b008 layer: implement BatchNorm layer 
See [Batch Normalization: Accelerating Deep Network Training by Reducing
Internal Covariate Shift](https://arxiv.org/pdf/1502.03167.pdf)
 2017-11-02 13:32:12 +08:00
Mike J Innes
e7a510da9a add cmudict dataset 2017-11-01 16:01:55 +00:00
Mike J Innes
0f8ba87dc6 treelike tuples 2017-10-31 16:37:41 +00:00
Mike J Innes
e943a39ee7 combine special cases 2017-10-31 16:37:33 +00:00
Iblis Lin
3d8b7250ae add scalar mean 2017-10-31 10:42:32 +00:00
Iblis Lin
c43bda019b TrackedArray: implement mean 
```julia
julia> p
Tracked 2×3 Array{Float64,2}:
 1.0  3.0  5.0
 2.0  4.0  6.0
```

Before
```julia
julia> @benchmark Flux.Tracker.back!(sum($p, 2) ./ size($p, 2), ones(2, 1))
BenchmarkTools.Trial:
  memory estimate:  3.44 KiB
  allocs estimate:  75
  --------------
  minimum time:     20.438 μs (0.00% GC)
  median time:      21.239 μs (0.00% GC)
  mean time:        22.354 μs (1.68% GC)
  maximum time:     3.811 ms (98.51% GC)
  --------------
  samples:          10000
  evals/sample:     1
```

After
```julia
julia> @benchmark Flux.Tracker.back!(mean($p, 2), ones(2, 1))
BenchmarkTools.Trial:
  memory estimate:  1008 bytes
  allocs estimate:  21
  --------------
  minimum time:     5.973 μs (0.00% GC)
  median time:      6.310 μs (0.00% GC)
  mean time:        6.630 μs (1.96% GC)
  maximum time:     680.709 μs (97.28% GC)
  --------------
  samples:          10000
  evals/sample:     6
```
 2017-10-30 16:21:02 +08:00
Mike J Innes
4c1b1eb18c Merge pull request #92 from CarloLucibello/drop 
add Dropout layer
 2017-10-26 12:07:28 +01:00
Mike J Innes
84efbbcc84 tracker predicate tweaks 2017-10-26 12:06:29 +01:00
Mike J Innes
cf6b930f63 reorganise 2017-10-26 11:46:12 +01:00
Mike J Innes
0df300299f clearer error message, fixes #93 2017-10-26 11:15:14 +01:00
GenaBitu
df06c3351d
Merge branch 'master' into cat-fix 2017-10-26 00:52:29 +02:00
CarloLucibello
711ea09d99 address comments 2017-10-25 02:35:27 +02:00
CarloLucibello
536ab3861d setmode! -> testmode! 2017-10-23 16:23:29 +02:00
CarloLucibello
00a9e5f01f construct TrackedScalar with params(1) 2017-10-23 10:49:45 +01:00
CarloLucibello
86c7c9246e add == and < for tracked arrays 2017-10-23 11:41:08 +02:00
CarloLucibello
2e1ed4c3fc add dropout 2017-10-23 10:12:53 +02:00
Mike J Innes
2a66545ef8 rnn state reset 2017-10-19 17:21:08 +01:00
Mike J Innes
99a7697d13 adam eta default arg 2017-10-19 14:31:34 +01:00
Mike J Innes
e5c8f6d835 only export known good optimisers 2017-10-19 11:26:11 +01:00
Mike J Innes
5b6a5667ed tracked array restructure 2017-10-18 22:54:58 +01:00
Mike J Innes
c8d4844da4 chunk util 2017-10-18 17:07:58 +01:00
Mike J Innes
07ad7cfa40 learning rate as default arg 2017-10-18 17:07:49 +01:00
Mike J Innes
e82428bb83 batching docs 2017-10-18 16:40:14 +01:00
Mike J Innes
b817ce632c syntax highlighting 2017-10-18 15:44:06 +01:00
Mike J Innes
fd249b773e rnn docs 2017-10-18 15:30:05 +01:00
Mike J Innes
190f48a709 nnlib docs 2017-10-18 14:40:58 +01:00
Mike J Innes
12944ae125 nnlib exports 2017-10-18 12:56:58 +01:00
Mike J Innes
0fbc8dff61 typoe 2017-10-18 12:48:58 +01:00
Mike J Innes
d6dd27dae5 dense layer example 2017-10-18 12:47:45 +01:00
Mike J Innes
7426faf37d optimiser docs 2017-10-18 12:09:48 +01:00
CarloLucibello
041079237e add docsting to train! 2017-10-17 21:04:18 +01:00
CarloLucibello
6d3a2a2210 change argument name for better clarity 2017-10-17 21:04:18 +01:00
Mike J Innes
23674b2555 logitcrossentropy tweaks 2017-10-17 17:58:32 +01:00
pevnak
4aa7741ba9 logit cross entropy 2017-10-17 17:57:46 +01:00
Mike J Innes
6dff8ca8d3 rename crossentropy loss 2017-10-17 17:36:18 +01:00
Mike J Innes
1800c8f523 deprecate mapparams 2017-10-17 17:35:30 +01:00
Mike J Innes
949fd9ba97 loss function tweaks 2017-10-17 17:30:11 +01:00
Mike J Innes
c764b74eba rename and fix mapleaves 2017-10-17 01:08:15 +01:00
Mike J Innes
7aa0b43ceb onehot sanity check 2017-10-17 00:07:58 +01:00
Mike J Innes
e02e320008 more general fmap 2017-10-17 00:07:15 +01:00
Mike J Innes
64e242e96c export param 2017-10-16 08:53:46 +01:00
Mike J Innes
d3db051ca0 flip 2017-10-16 08:53:39 +01:00
Mike J Innes
9a155abecd batch and batchseq apis 2017-10-15 23:44:40 +01:00
Mike J Innes
646720cd05 fix 2017-10-15 23:44:16 +01:00
Mike J Innes
c6556a29e6 order-stable params 2017-10-10 12:16:32 +01:00
GenaBitu
ef6d10886d Exposed all optimisers 2017-10-06 14:20:09 +01:00
GenaBitu
2084df96ae
Merge branch 'master' into cat-fix 2017-10-06 15:00:26 +02:00
pevnak
bfcc1ac25d exposing optimisers 2017-10-05 12:36:18 +01:00
Mike J Innes
1abc4febe6 more general adaptors 2017-10-04 18:55:56 +01:00
Dave Kleinschmidt
2b95aff158 actually use init argument in LSTMCell 2017-10-03 19:26:42 +01:00
Mike J Innes
5fd1b7d9a2 remove gc hack 2017-10-02 20:50:18 +01:00
Mike J Innes
1b91e6b38d store onehotmatrix height 2017-10-02 20:50:11 +01:00
Mike J Innes
7c8dba0b85 gc in training loop 2017-09-27 23:14:58 +01:00
Mike J Innes
a32ae4914c onehotmatrix cuda support 2017-09-27 22:51:00 +01:00
Mike J Innes
a60a754d68 beginnings of gpu support 2017-09-27 21:58:34 +01:00
Mike J Innes
120a6db2bb Merge branch 'master' of github.com:MikeInnes/Flux.jl 2017-09-27 21:16:23 +01:00
Mike J Innes
4bafa2b374 generic tree functions 2017-09-27 21:11:21 +01:00