TastyPancakes/Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,477 Commits 104 Branches 42 Tags 5.8 MiB
746e3310f1
Commit Graph

1597 Commits

Author SHA1 Message Date
Katharine Hyatt
b8b4bc48b9 Backticks and examples for normalise 2019-10-21 10:31:44 -04:00
Dhairya Gandhi
4477dd8d54 reviews 2019-10-10 20:27:11 +05:30
Dhairya Gandhi
f19066ee29 more docstrings 2019-10-10 16:48:12 +05:30
Dhairya Gandhi
fe52689cfe in depth docstrings 2019-10-09 16:16:11 +05:30
thebhatman
96a23c295c Changes to docs 2019-10-09 14:53:03 +05:30
bors[bot]
af0dcb2c63
Merge #882 
882: Check if CUDA availability changed during init. r=MikeInnes a=maleadt

With this PR, Flux checks using CUDAapi if CUDA is available during initialization, and forces recompilation if that does not agree with what was decided during precompilation. This avoids the scenario where Flux was precompiled without GPU support, consequently not allowing use of the GPU even if the user fixed his CUDA/GPU set-up because that does not force recompilation (and we can't add precompilation dependencies on stuff that doesn't exist).

However, we can't do the same for the case where we have a GPU/CUDA but CuArrays fails to import (checking if it imports during `__init__` would be much too expensive, if even possible), so this PR removes support for having CUDA/a GPU but CuArrays being broken. That's a little risky now that Flux depends on CuArrays, but the package is pretty mature and I haven't seen many bug reports failing to load it recently.

Fixes https://github.com/FluxML/Flux.jl/pull/852#issuecomment-538028314

cc @MikeInnes @xukai92

Co-authored-by: Tim Besard <tim.besard@gmail.com>
 2019-10-08 13:24:49 +00:00
Dhairya Gandhi
b503741651 expanded docstrings 2019-10-04 14:46:03 +05:30
Tim Besard
8aea15e6e0 Demote to const variables. 2019-10-03 21:28:55 +02:00
Tim Besard
2369b2b3fd Add an environment variable to disable CUDA usage. 2019-10-03 21:27:54 +02:00
Tim Besard
63d196aa37 Check if CUDA availability changed during init. 2019-10-03 20:05:32 +02:00
thebhatman
ec886c8ce8 Added docstring for hinge loss 2019-10-03 21:13:09 +05:30
Manjunath Bhat
2b30319a55
Merge branch 'master' into patch-6 2019-09-30 21:05:02 +05:30
thebhatman
6e289ef939 Merge branch 'patch-6' of https://github.com/thebhatman/Flux.jl into patch-6 2019-09-30 20:55:44 +05:30
Filippo Vicentini
606fe58854
Use <:Number 2019-09-29 12:33:02 +02:00
Filippo Vicentini
14e94c291e
Make it actually work 2019-09-29 12:28:01 +02:00
Filippo Vicentini
d91677f651
Fix params! to work with complex numbers 2019-09-29 12:23:41 +02:00
Dhairya Gandhi
8013c728b1 clearer optimiser docstrings 2019-09-28 16:09:00 +05:30
Dhairya Gandhi
0175485a80 fixup 2019-09-27 22:08:25 +05:30
Dhairya Gandhi
8bb0db7d0c opt docstrings 2019-09-27 22:04:53 +05:30
Mike Innes
b90b02872f Merge branch 'master' into tb/cuarrays_dnn 2019-09-27 14:58:32 +01:00
Mike Innes
46bc8e5e64 move pullbacks to CuArrays 2019-09-26 17:14:18 +01:00
Michael Abbott
806e0c5c57 line 2019-09-25 15:20:13 +02:00
Michael Abbott
4245d9acad eg 2019-09-25 15:18:40 +02:00
Michael Abbott
2de84ce79f simplify 2019-09-25 13:59:32 +02:00
Michael Abbott
1a1a96571a +Chain 2019-09-25 13:47:29 +02:00
Michael Abbott
19830c71b1 fix printing of SkipConnection 2019-09-25 13:37:01 +02:00
bors[bot]
acb6a89245
Merge #865 
865: Functor r=MikeInnes a=MikeInnes

This refactors our current `@treelike` infrastructure. It somewhat formalises what we're doing around the idea of a Flux model as a functor, i.e. something that can be mapped over.

This is much more flexible than what we had before, and avoids some issues. It allows layers to have state that isn't mappable; it allows for dispatch when walking the tree, which means layers like `BatchNorm` can have non-trainable parameters; and it also allows for zipped mapping like `fmap(+, xs, ys)`, which isn't implemented yet but will be useful for the new optimisers work.

The main downside is that the term `functor` has been previously used in the Julia community as a malapropism for "thing that behaves like a function"; but hopefully this can start to reduce that usage.

Co-authored-by: Mike Innes <mike.j.innes@gmail.com>
 2019-09-24 16:36:10 +00:00
Dhairya Gandhi
822288d63d merge conflicts 2019-09-24 00:31:44 +05:30
Dhairya Gandhi
6846551f57 fix cuda init 2019-09-22 22:02:05 +05:30
Mike Innes
b60df53ba1 pkg up 2019-09-19 18:33:33 +01:00
Mike Innes
cabb81e30b internal rename 2019-09-19 15:53:31 +01:00
Mike Innes
b951377426 fix normalisation layer params 2019-09-19 15:33:24 +01:00
Mike Innes
6529dbcbe6 functor refactor 2019-09-19 15:22:11 +01:00
Mike Innes
2c71fc282b rename functor.jl 2019-09-19 14:15:28 +01:00
Mike Innes
c5e56b7e04 move setweights and copy_transpose 2019-09-17 17:22:35 +01:00
Mike Innes
5baebf48f4 Merge branch 'master' into tb/cuarrays_dnn 2019-09-17 16:17:09 +01:00
Mike Innes
368b1f53b4 tuple support 2019-09-17 15:49:39 +01:00
Mike Innes
b348b20452 cudnn rnns + implicit gradients 2019-09-17 15:41:42 +01:00
Mike Innes
fe57215b7e test fillarray gradients 2019-09-17 15:21:03 +01:00
Tim Besard
4942d7fcfd Move functionality over to CuArrays. 2019-09-13 08:21:45 +02:00
Tim Besard
1e7ff4f65d Query the worksize. 2019-09-13 08:04:05 +02:00
Tim Besard
04fce70019 Move low-level CUDNN wrappers to CuArrays. 2019-09-13 08:04:05 +02:00
Mike Innes
de2049450b docs mostly fixed 2019-09-10 15:17:07 +01:00
Mike Innes
c8d460ff84 doctests passing 2019-09-10 15:02:43 +01:00
Mike J Innes
67c38b3099 Merge branch 'master' into zygote 2019-09-06 15:18:58 +01:00
thebhatman
ecc9ce9d64 Gradient on AlphaDropout now working 2019-09-06 16:34:19 +05:30
Mike J Innes
3c1ac84676
Merge pull request #842 from baggepinnen/patch-4 
Add RADAM optimizer
 2019-09-02 14:36:40 +01:00
Manjunath Bhat
c3cc4bf966
Remove double docstring 2019-08-31 01:35:40 +05:30
thebhatman
2f1a187665 Update AlphaDropout 2019-08-31 01:28:58 +05:30
Fredrik Bagge Carlson
cb3bfd72f3
Export RADAM from Optimise 2019-08-29 07:46:45 +08:00
Mike J Innes
9cd97f06f7 define has_cuarrays when no cuda 2019-08-27 15:06:04 +01:00
Tim Besard
4fef9d8508 Don't depend on unreleased CuArrays. 2019-08-27 09:40:22 +02:00
Tim Besard
6ad3cdd138 Replace Requires with direct CuArrays dependency. 2019-08-27 09:33:15 +02:00
janEbert
dec1b37e8e Merge remote-tracking branch 'origin/master' into HEAD 2019-08-24 12:23:10 +02:00
janEbert
978d7bf195 Fix CuArrays.libcudnn imports 2019-08-24 02:21:54 +02:00
Mike Innes
487000ac31 fix cuda code and tests 2019-08-19 16:56:48 +01:00
Mike Innes
6c67404398 update cleanup 2019-08-19 15:44:51 +01:00
Mike Innes
447fd9d604 conv docstring formatting 2019-08-19 15:30:59 +01:00
Mike Innes
2f7ad895aa test cleanups 2019-08-19 15:22:50 +01:00
Mike Innes
9590aa63e3 rm last uses of param/data 2019-08-19 15:14:42 +01:00
thebhatman
a76e4d128b Remove param from crosscor 2019-08-19 19:19:53 +05:30
Manjunath Bhat
8456b7ba45
Remove param from groupnorm 2019-08-19 19:16:21 +05:30
Mike Innes
3ecca436e4 formatting fix 2019-08-19 14:42:07 +01:00
Mike Innes
49044dff7c avoid adjoint on abstract type 2019-08-19 14:39:09 +01:00
Mike Innes
b8fabad337 deprecate param/data 2019-08-19 14:35:48 +01:00
Fredrik Bagge Carlson
3287cf23db
Add RADAM export 2019-08-19 13:07:39 +08:00
Fredrik Bagge Carlson
ebbad0d135
Add RADAM optimizer 2019-08-19 12:22:32 +08:00
Miguel Madrid Mencía
14affbc91b
Use CuArrays.ones instead cuones which is deprecated 2019-08-11 13:38:44 +02:00
Mike J Innes
7c111e7cde fixes #645 
fixes #831
 2019-08-09 13:53:11 +01:00
Moelf
4d00957b36
Fix CuArray zeros deprecation 2019-08-06 22:23:21 +02:00
Christopher Rackauckas
ed12d4e7c0
Momentum doesn't need params 2019-07-31 17:56:51 -04:00
Mike J Innes
f3551da5a2 dropout printing 2019-07-24 11:20:39 -04:00
thebhatman
faac0ff08b Updated InstanceNorm and GroupNorm to avoid mutation 2019-07-18 16:13:58 +05:30
Manjunath Bhat
b779d43aca
replaced trunc Int with div 2019-07-16 17:52:55 +05:30
thebhatman
2816fbb9b2 Fix for getindex error in BatchNorm 2019-07-12 22:19:41 +05:30
Mike Innes
a140c31f72 fix batchnorm 2019-07-12 16:09:42 +01:00
Mike Innes
1fc584102d fix dropout 2019-07-12 15:38:28 +01:00
Mike Innes
e2bf46b7fd gpu test fixes 2019-07-12 14:52:01 +01:00
Mike Innes
33c8d84a60 cuparam -> cuarray 2019-07-11 14:14:56 +01:00
Manjunath Bhat
11c9a8450c
Remove active from GroupNorm 2019-07-11 18:40:48 +05:30
Mike Innes
c2cd7dab91 re-export gradient 2019-07-11 13:55:12 +01:00
DrChainsaw
16d5f2bc24 Add x to seen in prefor to avoid infinite recursion if passed something self-referential 2019-07-08 23:11:35 +02:00
thebhatman
cf5bc801d3 Check for nothing in update step 2019-07-08 19:22:23 +05:30
thebhatman
8d78b437ff Merge branch 'sf/zygote_updated' of https://github.com/thebhatman/Flux.jl 2019-07-08 18:47:17 +05:30
thebhatman
812541f8d6 zeros replaced by fill to avoid nothing grad 2019-07-06 19:41:03 +05:30
thebhatman
3ee2a76f61 Removed .data from LSTMCell 2019-07-02 17:38:30 +05:30
thebhatman
b194e7e3a8 Callback being called now 2019-06-20 00:37:54 +05:30
Dhairya Gandhi
dd9cdbef14 remove uncessary call to beta 2019-06-16 19:09:50 +05:30
Dhairya Gandhi
67f18663d9 pick beta from state in NADAM 2019-06-16 19:06:59 +05:30
thebhatman
7ab9d8ed3d Minor update 2019-06-13 18:59:03 +05:30
thebhatman
ce11804dc1 CrossCor test passing, hopefully. 2019-06-13 01:21:58 +05:30
thebhatman
48ed93cdaa Silly error in Dropout corrected. 2019-06-12 23:16:15 +05:30
thebhatman
e9797408ec DepthwiseConv corrected again. 2019-06-12 23:01:51 +05:30
thebhatman
00a4f4c26d Correcting Dropout 2019-06-12 22:39:30 +05:30
thebhatman
bd7e3b1f41 Dropout with dims test passing. 2019-06-12 22:16:11 +05:30
thebhatman
c7c0ee2cbc Resolving Merge Conflicts 2019-06-12 21:34:42 +05:30
thebhatman
dfd2965e85 GroupNorm tests corrected 2019-06-11 22:32:54 +05:30
thebhatman
11073dcd25 GroupNorm made to use istraining() 2019-06-11 22:04:33 +05:30
thebhatman
ef63f80644 No ops defined for param and data 2019-06-10 18:24:18 +05:30
Mike J Innes
b98075817c
Merge branch 'master' into DenseBlock 2019-06-05 14:27:47 +01:00
ayush-1506
2161163a82 added crosscor 2019-05-14 02:52:28 -07:00
Bruno Hebling Vieira
796a2957c9 Added news and removed type annotation from SkipConnection structure 2019-05-13 16:33:31 -03:00
Bruno Hebling Vieira
e7d76b8423 Added the SkipConnection layer and constructor 
Added missing export

Corrected channel placement

Dimension 4 cannot be assumed to always be the Channel dimension

Deprecation of `treelike`

Code now makes use of `@treelike` macro instead of the deprecated `treelike` function (it worked on my end because I'm on Julia 0.7, while Julia 1.0 deprecated stuff)

Update basic.jl

Renaming to SkipConnection

* Update Flux.jl

* Update basic.jl

Updated `SkipConnection` with a `connection` field

I'm pretty sure I broke something now, but this PR should follow along these lines `cat` needs special treatment (the user can declare his own `concatenate` connection, but I foresee it's going to be used often so we can simply define special treatment)

Forgot to remove some rebasing text

Forgot to remove some more rebasing text

Removed local copy and default cat method from the function calls

Adjusted some more types for inference, could improve on this as well

Re-placed some left-over spaces
 2019-05-13 16:32:00 -03:00
bors[bot]
68ba6e4e2f Merge #563 
563: noise shape for dropout r=MikeInnes a=chengchingwen

I add the noise shape for dropout, similar to the `noise_shape` argument in [`tf.nn.dropout`](https://www.tensorflow.org/api_docs/python/tf/nn/dropout)

Co-authored-by: chengchingwen <adgjl5645@hotmail.com>
Co-authored-by: Peter <adgjl5645@hotmail.com>
 2019-05-13 17:16:10 +00:00
chengchingwen
2fc2a5282c Merge remote-tracking branch 'upstream/master' into drop_shape 2019-05-14 00:50:59 +08:00
Elliot Saba
2e6561bb6a Change DepthwiseConv() to use in=>out instead of in=>mult. 
This is an API change, but I think it makes more sense, and is more
consistent with our `Conv()` api.
 2019-05-12 11:20:24 -07:00
chengchingwen
5c5140683c make dims as field of Dropout 2019-05-10 23:45:50 +08:00
Mike J Innes
92ddc618f8 update for arrays 2019-05-02 18:57:52 -07:00
Mike J Innes
c70276ddfe rm more deprecations 2019-05-02 18:57:52 -07:00
Mike J Innes
256695262c rm optimiser deprecations 2019-05-02 18:54:01 -07:00
Mike J Innes
82ee61f5be implement #643 2019-05-02 18:52:09 -07:00
Mike J Innes
c313be8e95 rm data/param 2019-05-02 18:52:09 -07:00
Mike J Innes
aa4d221f8c break all the things 2019-05-02 18:50:52 -07:00
Avik Pal
a0be6fa837
Add missing activation function for batchnorm 2019-05-01 19:47:54 +05:30
Dhairya Gandhi
221670a2b1
Merge pull request #733 from thebhatman/expdecay-fix 
Fixed ExpDecay
 2019-05-01 18:58:37 +05:30
Dhairya Gandhi
9bbbd17e4b
Merge branch 'master' into onecold 2019-04-30 19:09:36 +05:30
Roger-luo
d63338c242 fix doctest 2019-04-26 18:12:14 +08:00
Mike J Innes
6c3a939133
Update src/onehot.jl 
Co-Authored-By: Roger-luo <hiroger@qq.com>
 2019-04-26 18:09:14 +08:00
Roger-luo
fabcd05ff2 add examples 2019-04-26 18:05:03 +08:00
Elliot Saba
732f97fe16 Split out conv_transpose_dims() so that Zygote can ignore it 2019-04-25 10:24:19 -07:00
Elliot Saba
6e22cd4931 Add asymmetric padding to convolutional layers 2019-04-25 09:55:23 -07:00
Elliot Saba
113ddc8760 Update Flux code for new NNlib branch 2019-04-25 09:55:23 -07:00
Hossein Pourbozorg
7f06b15f67 use https instead of http for web links 2019-04-25 11:04:03 +00:00
Jake Topping
ff7adda74b
Swap comma for full stop 
"ERROR: LoadError: UndefVarError: G not defined" caused by "gn,G" rather than "gn.G" in line 386. Swapping for full stop should fix this
 2019-04-22 17:08:36 +01:00
Zachary P Christensen
83eb5a1df6
Fix typo in Maxout 2019-04-19 17:02:26 -04:00
thebhatman
31a50ab16a Fixed ExpDecay 2019-04-11 17:28:06 +05:30
Mike J Innes
54d9229be9
Merge pull request #710 from johnnychen94/master 
naive implementation of activations
 2019-04-05 15:33:31 +01:00
Johnny Chen
a300376f71
fix a typo in comment 
`inplementation` --> `implementation`
 2019-04-05 19:19:30 +08:00
JohnnyChen
3cafbbad02 simplify the implementation 2019-04-05 18:44:00 +08:00
JohnnyChen
de7a5f4024 correct the function behavior; support Any type 2019-04-05 18:16:44 +08:00
thebhatman
b84ab7ac95 Removed logcosh 2019-04-05 03:16:54 +05:30
bors[bot]
bd9d73a941 Merge #655 
655: Added support for Float64 for DepthwiseConv r=dhairyagandhi96 a=thebhatman

DepthwiseConv was giving errors for Float64. This fixes the issue.

Co-authored-by: Manjunath Bhat <manjunathbhat9920@gmail.com>
 2019-04-04 17:25:52 +00:00
chengchingwen
261235311c change dims as unbroadcasted dims and keyword argument 2019-04-05 01:19:20 +08:00
Dhairya Gandhi
1963f30911
Merge pull request #726 from dhairyagandhi96/iris 
use cached iris dataset
 2019-04-04 22:46:21 +05:30
Dhairya Gandhi
9c8175b1c0 fixes 2019-04-04 22:32:01 +05:30
Dhairya Gandhi
4f754d33cb switch to http link 2019-04-04 22:18:38 +05:30
Dhairya Gandhi
38cc216a4b switch to azure 2019-04-04 22:03:01 +05:30
Dhairya Gandhi
77274b4af7 change iris link 2019-04-04 21:07:46 +05:30
Dhairya Gandhi
2952bcdab1 fixes 2019-04-04 19:28:40 +05:30
Dhairya Gandhi
5b9c53439b recreate OHV 2019-04-04 19:19:47 +05:30
Dhairya Gandhi
4f1336905f fix colon indexing 2019-04-04 19:16:14 +05:30
Shreyas
4cb7b9278b Minor changes to docstring according to guidelines 2019-03-30 00:28:23 +05:30
JohnnyChen
82595648e2 change 4-spaces tab to 2-spaces tab 2019-03-28 22:40:24 +08:00
Shreyas
b6fcd1d837 Added export to Maxout 2019-03-28 19:15:16 +05:30
JohnnyChen
13c58494ec add x into results 2019-03-28 19:28:59 +08:00
JohnnyChen
5c2a071713 add support for 0-element Chain 2019-03-28 17:20:41 +08:00
JohnnyChen
ccfe0f8720 naive implementation of activations 2019-03-28 17:07:04 +08:00
Shreyas
61c1fbd013 Made Requested Changes 2019-03-28 01:33:04 +05:30
Shreyas
671aed963e Made a few fixes. Added tests 2019-03-28 00:51:50 +05:30
thebhatman
4efcc69ba5 logcosh averaged 2019-03-26 23:23:02 +05:30