1064: Include cuda/cuda.jl during precompilation? r=CarloLucibello a=ianshmean
Loading `cuda/cuda.jl` at run-time during `__init__()` seems to be causing issues with PackageCompiler. (see error at bottom).
I'm wondering the cost of loading `cuda/cuda.jl` is negligible enough to just do it in all cases and get it precompiled. Setting `Flux.use_cuda[]` would stil be used for switching cuda on or off.
Load time in 1.3.1 on my mac (without cuda):
This PR:
```
julia> @time using Flux
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
37.313982 seconds (56.30 M allocations: 2.822 GiB, 2.52% gc time)
...
julia> @time using Flux
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
22.111054 seconds (52.93 M allocations: 2.663 GiB, 3.99% gc time)
```
Master:
```
julia> @time using Flux
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
35.750143 seconds (53.73 M allocations: 2.698 GiB, 2.51% gc time)
...
julia> @time using Flux
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
26.267999 seconds (52.92 M allocations: 2.660 GiB, 3.67% gc time)
```
I didn't make `include("cuda/cuda.jl")` dependent on `CuArrays.functional()` because I guess there could be a case where, say, a user doesn't have cuda installed, loads Flux, installs cuda, reloads Flux.. where the 2nd time the package isn't re-precompiled.
The PackageCompiler error, which doesn't happen every time. It just seems that the runtime loading of cuda.jl may be introducing dep tracking issues (?)
```
┌ Warning: Package Zygote does not have InteractiveUtils in its dependencies:
│ - If you have Zygote checked out for development and have
│ added InteractiveUtils as a dependency but haven't updated your primary
│ environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with Zygote
└ Loading InteractiveUtils into Zygote from project dependency, future warnings for Zygote are suppressed.
fatal: error thrown and no exception handler available.
#<null>
require at ./loading.jl:905
_jl_invoke at /home/ian/Documents/julia-kf-31156/src/gf.c:2161 [inlined]
jl_apply_generic at /home/ian/Documents/julia-kf-31156/src/gf.c:2328
jl_apply at /home/ian/Documents/julia-kf-31156/src/julia.h:1695 [inlined]
call_require at /home/ian/Documents/julia-kf-31156/src/toplevel.c:399 [inlined]
eval_import_path at /home/ian/Documents/julia-kf-31156/src/toplevel.c:436
eval_import_from at /home/ian/Documents/julia-kf-31156/src/toplevel.c:557
jl_toplevel_eval_flex at /home/ian/Documents/julia-kf-31156/src/toplevel.c:646
jl_eval_module_expr at /home/ian/Documents/julia-kf-31156/src/toplevel.c:181
jl_toplevel_eval_flex at /home/ian/Documents/julia-kf-31156/src/toplevel.c:640
jl_parse_eval_all at /home/ian/Documents/julia-kf-31156/src/ast.c:907
jl_load_rewrite at /home/ian/Documents/julia-kf-31156/src/toplevel.c:872
include at ./Base.jl:380
include at ./Base.jl:368 [inlined]
include at /home/ian/.julia/packages/Flux/p8ZLv/src/Flux.jl:1 [inlined]
__init__ at /home/ian/.julia/packages/Flux/p8ZLv/src/Flux.jl:56
jfptr___init___22072 at /home/ian/Documents/MyPackage.jl/dev/compilation/MyPackageSysImage.so (unknown line)
_jl_invoke at /home/ian/Documents/julia-kf-31156/src/gf.c:2161 [inlined]
jl_apply_generic at /home/ian/Documents/julia-kf-31156/src/gf.c:2328
jl_apply at /home/ian/Documents/julia-kf-31156/src/julia.h:1695 [inlined]
jl_module_run_initializer at /home/ian/Documents/julia-kf-31156/src/toplevel.c:74
_julia_init at /home/ian/Documents/julia-kf-31156/src/init.c:788
unknown function (ip: 0x5594b1667f)
__libc_start_main at /lib/aarch64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x5594b16733)
unknown function (ip: 0x5594b16733)
```
Co-authored-by: Ian <i.r.butterworth@gmail.com>
1066: fix travis for documentation build r=CarloLucibello a=johnnychen94
The previous build doesn't trigger the documentation stage because the matrix doesn't get expanded for the sole job.
Not very clear how Travis reads the config but this change fixes the issue.
😕 weird that it doesn't allow failures on nightly here... The one in my fork works as expected. https://github.com/johnnychen94/Flux.jl/runs/479502998
cc: @CarloLucibello
Co-authored-by: Johnny Chen <johnnychen94@hotmail.com>
1057: add Julia ecosystem doc section r=CarloLucibello a=CarloLucibello
Partially fixing #251, related to the discussion in #1051 .
Not exactly a poem that I wrote here, maybe someone could suggest a better rephrasing.
Suggestion for additional packages to add to the list also welcome
Co-authored-by: CarloLucibello <carlo.lucibello@gmail.com>
1062: docstring ensure signature code formatting r=CarloLucibello a=visr
by using a four space indent instead of two
Fixes issues seen here:
Where the type signature has no code formatting, and a code block is introduced that throws off the rest of the formatting.
Co-authored-by: Martijn Visser <mgvisser@gmail.com>
1044: Add testmode! back for normalization layers r=CarloLucibello a=darsnack
Fixed#909
I added `testmode!(m, mode)` back to Flux as per v0.9. Now the `mode` can be `false`, `true`, or `:auto`/`nothing` with the default being `:auto` for newly constructed layers. In `:auto` mode, the `istraining()` functions added in v0.10 are used to determine whether we are evaluating within an AD trace or not.
Also plan on adding a doc section in an additional commit.
Co-authored-by: Kyle Daruwalla <daruwalla@wisc.edu>
1051: add DataLoader r=CarloLucibello a=CarloLucibello
Fix#450
This adds a DataLoader type, largely adapted from the Knet one, therefore pinging @denizyuret to check if he is cool with this. Unfortunately, I cannot "unsee" his implementation, and in any case any reasonable alternative implementation will be pretty much similar I guess.
This is an initial implementation to get things going, possibly in the future we will also want a distributed and out-of-memory option as the one implemented by @staticfloat here
https://github.com/FluxML/Metalhead.jl/blob/sf/training/training/ImageNet/dataset.jl
Co-authored-by: CarloLucibello <carlo.lucibello@gmail.com>
1023: Feature: Added Boston Housing Dataset r=CarloLucibello a=pranjaldatta
[Boston Housing Dataset](https://archive.ics.uci.edu/ml/machine-learning-databases/housing/) is one of the most common datasets that are used by beginners. It is as popular as other datasets like Iris etc. Hence, it feels only natural that this dataset is a part of Flux.
Added src/data/housing.jl: code for downloading and loading the dataset
Edited src/data/Data.jl: To include and export housing.jl
Edited test/data.jl: Added test for the dataset.
*All tests in test/data.jl are passing*
Co-authored-by: pranjaldatta <pranjaldatta99@gmail.com>
Co-authored-by: Pranjal Datta <pranjaldatta99@gmail.com>
1041: add NNlib docs + misc docs improvements r=CarloLucibello a=CarloLucibello
Partially addressing https://github.com/FluxML/NNlib.jl/issues/137.
Also, I'm leaving out the `σ` activation and using its alias `sigmoid`, since `σ` conveys little information and it is also used to denote a generic activation in the Dense layer. I think we should deprecate `σ` in NNlib, has this been discussed already?
In an ideal world, before merging this, we should get NNlib to either unexport or add docs to its undocumented exports
Co-authored-by: Carlo Lucibello <carlo.lucibello@gmail.com>
Adarshkumar712
