Improvements like...
- fixing typos,
- removing trailing and double whitespaces,
- using `jldoctest` blocks where applicable,
- fixing, updating or correctly setting up existing doctests,
- improving consistency (for example, always use "# Examples" instead
of other variants),
- removing empty lines between docstrings and functions,
- instead of mentioning keywords, put them into the docstring,
- adding some missing but useful keywords,
- adding references (`@ref`),
- using LaTeX math where applicable, and
- linking papers.
Debatable stuff that is untouched:
- BE/AE s/z irregularities ("normalise" versus "normalize") since
most papers use the AE version while the Flux source code was
written with BE spelling.
- Names of normalization functions are capitalized
("Batch Normalization" instead of "batch normalization").
1080: CompatHelper: bump compat for "Colors" to "0.12" r=dhairyagandhi96 a=github-actions[bot]
This pull request changes the compat entry for the `Colors` package from `0.8, 0.9, 0.10, 0.11` to `0.8, 0.9, 0.10, 0.11, 0.12`.
This keeps the compat entries for earlier versions.
Note: I have not tested your package with this new compat entry. It is your responsibility to make sure that your package tests pass before you merge this pull request.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1078: CompatHelper: bump compat for "CodecZlib" to "0.7" r=CarloLucibello a=github-actions[bot]
This pull request changes the compat entry for the `CodecZlib` package from `0.5, 0.6` to `0.5, 0.6, 0.7`.
This keeps the compat entries for earlier versions.
Note: I have not tested your package with this new compat entry. It is your responsibility to make sure that your package tests pass before you merge this pull request.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1028: Common questions answered in docs r=CarloLucibello a=dhairyagandhi96
cc @MikeInnes
1070: Prevent breakage due to new `active` field in normalise layers r=CarloLucibello a=ianshmean
Prevents breakage where the normalise structs, such as `BatchNorm`, have been directly defined but missing the new `active` field
cc. @darsnack
Co-authored-by: Dhairya Gandhi <dhairya@juliacopmuting.com>
Co-authored-by: Dhairya Gandhi <dhairya@juliacomputing.com>
Co-authored-by: Ian <i.r.butterworth@gmail.com>
1053: Added Some Loss functions with some doc improvements r=CarloLucibello a=AdarshKumar712
Added the following loss functions with tests:
1. mae
2. mean squared logarithmic error
3. huber loss
4. squared hinge loss
5. dice coeff loss
6. tversky loss
Also added some documentation improvements for few other functions.
Co-authored-by: Adarsh Kumar <45385384+AdarshKumar712@users.noreply.github.com>
1064: Include cuda/cuda.jl during precompilation? r=CarloLucibello a=ianshmean
Loading `cuda/cuda.jl` at run-time during `__init__()` seems to be causing issues with PackageCompiler. (see error at bottom).
I'm wondering the cost of loading `cuda/cuda.jl` is negligible enough to just do it in all cases and get it precompiled. Setting `Flux.use_cuda[]` would stil be used for switching cuda on or off.
Load time in 1.3.1 on my mac (without cuda):
This PR:
```
julia> @time using Flux
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
37.313982 seconds (56.30 M allocations: 2.822 GiB, 2.52% gc time)
...
julia> @time using Flux
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
22.111054 seconds (52.93 M allocations: 2.663 GiB, 3.99% gc time)
```
Master:
```
julia> @time using Flux
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
35.750143 seconds (53.73 M allocations: 2.698 GiB, 2.51% gc time)
...
julia> @time using Flux
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
26.267999 seconds (52.92 M allocations: 2.660 GiB, 3.67% gc time)
```
I didn't make `include("cuda/cuda.jl")` dependent on `CuArrays.functional()` because I guess there could be a case where, say, a user doesn't have cuda installed, loads Flux, installs cuda, reloads Flux.. where the 2nd time the package isn't re-precompiled.
The PackageCompiler error, which doesn't happen every time. It just seems that the runtime loading of cuda.jl may be introducing dep tracking issues (?)
```
┌ Warning: Package Zygote does not have InteractiveUtils in its dependencies:
│ - If you have Zygote checked out for development and have
│ added InteractiveUtils as a dependency but haven't updated your primary
│ environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with Zygote
└ Loading InteractiveUtils into Zygote from project dependency, future warnings for Zygote are suppressed.
fatal: error thrown and no exception handler available.
#<null>
require at ./loading.jl:905
_jl_invoke at /home/ian/Documents/julia-kf-31156/src/gf.c:2161 [inlined]
jl_apply_generic at /home/ian/Documents/julia-kf-31156/src/gf.c:2328
jl_apply at /home/ian/Documents/julia-kf-31156/src/julia.h:1695 [inlined]
call_require at /home/ian/Documents/julia-kf-31156/src/toplevel.c:399 [inlined]
eval_import_path at /home/ian/Documents/julia-kf-31156/src/toplevel.c:436
eval_import_from at /home/ian/Documents/julia-kf-31156/src/toplevel.c:557
jl_toplevel_eval_flex at /home/ian/Documents/julia-kf-31156/src/toplevel.c:646
jl_eval_module_expr at /home/ian/Documents/julia-kf-31156/src/toplevel.c:181
jl_toplevel_eval_flex at /home/ian/Documents/julia-kf-31156/src/toplevel.c:640
jl_parse_eval_all at /home/ian/Documents/julia-kf-31156/src/ast.c:907
jl_load_rewrite at /home/ian/Documents/julia-kf-31156/src/toplevel.c:872
include at ./Base.jl:380
include at ./Base.jl:368 [inlined]
include at /home/ian/.julia/packages/Flux/p8ZLv/src/Flux.jl:1 [inlined]
__init__ at /home/ian/.julia/packages/Flux/p8ZLv/src/Flux.jl:56
jfptr___init___22072 at /home/ian/Documents/MyPackage.jl/dev/compilation/MyPackageSysImage.so (unknown line)
_jl_invoke at /home/ian/Documents/julia-kf-31156/src/gf.c:2161 [inlined]
jl_apply_generic at /home/ian/Documents/julia-kf-31156/src/gf.c:2328
jl_apply at /home/ian/Documents/julia-kf-31156/src/julia.h:1695 [inlined]
jl_module_run_initializer at /home/ian/Documents/julia-kf-31156/src/toplevel.c:74
_julia_init at /home/ian/Documents/julia-kf-31156/src/init.c:788
unknown function (ip: 0x5594b1667f)
__libc_start_main at /lib/aarch64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x5594b16733)
unknown function (ip: 0x5594b16733)
```
Co-authored-by: Ian <i.r.butterworth@gmail.com>
