957: Add some gradient checking tests on GPUs r=dhairyagandhi96 a=dhairyagandhi96
Good to add generic tests for tracking gradients through the various layers on the GPU.
Co-authored-by: Dhairya Gandhi <dhairya@juliacopmuting.com>
Co-authored-by: Dhairya Gandhi <dhairya@juliacomputing.com>
998: test restructure on the GPU r=CarloLucibello a=ChrisRackauckas
Requires https://github.com/FluxML/Zygote.jl/pull/474 to pass
Co-authored-by: Chris Rackauckas <accounts@chrisrackauckas.com>
865: Functor r=MikeInnes a=MikeInnes
This refactors our current `@treelike` infrastructure. It somewhat formalises what we're doing around the idea of a Flux model as a functor, i.e. something that can be mapped over.
This is much more flexible than what we had before, and avoids some issues. It allows layers to have state that isn't mappable; it allows for dispatch when walking the tree, which means layers like `BatchNorm` can have non-trainable parameters; and it also allows for zipped mapping like `fmap(+, xs, ys)`, which isn't implemented yet but will be useful for the new optimisers work.
The main downside is that the term `functor` has been previously used in the Julia community as a malapropism for "thing that behaves like a function"; but hopefully this can start to reduce that usage.
Co-authored-by: Mike Innes <mike.j.innes@gmail.com>
bors[bot]