1221: DataLoader with NamedTuple r=CarloLucibello a=cossio
Just a couple of small changes, so that `DataLoader` can be created with a `NamedTuple` of tensors instead of `Tuple`. This way the tensors can be referred to by name. For example
```
train_loader = DataLoader((images = Xtrain, labels = Ytrain), batchsize=16)
batch = first(train_loader)
y = model(batch.images)
logitcrossentropy(y, batch.labels)
```
If we only use tuples, then in datasets with multiple tensors one has to be careful about the order in which the tensors are fed into the `DataLoader` constructor and be consistent with this elsewhere. With `NamedTuples` one just have to be consistent about the names used, which I think is a minor improvement.
CC @CarloLucibello
### PR Checklist
- [x] Tests are added
- [x] Entry in NEWS.md
- [x] Documentation, if applicable
I don't think this qualifies as an API change. It's just a minor feature addition. So final review probably not required.
- [ ] Final review from `@MikeInnes` or `@dhairyagandhi96` (for API changes).
Co-authored-by: cossio <j.cossio.diaz@gmail.com>
Co-authored-by: cossio <cossio@users.noreply.github.com>
1231: use `ntuple` in conv r=MikeInnes a=MikeInnes
This is the right abstraction over `map`, and in particular is a bit easier to compile away in some cases.
As this is a trivial change from Flux's perspective it's not easy to test here, but there are downstream tests in XLA.jl.
Co-authored-by: Mike J Innes <mike.j.innes@gmail.com>
1218: Require weight and bias to be AbstractArrays r=CarloLucibello a=oxinabox
closes#1199
While in theory someone could be using Dense with weights and biases that are not abstract arrays, I would be surprised.
So allowing it is just leaving a food-gun laying around.
If it is common then we can instead close#1199 by adding a special constructor for `Number` subtypes that error if they are not integers, or something a long those lines.
### PR Checklist
- [x] Tests are added
- [x] Entry in NEWS.md
I think this is a bug-fix thus the following are not required:
- [ ] Documentation, if applicable
- [ ] Final review from `@MikeInnes` or `@dhairyagandhi96` (for API changes).
Co-authored-by: Lyndon White <lyndon.white@invenialabs.co.uk>
Co-authored-by: Lyndon White <oxinabox@ucc.asn.au>
1220: CompatHelper: bump compat for "Adapt" to "2.0" r=CarloLucibello a=github-actions[bot]
This pull request changes the compat entry for the `Adapt` package from `1` to `1, 2.0`.
This keeps the compat entries for earlier versions.
Note: I have not tested your package with this new compat entry. It is your responsibility to make sure that your package tests pass before you merge this pull request.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
1213: Fixing indentation in train! docstring r=CarloLucibello a=natema
One code block is not correctly displayed in the doc of [Flux.Optimise.train!
](https://fluxml.ai/Flux.jl/stable/training/training/#Flux.Optimise.train!).
Based on the previous code block, I guess it's an indentation problem.
Co-authored-by: natema <natema@users.noreply.github.com>
1152: extend dataloader r=CarloLucibello a=CarloLucibello
cfr discussion in #1149. Currently DataLoader interface supports
1. `for x in DataLoader(X)`
2. `for (x, y) in DataLoader(X, Y)`
This PR adds
3. `for (x,) in DataLoader((X,))`
4. `for (x, y) in DataLoader((X, Y))`
Edit:
the constructor in 2. is removed in this PR
Co-authored-by: CarloLucibello <carlo.lucibello@gmail.com>
1129: Added dropgrad in huber_loss r=CarloLucibello a=HenriDeh
Workaround to prevent `iterate(::nothing)` when working with CuArrays. See issue #1128
Co-authored-by: HenriDeh <47037088+HenriDeh@users.noreply.github.com>
1141: Speedup matmul of CuMatrix and OneHotMatrix r=CarloLucibello a=AStupidBear
This solves #189.
```julia
julia> using Flux
julia> using Flux: CuArrays
julia> A = zeros(300, 10000) |> gpu;
julia> B = Flux.onehotbatch(rand(1:10000, 256), 1:10000) |> gpu;
julia> A * B; CuArrays.@time A * B;
┌ Warning: Performing scalar operations on GPU arrays: This is very slow, consider disallowing these operations with `allowscalar(false)`
└ @ GPUArrays ~/shared/.julia/packages/GPUArrays/OXvxB/src/host/indexing.jl:43
0.002824 seconds (951 CPU allocations: 38.156 KiB) (2 GPU allocations: 301.000 KiB, 2.32% gc time of which 46.42% spent allocating)
julia> import Base: *
julia> A::AbstractMatrix * B::Flux.OneHotMatrix = @inbounds A[:, map(x->x.ix, B.data)]
* (generic function with 522 methods)
julia> A * B; CuArrays.@time A * B;
0.000343 seconds (169 CPU allocations: 5.000 KiB) (2 GPU allocations: 301.000 KiB, 15.53% gc time of which 65.97% spent allocating)
```
Co-authored-by: Yao Lu <luyaocns@gmail.com>
1211: Fixing syntax in onehot docstring r=CarloLucibello a=natema
`otherwise, it will error` -> `otherwise, it will raise an error`
Co-authored-by: natema <natema@users.noreply.github.com>
1208: Fixing output format for `onehot` r=dhairyagandhi96 a=natema
Currently `Flux.OneHotVector` is displayed as a binary vector (0/1) rather than a boolean one (true/false). This is also shown in successive examples in the same page.
I fixed the `onehot(:b, [:a, :b, :c])` and `onehot(:c, [:a, :b, :c])` outputs in the first example of the page accordingly.
Co-authored-by: natema <natema@users.noreply.github.com>
`Flux.OneHotVector` is displayed as a binary vector (0/1) rather than a boolean (true/false) one, as is also shown in successive examples in the same page, so I fixed the `onehot(:b, [:a, :b, :c])` and `onehot(:c, [:a, :b, :c])` output as given by the current Julia version 1.4.2.
1206: Fixing ambiguous remark in Preserve inputs' types r=dhairyagandhi96 a=natema
This PR is based on the [discussion in the forum](https://discourse.julialang.org/t/not-clear-what-0-01f0x-is-in-the-flux-docs/40553?u=mathematics) on the ambiguity of `0.01f0x` in the line
> While one could change the activation function (e.g. to use `0.01f0x`)
Co-authored-by: natema <natema@users.noreply.github.com>
1191: Pull Request Template r=MikeInnes a=MikeInnes
Hopefully makes it a little clearer what the requirements are, which will lead to easier review, and encourage things like NEWS.md that we want to be better in sync.
cc @dhairyagandhi96 and @CarloLucibello for thoughts.
Co-authored-by: Mike J Innes <mike.j.innes@gmail.com>
1190: Correcting advanced.md r=dhairyagandhi96 a=Sleort
To make the example consistent, it should be
```
julia> Flux.trainable(a::Affine) = (a.W,)
```
not
```
julia> Flux.trainable(a::Affine) = (a.W, a.b)
```
Co-authored-by: Troels Arnfred Bojesen <tr-ab@online.no>
To make the example consistent, it should be
```
julia> Flux.trainable(a::Affine) = (a.W,)
```
not
```
julia> Flux.trainable(a::Affine) = (a.W, a.b)
```
1185: Add some news r=dhairyagandhi96 a=dhairyagandhi96
cc @CarloLucibello please add to this list as well
Co-authored-by: Dhairya Gandhi <dhairya@juliacopmuting.com>
957: Add some gradient checking tests on GPUs r=dhairyagandhi96 a=dhairyagandhi96
Good to add generic tests for tracking gradients through the various layers on the GPU.
Co-authored-by: Dhairya Gandhi <dhairya@juliacopmuting.com>
Co-authored-by: Dhairya Gandhi <dhairya@juliacomputing.com>
1165: Fix docstring of logitcrossentropy r=dhairyagandhi96 a=cossio
Since `y` is a logit, there is no log (see the diff).
Co-authored-by: cossio <cossio@users.noreply.github.com>
bors[bot]