446: Added the SkipConnection layer and constructor r=MikeInnes a=bhvieira
I added a DenseBlock constructor, which allows one to train DenseNets (you can train ResNets and MixNets with this as well, only need change the connection, which is concatenation for DenseNets).
Disclaimer: I created the block for a 3D U-Net, so the assumption here is that whatever layer is inside the block, its output has the same spatial dimension (i.e. all array dimensions excluding the channel and minibatch dimensions) as the input, otherwise the connection wouldn't match. I'm not sure this matches the topology of every DenseNet there is out there, but I suppose this is a good starting point.
No tests yet, will add them as the PR evolve.
I'm open to suggestions! :)
Co-authored-by: Bruno Hebling Vieira <bruno.hebling.vieira@usp.br>
Co-authored-by: Mike J Innes <mike.j.innes@gmail.com>
762: CrossCor layer r=avik-pal a=ayush-1506
Same as #423 (which could be edited since I lost access to that github account).
Co-authored-by: ayush-1506 <ayush.shridhar1506@gmail.com>
Added missing export
Corrected channel placement
Dimension 4 cannot be assumed to always be the Channel dimension
Deprecation of `treelike`
Code now makes use of `@treelike` macro instead of the deprecated `treelike` function (it worked on my end because I'm on Julia 0.7, while Julia 1.0 deprecated stuff)
Update basic.jl
Renaming to SkipConnection
* Update Flux.jl
* Update basic.jl
Updated `SkipConnection` with a `connection` field
I'm pretty sure I broke something now, but this PR should follow along these lines `cat` needs special treatment (the user can declare his own `concatenate` connection, but I foresee it's going to be used often so we can simply define special treatment)
Forgot to remove some rebasing text
Forgot to remove some more rebasing text
Removed local copy and default cat method from the function calls
Adjusted some more types for inference, could improve on this as well
Re-placed some left-over spaces
563: noise shape for dropout r=MikeInnes a=chengchingwen
I add the noise shape for dropout, similar to the `noise_shape` argument in [`tf.nn.dropout`](https://www.tensorflow.org/api_docs/python/tf/nn/dropout)
Co-authored-by: chengchingwen <adgjl5645@hotmail.com>
Co-authored-by: Peter <adgjl5645@hotmail.com>
756: Change `DepthwiseConv()` to use `in=>out` instead of `in=>mult`. r=MikeInnes a=staticfloat
This is an API change, but I think it makes more sense, and is more consistent with our `Conv()` API. This also dumps the `DepthwiseConv((3,3), C_in)` API, as I'm not sure why you would want to specify only the input channel count and default the output to a channel multiplier of 1; if anything I would think you'd want to specify the channel output and leave the input to be default. In any case, I think consistency with `Conv()` is the best thing to chase after here.
Co-authored-by: Elliot Saba <staticfloat@gmail.com>
Viral B. Shah