Added news and removed type annotation from SkipConnection structure
This commit is contained in:
parent
c5fc2fb9a3
commit
796a2957c9
1
NEWS.md
1
NEWS.md
|
@ -1,5 +1,6 @@
|
|||
# v0.9.0
|
||||
* [Depthwise convolutional layer API changes](https://github.com/FluxML/Flux.jl/pull/756) from `in => mult` channel specification to `in => out` channel specification, and deprecates implicit `out` constructor.
|
||||
* New [SkipConnection](https://github.com/FluxML/Flux.jl/pull/446), which can be used to train residual neural network architectures.
|
||||
|
||||
# v0.8.0
|
||||
|
||||
|
|
|
@ -190,26 +190,29 @@ function (mo::Maxout)(input::AbstractArray)
|
|||
mapreduce(f -> f(input), (acc, out) -> max.(acc, out), mo.over)
|
||||
end
|
||||
|
||||
|
||||
"""
|
||||
SkipConnection(layers...)
|
||||
|
||||
|
||||
Creates a Skip Connection, which constitutes of a layer or Chain of consecutive layers
|
||||
and a shortcut connection linking the input to the block to the
|
||||
output through a user-supplied function.
|
||||
output through a user-supplied callable.
|
||||
|
||||
`SkipConnection` requires the output dimension to be the same as the input.
|
||||
|
||||
A 'ResNet'-type skip-connection with identity shortcut would simply be
|
||||
```julia
|
||||
SkipConnection(layer, (a,b) -> a + b)
|
||||
```
|
||||
"""
|
||||
|
||||
struct SkipConnection
|
||||
layers
|
||||
connection::Function #user can pass arbitrary connections here, such as `.+`
|
||||
connection #user can pass arbitrary connections here, such as (a,b) -> a + b
|
||||
end
|
||||
|
||||
@treelike SkipConnection
|
||||
|
||||
function (skip::SkipConnection)(input::AbstractArray)
|
||||
function (skip::SkipConnection)(input)
|
||||
#We apply the layers to the input and return the result of the application of the layers and the original input
|
||||
skip.connection(skip.layers(input), input)
|
||||
end
|
||||
|
|
Loading…
Reference in New Issue