Added news and removed type annotation from SkipConnection structure
This commit is contained in:
parent
c5fc2fb9a3
commit
796a2957c9
1
NEWS.md
1
NEWS.md
@ -1,5 +1,6 @@
|
|||||||
# v0.9.0
|
# v0.9.0
|
||||||
* [Depthwise convolutional layer API changes](https://github.com/FluxML/Flux.jl/pull/756) from `in => mult` channel specification to `in => out` channel specification, and deprecates implicit `out` constructor.
|
* [Depthwise convolutional layer API changes](https://github.com/FluxML/Flux.jl/pull/756) from `in => mult` channel specification to `in => out` channel specification, and deprecates implicit `out` constructor.
|
||||||
|
* New [SkipConnection](https://github.com/FluxML/Flux.jl/pull/446), which can be used to train residual neural network architectures.
|
||||||
|
|
||||||
# v0.8.0
|
# v0.8.0
|
||||||
|
|
||||||
|
@ -190,26 +190,29 @@ function (mo::Maxout)(input::AbstractArray)
|
|||||||
mapreduce(f -> f(input), (acc, out) -> max.(acc, out), mo.over)
|
mapreduce(f -> f(input), (acc, out) -> max.(acc, out), mo.over)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
||||||
"""
|
"""
|
||||||
SkipConnection(layers...)
|
SkipConnection(layers...)
|
||||||
|
|
||||||
|
|
||||||
Creates a Skip Connection, which constitutes of a layer or Chain of consecutive layers
|
Creates a Skip Connection, which constitutes of a layer or Chain of consecutive layers
|
||||||
and a shortcut connection linking the input to the block to the
|
and a shortcut connection linking the input to the block to the
|
||||||
output through a user-supplied function.
|
output through a user-supplied callable.
|
||||||
|
|
||||||
`SkipConnection` requires the output dimension to be the same as the input.
|
`SkipConnection` requires the output dimension to be the same as the input.
|
||||||
|
|
||||||
|
A 'ResNet'-type skip-connection with identity shortcut would simply be
|
||||||
|
```julia
|
||||||
|
SkipConnection(layer, (a,b) -> a + b)
|
||||||
|
```
|
||||||
"""
|
"""
|
||||||
|
|
||||||
struct SkipConnection
|
struct SkipConnection
|
||||||
layers
|
layers
|
||||||
connection::Function #user can pass arbitrary connections here, such as `.+`
|
connection #user can pass arbitrary connections here, such as (a,b) -> a + b
|
||||||
end
|
end
|
||||||
|
|
||||||
@treelike SkipConnection
|
@treelike SkipConnection
|
||||||
|
|
||||||
function (skip::SkipConnection)(input::AbstractArray)
|
function (skip::SkipConnection)(input)
|
||||||
#We apply the layers to the input and return the result of the application of the layers and the original input
|
#We apply the layers to the input and return the result of the application of the layers and the original input
|
||||||
skip.connection(skip.layers(input), input)
|
skip.connection(skip.layers(input), input)
|
||||||
end
|
end
|
||||||
|
Loading…
Reference in New Issue
Block a user