CarloLucibello
13b934c250
improve optimizers
2017-11-24 12:12:20 +01:00
Mike J Innes
dc1f08a709
Merge pull request #98 from FluxML/log
...
GPU-ready log function
2017-11-23 17:17:39 +00:00
Mike J Innes
9f5c4dd3e9
Merge pull request #104 from baggepinnen/patch-1
...
Allow array of optimisers to train!
2017-11-21 17:16:35 +01:00
Mike J Innes
feb35783e6
Merge pull request #95 from FluxML/layernorm
...
Layer Normalisation
2017-11-21 17:12:49 +01:00
Mike J Innes
351d3d4771
std derivative
2017-11-21 17:04:04 +01:00
Mike J Innes
b06884b912
LayerNorm tweaks
2017-11-21 16:32:36 +01:00
skariel
11d53781b2
adding layer normalization
2017-11-21 16:30:24 +01:00
Mike J Innes
979949d01a
style
2017-11-21 15:25:09 +01:00
Mike J Innes
785fbcf68e
Merge pull request #107 from baggepinnen/patch-2
...
Fix bug in rmsprop and adadelta
2017-11-21 15:24:11 +01:00
Mike J Innes
e51268caf5
mention treelike
2017-11-21 12:59:39 +01:00
Mike J Innes
187fddc11c
doc fixes
2017-11-21 12:29:02 +01:00
Fredrik Bagge Carlson
8991ce028c
Fix bug in rmsprop and adadelta
...
`@. p.Δ = η * p.Δ / √acc` parses correctly while `@. p.Δ /= √acc*η` seems to parse like `@. p.Δ /= (√acc*η)`, hence the step size was de facto interpreted as `1/η`
2017-11-14 17:32:16 +01:00
Mike J Innes
e0657d93ec
mv numeric.jl to nnlib
2017-11-09 15:06:29 +00:00
Mike J Innes
2cb94981a0
gpu-ready log
2017-11-09 15:04:01 +00:00
Mike J Innes
e5d99d784e
fixes #79
2017-11-09 14:53:26 +00:00
Mike J Innes
bdf02e42ae
test tweaks
2017-11-08 22:18:45 +00:00
Mike J Innes
fcd091e8f0
Ac_mul_B derivatives
2017-11-08 22:18:45 +00:00
Mike J Innes
d4229c4815
useful params method
2017-11-08 22:18:45 +00:00
Mike J Innes
d6423eefe5
matrix-vector fast path
2017-11-08 22:18:45 +00:00
Fredrik Bagge Carlson
97244e0a68
Allow array of optimisers to train!
...
This allows an array of optimisers to be sent to `train!`
2017-11-04 13:27:32 +01:00
Mike J Innes
efa51f02e7
basic batch type
2017-11-02 11:49:42 +00:00
Mike J Innes
21ea93ffcd
rename treelike
2017-11-02 11:47:34 +00:00
Mike J Innes
e7a510da9a
add cmudict dataset
2017-11-01 16:01:55 +00:00
Mike J Innes
0f8ba87dc6
treelike tuples
2017-10-31 16:37:41 +00:00
Mike J Innes
e943a39ee7
combine special cases
2017-10-31 16:37:33 +00:00
Iblis Lin
3d8b7250ae
add scalar mean
2017-10-31 10:42:32 +00:00
Mike J Innes
1186170542
Merge pull request #99 from iblis17/mean
...
TrackedArray: implement `mean`
2017-10-30 16:37:08 +00:00
Iblis Lin
c43bda019b
TrackedArray: implement mean
...
```julia
julia> p
Tracked 2×3 Array{Float64,2}:
1.0 3.0 5.0
2.0 4.0 6.0
```
Before
```julia
julia> @benchmark Flux.Tracker.back!(sum($p, 2) ./ size($p, 2), ones(2, 1))
BenchmarkTools.Trial:
memory estimate: 3.44 KiB
allocs estimate: 75
--------------
minimum time: 20.438 μs (0.00% GC)
median time: 21.239 μs (0.00% GC)
mean time: 22.354 μs (1.68% GC)
maximum time: 3.811 ms (98.51% GC)
--------------
samples: 10000
evals/sample: 1
```
After
```julia
julia> @benchmark Flux.Tracker.back!(mean($p, 2), ones(2, 1))
BenchmarkTools.Trial:
memory estimate: 1008 bytes
allocs estimate: 21
--------------
minimum time: 5.973 μs (0.00% GC)
median time: 6.310 μs (0.00% GC)
mean time: 6.630 μs (1.96% GC)
maximum time: 680.709 μs (97.28% GC)
--------------
samples: 10000
evals/sample: 6
```
2017-10-30 16:21:02 +08:00
Mike J Innes
4c1b1eb18c
Merge pull request #92 from CarloLucibello/drop
...
add Dropout layer
2017-10-26 12:07:28 +01:00
Mike J Innes
84efbbcc84
tracker predicate tweaks
2017-10-26 12:06:29 +01:00
Mike J Innes
cf6b930f63
reorganise
2017-10-26 11:46:12 +01:00
Mike J Innes
0df300299f
clearer error message, fixes #93
2017-10-26 11:15:14 +01:00
CarloLucibello
711ea09d99
address comments
2017-10-25 02:35:27 +02:00
CarloLucibello
536ab3861d
setmode! -> testmode!
2017-10-23 16:23:29 +02:00
Mike J Innes
6768afda29
minor wording
2017-10-23 13:07:07 +01:00
CarloLucibello
00a9e5f01f
construct TrackedScalar with params(1)
2017-10-23 10:49:45 +01:00
CarloLucibello
86c7c9246e
add == and < for tracked arrays
2017-10-23 11:41:08 +02:00
CarloLucibello
2e1ed4c3fc
add dropout
2017-10-23 10:12:53 +02:00
Mike J Innes
2a66545ef8
rnn state reset
2017-10-19 17:21:08 +01:00
Mike J Innes
99a7697d13
adam eta default arg
2017-10-19 14:31:34 +01:00
Mike J Innes
e5c8f6d835
only export known good optimisers
2017-10-19 11:26:11 +01:00
Mike J Innes
5b6a5667ed
tracked array restructure
2017-10-18 22:54:58 +01:00
Mike J Innes
c8d4844da4
chunk util
2017-10-18 17:07:58 +01:00
Mike J Innes
07ad7cfa40
learning rate as default arg
2017-10-18 17:07:49 +01:00
Mike J Innes
e82428bb83
batching docs
2017-10-18 16:40:14 +01:00
Mike J Innes
b817ce632c
syntax highlighting
2017-10-18 15:44:06 +01:00
Mike J Innes
fd249b773e
rnn docs
2017-10-18 15:30:05 +01:00
Mike J Innes
897f812055
remove this until nnlib release
2017-10-18 14:52:48 +01:00
Mike J Innes
190f48a709
nnlib docs
2017-10-18 14:40:58 +01:00
Mike J Innes
12944ae125
nnlib exports
2017-10-18 12:56:58 +01:00
