4 lines
72 KiB
JavaScript
4 lines
72 KiB
JavaScript
|
var documenterSearchIndex = {"docs":
|
|||
|
|
[{"location":"data/onehot/#One-Hot-Encoding-1","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"","category":"section"},{"location":"data/onehot/#","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"It's common to encode categorical variables (like true, false or cat, dog) in \"one-of-k\" or \"one-hot\" form. Flux provides the onehot function to make this easy.","category":"page"},{"location":"data/onehot/#","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"julia> using Flux: onehot, onecold\n\njulia> onehot(:b, [:a, :b, :c])\n3-element Flux.OneHotVector:\n false\n true\n false\n\njulia> onehot(:c, [:a, :b, :c])\n3-element Flux.OneHotVector:\n false\n false\n true","category":"page"},{"location":"data/onehot/#","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"The inverse is onecold (which can take a general probability distribution, as well as just booleans).","category":"page"},{"location":"data/onehot/#","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"julia> onecold(ans, [:a, :b, :c])\n:c\n\njulia> onecold([true, false, false], [:a, :b, :c])\n:a\n\njulia> onecold([0.3, 0.2, 0.5], [:a, :b, :c])\n:c","category":"page"},{"location":"data/onehot/#Batches-1","page":"One-Hot Encoding","title":"Batches","text":"","category":"section"},{"location":"data/onehot/#","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"onehotbatch creates a batch (matrix) of one-hot vectors, and onecold treats matrices as batches.","category":"page"},{"location":"data/onehot/#","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"julia> using Flux: onehotbatch\n\njulia> onehotbatch([:b, :a, :b], [:a, :b, :c])\n3×3 Flux.OneHotMatrix:\n false true false\n true false true\n false false false\n\njulia> onecold(ans, [:a, :b, :c])\n3-element Array{Symbol,1}:\n :b\n :a\n :b","category":"page"},{"location":"data/onehot/#","page":"One-Hot Encoding","title":"One-Hot Encoding","text":"Note that these operations returned OneHotVector and OneHotMatrix rather than Arrays. OneHotVectors behave like normal vectors but avoid any unnecessary cost compared to using an integer index directly. For example, multiplying a matrix with a one-hot vector simply slices out the relevant row of the matrix under the hood.","category":"page"},{"location":"models/regularisation/#Regularisation-1","page":"Regularisation","title":"Regularisation","text":"","category":"section"},{"location":"models/regularisation/#","page":"Regularisation","title":"Regularisation","text":"Applying regularisation to model parameters is straightforward. We just need to apply an appropriate regulariser, such as norm, to each model parameter and add the result to the overall loss.","category":"page"},{"location":"models/regularisation/#","page":"Regularisation","title":"Regularisation","text":"For example, say we have a simple regression.","category":"page"},{"location":"models/regularisation/#","page":"Regularisation","title":"Regularisation","text":"using Flux: crossentropy\nm = Dense(10, 5)\nloss(x, y) = crossentropy(softmax(m(x)), y)","category":"page"},{"location":"models/regularisation/#","page":"Regularisation","title":"Regularisation","text":"We can regularise this by taking the (L2) norm of the parameters, m.W and m.b.","category":"page"},{"location":"models/regularisation/#","page":"Regularisation","title":"Regularisation","text":"using LinearAlgebra\n\npenalty() = norm(m.W) + norm(m.b)\nloss(x, y) = crossentropy(softmax(m(x)), y) + penalty()","category":"page"},{"location":"models/regularisation/#","page":"Regularisation","title":"Regularisation","text":"When working with layers, Flux provides the params function to grab all parameters at once. We can easily penalise everything with sum(norm, params).","category":"page"},{"location":"models/regularisation/#","page":"Regularisation","title":"Regularisation","text":"julia> params(m)\n2-element Array{Any,1}:\n param([0.355408 0.533092; … 0.430459 0.171498])\n param([0.0, 0.0, 0.0, 0.0, 0.0])\n\njulia> sum(norm, params(m))\n26.01749952921026 (tracked)","ca
|
|||
|
|
}
|