TastyPancakes
/
Flux.jl
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge #1129 

1129: Added dropgrad in huber_loss r=CarloLucibello a=HenriDeh

Workaround to prevent `iterate(::nothing)` when working with CuArrays. See issue #1128

Co-authored-by: HenriDeh <47037088+HenriDeh@users.noreply.github.com>
This commit is contained in:
bors[bot] 2020-06-06 17:21:19 +00:00 committed by GitHub
parent 9ebbe8cb4c ac94754281
commit d9b07475b0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/layers/stateless.jl
View File

@ -46,9 +46,10 @@ given the prediction `ŷ` and true values `y`.
Huber loss = | Huber loss = |
| δ * (|ŷ - y| - 0.5 * δ), otherwise | δ * (|ŷ - y| - 0.5 * δ), otherwise
""" """
#TODO: remove dropgrad when Zygote can handle this function with CuArrays
function huber_loss(ŷ, y; δ=eltype(ŷ)(1)) function huber_loss(ŷ, y; δ=eltype(ŷ)(1))
abs_error = abs.(ŷ .- y) abs_error = abs.(ŷ .- y)
temp = abs_error .< δ temp = Zygote.dropgrad(abs_error .< δ)
x = eltype(ŷ)(0.5) x = eltype(ŷ)(0.5)
hub_loss = sum(((abs_error.^2) .* temp) .* x .+ δ*(abs_error .- x*δ) .* (1 .- temp)) * 1 // length(y) hub_loss = sum(((abs_error.^2) .* temp) .* x .+ δ*(abs_error .- x*δ) .* (1 .- temp)) * 1 // length(y)
end end