sparsemax
- 記事がでかくなりすぎなので分割
- 数式の解説をMathWillsに記載した
sparsemaxとは
In words, sparsemax returns the Euclidean projection of the input vector z onto the probability simplex.
This projection is likely to hit the boundary of the simplex, in which case sparsemax(z) becomes sparse.
We will see that sparsemax retains most of the important properties of softmax, having in addition the ability of producing sparse distributions.