Document Details

Clip: arXiv:1805.02867v2 [cs.PF] 28 Jul 2018 Online normalizer calculation for softmax Maxim Milakov NVIDIA mmilakov@nvidia.com Natalia Gimelshein NVIDIA ngimelshein@nvidia.com Abstract The Softmax function is ubiquitous in machine learning, multiple previous works suggested faster alternatives for it. In this paper we propose a way to compute classical Softmax with fewer memory accesses and hypothesize that this reduction in memory accesses should improve Softmax performance on actual hardware. The benchmarks confirm this hypothesis: Softmax accelerates by up to1.3x and Softmax+TopK combined and fused by up to5x.

Filename: 1805.02867v2.pdf

Notes: