When deep learning models on GPU can be accelerated by taking advantage of unstr

When deep learning models on GPU can be accelerated by taking advantage of unstructured sparsity

This paper is focused on the improvement the efficiency of the sparse convolutional neural networks (CNNs) layers on graphic processing units (GPU). The Nvidia deep neural network (cuDnn) library provides the most effective implementation of deep learning (DL) algorithms for GPUs.GPUs are one of the most efficient and commonly used accelerators for deep learning computations. The modern CNN models need megabytes of coefficients and needed millions MAC operations to perform convolution. One of the most common techniques for compressing CNN models is weight pruning. There are two main types of pruning: structural (based on removing whole weight channels) and non-structural (removing individual weights). The first enables much easier acceleration, but with this type it is difficult to achieve a sparsity level and accuracy as high as that obtained with the second type. Non-structural pruning with retraining can generate a matrix-weight up to $\sim90\%$ or more of sparsity in some deep CNN models. This work shows when is worth using a direct sparse operation to speed-up the calculation of the convolution layers. The VGG-16, CNN-non-static and 1x1 layers from ResNet models were used as a benchmarks. In addition, we present the impact of using reduced precision on time efficiency.

通过利用非结构化稀疏性可以加速GPU上的深度学习模型

本文的重点是提高图形处理单元（GPU）上的稀疏卷积神经网络（CNN）层的效率。Nvidia深度神经网络（cuDnn）库为GPU提供了最有效的深度学习（DL）算法实现。.. GPU是用于深度学习计算的最高效，最常用的加速器之一。现代的CNN模型需要兆字节的系数，并且需要数百万个MAC操作来执行卷积。压缩CNN模型最常用的技术之一是权重修剪。修剪的主要类型有两种：结构性的（基于删除整个权重通道）和非结构性的（删除单个权重）。前者使加速变得容易得多，但是使用这种类型，很难达到与第二种类型相同的稀疏度和准确性。通过再培训进行非结构性修剪可以生成矩阵权重，最高可达 〜90％ 或某些深度CNN模型中的稀疏性。这项工作表明何时应该使用直接稀疏运算来加快卷积层的计算速度。来自ResNet模型的VGG-16，CNN非静态和1x1层用作基准。此外，我们还介绍了降低精度对时间效率的影响。 （阅读更多）