Presentation
HAP: Efficient Quantization Harnessing Adaptive Precision for DNN Hardware Acceleration
DescriptionPost-training quantization (PTQ) avoids expensive training process and it is effective in accelerating Deep Neural Networks (DNNs). Since the representation capacity of a quantizer decreases exponentially with precision, PTQ encounters fundamental limitations in the low-precision domain. As an alternative, adaptive-precision quantization (APQ) leverages the sparsity in data distribution to reduce precision without sacrificing resolution. This is especially suitable for activations, as they tend to be sparser and more vulnerable than weights. However, existing APQ approaches require data to heavily cluster near zeros, which is not guaranteed in asymmetric quantization that is preferred for activations. Furthermore, the varying precision can lead to an unbalanced computational workload, making it difficult to effectively harvest the theoretical performance gain. To solve these problems, we propose a novel quantization and accelerator design, harnessing adaptive-precision (HAP). It leverages a per-group dynamic zero-point to generalize APQ. Moreover, its intra-channel grouping strategy makes it possible to balance variable-precision workload via reordering. Leveraging a channel-level dual-precision weight quantization scheme, it achieves superior accuracy compared with existing PTQ solutions at the level of 4 or 5 bits for a variety of DNN families. On hardware, a novel bit-serial accelerator featuring a lightweight reorder engine is developed. Results show it achieves a 2.65x speedup and a 55% energy reduction on average compared to existing accelerators.
Event Type
Research Manuscript
TimeTuesday, July 2812:06pm - 12:18pm PDT
LocationMtg Room 101A
Similar Presentations