Hardware Accelerators for Neural Processing

Abstract

There has been a great change in the computing environment after the introduction of deep learning systems in every day applications. The requirements of these systems are so vastly different from the conventional systems that a complete revision of the processor design strategies is necessary. Processors capable of streamed SIMD, MIMD, Matrix and systolic arrays do offer some solutions. As many new neural structures will be introduced over next years, new processor architectures need to evolve. In spite of the variability of Artificial Neural Network (ANN) structures, some feature will be common among them. We have tried to implement the hardware components required for most of the ANNs. This paper highlights some of the key issues related to hardware implementation of neural networks and suggests some possible solutions. However, the arena remains very open for innovation. Low precision arithmetic and approximation techniques suitable for acceleration of computational load of neural networks have been implemented and their results have been presented. We also show that for a given ratio of area occupied by serial multiplier to that of a parallel multiplier, a threshold exists beyond which the serial multipliers have a distinct performance advantage over parallel multipliers. Need for multi-operand operations and methods to implement them have been discussed.