block floating
182294915
High Performance FPGA Based CNN Accelerator With Block Floating Point Arithmetic
Convolutional neural networks (CNNs) are widely used and have achieved great success in computer vision and speech processing applications. However, deploying the large scale CNN model in the embedded system is subject to the constraints of computation and memory. An optimized block floating point (BFP) arithmetic is adopted in our accelerator for efficient inference of deep neural networks in this paper. The feature maps and model parameters are represented in 16 bit and 8 bit formats, respectively, in the off chip memory, which can reduce memory and off chip bandwidth requirements by 50% and 75% compared to the 32 bit FP counterpart. The proposed 8 bit BFP arithmetic with optimized rounding and shifting operation based quantization schemes improves the energy and hardware efficiency by three times. One CNN model can be deployed in our accelerator without retraining at the cost of an accuracy loss of not more than 0.12% The proposed reconfigurable accelerator with three parallelism dimensions, ping pong off chip DDR3 memory access, and an optimized on chip buffer group is implemented on the Xilinx VC709 evaluation board. Our accelerator achieves a performance of 760.83 GOP/s and 82.88 GOP/s/W under a 200 MHz working frequency, significantly outperforming previous accelerators.
IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Xiaocong Lian Zhenyu Liu Zhourui Song Jiwu Dai Wei Zhou Xiangyang Ji
4924362
Computation Error Analysis of Block Floating Point Arithmetic Oriented Convolution Neural Network Accelerator Design
The heavy burdens of computation and off chip traffic impede deploying the large scale convolution neural network on embedded platforms. As CNN is attributed to the strong endurance to computation errors, employing block floating point (BFP) arithmetics in CNN accelerators could save the hardware cost and data traffics efficiently, while maintaining the classification accuracy. In this paper, we verify the effects of word width definitions in BFP to the CNN performance without retraining. Several typical CNN models, including VGG16, ResNet 18, ResNet 50 and GoogLeNet, were tested in this paper. Experiments revealed that 8 bit mantissa, including sign bit, in BFP representation merely induced less than 0.3% accuracy loss. In addition, we investigate the computational errors in theory and develop the noise to signal ratio (NSR) upper bound, which provides the promising guidance for BFP based CNN engine design.
AAAI
Zhourui Song Zhenyu Liu Chunlu Wang Dongsheng Wang
53979157
Training DNNs with Hybrid Block Floating Point
The wide adoption of DNNs has given birth to unrelenting computing requirements, forcing datacenter operators to adopt domain specific accelerators to train them. These accelerators typically employ densely packed full precision floating point arithmetic to maximize performance per area. Ongoing research efforts seek to further increase that performance density by replacing floating point with fixed point arithmetic. However, a significant roadblock for these attempts has been fixed point's narrow dynamic range, which is insufficient for DNN training convergence. We identify block floating point (BFP) as a promising alternative representation since it exhibits wide dynamic range and enables the majority of DNN operations to be performed with fixed point logic. Unfortunately, BFP alone introduces several limitations that preclude its direct applicability. In this work, we introduce HBFP, a hybrid BFP FP approach, which performs all dot products in BFP and other operations in floating point. HBFP delivers the best of both worlds: the high accuracy of floating point at the superior hardware density of fixed point. For a wide variety of models, we show that HBFP matches floating point's accuracy while enabling hardware implementations that deliver up to 8.5 x higher throughput.
NeurIPS
Mario Paulo Drumond Tao Lin Martin Jaggi Babak Falsafi
52519942
Reconfigurable Acceleration of 3D CNNs for Human Action Recognition with Block Floating Point Representation
Human action recognition (HAR) has been widely employed in various applications such as autonomous cars and intelligent video surveillance. Among the algorithms proposed for HAR, the 3D CNNs algorithm can achieve the best accuracy. However, its algorithmic complexity imposes a substantial overhead over the speed of these networks, which limits their deployment in real life applications. This paper proposes a novel customizable architecture for 3D CNNs based on block floating point (BFP) arithmetic, where the utilization of BFP significantly reduces the bitwidth while eliminating the need to retrain the network. Optimizations such as locality exploration and block alignment with 3D blocking are performed to improve performance and accuracy. An analytical model and tool are developed to predict the optimized parameters for hardware customization based on user constraints such as FPGA resources or accuracy requirement. Experiments show that without retraining, a 15 bit mantissa design using single precision accumulation on a Xilinx ZC706 device can be 8.2 times faster than an Intel i7 950 processor at 3.07 GHz with only 0.4% accuracy loss.
2018 28th International Conference on Field Programmable Logic and Applications (FPL)
Hongxiang Fan Ho-Cheung Ng Shuanglong Liu Zhiqiang Que Xinyu Niu Wayne Luk
4732325
End to End DNN Training with Block Floating Point Arithmetic
DNNs are ubiquitous datacenter workloads, requiring orders of magnitude more computing power from servers than traditional workloads. As such, datacenter operators are forced to adopt domain specific accelerators that employ half precision floating point (FP) numeric representations to improve arithmetic density. Unfortunately, even these representations are not dense enough, and are, therefore, sub optimal for DNNs. We propose a hybrid approach that employs dense block floating point (BFP) arithmetic on dot product computations and FP arithmetic elsewhere. While using BFP improves the performance of dot product operations, that compose most of DNN computations, allowing values to freely float between dot product operations leads to a better choice of tensor exponents when converting values to back BFP. We show that models trained with hybrid BFP FP arithmetic either match or outperform their FP32 counterparts, leading to more compact models and denser arithmetic in computing platforms.
ArXiv
Mario Paulo Drumond Tao Lin Martin Jaggi Babak Falsafi
9981535
A block floating point treatment to finite precision realization of the adaptive decision feedback equalizer
A scheme for efficient realization of the adaptive decision feedback equalizer (ADFE) is presented using the block floating point (BFP) data format which enables the ADFE to process rapidly varying data over a wide dynamic range, at a fixed point like complexity. The proposed scheme adopts appropriate BFP format for the data as well as the filter weights and works out separate update relations for the filter weight mantissas and exponents. Overflows at the feed forward and the feedback filter output are prevented by certain dynamic scaling of the respective input, while overflow in weight update calculations is avoided by imposing certain upper bound on the algorithm step size u which is shown to be less than the convergence bound. The proposed scheme deploys mostly simple fixed point operations and is shown to achieve considerable computational gain over its floating point based counterpart. HighlightsA? Presents a block floating point (BFP) realization of adaptive decision feedback equalizer. A? Suitable BFP format chosen to represent feedforward and feedback coefficient vectors. A? Separate update relations for the coefficient mantissa vectors and exponent derived. A? Overflow controlled by certain dynamic scaling of data and new upper bound on step size. A? Deploys mostly fixed point operations, more efficient than a FP realization
Signal Process.
Rafi Ahamed Shaik Mrityunjoy Chakraborty
15387076
Reconfigurable Block Floating Point Processing Elements in Virtex Platforms
This paper describes a project undertaken to simplify the implementation of high throughput, low power, numerically intensive applications on Virtex platforms. The system is a pipeline composed of block floating point processing elements. These combine the advantages of fixed point and floating point implementations: improved data accuracy (compared to fixed point) while keeping the hardware resources to a minimum. The design is based on the DSP48E, a digital signal processing element or slice provided on certain Xilinx Virtex FPGAs. This implementation approach yields high throughput (550 MHz) and low power consumption. The key to this is implementing high performance shifting and rounding modules in Virtex 5 platforms. The implementation can be easily ported to Virtex 6 and 7 Series FPGA families.
2011 International Conference on Reconfigurable Computing and FPGAs
Guillermo Conde Gregory W Donohoe
10112899
Block Floating Point Interval ALU for Digital Signal Processing
Numerical analysis on real numbers is performed using either pointwise arithmetic or interval arithmetic. Today, interval analysis is a mature discipline and finds use not only in error analysis but also control applications such as robotics. The interval arithmetic hardware architecture proposed in the work [W. Edmonson, R. Gupte, J. Gianchandani, S. Ocloo, W. Alexander, Interval arithmetic logic unit for signal processing and control applications, Workshop on Reliable Engineering Computing, Savannah, GA, USA, 2006] for Digital Signal Processors (DSP) is prone to unreliability owing to overflow errors resulting from the small dynamic range offered by fixed point computations. In this paper, we present a solution to this problem in the form of a fixed point interval ALU that utilizes the concept of Block Floating Point (BFP) presented in [K. Kalliojarvi, J. Astola, [Roundoff errors in block floating point systems, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 44, pp. 783 790, 1996] to attain a higher dynamic range for interval as well as point wise computations. We add block floating point support to the ALU in the form of special instructions such as Exponent Detection and Normalization that aid the change of the dynamic range of the input. We also provide additional hardware to perform Conditional Block Floating point Scaling (CBFS) on the output endpoints of the interval result. We explore the design space across varying pipeline depths for best throughput which is characterized as the highest number of output samples processed per second. Our results show that the four stage highly pipelined architecture provides the highest throughput of about 86.1 Msamples/sec for interval operations.
Reliab. Comput.
Sandeep Hattangady William W Edmonson Winser E Alexander
126175927
Block floating point compression of signal data
A method and apparatus for compressing signal samples uses block floating point representations where the number of bits per mantissa is determined by the maximum magnitude sample in the group. The compressor defines groups of signal samples having a fixed number of samples per group. The maximum magnitude sample in the group determines an exponent value corresponding to the number of bits for representing the maximum sample value. The exponent values are encoded to form exponent tokens. Exponent differences between consecutive exponent values may be encoded individually or jointly. The samples in the group are mapped to corresponding mantissas, each mantissa having a number of bits based on the exponent value. Removing LSBs depending on the exponent value produces mantissas having fewer bits. Feedback control monitors the compressed bit rate and/or a quality metric. This abstract does not limit the scope of the invention as described in the claims.
2535685
Potential of using block floating point arithmetic in ASIP based GNSS receivers
As an alternative to floating point and fixed point arithmetics, block floating point arithmetic offers a compromise between computational accuracy and hardware complexity. Nevertheless, today practical applications are limited to Fast Fourier Transformation and digital filters. This work uses the block floating point format to realize the position estimation algorithm in Global Navigation Satellite System (GNSS) receivers, which is based on the Newton Rhapson method. The precision of this novel approach is quantified by extensive simulations using synthetic as well as real GNSS data. The implementation of the position estimation algorithm using block floating point format on an application specific processor is introduced and compared to implementations on a standard embedded processor and in standard floating point arithmetic in terms of performance and costs.
ASAP 2010 21st IEEE International Conference on Application specific Systems, Architectures and Processors
Emrah Tasdemir Gotz Kappen Tobias G Noll