Preview is not available for this file. Please download the file.
Description
Intel® Deep Learning Boost (Intel® DL Boost) uses bfloat16 format (BF16). This document describes the bfloat16 floating-point format. BF16 has several advantages over FP16: It can be seen as a short version of FP32, skipping the least significant 16 bits of mantissa. There is no need to support denormals; FP32, and therefore also BF16, offer more than enough range for deep learning training tasks. FP32 accumulation after the multiply is essential to achieve sufficient numerical behavior on an application level. Hardware exception handling is not needed as this is a performance optimization; industry is designing algorithms around checking inf/NaN.