FastPrepSymmetric INT8 Tensor Quantization

Symmetric INT8 Tensor Quantization

NVIDIA logoNVIDIA● MediumFULLTIMEPHONE SCREEN
Learn

Problem statement

Given a nonempty tensor represented by values, implement symmetric per-tensor INT8 quantization and return its integer codes in input order.

For this exercise, use the following calibration and rounding contract:

  • Let A = max(abs(values[i])). For A > 0, the scale factor is s = A / 127 and the zero-point is 0.
  • For each value x, calculate the magnitude floor(abs(x) / A * 127 + 0.5), restore the sign of x, and clamp the code to [-127,127]. This rounds half-way values away from zero. The code -128 is unused.
  • If all values are zero, use scale 1 and return only zero codes.

The inputs are finite FP32 values supplied through a floating-point array that represents each value exactly. The result is an integer array rather than a byte buffer. After a direct solution, optimize calibration so that it is shared across the tensor.

Function

quantizeInt8(values: double[]) → int[]

Examples

Example 1

values = [-2,-1,0,1,2]return = [-127,-64,0,64,127]

The peak is 2; magnitudes 63.5 round to 64, including negative values.

Example 2

values = [0,0,0]return = [0,0,0]

An all-zero tensor has scale 1 and only zero codes.

Example 3

values = [-3,-1.5,0.75,3]return = [-127,-64,32,127]

The scale is 3/127. Half-step -63.5 rounds away from zero to -64, while 31.75 rounds to 32.

Constraints

  • 1 <= values.length <= 10000.
  • Every input is a finite FP32 value represented exactly by the supplied double.
  • abs(values[i]) <= 10^6.

More NVIDIA problems

See NVIDIA hiring insights
public int[] quantizeInt8(double[] values) {
    // Write your code here
}
values[-2,-1,0,1,2]
expected[-127,-64,0,64,127]
Checking account…