profile
viewpoint

Ask questionsFine-Grained Control Over TOCO Quantization

Our (Syntiant Corp’s) neural network inference chips use quantized weights and biases in order to minimize storage and energy consumption. The new Tensorflow experimental quantization feature tf.contrib.quantize.experimental_create_training_graph supports quantizing weights to between 2 and n bits, but the tf.contrib.lite.toco_convert tool currently only supports 8 bit quantization. As a result, we have to internally fork the TFLite pipeline before generating the Flatbuffer.

Feature request: Update TOCO to support arbitrary (i.e., 2 to n bit) signed fixed point quantization of weights and biases for both symmetric and asymmetric quantization. Our desired solution would process the quantization specified at the op or Keras layer level and not involve quantization specification within the TOCO tool API.

tensorflow/tensorflow

Answer questions suharshs

We are working to make things more configurable in new converter iterations, will close this issue since its a bit stale.

useful!

Related questions

ModuleNotFoundError: No module named 'tensorflow.contrib'
Error occurred when finalizing GeneratorDataset iterator
ModuleNotFoundError: No module named 'tensorflow.contrib'
When importing TensorFlow, error loading Hadoop hot 4
The flag 'log_dir' is defined twice. hot 3
[TF 2.0] Dataset has no attribute 'make_one_shot_iterator' hot 3
Lossy conversion from float32 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning. hot 3
TF2.0 AutoGraph issue hot 3
Error loading tensorflow hot 3
AttributeError: module 'tensorflow' has no attribute 'set_random_seed' hot 3
No tf.lite.experimental.nn.bidirectional_dynamic_rnn ops is finded hot 3
Incorrect Error TypeError: padded_batch() missing 1 required positional argument: 'padded_shapes' hot 3
tensorflow2.0 detected 'xla_gpu' , but 'gpu' expected hot 2
Using tensorflow gpu 2.1 with Cuda 10.2 hot 2
Restoring Keras model fails inside a distribution strategy scope hot 2
Github User Rank List