profile
viewpoint

Ask questions[tflite][quantization][deeplabv3] Constant array MobilenetV2/expanded_conv_7/depthwise/depthwise_weights lacks MinMax information

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow):no
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Ubuntu 16.04
  • TensorFlow installed from (source or binary):Source
  • TensorFlow version (use command below):1.9.0
  • Python version:2.7.12
  • Bazel version (if compiling from source):0.12.0
  • GCC/Compiler version (if compiling from source):5.4.0
  • CUDA/cuDNN version:cuda-9.0/7.0
  • GPU model and memory:GeForce GTX 1080/8105MiB
  • Phone:xiaomi5 (Snapdragon 820)
  • Exact command to reproduce: bazel run --config=opt //tensorflow/contrib/lite/toco:toco -- --input_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb --output_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/kanul.tflite --inference_type=QUANTIZED_UINT8 --input_shape=1,513,513,3 --input_array=sub_7 --output_array=ResizeBilinear_3

Describe the problem I have tried to quantize MobileNetV2 for deeplabV3+ with TFlite. But I fail to convert the model. From the following issue, I saw that the operations were not supported for the option of quantization.

https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/model_zoo.md Checkpoint name: mobilenetv2_coco_voc_trainaug

As we can see graphs from the tensorboard, there is one big problem.

In "import/MobilenetV2/expanded_conv_7/depthwise/depthwise",

the operation of depthwise consists of the subgraph with 3 nodes: (depthwise) and BatchToSpaceND, SpaceToBatchND.

But, in "import/MobilenetV2/expanded_conv_6/depthwise/depthwise",

the operation of depthwise is DepthwiseConv2dNative itself.

From the difference, we can not quantize deeplabv3 based on mobilenetv2.

The one thing is that MobilenetV2/expanded_conv_7~16 does not have min/max value to be needed for quantization with tflite.

Although I implement the needed min/max value in hardcode_min_max.cc,

This model does not run well in mobile environments.

The ultimate problem is caused by the fact that depthwise_conv_7~16 consist of 3 nodes including BatchToSpaceND and SpaceToBatchND.

I request you to notify the method to resolve above issues.

Source code / logs

bazel run --config=opt //tensorflow/contrib/lite/toco:toco -- --input_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/frozen_inference_graph.pb --output_file=/external_home/data/model/deeplabv3_mnv2_pascal_train_aug/kanul.tflite --inference_type=QUANTIZED_UINT8 --input_shape=1,513,513,3 --input_array=sub_7 --output_array=ResizeBilinear_3

2018-07-11 04:40:01.330069: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before pre-quantization graph transformations: 166 operators, 340 arrays (1 quantized) 2018-07-11 04:40:01.330711: W tensorflow/contrib/lite/toco/graph_transformations/hardcode_min_max.cc:339] Tweaking the MinMax of array ResizeBilinear_1, which is an input to {Concatenation operator with output concat}, because we want all inputs and outputs of a Concatenation operator to have the same MinMax so that it can be implemented as a pure byte-copy, no arithmetic. 2018-07-11 04:40:01.332983: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After pre-quantization graph transformations pass 1: 111 operators, 285 arrays (1 quantized) 2018-07-11 04:40:01.335731: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before quantization graph transformations: 111 operators, 285 arrays (1 quantized) 2018-07-11 04:40:01.337575: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_7/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.337670: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_7/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.337695: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_7/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.338553: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_8/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.338711: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_8/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.338786: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_8/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.339777: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_9/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.339918: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_9/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.339985: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_9/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.340933: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_10/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.341034: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_10/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.341059: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_10/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.342497: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_11/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.342593: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_11/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.342620: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_11/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.344311: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_12/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.344422: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_12/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.344452: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_12/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.345978: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_13/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.346094: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_13/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.346122: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_13/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.349163: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_14/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.349318: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_14/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.349351: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_14/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.353356: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_15/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.353511: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_15/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.353545: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_15/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.357264: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_16/depthwise/depthwise_weights lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.357400: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_16/depthwise/BatchNorm/FusedBatchNorm_mul_0_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy. 2018-07-11 04:40:01.357432: W tensorflow/contrib/lite/toco/graph_transformations/quantize.cc:92] Constant array MobilenetV2/expanded_conv_16/depthwise/BatchNorm/FusedBatchNorm_add_param lacks MinMax information. To make up for that, we will now compute the MinMax from actual array elements. That will result in quantization parameters that probably do not match whichever arithmetic was used during training, and thus will probably be a cause of poor inference accuracy.

tensorflow/models

Answer questions suharshs

This should be resolved in the newest version of TF AFAIK.

useful!

Related questions

ERROR:root:Internal Python error in the inspect module. hot 6
AttributeError: module 'tensorflow' has no attribute 'contrib' hot 3
Train.py in object_detection crash. AttributeError: module 'tensorflow.contrib.slim.python.slim.data.tfexample_decoder' has no attribute 'BackupHandler' hot 3
ImportError: cannot import name 'string_int_label_map_pb2' hot 3
Cant convert from .pb to .tflite hot 2
Any plans to release EfficientNet based SSD for object detection ? [Question] hot 2
ResourceExhaustedError during retraining in Object detection API hot 2
batch_norm_trainable field in ssd mobilenet v2 coco hot 2
tensorflow.python.framework.errors_impl.NotFoundError: NewRandomAccessFile failed to Create/Open: data/Obj_det.pbtxt : The system cannot find the file specified. hot 2
ImportError: No module named nets hot 2
ValueError: Unknown meta architecture: None hot 2
Object Detection Train Error - ImportError: cannot import name 'dataset_builder' hot 2
from object_detection.protos import input_reader_pb2 ImportError: cannot import name 'input_reader_pb2' hot 2
Quantized SSD-MobileNet Checkpoints Missing Min/Max? hot 1
[Feature request] Example of TensorFlow Lite C++ and MobileNet SSD for Object Detection hot 1
Github User Rank List