profile
viewpoint

Ask questionsFL16 model run on GPU

System information

  • Host OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Linux Ubuntu 16.04
  • TensorFlow installed from (source or binary): binary
  • Tensorflow version (commit SHA if source): 1.15
  • Target platform (e.g. Arm Mbed OS, Arduino Nano 33 etc.):Android 9 api28 ,Mali-T864 GPU

Describe the problem We tried to run a post-quantized (to float16) model on a robot with GPU delegate according to https://www.tensorflow.org/lite/performance/gpu and https://medium.com/tensorflow/tensorflow-model-optimization-toolkit-float16-quantization-halves-model-size-cc113c75a2fa but it fails to run on GPU even after we graph transformed non-GPU supported operators in it. The logs is attached. Interesting thing is if we do not quantize it to fl16, all operators of the model can successfully run on GPU. Netron shows there are lots of 'dequantize' operators added to the graph after we use tflite converter to quantize the model to fl16. So what should we do to let the quantized fl16 model run on GPU entirely?

One more question is we found a parameter SetAllowFp16PrecisionForFp32 in tflite c++. What is the difference between 1).set this to true and use a fl32 model. 2). set this to true and use fl16 model. 3). set this to false and use fl32 model. 4) set this to false and use fl16 model?

Many thanks.

Model is uploaded in: https://drive.google.com/drive/folders/18B4Wx4BEPxfptsTmIEZySwILLZNXbE2v?usp=sharing Inputs are image of size 1933213

Please provide the exact sequence of commands/steps when you ran into the problem INFO: Initialized TensorFlow Lite runtime. INFO: Created TensorFlow Lite delegate for GPU. ERROR: Next operations are not supported by GPU delegate: CONV_2D: Expected 1 input tensor(s), but node has 3 runtime input(s). DEPTHWISE_CONV_2D: Expected 1 input tensor(s), but node has 3 runtime input(s). DEQUANTIZE: Operation is not supported. First 0 operations will run on the GPU, and the remaining 198 on the CPU.

tensorflow/tensorflow

Answer questions renjie-liu

Hi Sachin, can you help take a look?

useful!

Related questions

ModuleNotFoundError: No module named 'tensorflow.contrib' hot 9
Tf.Keras metrics issue hot 8
Error occurred when finalizing GeneratorDataset iterator hot 7
Error loading tensorflow hot 6
module 'tensorflow' has no attribute 'ConfigProto' hot 6
TF 2.0 'Tensor' object has no attribute 'numpy' while using .numpy() although eager execution enabled by default hot 6
tensorflow-gpu CUPTI errors
Lossy conversion from float32 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning.
ModuleNotFoundError: No module named 'tensorflow.contrib'
When importing TensorFlow, error loading Hadoop
OSError: SavedModel file does not exist at: saved_model_dir/{saved_model.pbtxt|saved_model.pb}
AttributeError: module 'tensorflow.python.framework.op_def_registry' has no attribute 'register_op_list'
tf.keras.layers.Conv1DTranspose ?
[TF 2.0] tf.keras.optimizers.Adam hot 4
TF2.0 AutoGraph issue hot 4
source:https://uonfu.com/
Github User Rank List