profile
viewpoint

Ask questionsQuestion about exporting an integer-only MobileBERT to TF-Lite format.

Hi, I'm trying to export a mobilebert model to tflite format.

Environment Docker (tensorflow/tensorflow:1.15.0-gpu-py3) image V100 16GB

As guided in README.md., I followed "Run Quantization-aware-training with Squad" then "Export an integer-only MobileBERT to TF-Lite format." However, I got an error while converting to quantized tflite model.

2020-07-15 10:26:10.934857: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:786] Optimization results for grappler item: graph_to_optimize 2020-07-15 10:26:10.934903: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:788] constant_folding: Graph size after: 4461 nodes (-1120), 4701 edges (-1124), time = 779.203ms. 2020-07-15 10:26:10.934931: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:788] constant_folding: Graph size after: 4461 nodes (0), 4701 edges (0), time = 374.792ms. Traceback (most recent call last): File "run_squad.py", line 1517, in <module> tf.app.run() File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299, in run _run_main(main, args) File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250, in _run_main sys.exit(main(argv)) File "run_squad.py", line 1508, in main tflite_model = converter.convert() File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/lite/python/lite.py", line 993, in convert inference_output_type) File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/lite/python/lite.py", line 239, in _calibrate_quantize_model inference_output_type, allow_float) File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/lite/python/optimize/calibrator.py", line 78, in calibrate_and_quantize np.dtype(output_type.as_numpy_dtype()).num, allow_float) File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/lite/python/optimize/tensorflow_lite_wrap_calibration_wrapper.py", line 115, in QuantizeModel return _tensorflow_lite_wrap_calibration_wrapper.CalibrationWrapper_QuantizeModel(self, input_py_type, output_py_type, allow_float) RuntimeError: Invalid quantization params for op GATHER at index 2 in subgraph 0

I used pre-trained weights (uncased_L-24_H-128_B-512_A-4_F-4_OPT) that mentioned in README.md. Is it required to distillation process before quantization-aware-training?

Regards, Dongjin.

google-research/google-research

Answer questions renjie-liu

I think the real issue is the model trained in 1.x world but the quantization needs 2.x. (it's easier for us to do internally)

@saberkun we probably need to migrate mobilebert to 2.x asap.

wdyt?

useful!

Related questions

dql_grasping: Training with default config did not converge to good success rate hot 1
Trouble training temporal cycle consistency model on a new dataset hot 1
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe7 in position 1: invalid continuation byte hot 1
dql_grasping: Training with default config did not converge to good success rate hot 1
source:https://uonfu.com/
Github User Rank List