profile
viewpoint

Ask questions[RNN] Converting network with LSTM layer to int8 sefaults whole python

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu or Google Colab
  • TensorFlow installed from (source or binary): pip install tf-nightly
  • TensorFlow version (or github SHA if from source): 2.4.0-dev20200710

Trying out conversion, I managed to reproducibly segfault. I am not sure whether I am supplying wrong inputs to representative_dataset but in any case, the code should not be segfaulting and at max throwing python errors. The issue is reproducible also on colab.

!pip install tf-nightly
import numpy as np
import tensorflow as tf
tf.__version__
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
x_train = x_train.astype(np.float32)
x_test = x_test.astype(np.float32)

_EPOCHS = 1
_TRAINING_DATA_COUNT = 1000
x_train = x_train[:_TRAINING_DATA_COUNT]
y_train = y_train[:_TRAINING_DATA_COUNT]

model.fit(x_train, y_train, epochs=_EPOCHS)
model.evaluate(x_test, y_test, verbose=0)

[1.8708466291427612, 0.531000018119812]

run_model = tf.function(lambda x: model(x))
# This is important, let's fix the input size.
BATCH_SIZE = 1
STEPS = 28
INPUT_SIZE = 28
concrete_func = run_model.get_concrete_function(
    tf.TensorSpec([BATCH_SIZE, STEPS, INPUT_SIZE], model.inputs[0].dtype))

# model directory.
MODEL_DIR = "keras_lstm"
model.save(MODEL_DIR, save_format="tf", signatures=concrete_func)

WARNING:tensorflow:AutoGraph could not transform <function <lambda> at 0x7f69aa50b730> and will run it as-is. Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: <module 'main'> is a built-in module To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert WARNING:tensorflow:AutoGraph could not transform <function <lambda> at 0x7f69aa50b730> and will run it as-is. Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: <module 'main'> is a built-in module To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert WARNING: AutoGraph could not transform <function <lambda> at 0x7f69aa50b730> and will run it as-is. Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: <module 'main'> is a built-in module To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert INFO:tensorflow:Assets written to: keras_lstm/assets INFO:tensorflow:Assets written to: keras_lstm/assets

def representative_dataset_gen():
    return [[x_train[:1]]] # Not sure why I need to wrap this in two lists (otherwise converter complains)

converter = tf.lite.TFLiteConverter.from_saved_model(MODEL_DIR)
converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
        
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

Running this last cell now segfaults python interpreter / crashes jupyter kernel

# This segfaults
tflite_model = converter.convert()
tensorflow/tensorflow

Answer questions renjie-liu

fused unidirectional_lstm does not support int8 yet (WIP), sorry about that.

if you use float, it should work.

useful!

Related questions

ModuleNotFoundError: No module named 'tensorflow.contrib' hot 9
Tf.Keras metrics issue hot 8
Error occurred when finalizing GeneratorDataset iterator hot 7
Error loading tensorflow hot 6
module 'tensorflow' has no attribute 'ConfigProto' hot 6
TF 2.0 'Tensor' object has no attribute 'numpy' while using .numpy() although eager execution enabled by default hot 6
tensorflow-gpu CUPTI errors
Lossy conversion from float32 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning.
ModuleNotFoundError: No module named 'tensorflow.contrib'
When importing TensorFlow, error loading Hadoop
OSError: SavedModel file does not exist at: saved_model_dir/{saved_model.pbtxt|saved_model.pb}
AttributeError: module &#39;tensorflow.python.framework.op_def_registry&#39; has no attribute &#39;register_op_list&#39;
tf.keras.layers.Conv1DTranspose ?
[TF 2.0] tf.keras.optimizers.Adam hot 4
TF2.0 AutoGraph issue hot 4
source:https://uonfu.com/
Github User Rank List