profile
viewpoint

Ask questionsCustom dataset op encounters refcount error

I'm trying to implement a customize dataset op so that developing extention of tf.data may not need to recompile the whole tensorflow codebase. And as a start, I'm implementing an identity dataset op that would do nothing but pass the data of its input.

After compiled according to the custom op tutorial, the dataset can successfully output data, but it will raise error on destruction. The error message is:

% python3 identity_dataset_op.py 
2020-05-29 19:09:04.290680: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-05-29 19:09:04.304126: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fd634565fb0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-05-29 19:09:04.304140: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
tf.Tensor(1, shape=(), dtype=int32)
2020-05-29 19:09:04.308089: F /usr/local/lib/python3.7/site-packages/tensorflow_core/include/tensorflow/core/lib/core/refcount.h:90] Check failed: ref_.load() == 0 (1 vs. 0)
zsh: abort      python3 identity_dataset_op.py

The problem is that the refcount has not been set to 0 when the program enters the destructor and this cause the destructor of RefCounted (which is the base class of all Dataset) panic.

However, I believe the destructor of a dataset will only be called when its refcount is set to 0. I wonder if this error is connected to some functionality that a custom op cannot use. Also, it will be really nice if you tell me whether it is possible to make a customize dataset op.

BTW, during debugging, I noticed that the MakeDataset function would be called twice. Is there any reason for this?

Thank you so much for your time on this issue.

The code for the op is in this zip file. custom_dataset_op.zip

To run the test, simple compile the .cc file to identity_dataset_op.zip and run python identity_dataset.py.

tensorflow/tensorflow

Answer questions aaudiber

I looked through the code and nothing stuck out that would cause the refcount issue. When you build with the tensorflow codebase, are you building from latest, or from branch 1.15? It's possible that there were some changes to reference counting since 1.15.

useful!

Related questions

ModuleNotFoundError: No module named 'tensorflow.contrib' hot 8
Error occurred when finalizing GeneratorDataset iterator hot 6
ModuleNotFoundError: No module named 'tensorflow.contrib'
When importing TensorFlow, error loading Hadoop
tf.keras.layers.Conv1DTranspose ?
tensorflow-gpu CUPTI errors hot 4
[TF 2.0] tf.keras.optimizers.Adam hot 4
Lossy conversion from float32 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning. hot 4
TF2.0 AutoGraph issue hot 4
Tf.Keras metrics issue hot 4
module 'tensorflow' has no attribute 'ConfigProto' hot 4
TF 2.0 'Tensor' object has no attribute 'numpy' while using .numpy() although eager execution enabled by default hot 4
ModuleNotFoundError: No module named 'tensorflow.examples.tutorials' hot 4
AttributeError: module 'tensorflow.python.framework.op_def_registry' has no attribute 'register_op_list' hot 4
tensorflow2.0 detected 'xla_gpu' , but 'gpu' expected hot 3
Github User Rank List