profile
viewpoint

Ask questionsDataset iterating different behavior in TF 2.1 and 2.2

System information

  • OS Platform and Distribution: Windows 10 Home
  • TensorFlow versions:
    • 2.1: v2.1.0-rc2-17-ge5bf8de410 2.1.0
    • 2.2: v2.2.0-rc4-8-g2b96f3662b 2.2.0
  • Python version: 3.7.6

I was not sure how to report this issue as it might be bug or just expected behavior. There are difference in TF 2.1 and 2.2. This is a code snippet to reproduce my issue:

import math
import numpy as np
import tensorflow as tf

# simple dataset with zeros
batch_size = 32
features = np.zeros((10000, 60, 2))
labels = np.zeros((10000, 1))
train_data = tf.data.Dataset.from_tensor_slices((features, labels)).batch(batch_size)
train_steps = int(math.ceil(features.shape[0] / batch_size))

# simple model with Dense layers
inputs = tf.keras.Input(shape=(features[0].shape[0], features[0].shape[1]))
x = tf.keras.layers.Dense(32, activation="relu")(inputs)
outputs = tf.keras.layers.Dense(1, activation="relu")(x)
model = tf.keras.Model(inputs, outputs, name="example_model")

# model fitting
model.compile(loss="mse", optimizer="adam", metrics=["mse"])
model.fit(train_data, epochs=100, steps_per_epoch=train_steps)

When I run this code in TF2.1 it will produce this error: https://pastebin.com/4M43SE44 After the first epoch, there are warnings about end of sequence, that my input ran out of data. And finally, as you can see in the pasted output, it raises ValueError: Empty training data.. When I change line with dataset creation to <pre> train_data = tf.data.Dataset.from_tensor_slices((features, labels)).batch(batch_size).<b>repeat()</b> </pre> than everything works as expected.

This is behavior I would expect. (Note steps_per_epoch attribute as I wan to control this by myself, of course when I do not use repeat and steps_per_epoch is set to None, it will work under TF2.1 as it will iterate whole dataset every epoch).

When I run the same code with TF2.2 (no repeat, train_steps are specified) it works without any issue. Is this behavior intentional? Why does it work in TF2.2 and not 2.1? Could anyone elaborate on this issue?

tensorflow/tensorflow

Answer questions tomerk

@omalleyt12 is this one of the known regressions that have fixes for 2.3 that didn't quite make it in to 2.2? Or is this something to add to the list?

useful!

Related questions

ModuleNotFoundError: No module named 'tensorflow.contrib' hot 8
Error occurred when finalizing GeneratorDataset iterator hot 6
ModuleNotFoundError: No module named 'tensorflow.contrib'
When importing TensorFlow, error loading Hadoop
tf.keras.layers.Conv1DTranspose ?
tensorflow-gpu CUPTI errors hot 4
[TF 2.0] tf.keras.optimizers.Adam hot 4
Lossy conversion from float32 to uint8. Range [0, 1]. Convert image to uint8 prior to saving to suppress this warning. hot 4
TF2.0 AutoGraph issue hot 4
Tf.Keras metrics issue hot 4
module 'tensorflow' has no attribute 'ConfigProto' hot 4
TF 2.0 'Tensor' object has no attribute 'numpy' while using .numpy() although eager execution enabled by default hot 4
ModuleNotFoundError: No module named 'tensorflow.examples.tutorials' hot 4
AttributeError: module &#39;tensorflow.python.framework.op_def_registry&#39; has no attribute &#39;register_op_list&#39; hot 4
tensorflow2.0 detected 'xla_gpu' , but 'gpu' expected hot 3
Github User Rank List