Is It possibile to setup a Discourse instance? It would be nice to have this instance authetnicated with your Keycloak instance as It will be useful also to discuss non Camelot topics.

@Palkos83 Yes try with 8.0.4

I put this under ecosystem review. If any project in TF ecosystem is interested in this or has this in It own internal roadmap we will try to review It here.

/cc @tanzhenyu @dynamicwebpaige for ecosystem check.

@bhack Can you please check @mihaimaruseac's comments and keep us posted ? Thanks!

I don't understand cause it seems to me that there Is still any plan on your side.

For this as It Is not a bug or feature request you need to close the ticket and open a support question at https://stackoverflow.com/questions/tagged/tensorflow

If you want to use from_tensor_slices you need to prepare something compatible

I think this Is another problem. You can try on stackoverflow

is cuda_version removed?

TF reference implementation at https://github.com/juntang-zhuang/Adabelief-Optimizer/blob/master/TensorFlow_Experiments/AdaBelief_tf.py

I'm using TensorFlow version: 2.3.1 and TFX version: 0.24.1.

I am developing machine learning workflow for timeseries using TFX. The ExampleGen, StatisticsGen, SchemaGen and Example validator and transform components worked fine. In the Trainer component, i used the tf.keras.preprocessing.timeseries_dataset_from_array API within the run_fn() of the Trainer component to generate the train and evaluation windows then i got a TypeError: dataset length is infinite. Is this the right way to create the train and eval data window. Below is the modification i made to the run_fn() in the Trainer module file and the error message;

def run_fn(fn_args: TrainerFnArgs):
    """Train the model based on given args.

    Args:
    fn_args: Holds args used to train the model as name/value pairs.
    """
    # Number of nodes in the first layer of the DNN
    first_dnn_layer_size = 100
    num_dnn_layers = 4
    dnn_decay_factor = 0.7

    sequence_length = 144
    batch_size = 1

    tf_transform_output = tft.TFTransformOutput(fn_args.transform_output)

    train_dataset = _input_fn(fn_args.train_files, fn_args.data_accessor, 
                            tf_transform_output, 40)
    
    # Generate timeseries sliding window for the training data
    train_dataset = tf.keras.preprocessing.timeseries_dataset_from_array(train_dataset, train_dataset, sequence_length=sequence_length , batch_size=batch_size)

    # Generate timeseries sliding window for the eval data
    eval_dataset = _input_fn(fn_args.eval_files, fn_args.data_accessor, 
                           tf_transform_output, 40)
    eval_dataset = tf.keras.preprocessing.timeseries_dataset_from_array(eval_dataset, eval_dataset, sequence_length=sequence_length , batch_size=batch_size)

    model = _build_keras_model(
      # Construct layers sizes with exponetial decay
      hidden_units=[
          max(2, int(first_dnn_layer_size * dnn_decay_factor**i))
          for i in range(num_dnn_layers)
      ])

    tensorboard_callback = tf.keras.callbacks.TensorBoard(
      log_dir=fn_args.model_run_dir, update_freq='batch')
    model.fit(
      train_dataset,
      steps_per_epoch=fn_args.train_steps,
      validation_data=eval_dataset,
      validation_steps=fn_args.eval_steps,
      callbacks=[tensorboard_callback])

    signatures = {
      'serving_default':
          _get_serve_tf_examples_fn(model,
                                    tf_transform_output).get_concrete_function(
                                        tf.TensorSpec(
                                            shape=[None],
                                            dtype=tf.string,
                                            name='examples')),
    }

trainer = Trainer(
    module_file=os.path.abspath(trainer_module_file),
    custom_executor_spec=executor_spec.ExecutorClassSpec(GenericExecutor),
    examples=transform.outputs['transformed_examples'],
    transform_graph=transform.outputs['transform_graph'],
    schema=schema_gen.outputs['schema'],
    train_args=trainer_pb2.TrainArgs(num_steps=10000),
    eval_args=trainer_pb2.EvalArgs(num_steps=5000))
context.run(trainer)

Error message:

INFO:absl:Running driver for Trainer
INFO:absl:MetadataStore with DB connection initialized
INFO:absl:Running executor for Trainer
INFO:absl:Train on the 'train' split when train_args.splits is not set.
INFO:absl:Evaluate on the 'eval' split when eval_args.splits is not set.
INFO:absl:Training model.
INFO:absl:We decided to produce LargeList and LargeBinary types.
INFO:absl:Feature Appliances_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature Press_mm_hg_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_1_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_2_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_3_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_4_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_5_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_6_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_7_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_8_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_9_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature RH_out_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T1_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T2_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T3_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T4_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T5_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T6_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T7_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T8_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T9_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature T_out_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature Tdewpoint_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature Visibility_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature Windspeed_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature lights_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature rv1_xf has a shape . Setting to DenseTensor.
INFO:absl:Feature rv2_xf has a shape . Setting to DenseTensor.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-21-cc5cfdc341eb> in <module>()
      7     train_args=trainer_pb2.TrainArgs(num_steps=10000),
      8     eval_args=trainer_pb2.EvalArgs(num_steps=5000))
----> 9 context.run(trainer)

7 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/dataset_ops.py in __len__(self)
    443     length = self.cardinality()
    444     if length.numpy() == INFINITE:
--> 445       raise TypeError("dataset length is infinite.")
    446     if length.numpy() == UNKNOWN:
    447       raise TypeError("dataset length is unknown.")
TypeError: dataset length is infinite.

Please reopen this if you are using Tensorflow addons.

I don't think so the old one is 2.4.0.dev20200902.

/cc @andresusanopinto

/cc Gently ping for @tanzhenyu @dynamicwebpaige for ecosystem pre-check

@gmacon If you insert print(t1.__call__.pretty_printed_concrete_signatures())

You will see:

ConcreteFunction __call__(x)
  Args:
    x: float32 Tensor, shape=(None, 3)
  Returns:
    float32 Tensor, shape=(None, 3)
__call__(x)
  Args:
    x: float32 Tensor, shape=(7, 3)
  Returns:
    float32 Tensor, shape=(7, 3)

When you save the model the saved concrete function could be in any order. E.g.

Tensor("StatefulPartitionedCall:0", shape=(7, 3), dtype=float32)
Tensor("StatefulPartitionedCall_1:0", shape=(None, 3), dtype=float32)
Tensor("StatefulPartitionedCall_2:0", shape=(None, 3), dtype=float32)

The logic is that it takes the first saved concrete function with a "compatible" in a loop over all the saved concrete functions:

https://github.com/tensorflow/tensorflow/blob/4e281e9a211729a9a57df04dad22d15c988f2919/tensorflow/python/saved_model/function_deserialization.py#L246-L253

And the compatibility check is: https://github.com/tensorflow/tensorflow/blob/4e281e9a211729a9a57df04dad22d15c988f2919/tensorflow/python/saved_model/function_deserialization.py#L92-L119

So as the order of the concrete functions change on save you could enter in any compatible concrete function.

Yes working with kwargs is still in the todo list. Why do you require to trace with t1(np.ones((7, 3), dtype=np.float32))?

On master you can try to make a look at

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/common_runtime/threadpool_device.cc#L45:L69

@gmacon Can you try with @tf.function(input_signature=(tf.TensorSpec(shape=[None, 3], dtype=tf.float32),))?

If you want to preserve the gradient you could take a look at https://www.tensorflow.org/probability/api_docs/python/tfp/math/clip_by_value_preserve_gradient

cosine_loss(y_true.astype(dtype=np.float32), y_pred).numpy()

@fisakhan Have you tried to control also OMP_THREAD_NUM?

I don't know what you mean by "migrate".

Yes I supposed that we had a related issue about this features as it was mandatory in our contribution policy. But in this case we have a PR directly.

For now, we could perhaps put something like this under tfp.experimental.stats, where we have other streaming metrics like covariance etc. Or you could modify to drop the streaming component and simply compute on a single tensor of observations, which would be more consistent w/ e.g. tfp.stats.covariance. We would likely want to eliminate the for loops for TFP. If you really need a loop, we usually use tf.while_loop or tf.scan to avoid bloating the tf.function graph.

The plan is ok for me

@sorensenjs Can you open the PR at https://github.com/tensorflow/probability/ ?

@jvdillon @brianwa84 Do you have permission to migrate this in TFP?

Description

Using absolute reference to other classes tfa.seq2seq. will create a link in the online documentation.

Type of change

• [ ] Bug fix
• [ ] New Tutorial
• [x] Updated or additional documentation
• [ ] Additional Testing
• [ ] New Activation and the changes conform to the activation contribution guidelines
• [ ] New Callback and the changes conform to the callback contribution guidelines
• [ ] New Image addition and the changes conform to the image op contribution guidelines
• [ ] New Layer and the changes conform to the layer contribution guidelines
• [ ] New Loss and the changes conform to the loss contribution guidelines
• [ ] New Metric and the changes conform to the metric contribution guidelines
• [ ] New Optimizer and the changes conform to the optimizer contribution guidelines
• [ ] New RNN Cell and the changes conform to the rnn contribution guidelines
• [ ] New Seq2seq addition and the changes conform to the seq2seq contribution guidelines
• [ ] New Text addition and the changes conform to the text op contribution guidelines

Checklist:

• [x] I've properly formatted my code according to the guidelines
  • [x] By running Black + Flake8
  • [ ] By running pre-commit hooks
• [ ] This PR addresses an already submitted issue for TensorFlow Addons
• [ ] I have made corresponding changes to the documentation
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] This PR contains modifications to C++ custom-ops

In general I think that is great and strategic to have this SIG cause Models and Hub are really one of the unique places in the ecosystem where all of the pieces converges together tocomposed our end2end goal. Also we often suffer more then the Pytorch ecosystem about availability of third party papers official (and unofficial) reference implementations in the github "universe".

But we really need to take this SIG as an occasion to improve our ecosystem about component re-usability and ownership disambiguation. It could be nice to have a short grace period for the new components (layer, loss, optimizer, preprocessing, postprocessing, etc.) that would be introduced in TF model. As we know external members don't know the public roadmap of the different internal teams (and this could happen also between different internal teams) you can find a recap thread at https://github.com/tensorflow/community/issues/29. This could create duplicates, unaware supersedes, implementation divergences and so on.. In Addons we have tried to Draft an early ecosystem-review process to notify the ecosystem that probably could be standardized and improved for general use to improve the coherence between internal teams and external members. By this point of view I see this SIG one of the most critical points to achieve this goal in our ecosystem.

The PR add Just an extra check but you need to use learning_rate not lr

Do you need to use tf.data.Dataset.from_tensor_slices(data[1][0])?

Yes this is about GPU memory OOM when allocating tensor with shape.

For nightly you need to do other related changes like probably to yield [x, y1, y2]

@QColeman97 It is working now. Do you meant with tf-nightly?

@AshTech25 Are you training/fine-tuning the model?

No I'm just using the model for my inferences

So without the training you will not got the error but the warning has the same origin

Check https://www.tensorflow.org/tutorials/distribute/save_and_load?hl=en#the_tfsaved_model_api And the glossary at https://www.tensorflow.org/api_docs/python/tf/distribute

Check https://github.com/tensorflow/tensorflow/pull/44295

Fix https://github.com/tensorflow/tensorflow/issues/44293 https://github.com/tensorflow/tensorflow/issues/44172

Please check https://github.com/tensorflow/tensorflow/issues/44172#issuecomment-712776248

There are many dups or semi-dups like https://github.com/tensorflow/addons/issues/1903#issuecomment-715897109 that confuse users and third party SIG maintainers.

We had this in https://github.com/tensorflow/models/blob/master/official/nlp/optimization.py#L112

/cc @philjd

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
• TensorFlow version and how it was installed (source or binary): binary 2.3
• TensorFlow-Addons version and how it was installed (source or binary): binary nightly
• Python version: 3.6
• Is GPU used? (yes/no): yes

Describe the bug

seem exclude_from_weight_decay is missed in tfa.optimizers.AdamW. in tfa.optimizers.LAMB, we have this parameter.

Code to reproduce the issue

Provide a reproducible test case that is the bare minimum necessary to generate the problem.

Other info / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

https://github.com/tensorflow/addons/issues/1903

This comment is connected to the models's reusable components topic. Some times ago we had a ticket about this at https://github.com/tensorflow/community/issues/223

The real question is also how to implement something like https://github.com/tensorflow/addons/issues/236. I don't know if this new SIG will have the resource to implement something like to solve this open topic. I think it would be really useful for all the SIGs but also for the TF ecosystem.

We are also trying to figure if a bot could do some keep-alive/MIA checks https://github.com/tensorflow/addons/pull/2024

What is the status of this? I don't know if this is also will cover the issue about optimizer with slots. In this case I want to add a warning somewhere in the meantime https://github.com/tensorflow/tensorflow/pull/42846

What about keras-cv/keras-nlp?

We have already something about this in SIG-addons in the approved RFC https://github.com/tensorflow/community/blob/master/rfcs/20190308-addons-proxy-maintainership.md#repository-growth-and-review /cc @seanpmorgan

Also for build/rebuild action we need to control platform flag https://www.docker.com/blog/multi-platform-docker-builds/

How this will fit with the multiarch case? https://github.com/multiarch/qemu-user-static

See also https://github.com/jitsi/jitsi-meet/issues/6864

@QColeman97 I modified you example also to run to the next TF version. I think that problem is that you are producing a loss: in the log dictionary with an array of batch size len. The monitor is expected to be a single value.

Are you using Tensorflow addons features?

You could require to use https://github.com/tensorflow/addons/blob/master/tools/install_so_files.sh

Check https://github.com/tensorflow/addons/pull/1909 /cc @seanpmorgan

Check https://github.com/tensorflow/tensorflow/issues/24548

@QColeman97 Can you try to run your example on tf-nightly cause It seems to have other errors.

@bamurtaugh Ok can you summarize this thread there?

@Gabrielllopes I don't think it is similar.

@mantonios107 Ok but I am quite sure that nobody will backport this on 2.1

The problem is that in the callback your current is an array and self.min_delta is just an int. https://github.com/tensorflow/tensorflow/blob/c587b9a2dcd6595b70b57bd8d8a4563968cdb128/tensorflow/python/keras/callbacks.py#L1762-L1765

TensorFlow installed from (source or binary): source

Are you compiling TF from source?

Cause this one is running correctly on colab.

Yes I meant can I reproduce your issue with your code?

from tensorflow.keras.losses import Loss
import tensorflow as tf
import tensorflow.keras.backend as K
import numpy as np


class BatchMeanSquaredError(Loss):

    def __init__(self, reduction='auto', name='batch_mean_squared_error'):
        super().__init__(reduction=reduction, name=name)

    def call(self, y_true, y_pred):
        y_pred = tf.convert_to_tensor(y_pred)
        y_true = tf.cast(y_true, y_pred.dtype)
        L = K.mean((y_pred - y_true) ** 2, axis=0)
        return L

X = np.random.random((1000, 3))
y = np.ones(shape=(1000, 3))

model = tf.keras.Sequential(
    [
        tf.keras.layers.Dense(3, activation='relu'),
        tf.keras.layers.Dense(3, activation='relu'),
        tf.keras.layers.Dense(3)
    ]
)

bmse = BatchMeanSquaredError()
model.compile(loss=bmse, optimizer='sgd')

model.fit(X, y, batch_size=10, epochs=5)

tf.keras.models.save_model(model=model, filepath='model.h5')

custom_objects = {'BatchMeanSquaredError': BatchMeanSquaredError}
tf.keras.models.load_model('model.h5', custom_objects=custom_objects)

Can I reproduce with Standalone code to reproduce the issue?

https://docs.python.org/3/library/ast.html Deprecated since version 3.9: Old classes ast.Index and ast.ExtSlice are still available, but they will be removed in future Python releases. In the meantime, instantiating them will return an instance of a different class.

I think this is a different ticket as the version was already reverted https://github.com/tensorflow/tensorflow/commit/710f3c83b4147eb76e748efcc218325c4978726c. /cc @mihaimaruseac

/cc @MarkDaoust What do you think?

This solved the problem! Thank you so much. Maybe this could be added to the documentation at https://www.tensorflow.org/api_docs/python/tf/keras/layers/experimental/preprocessing/Normalization ? I'm not sure why the map step is necessary, but in either case it does solve the issue. Thanks once again!

Can you open a new documentation issue? I think that probably we could also raise an exception.

For the bugs, we can create an issue based on the description. Although it's more work, we need to go through the feedbacks anyway to aggregate them and sort them by count. Same for the most requested features

I suppose It will be quite time consuming on your side if you will start to do It formally.

If we don't have still enought github users around the project we could try to build up at least a prosumers community around the project. I think that entry level users probably will still use the feedback form.

Do you think It is hard to set-up a discourse instance and configure it with Rosseau auth?

Yes the problem seems that It cannot find Cudnn files in your system

Can you configure in the same python3 env?

“ ImportError: No module named pip._internal.req”

What Is the error now?

I think that you need to use something like:

feature_ds = sample_dataset.map(lambda x, y: x)
normalizer = preprocessing.Normalization()
normalizer.adapt(feature_ds)

@AshTech25 Are you training/fine-tuning the model?

As I see It was a wrongly updated dependency

Description

See #2197 for additional context.

Type of change

• [ ] Bug fix
• [ ] New Tutorial
• [x] Updated or additional documentation
• [ ] Additional Testing
• [ ] New Activation and the changes conform to the activation contribution guidelines
• [ ] New Callback and the changes conform to the callback contribution guidelines
• [ ] New Image addition and the changes conform to the image op contribution guidelines
• [ ] New Layer and the changes conform to the layer contribution guidelines
• [ ] New Loss and the changes conform to the loss contribution guidelines
• [ ] New Metric and the changes conform to the metric contribution guidelines
• [ ] New Optimizer and the changes conform to the optimizer contribution guidelines
• [ ] New RNN Cell and the changes conform to the rnn contribution guidelines
• [x] New Seq2seq addition and the changes conform to the seq2seq contribution guidelines
• [ ] New Text addition and the changes conform to the text op contribution guidelines

Checklist:

• [x] I've properly formatted my code according to the guidelines
  • [x] By running Black + Flake8
  • [ ] By running pre-commit hooks
• [ ] This PR addresses an already submitted issue for TensorFlow Addons
• [ ] I have made corresponding changes to the documentation
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] This PR contains modifications to C++ custom-ops

How Has This Been Tested?

Ran the existing tests.

@smehra34 If you try with tf-nightly you will see a new warning: WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines tf.data is recommended.

Can you try in python:

from pip._internal.req import parse_requirements

I think you have problem with pip. Try to force reinstall pip.

In the meantime you can use pip install gast==0.3.3 with tf-nightly.

Please check https://github.com/serge-sans-paille/gast/issues/53#issuecomment-712421906

I think is a dup of https://github.com/tensorflow/tensorflow/issues/40166 /cc @andresp

Do you have a very minimal but runnable (copy, paste and run) example to reproduce this?

Please check https://github.com/tensorflow/tensorflow/blob/fcc4b966f1265f466e82617020af93670141b009/tensorflow/python/keras/optimizer_v2/adam.py#L362-L363

https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html

What about desktop support?

This file has many differences compared to the existing dasiamrpn.py, while they are actually very similar algorithm. I think it would be better if we create a 'tracker' dir under https://github.com/opencv/opencv/blob/master/samples/dnn/ , and move both file inside, so that we could make them share a utils.py which generate anchors etc.

@ZhiyuanChen I think it is ok. Are you interested to open a new PR?

/cc @jinyup100