profile
viewpoint
Saurabh Saxena saxenasaurabh @google Mountain View, CA saurabhsaxena.info

saxenasaurabh/shopassist 1

Assistive shopping

saxenasaurabh/3dcloud-web-viewer 0

A WebGL based visualizer for visualzing 3D point clouds

saxenasaurabh/community 0

Stores documents used by the TensorFlow developer community

saxenasaurabh/drawpath 0

An assisted path drawing app

saxenasaurabh/quiver-dart 0

A set of utility libraries for Dart

saxenasaurabh/tensorflow 0

An Open Source Machine Learning Framework for Everyone

issue commenttensorflow/tensorflow

AlreadyExistsError exception while running GRU layer on Tensorflow 2.2.0rc_3

@rmlarsen mentioned here that this may be due a bug in grappler generating non-unique names. @rmlarsen @ezhulenev do you know if this was fixed?

jakublipinski

comment created time in a month

issue commenttensorflow/tensorflow

tf.name_scope has no effect when used with tf.cond and autograph

Thanks for trying that. I have a fix that should make the eager tensorboard consistent with the graph one. I am testing it out.

sumanthratna

comment created time in a month

issue commenttensorflow/tensorflow

tf.name_scope has no effect when used with tf.cond and autograph

Could you try this in nightly? 8c1ead8 partially fixes this by propagating the parent name_scope. Although, I think there might still be a difference between graph and eager mode because graph mode adds an additional name_scope "cond". We should reconcile those.

sumanthratna

comment created time in a month

issue closedtensorflow/tensorflow

Default type for complex sparse tensor is complex128

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
  • TensorFlow installed from (source or binary): - TensorFlow version (use command below): pip

Describe the current behavior When I instantiate a complex sparse tensor, the default type is complex128, whereas with a float sparse tensor it is float32.

Describe the expected behavior The default type for complex sparse tensor should probably be complex64, since we can't set the type in the tensor creation.

Standalone code to reproduce the issue

a = tf.sparse.SparseTensor(indices=[[0, 0],], values=[1.+ 0j], dense_shape=[2, 2])
print(a.dtype)

This gives tf.complex128.

Other info / logs This can be circumvented using numpy to fix the type of the values but it isn't ideal imho.

closed time in a month

zaccharieramzi

issue commenttensorflow/tensorflow

Default type for complex sparse tensor is complex128

This is consistent with the behavior in numpy:

>>> np.array([1.+ 0j]).dtype
dtype('complex128')
zaccharieramzi

comment created time in a month

issue closedtensorflow/tensorflow

Cannot execute the substraction op with the broadcast mechanism

import tensorflow as tf
import numpy as np
x = tf.constant(np.random.random((500, 6)))
y = x - x[:,0]

This code raises the issue of not supporting [500,6] - [500] in the sub OP. I suppose that it should compute correctly, using the broadcast automatically? Is it a bug? version: tf2.0

closed time in 2 months

TomorrowIsAnOtherDay

issue commenttensorflow/tensorflow

Cannot execute the substraction op with the broadcast mechanism

This is consistent with the broadcast semantics of numpy:

>>> x=np.random.random((500, 6))
>>> x-x[:,0]
ValueError: operands could not be broadcast together with shapes (500,6) (500,) 
TomorrowIsAnOtherDay

comment created time in 2 months

issue commenttensorflow/tensorflow

log(diag_part(x)) vs diag_part(log(x)) difference with Cholesky gradients

Thanks @sethnabarro for reporting this. This seems to be an artifact of how the gradient of tf.diag_part is implemented. We fill the non-diagonal values with zeros and if those entries in L are zeros as well you would get a 0/0. I don't see an easy way to fix this for now (unless we add some fused APIs which does not seem combinatorially feasible) but I will discuss possible solutions with other folks.

sethnabarro

comment created time in 2 months

issue commenttensorflow/tensorflow

No gradient defined for op

Gradient for Xlogy is not implemented yet. But you are welcome to send a PR! You can take a look at the existing C++ gradient implementations here and the python gradient for Xlogy here.

minjac

comment created time in 2 months

issue closedtensorflow/tensorflow

[TF 2.0] GRU layer doesn't work when called from tf.function

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Darwin Kernel Version 18.6.0
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
  • TensorFlow installed from (source or binary): binary
  • TensorFlow version (use command below): 2.0.0-dev20190826
  • Python version: Python 3.6.8 :: Anaconda, Inc.
  • Bazel version (if compiling from source): N/A
  • GCC/Compiler version (if compiling from source): N/A
  • CUDA/cuDNN version: N/A
  • GPU model and memory: N/A

Describe the current behavior

The code below raises error:

Tried to convert 'tensor' to a tensor and failed. Error: None values not supported.

As noted in comments, the bug disappears if we don't use tf.function, or set persistent=False in gradient tape.

Code to reproduce the issue

import numpy as np
import tensorflow as tf

x = tf.cast(np.random.randn(1, 100), tf.float32)
y = tf.cast(np.random.randn(1, 100, 100), tf.float32)
z = tf.cast(np.random.randn(1, 100), tf.float32)

class Model(tf.keras.Model):
    def __init__(self):
        super(Model, self).__init__()
        self.layer = tf.keras.layers.GRU(100)

    @tf.function # remove this and it works fine
    def call(self, x, y):
        z = self.layer(y, initial_state=x)
        return z

model = Model()

with tf.GradientTape(persistent=True) as tape: # if persistent=False it works fine
    loss = tf.norm(model(x, y) - z)
grads = tape.gradient(loss, model.trainable_variables)

closed time in 2 months

David-Mao

issue commenttensorflow/tensorflow

[TF 2.0] GRU layer doesn't work when called from tf.function

Sorry I don't understand. Should we close this since this seems to be fixed in 2.1? Are you asking if this will be fixed in 2.0? I don't think we would want to do that now since the fix is already available in a public release.

David-Mao

comment created time in 2 months

issue commenttensorflow/tensorflow

bug in C++ API "AddSymbolicGradients"

Thanks for debugging this! Your proposal looks good. Would you like to send a PR for the fix?

buaasun

comment created time in 2 months

issue closedtensorflow/tensorflow

How to override gradient in a FuncGraph created by tf.function

With tensorflow 1.x API, I can override gradients with a gradient_override_map during building a graph as follows

@tf.RegisterGradient('CustZero')
def _custom_grad(op, grad):
    return tf.zeros_like(op.inputs[0])
    
with tf.compat.v1.Session() as sess:
    with sess.graph.as_default() as g:
        x = tf.convert_to_tensor([-1.0, 0.0, 1.0, 2.0])
        with g.gradient_override_map({'Relu': 'CustZero'}):
            y = tf.nn.relu(x)
    dy = tf.gradients(y, x)

# dy: [0., 0., 0., 0.]

As we are moving to TF2, I'd like to use native TF2 API to do the same task with tf.function and tf.GradientTape. As I have learnt, tf.function creates a FuncGraph for each different input_signature, and I can use get_concrete_function to create one ConcreteFunction with a specific input_signature. Then I can get the FuncGraph associated with the ConcreteFunction. This is what I have tried, which failed to override gradients.

# create a concrete function with the specified input signatures
@tf.function(input_signature=(tf.TensorSpec(shape=(None,), dtype=tf.float32),))
def my_relu(x):
    return tf.nn.relu(x)
my_relu_conc = my_relu.get_concrete_function()

x = tf.constant([-1.0, 0.0, 1.0, 2.0], dtype=tf.float32)
with tf.GradientTape() as tape:
    with my_relu_conc.graph.gradient_override_map({'Relu': 'CustZero'}):
        tape.watch(x)
        y = my_relu_conc(x)
dy = tape.gradient(y, x).numpy()

# dy: [0., 0., 1., 1.]

From my understanding, since the FuncGraph is already traced, the graph cannot be changed and I cannot override with custom gradients.

My question is: are there any methods to override gradient using native TF2 APIs?

If not, could we add an additional parameter named gradient_override_map to tf.function or get_concrete_function such that we can override gradients in the process where a FuncGraph is created?

@tf.function(
    input_signature=(tf.TensorSpec(shape=(None,), dtype=tf.float32),), 
    gradient_override_map={'Relu': 'CustZero'})
def my_relu(x):
    return tf.nn.relu(x)
my_relu_conc = my_relu.get_concrete_function()

or

@tf.function()
def my_relu(x):
    return tf.nn.relu(x)
my_relu_conc = my_relu.get_concrete_function(
    input_signature=(tf.TensorSpec(shape=(None,), dtype=tf.float32),), 
    gradient_override_map={'Relu': 'CustZero'})

closed time in 3 months

lffan

issue commenttensorflow/tensorflow

How to override gradient in a FuncGraph created by tf.function

Could you try using tf.custom_gradient

@tf.function(input_signature=(tf.TensorSpec(shape=(None,), dtype=tf.float32),))
def my_relu(x):
  @tf.custom_gradient
  def relu_with_zeros_grad(x):
    output = tf.nn.relu(x)
    def grad_fn(grad):
       return tf.zeros_like(output.op.inputs[0])
    return output, grad_fn
  return relu_with_zeros_grad(x)
lffan

comment created time in 3 months

create barnchsaxenasaurabh/tensorflow

branch : cherrypicks_4KI0J

created branch time in 3 months

PR opened tensorflow/tensorflow

Make while_v2_indexed_slices_writers compatible with tensor equality.

PiperOrigin-RevId: 299619903 Change-Id: Ia829ce5942d08fb6c55d83d5180f0a49e80bfdc3

+46 -19

0 comment

2 changed files

pr created time in 3 months

more