profile
viewpoint
Saurabh Saxena saxenasaurabh @google Mountain View, CA saurabhsaxena.info

saxenasaurabh/shopassist 1

Assistive shopping

saxenasaurabh/3dcloud-web-viewer 0

A WebGL based visualizer for visualzing 3D point clouds

saxenasaurabh/community 0

Stores documents used by the TensorFlow developer community

saxenasaurabh/drawpath 0

An assisted path drawing app

saxenasaurabh/quiver-dart 0

A set of utility libraries for Dart

saxenasaurabh/tensorflow 0

An Open Source Machine Learning Framework for Everyone

issue commenttensorflow/tensorflow

tf.range + for x,y in dataset issue

I see. Yeah that would be simpler than trying to handle this in the placement code I agree.

SSSxCCC

comment created time in 11 days

issue commenttensorflow/tensorflow

tf.range + for x,y in dataset issue

@alextp I don't understand your suggestion. IIUC we must place the lowered control flow ops so we will still need update the placement logic?

@ezhulenev probably understands the placement logic better. Looking at the example here it seems we need to recognize patterns like Input(CPU) -> Enter -> Merge -> Switch -> Identity -> Consumer (CPU) and if so place the entire chain(-s in case of nesting) on CPU?

SSSxCCC

comment created time in 11 days

Pull request review commenttensorflow/tensorflow

Implement Hessian for sparse softmax cross entropy

 bool OpGradientDoesntRequireInputIndices(           {"SparseSegmentSumWithNumSegments", {false, {3}}},           {"SparseSlice", {false, {2, 4}}},           {"SparseSoftmax", {false, {1}}},-          {"SparseSoftmaxCrossEntropyWithLogits", {true, {}}},+          {"SparseSoftmaxCrossEntropyWithLogits", {false, {1}}},

My bad. You are both correct. Sorry for the confusion.

MichaelKonobeev

comment created time in 14 days

Pull request review commenttensorflow/tensorflow

Implement Hessian for sparse softmax cross entropy

 bool OpGradientDoesntRequireInputIndices(           {"SparseSegmentSumWithNumSegments", {false, {3}}},           {"SparseSlice", {false, {2, 4}}},           {"SparseSoftmax", {false, {1}}},-          {"SparseSoftmaxCrossEntropyWithLogits", {true, {}}},+          {"SparseSoftmaxCrossEntropyWithLogits", {false, {1}}},

Should be {0}?

MichaelKonobeev

comment created time in 14 days

Pull request review commenttensorflow/tensorflow

Implement Hessian for sparse softmax cross entropy

 bool OpGradientDoesntRequireOutputIndices(           {"SparseSegmentSum", {true, {}}},           {"SparseSegmentSumWithNumSegments", {true, {}}},           {"SparseSlice", {false, {1, 2}}},-          {"SparseSoftmaxCrossEntropyWithLogits", {false, {0}}},+          {"SparseSoftmaxCrossEntropyWithLogits", {false, {}}},

Yep, I think you got the inputs and outputs switched. +1 to running the update script. There is a test that checks that pywrap_gradient_exclusions.cc matches the update script output so that would fail if there isn't an exact match. (Hopefully we will get rid of the circular dependencies and convert this into a genrule).

Thanks!

MichaelKonobeev

comment created time in 14 days

push eventsaxenasaurabh/community

Saurabh Saxena

commit sha 1f28c31d44ae3d086f276c44d91b374cb1db3e2e

Resolve keras open question and mark RFC as accepted

view details

push time in a month

pull request commenttensorflow/community

RFC: Single python code path for eager and graph

@brijk7 could you help merge this RFC please? Thanks!

saxenasaurabh

comment created time in a month

push eventsaxenasaurabh/community

Saurabh Saxena

commit sha ca37121b46b785a5f64a0e735a5c0793c4ca942b

Update RFC #

view details

push time in a month

pull request commenttensorflow/community

RFC: Single python code path for eager and graph

Design Review Notes:

  • Changes:
    • TF_EagerContextDeleteGraph API added to delete the graph and metadata such as captures
    • TF_NewAbstractOp extended to include the op_type
  • Discussion on whether to expose C or C++ APIs
    • Claim is that python language binding already uses C++ and thus results in a C++ -> C -> C++ switch
    • Language bindings such as Java & JS require a C API
    • C API could be of concern if it resulted in poorer performance due to API constraints. However that doesn’t seem to be the case.
    • ABI stability is important across versions which is provided by C
    • Consensus is to stick with a C API as the benefits of C++ are not clear and C API has critical use cases.
    • Short discussion that we can possibly deprecate APIs such as TFE_Execute due to limited usage (1 in courier, 1 in swift, some experimental). We could also make existing APIs call into new APIs
  • Question about API change TF_TensorHandleToTensor to be TF_TensorHandleToEagerTensor?
    • Unresolved. However, no strong objections to change the name.
  • Do we support different ordering of tape popping
    • Forward prop cares about higher level ops to not watch low level ops.
    • Agreement is to enforce a stack.
  • Python gradients will still be supported through callback mechanism.
    • New mechanism will be a gradual migration of python gradients.
    • Registration avoids gradient permanently associated in the name.
    • No being differentiable - op
      • Register Gradient function
      • Implicitly not a good idea because you can get the wrong gradient
      • Python Registry still needed for interop
  • We plan on using pybind11
  • Debugability
    • Don’t need the debuggability at this layer pretty low level
    • We might have compositions
      • Complicated debugging
      • Tfdbg might need to be extended to show the internal operations that it executed
  • Function Registration
    • Local graph and trickle it out - will just use a global registry since that’s what we do anyways
  • Outstanding Keras question
    • Protocol buffer constructor from a nodedef convenient but new APIs will offer a series of APIs for the same functionality
  • We should make it clear that this API is not final and subject to change.
saxenasaurabh

comment created time in a month

push eventsaxenasaurabh/community

Saurabh Saxena

commit sha ee02db84fc90ac0060984b985f62d3bc45f32956

Minor API updates.

view details

push time in a month

issue commenttensorflow/tensorflow

TensorArray objects improperly handled as tf.function arguments or return values

TensorArrays have completely different representations in graph and eager modes. Inside graphs (tf.function) a TensorArray is a DT_VARIANT type tensor wrapping a vector<Tensor*> whereas in eager mode it is just a python list of EagerTensors so they are not really interoperable. This was done to avoid keeping a copy of the entire TensorArray when performing an operation in eager mode.

A workaround for now maybe to return arr.stack() from the tf.function and unstack the returned value using tf.TensorArray(tf.float32, num_rows).unstack(accumulate_error_graph(a)). If we support returning TensorArrays from tf.function this is what we might end up doing under the hood. Even if we don't support this, we should definitely raise a better error message at function tracing time maybe.

diNatale

comment created time in a month

issue commenttensorflow/tensorflow

Deadlock on recursive tf.function-decorated function

Thanks @mayou36 for the detailed description and repro. tf.function does not support recursion. Using a RLock would lead to stack overflows in the case where the arg is a symbolic tensor. I agree we should raise a better error message than deadlocking in this case.

mayou36

comment created time in a month

issue commenttensorflow/tensorflow

Gradient of matmul in while_loop works when run eagerly but not as tf.function

Wrapping in tf.function sounds reasonable till we have a proper fix. Runtime performance would be similar since we would inline the inner tf.function anyway.

mjwatkins2

comment created time in 2 months

PR opened tensorflow/community

Reviewers
Single python code path for eager and graph

This proposal discusses merging the graph building and eager op-dispatch code-paths in python and moving the FuncGraph capturing logic and gradient tape bookkeeping into C++.

+438 -0

0 comment

2 changed files

pr created time in 3 months

push eventsaxenasaurabh/community

Saurabh Saxena

commit sha 78d2a13462f719e67508cbde9c4d43d5ba506d0e

Remove haning tildas.

view details

push time in 3 months

push eventsaxenasaurabh/community

Saurabh Saxena

commit sha 60cf538c4f8138b0f9685b3d1d9584478a7d7b6d

Correctly format diffed code.

view details

push time in 3 months

push eventsaxenasaurabh/community

Saurabh Saxena

commit sha 789ce192a6324f54240c8262e70014bd65d0d4a9

Add RFC and FuncGraph CUJs addendum doc.

view details

push time in 3 months

push eventsaxenasaurabh/community

Saurabh Saxena

commit sha a920a4d902bef24aa259496f6d063021a4bf3019

Move to rfcs/

view details

push time in 3 months

create barnchsaxenasaurabh/community

branch : eager

created branch time in 3 months

fork saxenasaurabh/community

Stores documents used by the TensorFlow developer community

fork in 3 months

issue closedtensorflow/tensorflow

If a branch contains summaries, tf.cond doesn't support Operations as branches

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS Mojave (10.14.2)
  • TensorFlow installed from (source or binary): binary
  • TensorFlow version (use command below): v1.12.0-rc2-3-ga6d8ffae09 1.12.0
  • Python version: 3.6.6 (Anaconda)

Describe the current behavior The code below produces a TypeError. The bug seems to rely on a combination of:

  1. tf.cond
  2. a tf.contrib.summary in one of the branches
  3. tf.group

Describe the expected behavior The code should run successfully.

Code to reproduce the issue

import tensorflow as tf

writer = tf.contrib.summary.create_file_writer('tb')
with writer.as_default(), tf.contrib.summary.always_record_summaries():
    op = tf.cond(
        tf.random.normal([]) >= 0,
        lambda: tf.group(tf.contrib.summary.scalar('loss', 0.2)),
        tf.no_op)

sess = tf.Session()
sess.run(tf.global_variables_initializer())
sess.run(tf.contrib.summary.summary_writer_initializer_op())
sess.run(op)

Other info / logs Here's the traceback:

Traceback (most recent call last):
  File "repro.py", line 8, in <module>
    tf.no_op)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2086, in cond
    orig_res_t, res_t = context_t.BuildCondBranch(true_fn)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1941, in BuildCondBranch
    original_result)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 381, in map_structure
    structure[0], [func(*x) for x in entries])
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 381, in <listcomp>
    structure[0], [func(*x) for x in entries])
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 81, in identity
    return gen_array_ops.identity(input, name=name)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 3454, in identity
    "Identity", input=input, name=name)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 513, in _apply_op_helper
    raise err
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
    preferred_dtype=default_dtype)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1146, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/Users/dmz/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 6168, in _operation_conversion_error
    name, as_ref))
TypeError: Can't convert Operation 'cond/group_deps' to Tensor (target dtype=None, name='input', as_ref=False)

closed time in 3 months

daniel-ziegler

issue commenttensorflow/tensorflow

If a branch contains summaries, tf.cond doesn't support Operations as branches

@lsgrep fantastic! Thank you!

daniel-ziegler

comment created time in 3 months

issue commenttensorflow/tensorflow

ValueError when passing tensors to keras subclass model calls

The shape of feature_slice is different in both branches of if. It seems that only one branch is expected to be used during training, however the graph building code does not know this and tries to build gradient functions for both branches and it sees an inconsistency in the shape of the output gradient.

I didn't dig too deep but the reason this works with adding tf.function is that somehow the input gradient wrt feature_slice has an undefined size for the batch dim in that case. So now the output gradients wrt feature from both branches has undefined size for the batch dim which are compatible. The incomplete shape here is probably a bug.

I think you should be using a python bool here.

One thing I am unclear about is why we allow building a tf.cond with incompatible output shapes in the first place. I will look into that.

sghoshjr

comment created time in 4 months

issue commenttensorflow/tensorflow

Tensorflow Java API and @tf.function signatures

Feeding values and fetching values of tensors defined inside tf.function is not allowed. cc: @k-w-w @allenlavoie

bqm

comment created time in 4 months

issue closedtensorflow/tensorflow

AttributeError when passing tf.Variable into tf.function

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: no
  • TensorFlow installed from (source or binary): source
  • TensorFlow version (use command below): v2.0.0-rc2
  • Python version: 3.7.2
  • Bazel version (if compiling from source): 0.25.2
  • GCC/Compiler version (if compiling from source): 5.4.0 20160609
  • CUDA/cuDNN version: None
  • GPU model and memory: None

Describe the current behavior

import tensorflow as tf
@tf.function
def foo(v):
    pass

foo(tf.Variable(1))
foo(tf.Variable(1))

Using this code piece, tensorflow throws an error: AttributeError: 'NoneType' object has no attribute 'shape'

Describe the expected behavior No error occured.

Code to reproduce the issue Provided above

Other info / logs None

closed time in 4 months

dovahcrow

issue commenttensorflow/tensorflow

AttributeError when passing tf.Variable into tf.function

Sorry for the late update. This was fixed in 4a5b3dbee4e2e5b60f58fe61530b73664e36d87e and should land in 2.1.

dovahcrow

comment created time in 4 months

issue commenttensorflow/serving

Start serving failed with uninitialized value since path variable and lookup table can't initialize in order

Could you try changing your code to the following

with tf.control_dependencies([assign_index_asset_path]):
  vocabulary_file = index_asset_path.read_value()
table = tf.contrib.lookup.index_to_string_table_from_file(vocabulary_file=vocabulary_file, default_value="UNKNOWN")

Then you can just use the following initializer:

init_op = tf.group(tf.global_variables_initializer(), tf.tables_initializer())

The problem was that when we create a variable we create a default "read" op that has no control dependency. When you try to run the table's initializer TensorFlow's executor, which executes greedily, tries to run that "read" op first and fails because the Assign op has not been run yet. In my suggestion above you explicitly call read_value which create a new "read" and ensure it runs after the Assign so things should work.

BenZstory

comment created time in 5 months

issue commenttensorflow/tensorflow

If a branch contains summaries, tf.cond doesn't support Operations as branches

Please send a PR. I'd be happy to review.

daniel-ziegler

comment created time in 5 months

issue commenttensorflow/tensorflow

[TF 2.0] GRU layer doesn't work when called from tf.function

Yeah this seems to be failing when building the gradient of TensorListPopBack. It seems like with persistent=True this takes the path that outputs all necessary intermediates to support higher-order derivatives which triggers taking the derivative of TensorListPopBack. That path is not taken with persistent=False iiuc. I will look into a fix.

David-Mao

comment created time in 5 months

PR opened tensorflow/tensorflow

Fix shape inference for default gradients of resources

Fix shape inference for default gradients of resources which are usually of the form:

tf.zeros(gen_resource_variable_ops.variable_shape(resource), dtype) This adds support for VariableShape to ShapeRefiner::ConstantPartialShape. This change should provide better graph building shape inference but should not affect graph-building/runtime behavior.

PiperOrigin-RevId: 267681703

+17 -0

0 comment

2 changed files

pr created time in 6 months

create barnchsaxenasaurabh/tensorflow

branch : cherrypicks_PE4RX

created branch time in 6 months

create barnchsaxenasaurabh/tensorflow

branch : cherrypicks_JUPKK

created branch time in 6 months

more