profile
viewpoint

tensorflow/probability 2719

Probabilistic reasoning and statistical analysis in TensorFlow

tensorflow/agents 1184

TF-Agents is a library for Reinforcement Learning in TensorFlow

ebrevdo/synchrosqueezing 42

The MATLAB Synchrosqueezing Toolbox

ebrevdo/tensorflow 2

Computation using data flow graphs for scalable machine learning

ebrevdo/arduino 1

Arduino projects and libraries

ebrevdo/incanter 1

Clojure-based, R-like statistical computing and graphics environment for the JVM

ebrevdo/mahout 1

Mirror of Apache Mahout

ebrevdo/ansible 0

Ansible is a radically simple IT orchestration engine that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems.

ebrevdo/Arduino_Rest_Server 0

Simple rest-inspired server for Arduino that works with Serial or Ethernet connections.

Pull request review commenttensorflow/agents

added support for Dict action space

 def _compute_next_q_values(self, next_time_steps):     # action constraints are respected and helps centralize the greedy logic.     greedy_actions = self._target_greedy_policy.action(         next_time_steps, dummy_state).action-+    greedy_actions = tf.nest.flatten(greedy_actions)[0]

revert this file please.

JacobHanouna

comment created time in 4 days

PullRequestEvent

pull request commenttensorflow/agents

added support for Dict action space

Ah I didn't see the changes to epsilon_greedy. That change is fine! Can you revert the change to DQN agent?

JacobHanouna

comment created time in 4 days

Pull request review commenttensorflow/tensorflow

Speedup

 def ExtractBitsFromFloat16(x):   def SlowAppendFloat16ArrayToTensorProto(tensor_proto, proto_values):-  tensor_proto.half_val.extend(-      [ExtractBitsFromFloat16(x) for x in proto_values])+  tensor_proto.half_val.extend(np.asarray(proto_values, dtype=np.float16).view(np.uint16))

you can put the test here:

https://github.com/tensorflow/tensorflow/blob/dd000bf0222797e71267e442e9aaa54e189e08cf/tensorflow/python/framework/tensor_util_test.py

see for example the function "testHalf".

SmokerX

comment created time in 4 days

Pull request review commenttensorflow/agents

Peter zhizhin: Added an option for PPO to train on incomplete episode

 def __init__(self,       reward_norm_clipping: Value above and below to clip normalized reward.       normalize_observations: If true, keeps moving mean and variance of         observations and normalizes incoming observations.+      train_on_partial_episodes: If True (default False), then data from

let's remove this arg and assume it's always True; per oscar's comment on original PR.

tfboyd

comment created time in 14 days

pull request commenttensorflow/agents

added support for Dict action space

(a good followup PR would be to disallow nested action_specs in this agent in the first place).

JacobHanouna

comment created time in 19 days

PR closed tensorflow/agents

Reviewers
added support for Dict action space cla: yes

Epsilon greedy policy error when generating new action from Dict Action space. #276

+6 -4

6 comments

2 changed files

JacobHanouna

pr closed time in 19 days

pull request commenttensorflow/agents

added support for Dict action space

We're trying to tighten up the contract that Agents present to the world, in particular we don't want to allow arbitrary nested structures "so long as its only got one entry", if the agent does not support more than one action in the action space. Better if you just add a wrapper that converts the tensor to a dict.

If we start allowing this kind of soft contract, it makes it much harder for future developers to know which agents really support multiple action spaces; and makes it harder for us to add pytype to tf-agents. Sorry; will have to reject this PR.

JacobHanouna

comment created time in 19 days

issue commentgrpc/grpc

Error assertion failed: g_state != nullptr

Followup: gpr_once is a pthread_once_t; which sounds similar to absl::once_flag; either way, it's not clear that these kinds of flags are safe to use when multiple DSOs link your library.

kanagunbr

comment created time in 19 days

issue commentgrpc/grpc

Error assertion failed: g_state != nullptr

The problem seems closely related to the ResolverRegistry having static state, essentially set at link-time:

https://github.com/grpc/grpc/blob/master/src/core/ext/filters/client_channel/resolver_registry.h#L34

And the various plugins being re-registered in .so because grpc_init is called e.g. here and the std::atomic (or whatever it is) g_basic_init is not shared by the different .so files.

kanagunbr

comment created time in 19 days

pull request commenttensorflow/agents

Added an option for PPO to train on incomplete episodes.

This looks great; I just want to see if we can always train from partial episodes. If so we can get rid of one of the init args.

PeterZhizhin

comment created time in 20 days

Pull request review commenttensorflow/agents

Added an option for PPO to train on incomplete episodes.

 def __init__(self,       reward_norm_clipping: Value above an below to clip normalized reward.       normalize_observations: If true, keeps moving mean and variance of         observations and normalizes incoming observations.+      train_on_partial_episodes: If True (default False), then data from

i wonder if we can just always train on partial episodes. @oars wdyt? can we get rid of this arg and always assume true?

PeterZhizhin

comment created time in 20 days

issue commenttensorflow/agents

observation_and_action_constraint_splitter behavior

@nealwu ptal.

abstractpaper

comment created time in a month

issue closedtensorflow/agents

we need install x11-utils for tutorials/1_dqn_tutorial.ipynb

Hi,

When I run the tutorials/1_dqn_tutorial.ipynb in colab(https://colab.research.google.com/), I found it always report errors that: no xdpyinfo found.

After I installled x11-utils in it, it works. So please change the first section of 1_dqn_tutorial.ipynb from:

# Note: If you haven't installed the following dependencies, run:
!apt-get install -y xvfb
!pip install 'gym==0.10.11'
!pip install 'imageio==2.4.0'
!pip install PILLOW
!pip install 'pyglet==1.3.2'
!pip install pyvirtualdisplay
!pip install --upgrade tensorflow_probability
!pip install tf-agents
try:
  %tensorflow_version 2.x
except:
  pass

To:

# Note: If you haven't installed the following dependencies, run:
!apt-get install x11-utils
!apt-get install -y xvfb
!pip install 'gym==0.10.11'
!pip install 'imageio==2.4.0'
!pip install PILLOW
!pip install 'pyglet==1.3.2'
!pip install pyvirtualdisplay
!pip install --upgrade tensorflow_probability
!pip install tf-agents
try:
  %tensorflow_version 2.x
except:
  pass

closed time in a month

clockzhong

issue commenttensorflow/tensorflow

Define variable shape at restore/load, allow direct restoring of variables prior to calling __build__ (non-lazy variable loading from checkpoint)

This is partly a question for the TF-Agents repo, and partly for Keras (CC @fchollet). On the TF-Agents side, it's true you need to do a forward pass to create the appropriate Variables. The easiest way to do this is to take your TF-Agents network objects, and call net.create_variables() on each of them (you may want to pass training=True to this call in case you have batch norm or dropout).

That takes care of feeding appropriate random inputs. If that doesn't work, you should file a bug in tensorflow/agents instead of this repo.

Regarding Keras models, we rely on the lower level Keras Network class to represent computation in TF-Agents, so I can't help much there. I hope @fchollet can discuss your ideas on variable initialization.

mjlbach

comment created time in a month

issue commenttensorflow/agents

DQN Agent Issue With Custom Environment

Does that resolve your issue?

IbraheemNofal

comment created time in a month

issue commenttensorflow/agents

How do I adjust ddpg example to fit my environment?

Were you able to resolve this issue?

mauricerad

comment created time in a month

issue closedtensorflow/agents

is TFUniformReplayBuffer thread safe?

I am planning to spawn a new thread (process) to collect data from environment during training. Is the replaybuffer thread safe?

closed time in a month

wsun66

issue commenttensorflow/agents

Wrapping a loaded policy model with PyTFPolicy

It's expecting batched inputs with batch_size 1. Try passing reward=[0], discount=[1.0] and (i think) np.zeros((1, 19)) as inputs.

gokhansolak

comment created time in a month

issue commenttensorflow/agents

Epsilon greedy policy error when generating new action from Dict Action space.

That line needs to be rewritten as:

action = tf.nest.map_structure(lambda g, r: tf.compat.v1.where(cond, g, r), greedy_action.action, random_action.action)

Report back and let us know if this works. We can patch it on our end.

JacobHanouna

comment created time in a month

issue commenttensorflow/agents

tf-agents SAC 10x slower than stable-baselines on same hardware

@tfboyd quick ping on this one.

pirobot

comment created time in a month

issue commenttensorflow/agents

is it possible to convert tf-agents to tf-lite and run on android device

Yes; you should be able to do this. I'm guessing you care about inference (running a policy) more than training (since tflite doesn't support that anyway).

See the PolicySaver class. You can use it to export a SavedModel. You can then use the TFLite converter to convert that SavedModel to a TFLite model.

Please report back and let us know if this works for you!

ssujr

comment created time in a month

issue commenttensorflow/agents

dqn_tutorial - initial_collect_steps is defined to be 1000 but not used anywhere...

Thanks for catching this. @adammichaelwood could you send a fix?

i guess we also need to set initial_collect_steps to 100 so the tutorial stays consistent.

guyk1971

comment created time in a month

issue commenttensorflow/agents

ParallelPyEnvironment doesn't work when subprocess calling a tensorflow model

The error is pretty opaque though, @oars is it possible there's an error message from the other process that we are missing when this is raised? and it gets hidden by this error?

LLouice

comment created time in a month

issue commenttensorflow/agents

ParallelPyEnvironment doesn't work when subprocess calling a tensorflow model

Since you're relying on tensorflow to perform the reward calculation, you get GIL-free multithreading for free. This means you should be able to get away with using the BatchedPyEnvironment - this uses python multithreading instead of multiprocessing. Since running TF calls releases the GIL, you won't get any advantage (and only overhead) from using the ParallelPyEnvironment.

LLouice

comment created time in a month

issue commenttensorflow/agents

Problem when running dqn on FrozenLake

Great; were you able to submit the PR? If so, we can close this bug.

guyk1971

comment created time in a month

issue closedtensorflow/agents

concatenation layer and replay buffers

Hi, I am trying to run a dqn agent with an environment where my observation_spec is a collection of specs (list or tuple). my q-network thus needs a preprocessing_combiner. First i chose tf.keras.layers.Concatenate(), which only works if my observation_spec is a list. However, when trying to implement a replaybuffer (as done in https://github.com/tensorflow/agents/blob/master/docs/tutorials/1_dqn_tutorial.ipynb), ReplayBuffer.as_dataset() tells me to use tuples for my spec collections.
My current solution is switching out the preprocessing_combiner to tf.keras.layers.Lambda(lambda x:tf.keras.layers.Concatenate()(list(x))), but this feels more like a workaround than a fix..

I would be surprised if i was the first one with this problem, unless i am doing something conceptually wrong?

closed time in a month

niklasnolte

issue commenttensorflow/agents

concatenation layer and replay buffers

Yes; it's a bit sad that we're forced to use tuples but that's a limitation of tf.data - arrays are not allowed because they get converted to tensors. Your other options are to add preprocessing_layers which are just tf.keras.layers.Layer() - this will be an identity pass-through, and the resulting tensors passed to the combiner will be an actual list.

Let me know if this doesn't solve the problem.

niklasnolte

comment created time in a month

issue commenttensorflow/agents

TRAIN TF-AGENTS WITH MULTIPLE GPUs

@oars any advice for how to make the MirroredStrategy work here?

i wonder if you need to build the iterator inside the MirroredStrategy.

JCMiles

comment created time in a month

issue commenttensorflow/agents

we need install x11-utils for tutorials/1_dqn_tutorial.ipynb

@adammichaelwood ptal.

clockzhong

comment created time in a month

issue commenttensorflow/agents

Somthing wrong in tutorials/4_drivers_tutorial.ipynb

What version of tensorflow_probability do you have installed? @kbanoop can you take a look at the metrics issue?

clockzhong

comment created time in a month

issue commenttensorflow/tensorflow

Gradient computation occupies too much memories in "cnn (using while_loop) + lstm" network

Gradient of tf.gather has the same exact issues except when you're gathering from a tf.Variable directly. Which you're probably not.

On Tue, Jan 21, 2020 at 8:43 PM Max Shek-wai Chu notifications@github.com wrote:

Both tf.slice and the indexing operator take a big matrix and take out a chunk which is smaller. Since the output of the backprop of this operation must have the same shape as the original big matrix, it means it takes the gradient of the slice and pads it with zeros so it's the same shape as the input into the slice/indexing operator. Do this once for each row of the original matrix and you've turned your problem into O(n^2) compute and memory (sometimes O(n) memory if you compile with XLA). TensorArray avoids this by first slicing your big input tensor into rows, and forward and backprop shapes are the shapes of the rows of your original tensor. So you should try to use TensorArray when you're iterating over slices of any tensor, especially if performing backprop into those rows. … <#m_-8721515996843760003_> On Fri, Jan 17, 2020 at 10:36 PM Max Shek-wai Chu @.***> wrote: btw; one of the main reasons your memory is blowing up may be this line: single_output = vgg_m(input_tensor[:, t, :, :, :], reuse=True) transpose input_tensor to time, major, unstack it into a TensorArray, and use .read(t) on the TensorArray. this should significantly reduce memory usage if you backprop into input_tensor. I had the same problem but i dont know why it will make the memory explode at compute_gradients or minimize. Can you explain why it will happen? Also, can i use tf.slice instead of the indexing operator? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#15041 https://github.com/tensorflow/tensorflow/issues/15041?email_source=notifications&email_token=AANWFG2U6TK4EJAI7XGG4EDQ6KPQNA5CNFSM4EGJ2JQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJJRYJA#issuecomment-575872036>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFGYSKB7VA3B3TUJK5CLQ6KPQNANCNFSM4EGJ2JQA .

Also, how about tf.gather() instead of TensorArray?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/15041?email_source=notifications&email_token=AANWFGZ6QS7XNTLQBQUKZ2LQ67FI5A5CNFSM4EGJ2JQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJSGLVY#issuecomment-577005015, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFG4Z36ZBSZW5E5MZIF3Q67FI5ANCNFSM4EGJ2JQA .

MeteorsHub

comment created time in a month

issue openedbazelbuild/bazel

Unclear error in remote builds when authentication hasn't been properly set up

ATTENTION! Please read and follow:

  • if this is a question about how to build / test / query / deploy using Bazel, or a discussion starter, send it to bazel-discuss@googlegroups.com
  • if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

Bazel errors out with the opaque message

ERROR: Error initializing RemoteModule

When performing a remote build, if I haven't set up authentication properly. In particular, in my case I needed to run:

gcloud auth application-default login

to set up a default application auth, before I was able to progress in my testing.

What operating system are you running Bazel on?

linux

What's the output of bazel info release?

release 1.2.1

Have you found anything relevant by searching the web?

some details on how to get verbose logging out of the bazel build, but nothing that worked.

created time in a month

issue commenttensorflow/tensorflow

saved_model_cli breaks nightly packages

@byronyi let me know if this doesn't work for you.

byronyi

comment created time in a month

issue commenttensorflow/tensorflow

saved_model_cli breaks nightly packages

I have a solution; gonna get it through internal reviews and then will ping you when it's available.

byronyi

comment created time in a month

issue commenttensorflow/tensorflow

saved_model_cli breaks nightly packages

I think we can move pywrap_tfcompile to python/

byronyi

comment created time in a month

issue commenttensorflow/tensorflow

Gradient computation occupies too much memories in "cnn (using while_loop) + lstm" network

Both tf.slice and the indexing operator take a big matrix and take out a chunk which is smaller. Since the output of the backprop of this operation must have the same shape as the original big matrix, it means it takes the gradient of the slice and pads it with zeros so it's the same shape as the input into the slice/indexing operator. Do this once for each row of the original matrix and you've turned your problem into O(n^2) compute and memory (sometimes O(n) memory if you compile with XLA). TensorArray avoids this by first slicing your big input tensor into rows, and forward and backprop shapes are the shapes of the rows of your original tensor. So you should try to use TensorArray when you're iterating over slices of any tensor, especially if performing backprop into those rows.

On Fri, Jan 17, 2020 at 10:36 PM Max Shek-wai Chu notifications@github.com wrote:

btw; one of the main reasons your memory is blowing up may be this line:

single_output = vgg_m(input_tensor[:, t, :, :, :], reuse=True)

transpose input_tensor to time, major, unstack it into a TensorArray, and use .read(t) on the TensorArray. this should significantly reduce memory usage if you backprop into input_tensor.

I had the same problem but i dont know why it will make the memory explode at compute_gradients or minimize. Can you explain why it will happen? Also, can i use tf.slice instead of the indexing operator?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/15041?email_source=notifications&email_token=AANWFG2U6TK4EJAI7XGG4EDQ6KPQNA5CNFSM4EGJ2JQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJJRYJA#issuecomment-575872036, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFGYSKB7VA3B3TUJK5CLQ6KPQNANCNFSM4EGJ2JQA .

MeteorsHub

comment created time in a month

pull request commenttensorflow/tensorflow

[ROCm] Fix for the ROCm CSB breakage - 200113 - 1

I prefer just the change to types.cc file, but am OK with the fix in both places.

deven-amd

comment created time in a month

issue commenttensorflow/tensorflow

No example provided for using tf.nn.ctc_loss

Are you using GPU? I believe the latter approach will work if you use densified labels.

On Sat, Dec 28, 2019, 11:45 AM Gabriel Meseguer-Brocal < notifications@github.com> wrote:

Do you mean something like this (my_ctc_loss returns a scalar -tf.reduce_mean- not a tensor)?

def my_ctc_loss( labels, logits, label_length, logit_length, logits_time_major, blank_index, sample_weight ): return tf.reduce_mean(tf.nn.ctc_loss( labels=labels, logits=logits, label_length=label_length, logit_length=logit_length, logits_time_major=logits_time_major, blank_index=blank_index ))

INPUTS

inputs = Input(shape=[128, 64, 1], batch_size=32) # [frames, num_labels, channels] labels = Input(shape=[128], batch_size=32, dtype=tf.int32) label_length = tf.constant(np.ones((32)), dtype=tf.int32) logit_length = tf.constant(np.ones((32)), dtype=tf.int32)

MODEL

x = Conv2D(1, kernel_size=(5, 5), padding='same')(inputs) logits = Lambda(lambda z: tf.squeeze(z, [-1]))(x) model = Model(inputs, logits)

LOSSS

ctc_loss_emb = functools.partial( my_ctc_loss, labels, # labels logits, # logits label_length, # label_length logit_length, # logit_length False, # logits_time_major -1, # blank_index None, # sample_weight ) model.compile(optimizer=Adam(lr=0.001), loss=ctc_loss_emb)

Not working either:


TypeError Traceback (most recent call last) <ipython-input-3-d100089d6156> in <module> 20 None, # sample_weight 21 ) ---> 22 model.compile(optimizer=Adam(lr=0.001), loss=ctc_loss_emb)

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs) 455 self._self_setattr_tracking = False # pylint: disable=protected-access 456 try: --> 457 result = method(self, *args, **kwargs) 458 finally: 459 self._self_setattr_tracking = previous_value # pylint: disable=protected-access

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, distribute, **kwargs) 371 372 # Creates the model loss and weighted metrics sub-graphs. --> 373 self._compile_weights_loss_and_weighted_metrics() 374 375 # Functions for train, test and predict will

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs) 455 self._self_setattr_tracking = False # pylint: disable=protected-access 456 try: --> 457 result = method(self, *args, **kwargs) 458 finally: 459 self._self_setattr_tracking = previous_value # pylint: disable=protected-access

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in _compile_weights_loss_and_weighted_metrics(self, sample_weights) 1651 # loss_weight_2 * output_2_loss_fn(...) + 1652 # layer losses. -> 1653 self.total_loss = self._prepare_total_loss(masks) 1654 1655 def _prepare_skip_target_masks(self):

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in _prepare_total_loss(self, masks) 1732 # differentiate between use case where a custom optimizer 1733 # expects a vector loss value vs unreduced per-sample loss value. -> 1734 output_loss = loss_fn(y_true, y_pred, sample_weight=sample_weight) 1735 loss_reduction = losses_utils.ReductionV2.SUM_OVER_BATCH_SIZE 1736

TypeError: my_ctc_loss() got multiple values for argument 'sample_weight'

I've tried this:

@tf.function(autograph=False) def sparse_tuple_from(data): def py_sparse_tuple_from(sequences): """ """ indices = [] values = [] for n, seq in enumerate(sequences): indices.extend(zip([n] * len(seq), range(len(seq)))) values.extend(seq) indices = np.asarray(indices, dtype=np.int64) values = np.asarray(values, dtype=np.int32) shape = np.asarray( [len(sequences), np.asarray(indices).max(0)[1] + 1], dtype=np.int64) return indices, values, shape return tf.py_function( py_sparse_tuple_from, [data], (tf.int64, tf.int32, tf.int64))

class MyCTCLoss(tensorflow.losses.Loss): def init(self, logit_length, blank_index=-1, logits_time_major=False): super(MyCTCLoss, self).init() self.logit_length = logit_length self.blank_index = blank_index self.logits_time_major = logits_time_major

def call(self, y_true, y_pred):
    indices, values, shape = sparse_tuple_from(y_true)
    labels_ctc = tf.sparse.SparseTensor(
        indices=indices, values=values, dense_shape=shape)
    return tf.reduce_mean(tf.nn.ctc_loss(
        labels=labels_ctc, logits=y_pred, label_length=None,
        logit_length=logit_length,
        logits_time_major=self.logits_time_major,
        blank_index=self.blank_index
    ))

my_loss = MyCTCLoss(32) # 32 batch size model.compile(optimizer=Adam(lr=0.001), loss=my_loss)

Which at least compiles but not sure if trains or not. Additionally, as far as I understand the doc https://www.tensorflow.org/api_docs/python/tf/nn/ctc_loss?version=stable, using a SparseTensor force the loss to be computed in the CPU not the GPU:

  • On TPU and GPU: Only dense padded labels are supported.
  • On CPU: Caller may use SparseTensor or dense padded labels but calling with a SparseTensor will be significantly faster.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/35063?email_source=notifications&email_token=AANWFGYKXRCPYNJFEBHYCRLQ26UEXA5CNFSM4J2DC2K2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYQ3ZQ#issuecomment-569445862, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFG5FVJ7KMJ4VEETJKJTQ26UEXANCNFSM4J2DC2KQ .

gabolsgabs

comment created time in 2 months

issue commenttensorflow/tensorflow

No example provided for using tf.nn.ctc_loss

Hmm; good point. You could write a loss function that calls ctc_loss, accepts an extra sample_weights argument (ignores it), and returns the reduce_sum(ctc_loss(...)); and pass that as the loss to compile.

On Sat, Dec 28, 2019 at 10:48 AM Gabriel Meseguer-Brocal < notifications@github.com> wrote:

Do you mean something like?

def ctc_layer(labels, label_length, logit_length, blank_index=-1, logits_time_major=False): def func(x): return tf.nn.ctc_loss( logits=x, labels=labels, label_length=label_length, logit_length=logit_length, blank_index=blank_index, logits_time_major=logits_time_major) return Lambda(func)

INPUTS

inputs = Input(shape=[128, 64, 1], batch_size=32) # [frames, num_labels, channels] labels = Input(shape=[128], batch_size=32, dtype=tf.int32) label_length = tf.constant(np.ones((32)), dtype=tf.int32) logit_length = tf.constant(np.ones((32)), dtype=tf.int32)

MODEL

x = Conv2D(1, kernel_size=(5, 5), padding='same')(inputs) logits = Lambda(lambda z: tf.squeeze(z, [-1]))(x) x = ctc_layer(labels, label_length, logit_length)(logits) model = Model(inputs, x) model.summary()


Layer (type) Output Shape Param #

input_13 (InputLayer) [(32, 128, 64, 1)] 0


conv2d_6 (Conv2D) (32, 128, 64, 1) 26


lambda_10 (Lambda) (32, 128, 64) 0


lambda_11 (Lambda) (32,) 0

Total params: 26 Trainable params: 26 Non-trainable params: 0


But then, in the compiler I don't know what to use as loss. I've checked the doc https://www.tensorflow.org/api_docs/python/tf/keras/losses and all the losses need a y_true (labels in the ctc_loss and already included in the ctc_layer). I tried with just reducing the output tensor with:

model.compile(optimizer=Adam(lr=0.001), loss=tf.reduce_mean) model.compile(optimizer=Adam(lr=0.001), loss=tf.reduce_sum)

But as expected is not working:


TypeError Traceback (most recent call last) <ipython-input-26-3091b5b8ce35> in <module> ----> 1 model.compile(optimizer=Adam(lr=0.001), loss=tf.reduce_sum)

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs) 455 self._self_setattr_tracking = False # pylint: disable=protected-access 456 try: --> 457 result = method(self, *args, **kwargs) 458 finally: 459 self._self_setattr_tracking = previous_value # pylint: disable=protected-access

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, distribute, **kwargs) 371 372 # Creates the model loss and weighted metrics sub-graphs. --> 373 self._compile_weights_loss_and_weighted_metrics() 374 375 # Functions for train, test and predict will

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/training/tracking/base.py in _method_wrapper(self, *args, **kwargs) 455 self._self_setattr_tracking = False # pylint: disable=protected-access 456 try: --> 457 result = method(self, *args, **kwargs) 458 finally: 459 self._self_setattr_tracking = previous_value # pylint: disable=protected-access

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in _compile_weights_loss_and_weighted_metrics(self, sample_weights) 1651 # loss_weight_2 * output_2_loss_fn(...) + 1652 # layer losses. -> 1653 self.total_loss = self._prepare_total_loss(masks) 1654 1655 def _prepare_skip_target_masks(self):

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in _prepare_total_loss(self, masks) 1711 1712 if hasattr(loss_fn, 'reduction'): -> 1713 per_sample_losses = loss_fn.call(y_true, y_pred) 1714 weighted_losses = losses_utils.compute_weighted_loss( 1715 per_sample_losses,

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/keras/losses.py in call(self, y_true, y_pred) 219 y_pred, y_true = tf_losses_util.squeeze_or_expand_dimensions( 220 y_pred, y_true) --> 221 return self.fn(y_true, y_pred, **self._fn_kwargs) 222 223 def get_config(self):

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py in wrapper(*args, **kwargs) 178 """Call target, and fall back on dispatchers if there is a TypeError.""" 179 try: --> 180 return target(*args, **kwargs) 181 except (TypeError, ValueError): 182 # Note: convert_to_eager_tensor currently raises a ValueError, not a

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py in reduce_sum(input_tensor, axis, keepdims, name) 1573 gen_math_ops._sum( 1574 input_tensor, _ReductionDims(input_tensor, axis), keepdims, -> 1575 name=name)) 1576 1577

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py in _sum(input, axis, keep_dims, name) 11166 _, _, _op = _op_def_lib._apply_op_helper( 11167 "Sum", input=input, reduction_indices=axis, keep_dims=keep_dims,

11168 name=name) 11169 _result = _op.outputs[:] 11170 _inputs_flat = _op.inputs

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords) 629 _SatisfiesTypeConstraint(base_type, 630 _Attr(op_def, input_arg.type_attr), --> 631 param_name=input_name) 632 attrs[input_arg.type_attr] = attr_value 633 inferred_from[input_arg.type_attr] = input_name

~/.virtualenvs/phd/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py in _SatisfiesTypeConstraint(dtype, attr_def, param_name) 58 "allowed values: %s" % 59 (param_name, dtypes.as_dtype(dtype).name, ---> 60 ", ".join(dtypes.as_dtype(x).name for x in allowed_list))) 61 62

TypeError: Value passed to parameter 'reduction_indices' has DataType float32 not in list of allowed values: int32, int64

Any other suggestion? Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/35063?email_source=notifications&email_token=AANWFG37P4XUGSO7MGDJJVDQ26NPFA5CNFSM4J2DC2K2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYP5OI#issuecomment-569441977, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFGYXHMX3IPPG76YP7KTQ26NPFANCNFSM4J2DC2KQ .

gabolsgabs

comment created time in 2 months

issue commenttensorflow/tensorflow

No example provided for using tf.nn.ctc_loss

Can you try it as part of a Lambda layer and use a loss with compile that does a reduce_sum? CTC loss is not a keras loss and returns a per batch entry loss vector.

On Mon, Dec 23, 2019, 3:28 PM Yasir Modak notifications@github.com wrote:

Assigned #35063 https://github.com/tensorflow/tensorflow/issues/35063 to @ebrevdo https://github.com/ebrevdo.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/35063?email_source=notifications&email_token=AANWFGYWVADJ7MAWKQMUZODQ2FCRTA5CNFSM4J2DC2K2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOVUZUULY#event-2905819695, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFG6ZWV756Z7SSORERE3Q2FCRTANCNFSM4J2DC2KQ .

gabolsgabs

comment created time in 2 months

Pull request review commenttensorflow/community

RFC: Keras categorical inputs

+# Keras categorical inputs++| Status        | Proposed                                             |+:-------------- |:---------------------------------------------------- |+| **Author(s)** | Zhenyu Tan (tanzheny@google.com), Francois Chollet (fchollet@google.com)|+| **Sponsor**   | Karmel Allison (karmel@google.com), Martin Wicke (wicke@google.com) |+| **Updated**   | 2019-12-12                                           |++## Objective++This document proposes 4 new preprocessing Keras layers (`CategoryLookup`, `CategoryCrossing`, `CategoryEncoding`, `CategoryHashing`), and 1 additional op (`to_sparse`) to allow users to:+* Perform feature engineering for categorical inputs+* Replace feature columns and `tf.keras.layers.DenseFeatures` with proposed layers+* Introduce sparse inputs that work with Keras linear models and other layers that support sparsity++Other proposed layers for replacement of feature columns such as `tf.feature_column.bucketized_column` and `tf.feature_column.numeric_column` has been discussed [here](https://github.com/keras-team/governance/blob/master/rfcs/20190502-preprocessing-layers.md) and are not the focus of this document.++## Motivation++Specifically, by introducing the 4 layers, we aim to address these pain points:+* Users have to define both feature columns and Keras Inputs for the model, resulting in code duplication and deviation from DRY (Do not repeat yourself) principle. See this [Github issue](https://github.com/tensorflow/tensorflow/issues/27416).+* Users with large dimension categorical inputs will incur large memory footprint and computation cost, if wrapped with indicator column through `tf.keras.layers.DenseFeatures`.+* Currently there is no way to correctly feed Keras linear model or dense layer with multivalent categorical inputs or weighted categorical inputs.++## User Benefit++We expect to get rid of the user painpoints once migrating off feature columns.++## Example Workflows++Two example workflows are presented below. These workflows can be found at this [colab](https://colab.sandbox.google.com/drive/1cEJhSYLcc2MKH7itwcDvue4PfvrLN-OR#scrollTo=22sa0D19kxXY).++### Workflow 1++The first example gives an equivalent code snippet to canned `LinearEstimator` [tutorial](https://www.tensorflow.org/tutorials/estimator/linear) on the Titanic dataset:++```python+dftrain = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/train.csv')+y_train = dftrain.pop('survived')++CATEGORICAL_COLUMNS = ['sex', 'n_siblings_spouses', 'parch', 'class', 'deck', 'embark_town', 'alone']+NUMERICAL_COLUMNS = ['age', 'fare']+# input list to create functional model.+model_inputs = []+# input list to feed linear model.+linear_inputs = []+for feature_name in CATEGORICAL_COLUMNS:+	feature_input = tf.keras.Input(shape=(1,), dtype=tf.string, name=feature_name, sparse=True)+	vocab_list = sorted(dftrain[feature_name].unique())+	# Map string values to indices+	x = tf.keras.layers.CategoryLookup(vocabulary=vocab_list, name=feature_name)(feature_input)+  x = tf.keras.layers.CategoryEncoding(num_categories=len(vocab_list))(x)+	linear_inputs.append(x)+	model_inputs.append(feature_input)++for feature_name in NUMERICAL_COLUMNS:+	feature_input = tf.keras.Input(shape=(1,), name=feature_name)+	linear_inputs.append(feature_input)+	model_inputs.append(feature_input)++linear_model = tf.keras.experimental.LinearModel(units=1)+linear_logits = linear_model(linear_inputs)+model = tf.keras.Model(model_inputs, linear_logits)++model.compile('sgd', loss=tf.keras.losses.BinaryCrossEntropy(from_logits=True), metrics=['accuracy'])++dataset = tf.data.Dataset.from_tensor_slices((+	(tf.to_sparse(dftrain.sex, "Unknown"), tf.to_sparse(dftrain.n_siblings_spouses, -1),+	tf.to_sparse(dftrain.parch, -1), tf.to_sparse(dftrain['class'], "Unknown"), tf.to_sparse(dftrain.deck, "Unknown"),+	tf.expand_dims(dftrain.age, axis=1), tf.expand_dims(dftrain.fare, axis=1)),+	y_train)).batch(bach_size).repeat(n_epochs)++model.fit(dataset)+```++### Workflow 2++The second example gives an instruction on how to transition from categorical feature columns to the proposed layers. Note that one difference for vocab categorical column is that, instead of providing a pair of mutually exclusive `default_value` and `num_oov_buckets` where `default_value` represents the value to map input to given out-of-vocab value, and `num_oov_buckets` represents value range of [len(vocab), len(vocab)+num_oov_buckets) to map input to from a hashing function given out-of-vocab value. In practice, we believe out-of-vocab values should be mapped to the head, i.e., [0, num_oov_tokens), and in-vocab values should be mapped to [num_oov_tokens, num_oov_tokens+len(vocab)).++1. Categorical vocab list column++Original:+```python+fc = tf.feature_column.categorical_feature_column_with_vocabulary_list(+	   key, vocabulary_list, dtype, default_value, num_oov_buckets)+```+Proposed:+```python+x = tf.keras.Input(shape=(1,), name=key, dtype=dtype)+layer = tf.keras.layers.CategoryLookup(+            vocabulary=vocabulary_list, num_oov_tokens=num_oov_buckets)+out = layer(x)+```++2. categorical vocab file column++Original:+```python+fc = tf.feature_column.categorical_column_with_vocab_file(+       key, vocabulary_file, vocabulary_size, dtype,+       default_value, num_oov_buckets)+```+Proposed:+```python+x = tf.keras.Input(shape=(1,), name=key, dtype=dtype)+layer = tf.keras.layers.CategoryLookup(

Ah I see, so here you allow the user to provide a vocabulary file directly, so the tf.data example may not be necessary. May still be useful if users have vocab that needs to be munged a bit before reading directly. But less important.

tanzhenyu

comment created time in 2 months

Pull request review commenttensorflow/community

RFC: Keras categorical inputs

+# Keras categorical inputs++| Status        | Proposed                                             |+:-------------- |:---------------------------------------------------- |+| **Author(s)** | Zhenyu Tan (tanzheny@google.com), Francois Chollet (fchollet@google.com)|+| **Sponsor**   | Karmel Allison (karmel@google.com), Martin Wicke (wicke@google.com) |+| **Updated**   | 2019-12-12                                           |++## Objective++This document proposes 4 new preprocessing Keras layers (`CategoryLookup`, `CategoryCrossing`, `CategoryEncoding`, `CategoryHashing`), and 1 additional op (`to_sparse`) to allow users to:+* Perform feature engineering for categorical inputs+* Replace feature columns and `tf.keras.layers.DenseFeatures` with proposed layers+* Introduce sparse inputs that work with Keras linear models and other layers that support sparsity++Other proposed layers for replacement of feature columns such as `tf.feature_column.bucketized_column` and `tf.feature_column.numeric_column` has been discussed [here](https://github.com/keras-team/governance/blob/master/rfcs/20190502-preprocessing-layers.md) and are not the focus of this document.++## Motivation++Specifically, by introducing the 4 layers, we aim to address these pain points:+* Users have to define both feature columns and Keras Inputs for the model, resulting in code duplication and deviation from DRY (Do not repeat yourself) principle. See this [Github issue](https://github.com/tensorflow/tensorflow/issues/27416).+* Users with large dimension categorical inputs will incur large memory footprint and computation cost, if wrapped with indicator column through `tf.keras.layers.DenseFeatures`.+* Currently there is no way to correctly feed Keras linear model or dense layer with multivalent categorical inputs or weighted categorical inputs.++## User Benefit++We expect to get rid of the user painpoints once migrating off feature columns.++## Example Workflows++Two example workflows are presented below. These workflows can be found at this [colab](https://colab.sandbox.google.com/drive/1cEJhSYLcc2MKH7itwcDvue4PfvrLN-OR#scrollTo=22sa0D19kxXY).++### Workflow 1

Would it make sense to provide an example workflow where you have to get the vocabulary from e.g. a csv file using tf.data and in particular, tf.data.experimental.unique and tf.data.experimental.get_single_element to read out the tensor? @jsimsa wdyt?

This is gonna be very common, i think, in real use cases.

tanzhenyu

comment created time in 2 months

Pull request review commenttensorflow/community

RFC: Keras categorical inputs

+# Keras categorical inputs++| Status        | Proposed                                             |+:-------------- |:---------------------------------------------------- |+| **Author(s)** | Zhenyu Tan (tanzheny@google.com), Francois Chollet (fchollet@google.com)|+| **Sponsor**   | Karmel Allison (karmel@google.com), Martin Wicke (wicke@google.com) |+| **Updated**   | 2019-12-12                                           |++## Objective++This document proposes 4 new preprocessing Keras layers (`CategoryLookup`, `CategoryCrossing`, `CategoryEncoding`, `CategoryHashing`), and 1 additional op (`to_sparse`) to allow users to:+* Perform feature engineering for categorical inputs+* Replace feature columns and `tf.keras.layers.DenseFeatures` with proposed layers+* Introduce sparse inputs that work with Keras linear models and other layers that support sparsity++Other proposed layers for replacement of feature columns such as `tf.feature_column.bucketized_column` and `tf.feature_column.numeric_column` has been discussed [here](https://github.com/keras-team/governance/blob/master/rfcs/20190502-preprocessing-layers.md) and are not the focus of this document.++## Motivation++Specifically, by introducing the 4 layers, we aim to address these pain points:+* Users have to define both feature columns and Keras Inputs for the model, resulting in code duplication and deviation from DRY (Do not repeat yourself) principle. See this [Github issue](https://github.com/tensorflow/tensorflow/issues/27416).+* Users with large dimension categorical inputs will incur large memory footprint and computation cost, if wrapped with indicator column through `tf.keras.layers.DenseFeatures`.+* Currently there is no way to correctly feed Keras linear model or dense layer with multivalent categorical inputs or weighted categorical inputs.++## User Benefit++We expect to get rid of the user painpoints once migrating off feature columns.++## Example Workflows++Two example workflows are presented below. These workflows can be found at this [colab](https://colab.sandbox.google.com/drive/1cEJhSYLcc2MKH7itwcDvue4PfvrLN-OR#scrollTo=22sa0D19kxXY).++### Workflow 1++The first example gives an equivalent code snippet to canned `LinearEstimator` [tutorial](https://www.tensorflow.org/tutorials/estimator/linear) on the Titanic dataset:++```python+dftrain = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/train.csv')+y_train = dftrain.pop('survived')++CATEGORICAL_COLUMNS = ['sex', 'n_siblings_spouses', 'parch', 'class', 'deck', 'embark_town', 'alone']+NUMERICAL_COLUMNS = ['age', 'fare']+# input list to create functional model.+model_inputs = []+# input list to feed linear model.+linear_inputs = []+for feature_name in CATEGORICAL_COLUMNS:+	feature_input = tf.keras.Input(shape=(1,), dtype=tf.string, name=feature_name, sparse=True)+	vocab_list = sorted(dftrain[feature_name].unique())+	# Map string values to indices+	x = tf.keras.layers.CategoryLookup(vocabulary=vocab_list, name=feature_name)(feature_input)+  x = tf.keras.layers.CategoryEncoding(num_categories=len(vocab_list))(x)+	linear_inputs.append(x)+	model_inputs.append(feature_input)++for feature_name in NUMERICAL_COLUMNS:+	feature_input = tf.keras.Input(shape=(1,), name=feature_name)+	linear_inputs.append(feature_input)+	model_inputs.append(feature_input)++linear_model = tf.keras.experimental.LinearModel(units=1)+linear_logits = linear_model(linear_inputs)+model = tf.keras.Model(model_inputs, linear_logits)++model.compile('sgd', loss=tf.keras.losses.BinaryCrossEntropy(from_logits=True), metrics=['accuracy'])++dataset = tf.data.Dataset.from_tensor_slices((+	(tf.to_sparse(dftrain.sex, "Unknown"), tf.to_sparse(dftrain.n_siblings_spouses, -1),+	tf.to_sparse(dftrain.parch, -1), tf.to_sparse(dftrain['class'], "Unknown"), tf.to_sparse(dftrain.deck, "Unknown"),+	tf.expand_dims(dftrain.age, axis=1), tf.expand_dims(dftrain.fare, axis=1)),+	y_train)).batch(bach_size).repeat(n_epochs)++model.fit(dataset)+```++### Workflow 2++The second example gives an instruction on how to transition from categorical feature columns to the proposed layers. Note that one difference for vocab categorical column is that, instead of providing a pair of mutually exclusive `default_value` and `num_oov_buckets` where `default_value` represents the value to map input to given out-of-vocab value, and `num_oov_buckets` represents value range of [len(vocab), len(vocab)+num_oov_buckets) to map input to from a hashing function given out-of-vocab value. In practice, we believe out-of-vocab values should be mapped to the head, i.e., [0, num_oov_tokens), and in-vocab values should be mapped to [num_oov_tokens, num_oov_tokens+len(vocab)).++1. Categorical vocab list column++Original:+```python+fc = tf.feature_column.categorical_feature_column_with_vocabulary_list(+	   key, vocabulary_list, dtype, default_value, num_oov_buckets)+```+Proposed:+```python+x = tf.keras.Input(shape=(1,), name=key, dtype=dtype)+layer = tf.keras.layers.CategoryLookup(+            vocabulary=vocabulary_list, num_oov_tokens=num_oov_buckets)+out = layer(x)+```++2. categorical vocab file column++Original:+```python+fc = tf.feature_column.categorical_column_with_vocab_file(+       key, vocabulary_file, vocabulary_size, dtype,+       default_value, num_oov_buckets)+```+Proposed:+```python+x = tf.keras.Input(shape=(1,), name=key, dtype=dtype)+layer = tf.keras.layers.CategoryLookup(+            vocabulary=vocabulary_file, num_oov_tokens=num_oov_buckets)+out = layer(x)+```+Note: `vocabulary_size` is only valid if `adapt` is called. Otherwise if user desires to lookup for the first K vocabularies in vocab file, then shrink the vocab file by only having the first K lines.++3. categorical hash column++Original:+```python+fc = tf.feature_column.categorical_column_with_hash_bucket(+       key, hash_bucket_size, dtype)+```+Proposed:+```python+x = tf.keras.Input(shape=(1,), name=key, dtype=dtype)+layer = tf.keras.layers.CategoryHashing(num_bins=hash_bucket_size)+out = layer(x)+```++4. categorical identity column++Original:+```python+fc = tf.feature_column.categorical_column_with_identity(+       key, num_buckets, default_value)+```+Proposed:+```python+x = tf.keras.Input(shape=(1,), name=key, dtype=dtype)+layer = tf.keras.layers.Lambda(lambda x: tf.where(tf.logical_or(x < 0, x > num_buckets), tf.fill(dims=tf.shape(x), value=default_value), x))+out = layer(x)+```++5. cross column++Original:+```python+fc_1 = tf.feature_column.categorical_column_with_vocabulary_list(key_1, vocabulary_list, +         dtype, default_value, num_oov_buckets)+fc_2 = tf.feature_column.categorical_column_with_hash_bucket(key_2, hash_bucket_size,+         dtype)+fc = tf.feature_column.crossed_column([fc_1, fc_2], hash_bucket_size, hash_key)+```+Proposed:+```python+x1 = tf.keras.Input(shape=(1,), name=key_1, dtype=dtype)+x2 = tf.keras.Input(shape=(1,), name=key_2, dtype=dtype)+layer1 = tf.keras.layers.CategoryLookup(+           vocabulary=vocabulary_list,  +           num_oov_tokens=num_oov_buckets)+x1 = layer1(x1)+layer2 = tf.keras.layers.CategoryHashing(+           num_bins=hash_bucket_size)+x2 = layer2(x2)+layer = tf.keras.layers.CategoryCrossing(num_bins=hash_bucket_size)+out = layer([x1, x2])+```++6. weighted categorical column++Original:+```python+fc = tf.feature_column.categorical_column_with_vocab_list(key, vocabulary_list,+         dtype, default_value, num_oov_buckets)+weight_fc = tf.feature_column.weighted_categorical_column(fc, weight_feature_key, +         dtype=weight_dtype)+linear_model = tf.estimator.LinearClassifier(units, feature_columns=[weight_fc])+```+Proposed:+```python+x1 = tf.keras.Input(shape=(1,), name=key, dtype=dtype)+x2 = tf.keras.Input(shape=(1,), name=weight_feature_key, dtype=weight_dtype)+layer = tf.keras.layers.CategoryLookup(+           vocabulary=vocabulary_list,   +           num_oov_tokens=num_oov_buckets)+x1 = layer(x1)+x = tf.keras.layers.CategoryEncoding(num_categories=len(vocabulary_list)+num_oov_buckets)([x1, x2])+linear_model = tf.keras.premade.LinearModel(units)+linear_logits = linear_model(x)+```++## Design Proposal+We propose a CategoryLookup layer to replace `tf.feature_column.categorical_column_with_vocabulary_list` and `tf.feature_column.categorical_column_with_vocabulary_file`, a `CategoryHashing` layer to replace `tf.feature_column.categorical_column_with_hash_bucket`, a `CategoryCrossing` layer to replace `tf.feature_column.crossed_column`, and another `CategoryEncoding` layer to convert the sparse input to the format required by linear models.++```python+`tf.keras.layers.CategoryLookup`+CategoryLookup(PreprocessingLayer):

Please don't forget to implement the correct compute_output_signature for these classes, since they will accept SparseTensorSpecs, and must emit SparseTensorSpecs in this case.

tanzhenyu

comment created time in 2 months

pull request commenttensorflow/agents

Support for 'training' flag (for dropout) in DQN Agent.

I'm adding this to all agents, after which perhaps you could rebase and add in your unit tests?

mhe500

comment created time in 2 months

issue commenttensorflow/agents

DQN Agent Issue With Custom Environment

Have you tried ensuring that your observations are properly batched? If right now you're using a single py environment, and wrapping it in a TFPyEnvironment, try adding a batch dimension: first wrap the py environment in a batched env: py_env = BatchedPyEnvironment([py_env]).

IbraheemNofal

comment created time in 2 months

pull request commenttensorflow/tensorflow

Implement Hessian for sparse softmax cross entropy

Yes; this is the problem I ran into :( Serializing custom gradients (as tf.functions?) would be required to do this; we don't currently have a design for this. If it's something you'd like to work on then I believe TF team would welcome a short design.

On Tue, Dec 10, 2019 at 2:32 PM MichaelKonobeev notifications@github.com wrote:

I noticed that when SparseSoftmaxCrossEntropyWithLogits from nn_ops_grad.py is removed, using tf.custom_gradient and then wrapping and decorating the function with tf.function causes an exception if working with persistent tape. Consider the following code:

@tf.custom_gradient def xent(logits, labels): loss, grad = gen_nn_ops.sparse_softmax_cross_entropy_with_logits( logits, labels) def grad_fn(dy): return tf.expand_dims(dy, -1) * grad, None return loss, grad_fn

@tf.function def function(logits, lables): return xent(logits, labels)

logits = tf.Variable([[1., 2.]]) labels = tf.Variable([1]) module = Module() with tf.GradientTape(persistent=True) as tape: loss = function(logits, labels)

grad = tape.gradient(loss, logits)

This will lead to LookupError from here https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/gradients_util.py#L623. If the tape is not persistent or xent is called directly, then it works fine. Is this expected behavior? Seems like it is necessary to support this case before moving to savable models as they work with tf.functions.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/pull/31700?email_source=notifications&email_token=AANWFGZRHSZ4DXY64RNIORDQYAKGLA5CNFSM4IMLTVRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGRGC2I#issuecomment-564289897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFG2XHEW4RCJLSZHTBC3QYAKGLANCNFSM4IMLTVRA .

MichaelKonobeev

comment created time in 2 months

pull request commenttensorflow/agents

port tf_agents.agents.dqn.examples.v2.train_eval_test.py to TF2.0

I'm confused @tagomatech , but i thought the v2 version is supposed to work with TF2. Did you have issues with DQN v2 train_eval_test in TF2?

tagomatech

comment created time in 2 months

pull request commenttensorflow/agents

Add trajectory arg to call fn for CounterMetric

@kbanoop do you recommend we close this pull request? @krishpop is Anoop's answer sufficient to revert this?

krishpop

comment created time in 2 months

pull request commenttensorflow/agents

Account for episode boundary with non-zero discount in DQN n-step update

Sorry for the delay. @oars perhaps your PR will resolve this?

mhe500

comment created time in 2 months

issue commenttensorflow/agents

DQN Agent Issue With Custom Environment

For a better answer, we'd need to look at your copy of the repo; so we can understand the diff. Also ensure that you can run the original example before you made any changes and that it doesn't lead to an error.

IbraheemNofal

comment created time in 2 months

issue commenttensorflow/agents

DQN Agent Issue With Custom Environment

I would like to see a full trace of your code, but I believe the problem is that you're performing training (correct me if I'm wrong), which requires getting batches of data from your environment. So for example, if the observatoin spec is TensorSpec(..., shape=(16,), dtype=tf.float32) then at training time you should be passing tensors of shape [batch_size, 2, 16]; where the first dimension corresponds to batch size (whatever it is) and the second corresponds to the fact that DQN requires seeing 2 time steps. The inner dimension matches the spec.

IbraheemNofal

comment created time in 2 months

issue closedtensorflow/agents

Issue with test ddpg

When I try to run the example DDPG test and eval scripts, I get the following issue: module 'tensorflow' has no attribute 'enable_resource_variables'

If I change this line to: tf.compat.v1.enable_resource_variables()

Then I get the following error:

RuntimeError: dataset.make_initializable_iterator is not supported when eager execution is enabled. Use for element in dataset instead. In call to configurable 'train_eval' (<function train_eval at 0x7f10910917a0>)

I am trying to avoid making too many changes to the provided script, but I had to change the environment due to not having mujoco: env_name="Pendulum-v0", env_load_fn=suite_gym.load,

closed time in 2 months

vedantammihir

issue commenttensorflow/agents

Issue with test ddpg

As @kuanghuei said; you should be instead running the examples from this directory. If those still pose a problem, please reopen this issue.

vedantammihir

comment created time in 2 months

issue commenttensorflow/agents

[Question] call() in ddpg critic network

@kbanoop would know best; I believe he wrote this code.

junhyeokahn

comment created time in 2 months

issue commenttensorflow/agents

How do I adjust ddpg example to fit my environment?

Assigning Oscar to triage. Perhaps Sam would be able to help with this?

mauricerad

comment created time in 2 months

issue commenttensorflow/agents

How do I adjust ddpg example to fit my environment?

@oars do you know who would be able to help with this?

@mauricerad do you have a git repo showing your setup? or are you just checking out tf_agents and modifying the existing ddpg train_eval example?

mauricerad

comment created time in 2 months

issue commenttensorflow/agents

ParallelPyEnvironments do not work on Windows

Unfortunately I'm not sure we have many dev boxes for testing on Windows. Have you had any experience with python subprocess pipes in windows, specifically know any gotchas to look out for when it comes to race conditions, opening pipes, etc?

NeoExtended

comment created time in 2 months

issue commenttensorflow/agents

ParallelPyEnvironments do not work on Windows

What kind of environment are you working in? Is it compute heavy? If not, you can try BatchedPyEnvironment for now.

NeoExtended

comment created time in 3 months

issue closedtensorflow/agents

Clarification on defining discrete, nominal actions

Hi! I am transitioning from keras-rl to tf_agents and would really appreciate some clarifications on defining discrete action specs.

I am buliding a custom environment in which the 9 possible actions are discrete and nominal (i.e. not ordinal) and I hope to run DQN on it. Does setting the action spec as an integer as follows assume any ordinal relationship?

self._action_spec = array_spec.BoundedArraySpec( shape=(), dtype=np.int32, minimum=0, maximum=8, name='action')

If that's the case, can I define the action space to be some one-hot encoded matrix as follows? How could I enforce the requirement that there must only be one 1 and eight 0s?

self._action_spec = array_spec.BoundedArraySpec( shape=((3, 3)), dtype=np.int32, minimum=0, maximum=1, name='action')

Thanks in advance!

closed time in 3 months

matthew-hsr

issue commenttensorflow/agents

Clarification on defining discrete, nominal actions

Please reopen if there is still confusion.

matthew-hsr

comment created time in 3 months

issue commenttensorflow/tensorflow

Strange behaviour of tf.sigmoid

Googlers: This is internal bug b/143090143.

On Thu, Nov 14, 2019 at 11:25 AM Rasmus Munk Larsen rmlarsen@google.com wrote:

This particular optimization is problematic, since it gives values that are very far off for gradients of values below -14. I have a TODO to remove it and just rely on the vectorized version of division, addition and exp.

On Thu, Nov 14, 2019 at 11:16 AM Eugene Brevdo ebrevdo@google.com wrote:

+Alexandre Passos apassos@google.com +Rasmus Larsen rmlarsen@google.com this is a performance optimization I think. Perhaps we should add a precision attr or graph config like xla does?

On Thu, Nov 14, 2019, 4:32 AM Gael Guennebaud notifications@github.com wrote:

See: https://eigen.tuxfamily.org/bz/show_bug.cgi?id=1777

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/33878?email_source=notifications&email_token=AANWFG2WJCOAVTJA34737U3QTVOQ3A5CNFSM4JHJMTK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEECAWGY#issuecomment-553913115, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFGYC2NCJELAOBB32VOTQTVOQ3ANCNFSM4JHJMTKQ .

georgesterpu

comment created time in 3 months

issue commenttensorflow/tensorflow

Strange behaviour of tf.sigmoid

This particular optimization is problematic, since it gives values that are very far off for gradients of values below -14. I have a TODO to remove it and just rely on the vectorized version of division, addition and exp.

On Thu, Nov 14, 2019 at 11:16 AM Eugene Brevdo ebrevdo@google.com wrote:

+Alexandre Passos apassos@google.com +Rasmus Larsen rmlarsen@google.com this is a performance optimization I think. Perhaps we should add a precision attr or graph config like xla does?

On Thu, Nov 14, 2019, 4:32 AM Gael Guennebaud notifications@github.com wrote:

See: https://eigen.tuxfamily.org/bz/show_bug.cgi?id=1777

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/33878?email_source=notifications&email_token=AANWFG2WJCOAVTJA34737U3QTVOQ3A5CNFSM4JHJMTK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEECAWGY#issuecomment-553913115, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFGYC2NCJELAOBB32VOTQTVOQ3ANCNFSM4JHJMTKQ .

georgesterpu

comment created time in 3 months

issue commenttensorflow/tensorflow

Strange behaviour of tf.sigmoid

+Alexandre Passos apassos@google.com +Rasmus Larsen rmlarsen@google.com this is a performance optimization I think. Perhaps we should add a precision attr or graph config like xla does?

On Thu, Nov 14, 2019, 4:32 AM Gael Guennebaud notifications@github.com wrote:

See: https://eigen.tuxfamily.org/bz/show_bug.cgi?id=1777

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/33878?email_source=notifications&email_token=AANWFG2WJCOAVTJA34737U3QTVOQ3A5CNFSM4JHJMTK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEECAWGY#issuecomment-553913115, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFGYC2NCJELAOBB32VOTQTVOQ3ANCNFSM4JHJMTKQ .

georgesterpu

comment created time in 3 months

issue commenttensorflow/tensorflow

TensorFlow test execution fails with Bazel v0.24.1 on s390x

I believe someone sent a PR to guard.

On Fri, Nov 8, 2019, 7:06 AM Gunhan Gulsoy notifications@github.com wrote:

ping! any updates here? Is this still a problem with newer bazel versions?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/28553?email_source=notifications&email_token=AANWFG2AI6TMHQ76QXBT2F3QSWMAHA5CNFSM4HL24AGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDSXUXY#issuecomment-551909983, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFG5VFSBHTUPM7UQNYCTQSWMAHANCNFSM4HL24AGA .

Nayana-ibm

comment created time in 4 months

issue commenttensorflow/tensorflow

Improved documentation for TensorArray

Sorry; in TF2 does one of two things:

  • In eager mode, it deletes the underlying values in the tensorarray (essentially releases memory)
  • In tf.function building mode, it does nothing. The underlying data is released when there are no more ops referencing it.

This is kinda inconsistent. In eager mode, reusing the TensorArray after close will lead to errors. In tf.function mode, it'll work just fine (and in that case close has no effect). @alextp if we care enough about this inconsistency.

alquraishi

comment created time in 4 months

issue commenttensorflow/tensorflow

[Feature] sparse_tensor_dense_matmul with two SparseTensor

It's available in tf2 nightlies under the tf.linalg.experimental module

On Sat, Nov 2, 2019, 4:34 PM Daniele Grattarola notifications@github.com wrote:

@chpohl https://github.com/chpohl I gave up after @ebrevdo https://github.com/ebrevdo told me that they already have the functionality implemented, but it never got released. Unfortunately, TF2 and the new Keras broke most practical uses of sparse matrices, so I don't think the the tf devs have any interest in supporting the feature moving forward.

If you want to implement sparse-sparse matmul for some older version of TF drop me an email, maybe we can look at it together.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/10375?email_source=notifications&email_token=AANWFG42DJNBCOFX556DYPDQRYFAHA5CNFSM4DNUVAP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC5HJYY#issuecomment-549090531, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANWFGYSLXJE2FJ6RI4DT23QRYFAHANCNFSM4DNUVAPQ .

danielegrattarola

comment created time in 4 months

pull request commenttensorflow/agents

Extend networks to handle complex observations

@witwolf ValueNetwork and ActorDistributionNetwork are not public-facing classes but base classes; aren't they? as such, they expose __init__ args because they cannot infer them from other arguments. But some of these other classes can infer the values that these args should be when they call super(), from their own __init__ arguments. right?

witwolf

comment created time in 4 months

push eventj0rd1smit/agents

ebrevdo

commit sha 4b3f5b8f22261fbddd8aedce77e9a609eafe3d49

Update array_spec.py

view details

push time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,         each item is the number of units in the layer. This is applied after the         LSTM cell.       activation_fn: Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...+      dtype: The dtype to use by the convolution and fully connected layers.       name: A string representing name of the network.      Returns:       A tf.float32 Tensor of q-values.      Raises:-      ValueError: If `observation_spec` or `action_spec` contains more than one+      ValueError: If `action_spec` contains more than one         item.     """     observation_spec, action_spec = input_tensor_spec -    if len(tf.nest.flatten(observation_spec)) > 1:-      raise ValueError(-          'Only a single observation is supported by this network.')+    kernel_initializer = tf.compat.v1.variance_scaling_initializer(+      scale=2.0, mode='fan_in', distribution='truncated_normal')++    obs_encoder = encoding_network.EncodingNetwork(+        observation_spec,+        preprocessing_layers=observation_preprocessing_layers,+        preprocessing_combiner=observation_preprocessing_combiner,+        conv_layer_params=observation_conv_layer_params,+        fc_layer_params=observation_fc_layer_params,+        activation_fn=activation_fn,+        kernel_initializer=kernel_initializer,+        dtype=dtype,+        name='obs_encoding')      if len(tf.nest.flatten(action_spec)) > 1:       raise ValueError('Only a single action is supported by this network.') -    observation_layers = utils.mlp_layers(-        observation_conv_layer_params,-        observation_fc_layer_params,-        activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='observation_encoding')--    action_layers = utils.mlp_layers(+    action_layers = tf.keras.Sequential(utils.mlp_layers(         None,         action_fc_layer_params,         activation_fn=activation_fn,         kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(             scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='action_encoding')--    joint_layers = utils.mlp_layers(-        None,-        joint_fc_layer_params,+        name='action_encoding'))++    obs_encoding_spec = tf.TensorSpec(+        shape=(observation_fc_layer_params[-1],), dtype=tf.float32)+    lstm_encoder = lstm_encoding_network.LSTMEncodingNetwork(+        input_tensor_spec=(obs_encoding_spec, action_spec),+        preprocessing_layers=(tf.keras.layers.Flatten(), action_layers),+        preprocessing_combiner=tf.keras.layers.Concatenate(axis=-1),+        input_fc_layer_params=joint_fc_layer_params,+        lstm_size=lstm_size,+        output_fc_layer_params=output_fc_layer_params,         activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='joint_mlp')--    # Create RNN cell-    if len(lstm_size) == 1:-      cell = tf.keras.layers.LSTMCell(lstm_size[0])-    else:-      cell = tf.keras.layers.StackedRNNCells(-          [tf.keras.layers.LSTMCell(size) for size in lstm_size])+        dtype=dtype,+        name='lstm') -    state_spec = tf.nest.map_structure(-        functools.partial(-            tensor_spec.TensorSpec, dtype=tf.float32,-            name='network_state_spec'), list(cell.state_size))--    output_layers = utils.mlp_layers(fc_layer_params=output_fc_layer_params,-                                     name='output')--    output_layers.append(+    output_layers = [         tf.keras.layers.Dense(             1,             activation=None,             kernel_initializer=tf.keras.initializers.RandomUniform(                 minval=-0.003, maxval=0.003),-            name='value'))+            name='value')]      super(CriticRnnNetwork, self).__init__(         input_tensor_spec=input_tensor_spec,-        state_spec=state_spec,+        state_spec=lstm_encoder.state_spec,         name=name) -    self._observation_layers = observation_layers-    self._action_layers = action_layers-    self._joint_layers = joint_layers-    self._dynamic_unroll = dynamic_unroll_layer.DynamicUnroll(cell)+    self._obs_encoder = obs_encoder+    self._lstm_encoder = lstm_encoder     self._output_layers = output_layers    # TODO(kbanoop): Standardize argument names across different networks.   def call(self, inputs, step_type, network_state=None):+    outer_rank = nest_utils.get_outer_rank(inputs, self.input_tensor_spec)+    batch_squash = utils.BatchSquash(outer_rank)+     observation, action = inputs-    observation_spec, _ = self.input_tensor_spec-    num_outer_dims = nest_utils.get_outer_rank(observation,-                                               observation_spec)-    if num_outer_dims not in (1, 2):-      raise ValueError(-          'Input observation must have a batch or batch x time outer shape.')--    has_time_dim = num_outer_dims == 2-    if not has_time_dim:-      # Add a time dimension to the inputs.-      observation = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1),-                                          observation)-      action = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1), action)-      step_type = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1),-                                        step_type)--    observation = tf.cast(tf.nest.flatten(observation)[0], tf.float32)-    action = tf.cast(tf.nest.flatten(action)[0], tf.float32)+    observation, _ = self._obs_encoder(+        observation,+        step_type=step_type,+        network_state=network_state) -    batch_squash = utils.BatchSquash(2)  # Squash B, and T dims.-    observation = batch_squash.flatten(observation)  # [B, T, ...] -> [BxT, ...]+    observation = batch_squash.flatten(observation)

please keep the comments describing what's going on here. same elsehwere you're using BatchSquash.

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,         each item is the number of units in the layer. This is applied after the         LSTM cell.       activation_fn: Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...+      dtype: The dtype to use by the convolution and fully connected layers.       name: A string representing name of the network.      Returns:       A tf.float32 Tensor of q-values.      Raises:-      ValueError: If `observation_spec` or `action_spec` contains more than one+      ValueError: If `action_spec` contains more than one         item.     """     observation_spec, action_spec = input_tensor_spec -    if len(tf.nest.flatten(observation_spec)) > 1:-      raise ValueError(-          'Only a single observation is supported by this network.')+    kernel_initializer = tf.compat.v1.variance_scaling_initializer(+      scale=2.0, mode='fan_in', distribution='truncated_normal')++    obs_encoder = encoding_network.EncodingNetwork(+        observation_spec,+        preprocessing_layers=observation_preprocessing_layers,+        preprocessing_combiner=observation_preprocessing_combiner,+        conv_layer_params=observation_conv_layer_params,+        fc_layer_params=observation_fc_layer_params,+        activation_fn=activation_fn,+        kernel_initializer=kernel_initializer,+        dtype=dtype,+        name='obs_encoding')      if len(tf.nest.flatten(action_spec)) > 1:       raise ValueError('Only a single action is supported by this network.') -    observation_layers = utils.mlp_layers(-        observation_conv_layer_params,-        observation_fc_layer_params,-        activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='observation_encoding')--    action_layers = utils.mlp_layers(+    action_layers = tf.keras.Sequential(utils.mlp_layers(         None,         action_fc_layer_params,         activation_fn=activation_fn,         kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(             scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='action_encoding')--    joint_layers = utils.mlp_layers(-        None,-        joint_fc_layer_params,+        name='action_encoding'))++    obs_encoding_spec = tf.TensorSpec(+        shape=(observation_fc_layer_params[-1],), dtype=tf.float32)+    lstm_encoder = lstm_encoding_network.LSTMEncodingNetwork(+        input_tensor_spec=(obs_encoding_spec, action_spec),+        preprocessing_layers=(tf.keras.layers.Flatten(), action_layers),+        preprocessing_combiner=tf.keras.layers.Concatenate(axis=-1),+        input_fc_layer_params=joint_fc_layer_params,+        lstm_size=lstm_size,+        output_fc_layer_params=output_fc_layer_params,         activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='joint_mlp')--    # Create RNN cell-    if len(lstm_size) == 1:-      cell = tf.keras.layers.LSTMCell(lstm_size[0])-    else:-      cell = tf.keras.layers.StackedRNNCells(-          [tf.keras.layers.LSTMCell(size) for size in lstm_size])+        dtype=dtype,+        name='lstm') -    state_spec = tf.nest.map_structure(-        functools.partial(-            tensor_spec.TensorSpec, dtype=tf.float32,-            name='network_state_spec'), list(cell.state_size))--    output_layers = utils.mlp_layers(fc_layer_params=output_fc_layer_params,-                                     name='output')--    output_layers.append(+    output_layers = [         tf.keras.layers.Dense(             1,             activation=None,             kernel_initializer=tf.keras.initializers.RandomUniform(                 minval=-0.003, maxval=0.003),-            name='value'))+            name='value')]      super(CriticRnnNetwork, self).__init__(         input_tensor_spec=input_tensor_spec,-        state_spec=state_spec,+        state_spec=lstm_encoder.state_spec,         name=name) -    self._observation_layers = observation_layers-    self._action_layers = action_layers-    self._joint_layers = joint_layers-    self._dynamic_unroll = dynamic_unroll_layer.DynamicUnroll(cell)+    self._obs_encoder = obs_encoder+    self._lstm_encoder = lstm_encoder     self._output_layers = output_layers    # TODO(kbanoop): Standardize argument names across different networks.   def call(self, inputs, step_type, network_state=None):+    outer_rank = nest_utils.get_outer_rank(inputs, self.input_tensor_spec)+    batch_squash = utils.BatchSquash(outer_rank)+     observation, action = inputs-    observation_spec, _ = self.input_tensor_spec-    num_outer_dims = nest_utils.get_outer_rank(observation,-                                               observation_spec)-    if num_outer_dims not in (1, 2):-      raise ValueError(-          'Input observation must have a batch or batch x time outer shape.')--    has_time_dim = num_outer_dims == 2-    if not has_time_dim:-      # Add a time dimension to the inputs.-      observation = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1),-                                          observation)-      action = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1), action)-      step_type = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1),-                                        step_type)--    observation = tf.cast(tf.nest.flatten(observation)[0], tf.float32)-    action = tf.cast(tf.nest.flatten(action)[0], tf.float32)+    observation, _ = self._obs_encoder(+        observation,+        step_type=step_type,+        network_state=network_state) -    batch_squash = utils.BatchSquash(2)  # Squash B, and T dims.-    observation = batch_squash.flatten(observation)  # [B, T, ...] -> [BxT, ...]+    observation = batch_squash.flatten(observation)+    action = tf.cast(tf.nest.flatten(action)[0], tf.float32)     action = batch_squash.flatten(action) -    for layer in self._observation_layers:-      observation = layer(observation)--    for layer in self._action_layers:-      action = layer(action)--    joint = tf.concat([observation, action], -1)-    for layer in self._joint_layers:-      joint = layer(joint)--    joint = batch_squash.unflatten(joint)  # [B x T, ...] -> [B, T, ...]--    with tf.name_scope('reset_mask'):-      reset_mask = tf.equal(step_type, time_step.StepType.FIRST)-    # Unroll over the time sequence.-    joint, network_state = self._dynamic_unroll(-        joint,-        reset_mask,-        initial_state=network_state)--    output = batch_squash.flatten(joint)  # [B, T, ...] -> [B x T, ...]+    output, network_state = self._lstm_encoder(+        inputs=(observation, action),+        step_type=step_type,+        network_state=network_state)      for layer in self._output_layers:       output = layer(output)      q_value = tf.reshape(output, [-1])-    q_value = batch_squash.unflatten(q_value)  # [B x T, ...] -> [B, T, ...]-    if not has_time_dim:-      q_value = tf.squeeze(q_value, axis=1)+    q_value = batch_squash.unflatten(q_value)

add back the comment explaining how things are being unsquashed.

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,         each item is the number of units in the layer. This is applied after the         LSTM cell.       activation_fn: Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...+      dtype: The dtype to use by the convolution and fully connected layers.       name: A string representing name of the network.      Returns:       A tf.float32 Tensor of q-values.      Raises:-      ValueError: If `observation_spec` or `action_spec` contains more than one+      ValueError: If `action_spec` contains more than one         item.     """     observation_spec, action_spec = input_tensor_spec -    if len(tf.nest.flatten(observation_spec)) > 1:-      raise ValueError(-          'Only a single observation is supported by this network.')+    kernel_initializer = tf.compat.v1.variance_scaling_initializer(+      scale=2.0, mode='fan_in', distribution='truncated_normal')++    obs_encoder = encoding_network.EncodingNetwork(+        observation_spec,+        preprocessing_layers=observation_preprocessing_layers,+        preprocessing_combiner=observation_preprocessing_combiner,+        conv_layer_params=observation_conv_layer_params,+        fc_layer_params=observation_fc_layer_params,+        activation_fn=activation_fn,+        kernel_initializer=kernel_initializer,+        dtype=dtype,+        name='obs_encoding')      if len(tf.nest.flatten(action_spec)) > 1:       raise ValueError('Only a single action is supported by this network.') -    observation_layers = utils.mlp_layers(-        observation_conv_layer_params,-        observation_fc_layer_params,-        activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='observation_encoding')--    action_layers = utils.mlp_layers(+    action_layers = tf.keras.Sequential(utils.mlp_layers(         None,         action_fc_layer_params,         activation_fn=activation_fn,         kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(             scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='action_encoding')--    joint_layers = utils.mlp_layers(-        None,-        joint_fc_layer_params,+        name='action_encoding'))++    obs_encoding_spec = tf.TensorSpec(+        shape=(observation_fc_layer_params[-1],), dtype=tf.float32)+    lstm_encoder = lstm_encoding_network.LSTMEncodingNetwork(+        input_tensor_spec=(obs_encoding_spec, action_spec),+        preprocessing_layers=(tf.keras.layers.Flatten(), action_layers),+        preprocessing_combiner=tf.keras.layers.Concatenate(axis=-1),+        input_fc_layer_params=joint_fc_layer_params,+        lstm_size=lstm_size,+        output_fc_layer_params=output_fc_layer_params,         activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='joint_mlp')--    # Create RNN cell-    if len(lstm_size) == 1:-      cell = tf.keras.layers.LSTMCell(lstm_size[0])-    else:-      cell = tf.keras.layers.StackedRNNCells(-          [tf.keras.layers.LSTMCell(size) for size in lstm_size])+        dtype=dtype,+        name='lstm') -    state_spec = tf.nest.map_structure(-        functools.partial(-            tensor_spec.TensorSpec, dtype=tf.float32,-            name='network_state_spec'), list(cell.state_size))--    output_layers = utils.mlp_layers(fc_layer_params=output_fc_layer_params,-                                     name='output')--    output_layers.append(+    output_layers = [         tf.keras.layers.Dense(             1,             activation=None,             kernel_initializer=tf.keras.initializers.RandomUniform(                 minval=-0.003, maxval=0.003),-            name='value'))+            name='value')]      super(CriticRnnNetwork, self).__init__(         input_tensor_spec=input_tensor_spec,-        state_spec=state_spec,+        state_spec=lstm_encoder.state_spec,         name=name) -    self._observation_layers = observation_layers-    self._action_layers = action_layers-    self._joint_layers = joint_layers-    self._dynamic_unroll = dynamic_unroll_layer.DynamicUnroll(cell)+    self._obs_encoder = obs_encoder+    self._lstm_encoder = lstm_encoder     self._output_layers = output_layers    # TODO(kbanoop): Standardize argument names across different networks.   def call(self, inputs, step_type, network_state=None):+    outer_rank = nest_utils.get_outer_rank(inputs, self.input_tensor_spec)+    batch_squash = utils.BatchSquash(outer_rank)+     observation, action = inputs-    observation_spec, _ = self.input_tensor_spec-    num_outer_dims = nest_utils.get_outer_rank(observation,-                                               observation_spec)-    if num_outer_dims not in (1, 2):-      raise ValueError(-          'Input observation must have a batch or batch x time outer shape.')--    has_time_dim = num_outer_dims == 2-    if not has_time_dim:-      # Add a time dimension to the inputs.-      observation = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1),-                                          observation)-      action = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1), action)-      step_type = tf.nest.map_structure(lambda t: tf.expand_dims(t, 1),-                                        step_type)--    observation = tf.cast(tf.nest.flatten(observation)[0], tf.float32)-    action = tf.cast(tf.nest.flatten(action)[0], tf.float32)+    observation, _ = self._obs_encoder(+        observation,+        step_type=step_type,+        network_state=network_state) -    batch_squash = utils.BatchSquash(2)  # Squash B, and T dims.-    observation = batch_squash.flatten(observation)  # [B, T, ...] -> [BxT, ...]+    observation = batch_squash.flatten(observation)+    action = tf.cast(tf.nest.flatten(action)[0], tf.float32)

should this be casting to a user-defined dtype (or if that's not available, then tf.float32)?

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,         each item is the number of units in the layer. This is applied after the         LSTM cell.       activation_fn: Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...+      dtype: The dtype to use by the convolution and fully connected layers.       name: A string representing name of the network.      Returns:       A tf.float32 Tensor of q-values.      Raises:-      ValueError: If `observation_spec` or `action_spec` contains more than one+      ValueError: If `action_spec` contains more than one         item.     """     observation_spec, action_spec = input_tensor_spec -    if len(tf.nest.flatten(observation_spec)) > 1:-      raise ValueError(-          'Only a single observation is supported by this network.')+    kernel_initializer = tf.compat.v1.variance_scaling_initializer(+      scale=2.0, mode='fan_in', distribution='truncated_normal')++    obs_encoder = encoding_network.EncodingNetwork(+        observation_spec,+        preprocessing_layers=observation_preprocessing_layers,+        preprocessing_combiner=observation_preprocessing_combiner,+        conv_layer_params=observation_conv_layer_params,+        fc_layer_params=observation_fc_layer_params,+        activation_fn=activation_fn,+        kernel_initializer=kernel_initializer,+        dtype=dtype,+        name='obs_encoding')      if len(tf.nest.flatten(action_spec)) > 1:       raise ValueError('Only a single action is supported by this network.') -    observation_layers = utils.mlp_layers(-        observation_conv_layer_params,-        observation_fc_layer_params,-        activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='observation_encoding')--    action_layers = utils.mlp_layers(+    action_layers = tf.keras.Sequential(utils.mlp_layers(

use our SequentialLayer instead of keras one

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,         each item is the number of units in the layer. This is applied after the         LSTM cell.       activation_fn: Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...+      dtype: The dtype to use by the convolution and fully connected layers.       name: A string representing name of the network.      Returns:       A tf.float32 Tensor of q-values.      Raises:-      ValueError: If `observation_spec` or `action_spec` contains more than one+      ValueError: If `action_spec` contains more than one         item.     """     observation_spec, action_spec = input_tensor_spec -    if len(tf.nest.flatten(observation_spec)) > 1:-      raise ValueError(-          'Only a single observation is supported by this network.')+    kernel_initializer = tf.compat.v1.variance_scaling_initializer(+      scale=2.0, mode='fan_in', distribution='truncated_normal')++    obs_encoder = encoding_network.EncodingNetwork(+        observation_spec,+        preprocessing_layers=observation_preprocessing_layers,+        preprocessing_combiner=observation_preprocessing_combiner,+        conv_layer_params=observation_conv_layer_params,+        fc_layer_params=observation_fc_layer_params,+        activation_fn=activation_fn,+        kernel_initializer=kernel_initializer,+        dtype=dtype,+        name='obs_encoding')      if len(tf.nest.flatten(action_spec)) > 1:       raise ValueError('Only a single action is supported by this network.') -    observation_layers = utils.mlp_layers(-        observation_conv_layer_params,-        observation_fc_layer_params,-        activation_fn=activation_fn,-        kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(-            scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='observation_encoding')--    action_layers = utils.mlp_layers(+    action_layers = tf.keras.Sequential(utils.mlp_layers(         None,         action_fc_layer_params,         activation_fn=activation_fn,         kernel_initializer=tf.compat.v1.keras.initializers.VarianceScaling(             scale=1. / 3., mode='fan_in', distribution='uniform'),-        name='action_encoding')--    joint_layers = utils.mlp_layers(-        None,-        joint_fc_layer_params,+        name='action_encoding'))++    obs_encoding_spec = tf.TensorSpec(+        shape=(observation_fc_layer_params[-1],), dtype=tf.float32)+    lstm_encoder = lstm_encoding_network.LSTMEncodingNetwork(+        input_tensor_spec=(obs_encoding_spec, action_spec),

can you add a comment explaining why you're encoding the action with observation via the LSTM encoder here? something seems off about this. then again, i'm unfamiliar with this part of the code. @oars any idea?

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 class CriticRnnNetwork(network.Network):    def __init__(self,                input_tensor_spec,+               observation_preprocessing_layers=None,+               observation_preprocessing_combiner=None,                observation_conv_layer_params=None,                observation_fc_layer_params=(200,),                action_fc_layer_params=(200,),                joint_fc_layer_params=(100),                lstm_size=(40,),                output_fc_layer_params=(200, 100),                activation_fn=tf.keras.activations.relu,+               dtype=tf.float32,

default value None?

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

-# coding=utf-8

(separately; i think we should get rid of the # coding=utf-8 line)

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

-# coding=utf-8

any idea why this entire file shows a diff? its hard to see what changes you made.

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,                joint_fc_layer_params=None,                joint_dropout_layer_params=None,                activation_fn=tf.nn.relu,+               kernel_initializer=None,+               batch_squash=True,+               dtype=tf.float32,

should this default to None, and if not set the layers use whatever dtype the preprocessing layers emitted?

same for occurrences of dtype elsewhere?

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,                joint_fc_layer_params=None,                joint_dropout_layer_params=None,                activation_fn=tf.nn.relu,+               kernel_initializer=None,+               batch_squash=True,

again, not clear why batch_squash should be exposed.

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def testDictOfSingleAction(self):                         [batch_size] + action_spec['motor'].shape.as_list())     self.assertEqual(len(actor_net.trainable_variables), 2) +  @test_util.run_in_graph_and_eager_modes()+  def testHandlePreprocessingLayers(self):+    observation_spec = (tensor_spec.TensorSpec([1], tf.float32),+                        tensor_spec.TensorSpec([], tf.float32))+    time_step_spec = ts.time_step_spec(observation_spec)+    time_step = tensor_spec.sample_spec_nest(time_step_spec, outer_dims=(3,))++    action_spec = tensor_spec.BoundedTensorSpec((2,), tf.float32, 2, 3)++    preprocessing_layers = (tf.keras.layers.Dense(4),+                            tf.keras.Sequential([

use TF-Agents Sequential layer instead, it handles serialization better.

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,         each item is a length-three tuple indicating (filters, kernel_size,         stride).       activation_fn: Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...+      kernel_initializer: Initializer to use for the kernels of the conv and+        dense layers. If none is provided a default glorot_uniform+      batch_squash: If True the outer_ranks of the observation are squashed into

i'm not sure this arg should be exposed to the user. why is it necessary? the network will know if it's an RNN network or not, and can perform batch squash as necessary.

witwolf

comment created time in 4 months

Pull request review commenttensorflow/agents

Extend networks to handle complex observations

 def __init__(self,         each item is a length-three tuple indicating (filters, kernel_size,         stride).       activation_fn: Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...+      kernel_initializer: Initializer to use for the kernels of the conv and+        dense layers. If none is provided a default glorot_uniform

this looks unfinished.

witwolf

comment created time in 4 months

pull request commenttensorflow/agents

Remove global_step argument

I thought we have a PR that selectively passes global_step iff the optimizer is a v1 optimizer.

tagomatech

comment created time in 4 months

issue commenttensorflow/agents

TF agents & TF 2.0 contrib dependency for current pip package.

@oars @sguada could you push a new pip?

tenser-flow

comment created time in 4 months

issue closedtensorflow/agents

save replay buffer

Hello, I could not find out which data structure is used save the trajectories in the replay buffer So if I continue training I can load the buffer with the old buffer content

Thank you for your help

closed time in 4 months

ChrisProgramming2018

issue commenttensorflow/agents

save replay buffer

Hi; the replay buffer subclasses tf.Module, so you should be able to save and load it using tf.train.Checkpoint:

rb = tf_uniform_replay_buffer.TFUniformReplayBuffer(tf.TensorSpec([], tf.float32), 1)
...
checkpointed = tf.train.Checkpoint(rb=rb)
checkpointed.save('/tmp/blah')
rb.clear()  # reset RB
checkpointed.restore('/tmp/blah-1')  # assuming checkpoint was to /tmp/blah-1
rb.get_next()  # should return data that was in RB before checkpointing

Note you can create an empty RB, associated it with a Checkpoint object, and restore into it.

ChrisProgramming2018

comment created time in 4 months

pull request commenttensorflow/tensorflow

Implement Hessian for sparse softmax cross entropy

A second component that will stop us from being able to solve this problem using tf.custom_gradient is the fact that custom_gradient is not serialized with the graph, so serializing a savedmodel containing a cross entropy loss using tf.cusotm_gradient, when you deserialize you can get either errors or incorrect behavior if taking gradients of the resulting tensor. So there are two major issues here.

MichaelKonobeev

comment created time in 4 months

pull request commenttensorflow/community

RFC: TensorCord Variant Object

As a followup: the proposal to merge tstring with TensorCord was reversed because of the problems this would create for users of tstring:

  • tstring would no longer represent a single contiguous string; so operations like array-type access and cast to string_view would no longer be possible. similarly, the "fast path" that most programs rely on when dealing with string buffers, by accessing buffer.data(), would on longer be available.

  • the idea of separating the view (a tstring with multiple pointers to memory it has no control over) from the deallocation operation (which would sit elsewhere in a TensorBuffer) was problematic.

ebrevdo

comment created time in 4 months

issue commenttensorflow/tensorflow

TypeError: can't pickle _thread.lock objects

One way is to use the model saving functionality in TensorFlow: tf.keras.models.save_model https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model or calling model.save https://www.tensorflow.org/guide/keras/save_and_serialize. Don't use pickle.

On Mon, Oct 7, 2019 at 7:19 AM SumitNikam notifications@github.com wrote:

@ebrevdo https://github.com/ebrevdo

import pickle from keras.layers import Dense from keras.layers import LSTM from keras.models import Sequential from keras.metrics import categorical_accuracy

model = Sequential() model.add(LSTM(20, return_sequences=True, stateful=False, batch_input_shape=(10, 20, 4))) model.add(Dense(3, activation='softmax'))

model.compile(loss="categorical_crossentropy", optimizer='adam', metrics=[categorical_accuracy], sample_weight_mode='temporal')

data_path = '/home/ubuntu/invoice/data/' #any path to store pickle dump output_file_path = data_path + 'model.dat' with open(output_file_path, 'wb') as f: pickle.dump(model, f)

Error Msg:

Traceback (most recent call last): File "<input>", line 19, in <module> TypeError: can't pickle _thread.lock objects

Do you solve this issue ? If you solve please suggest me to modifications please

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/issues/11157?email_source=notifications&email_token=AANWFGYP3ZQWRMDTSASYNE3QNNAPXA5CNFSM4DRIHGE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAQQGCA#issuecomment-539034376, or mute the thread https://github.com/notifications/unsubscribe-auth/AANWFG4B6FWNJ4EPBETPMKTQNNAPXANCNFSM4DRIHGEQ .

mattfeel

comment created time in 5 months

pull request commenttensorflow/tensorflow

Backport https://github.com/tensorflow/addons/pull/306 into master

Even for bugfixes?

On Fri, Sep 27, 2019 at 11:05 AM Mihai Maruseac notifications@github.com wrote:

The window for 1.15 cherry-picks has also closed, unfortunately.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tensorflow/pull/32096?email_source=notifications&email_token=AANWFG64HPPBJHCWOLZA4KLQLZDNXA5CNFSM4ISHFPR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7ZVOXA#issuecomment-536041308, or mute the thread https://github.com/notifications/unsubscribe-auth/AANWFG4XUWO2ZJAWVTQG6A3QLZDNXANCNFSM4ISHFPRQ .

sjmielke

comment created time in 5 months

Pull request review commenttensorflow/tensorflow

Speedup

 def ExtractBitsFromFloat16(x):   def SlowAppendFloat16ArrayToTensorProto(tensor_proto, proto_values):-  tensor_proto.half_val.extend(-      [ExtractBitsFromFloat16(x) for x in proto_values])+  tensor_proto.half_val.extend(np.asarray(proto_values, dtype=np.float16).view(np.uint16))

Unit tests are there to ensure that future changes don't cause a regression. It would take you less time to add one than to write that comment ;)

SmokerX

comment created time in 5 months

PR closed tensorflow/tensorflow

Reviewers
Backport https://github.com/tensorflow/addons/pull/306 into master cla: yes size:M

This is fixing the conditional in tf.contrib.seq2seq.beam_search_decoder's _check_static_batch_beam_maybe, as reported in https://github.com/tensorflow/addons/issues/289 .

+38 -12

4 comments

2 changed files

sjmielke

pr closed time in 5 months

pull request commenttensorflow/tensorflow

Backport https://github.com/tensorflow/addons/pull/306 into master

Looks like we missed the window; since contrib is no longer in master. I suggest just pushing this to the tf 1.15 branch manually. You'll still only need to have one PR, it just won't be to master...

sjmielke

comment created time in 5 months

issue commenttensorflow/addons

Backport "Fix handling of sequence_length w/ beam search decoder"

@qlzh727 thanks so much for handling this!

guillaumekln

comment created time in 5 months

Pull request review commenttensorflow/tensorflow

TPU support for training with scheduled sampling

 def next_inputs(self, time, outputs, state, sample_ids, name=None):        def maybe_sample():         """Perform scheduled sampling."""-        where_sampling = math_ops.cast(-            array_ops.where(sample_ids > -1), dtypes.int32)-        where_not_sampling = math_ops.cast(-            array_ops.where(sample_ids <= -1), dtypes.int32)-        sample_ids_sampling = array_ops.gather_nd(sample_ids, where_sampling)-        inputs_not_sampling = array_ops.gather_nd(-            base_next_inputs, where_not_sampling)-        sampled_next_inputs = self._embedding_fn(sample_ids_sampling)-        base_shape = array_ops.shape(base_next_inputs)-        return (array_ops.scatter_nd(indices=where_sampling,-                                     updates=sampled_next_inputs,-                                     shape=base_shape)-                + array_ops.scatter_nd(indices=where_not_sampling,-                                       updates=inputs_not_sampling,-                                       shape=base_shape))+        sampling_mask = math_ops.cast(sample_ids > -1, base_next_inputs.dtype)+        # Embedding lookup will fail for negative samples for some embedding_fn.+        outputs_sampled = self._embedding_fn(sample_ids + math_ops.cast(1 - sampling_mask, dtypes.int32))+        sampling_mask = array_ops.expand_dims(sampling_mask, axis=-1)+        sampled_masked = sampling_mask * outputs_sampled

Can you use tertiary where instead of a multiplier mask?

akademi4eg

comment created time in 5 months

pull request commenttensorflow/tensorflow

(re-re-submission) Improve Flatten to avoid using dynamic shapes when possible

OK; the windows failure is not related. Let's merge.

pooyadavoodi

comment created time in 6 months

issue commenttensorflow/tensorflow

One hot embedding?

You first convert the strings to ints via one of the helper classes/functions in tf.lookup. You will need to have a vocabulary in mind that maps strings to integers.

vladfi1

comment created time in 6 months

more