profile
viewpoint
Jonathan Hseu jhseu @tensorflow @google San Francisco, CA Google Brain

tensorflow/rust 2281

Rust language bindings for TensorFlow

tensorflow/ecosystem 1034

Integration of TensorFlow with other open-source frameworks

jhseu/nca 35

Neighborhood Components Analysis in C++

vladfi1/DeepSmash 10

Attempt at phillip rewrite in rllib. Also attempts at training from slippi replays.

jhseu/pycassa 2

Simple python library for Cassandra

jhseu/tensorflow 1

Computation using data flow graphs for scalable machine learning

guptapriya/community 0

Stores documents used by the TensorFlow developer community

jhseu/buildifier 0

A bazel BUILD file formatter and editor

jhseu/dolphin 0

Dolphin is a GameCube / Wii emulator, allowing you to play games for these two platforms on PC with improvements.

jhseu/ecosystem 0

Integration of TensorFlow with other Open-source Frameworks

pull request commenttensorflow/tensorflow

Go: NewTensor & Value performance improvement

Seems like it hit one of the throws in this function: https://github.com/golang/go/blob/master/src/runtime/checkptr.go#L22

philpearl

comment created time in 2 days

pull request commenttensorflow/tensorflow

Go: NewTensor & Value performance improvement

This was rolled back due to MemorySanitizer complaining about unsafe pointer arithmetic in TestSessionRunNeg from session_test.go. I haven't looked closely at it yet.

philpearl

comment created time in 2 days

starteddeepmind/haiku

started time in 3 days

starteddeepmind/rlax

started time in 3 days

pull request commenttensorflow/tensorflow

Added an argument to control the padding for flatten_atrous

I don't really think I know enough to review this change. Maybe someone from TFLite/mobile?

yuzhang-deepmap

comment created time in 4 days

Pull request review commenttensorflow/tensorflow

Go: NewTensor & Value performance improvement

 func NewTensor(value interface{}) (*Tensor, error) { 	raw := tensorData(t.c) 	buf := bytes.NewBuffer(raw[:0:len(raw)]) 	if dataType != String {-		if err := encodeTensor(buf, val, shape); err != nil {-			return nil, err+		if isAllArray(val.Type()) {

I'm not seeing where scalars are handled in this code. (Am I missing something?)

philpearl

comment created time in 9 days

startedgoogle/tcmalloc

started time in 9 days

startedmarktext/marktext

started time in 12 days

startedopenai/procgen

started time in 12 days

issue commenttensorflow/tensorflow

Go API performance concerns with NewTensor & Value

Definitely open to taking fixes for this. I'm having trouble understanding the proposed fix, though. The allocations here are user-side. Are you proposing constructing the tensor and then allowing the user to directly write to it?

philpearl

comment created time in 15 days

PR opened tensorflow/tensorflow

Reviewers
Check in generated protobufs for Go

To make it more idiomatic and easier to install. See #35133

Gunhan, any objections to doing this? It adds 1.7 MB of generated files to the repo.

+25762 -0

0 comment

59 changed files

pr created time in 17 days

create barnchjhseu/tensorflow

branch : vendored_protobufs

created branch time in 17 days

push eventjhseu/tensorflow

Trent Lo

commit sha b33c788a2f479a4753f49b566f08079692c75af2

Implement horizontal fusion. - It reduces kernel launch overhead and increases lauch dims by horizontally fusing indepedent computations.

view details

Trent Lo

commit sha cd68827e01d454937399bafcdb1eb4b9a116678a

Minor cleanup for horizontal fusion.

view details

Trent Lo

commit sha cb9ab8bee96530c9973d5e295b53d936cbf8ef72

Polishing coding style and comments.

view details

Trent Lo

commit sha 1876f2acc02dee840b3a8b6ab59f950b5a3bbf4f

Factor out lambdas in HorizontalFusionImpl.

view details

Trent Lo

commit sha 86bd5bf3e75cb5d14d24194a2d1e2d8f60753b03

Comment polishing.

view details

Trent Lo

commit sha 474e79985f722afa57d12447fb2f4dc30e890d06

Add some more unittests for horizontal fusion. In addition, we record the execution time of the tests here, showing the optimization effects of horizontal fusion, measured by --xla_hlo_profile. The accumulated kernel execution time in GradientDescentOptimizerLike is reduced from 2.39ms to 311us; the execution time in RMSProp is reduced from 980us to 112us. Before horizontal fusion: 2019-12-10 22:05:45.215015: I tensorflow/compiler/xla/service/executable.cc:208] Execution profile for GradientDescentOptimizerLike: (2.39 ms @ f_nom) 2019-12-10 22:05:48.877372: I tensorflow/compiler/xla/service/executable.cc:208] Execution profile for RMSPropLike: (980 us @ f_nom) After horizontal fusion: 2019-12-10 22:05:03.831600: I tensorflow/compiler/xla/service/executable.cc:208] Execution profile for GradientDescentOptimizerLike: (311 us @ f_nom) 2019-12-10 22:05:13.513901: I tensorflow/compiler/xla/service/executable.cc:208] Execution profile for RMSPropLike: (112 us @ f_nom)

view details

Trent Lo

commit sha a629a452bff5b7f7f2688086483d7eb8d3d02420

Polishing comments and coding styles.

view details

Stephan Uphoff

commit sha 7813cb00f35d6fc6d8ad8421021c1535f3e8c029

lite/micro: Add feature buffer to micro_speech example. This fixes #35117 Accumulate feature slices in separate buffer. The input tensor is not suitable for keeping state across interference as it has limited lifetime and the buffer space may be reused.

view details

Trent Lo

commit sha 7abde726e4706df2fa83c2ec3c89ef9fb5c99228

Polish coding styles and comments based on review feedback. In addition, use hlo_matcher to verify resultant DAGs instead of LLVM filecheck.

view details

Trent Lo

commit sha 5f5aa78f86a43d073663cc0f96acb3926d621e42

Merge branch 'upstream_master_dec19' into horizontal_fusion_github

view details

Eugene Kuznetsov

commit sha 968a674ecb6db34e5d2e09068a8d9ca5ca4e3e24

Enable //tensorflow/python:stateful_random_ops_test

view details

Eugene Kuznetsov

commit sha f7b28191777b6ae86c0dbdab7a74b8370e53eaa8

Fix for //tensorflow/python:stateful_random_ops_test: Pack arguments of UpdateVariableAndFill_Philox into a struct

view details

Eugene Kuznetsov

commit sha eee5851777b842945b12937600b005a58aae0f2c

Fix for //tensorflow/python:stateful_random_ops_test: Move the thread counter into the global namespace

view details

Trent Lo

commit sha 47ba0995d9838e5f9aa634abc59f4569c4a37375

Fix a buildifier format issue.

view details

Trent Lo

commit sha eab6c5e84d44afbfd4e2b80c5dd59a6b090ed3bf

Do not fuse fusions with different output types. It is forbidden as the concatenate (inserted for horizontal fusion) requires the input operands to have the same type.

view details

Trent Lo

commit sha e09da4f3dc39efda5a8e68539bea894c88831143

Minor polishing.

view details

Trent Lo

commit sha e2989a34af44c39624cffa36cf319c66615c2483

Add a help function GetUniqueOutputTypeOfFusion. It is safer and clearer for getting the unique output type of fusion.

view details

Trent Lo

commit sha 88521dae35d13c7b25b826e5dd4da2f2d26d6013

Minor error message polishing.

view details

Trent Lo

commit sha 30b1943d43ac83fde73069d8546ab6a5c1e68372

Minor comment polishing.

view details

Harry Slatyer

commit sha 2aaa53e21f5d12c6de74a7d73525f9fc227b13bb

Support Identity in tensor_util.constant_value. This looks just the same as StopGradient, since for the purposes of forward-propagated values the two are identical.

view details

push time in 17 days

issue commenttensorflow/tensorflow

Installing tensorflow go failed

You can actually ignore that error for now and run:

$ go generate github.com/tensorflow/tensorflow/tensorflow/go/op
$ go test github.com/tensorflow/tensorflow/tensorflow/go
ok  	github.com/tensorflow/tensorflow/tensorflow/go	0.182s
freemanliu

comment created time in 17 days

issue commenttensorflow/tensorflow

Cannot install tensorflow go

Please follow the Go README: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.md

You need to generate the proto files. That means you must use go get -d to download only and then use go generate to BUILD the proto .go files.

glarchev

comment created time in 18 days

issue commenttensorflow/tensorflow

Installing tensorflow go failed

@glarchev Please follow the instructions in the README: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.md

You need to use go get -d to download-only and then use go generate to create the protobuf files.

freemanliu

comment created time in 18 days

issue commenttensorflow/tensorflow

Installing tensorflow go failed

(As a sidenote, please follow the instructions on the Go README which differs slightly from the documentation on tensorflow.org. I'll work on updating the official docs): https://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.md

freemanliu

comment created time in 18 days

issue commenttensorflow/tensorflow

Golang tensorflow no longer builds due to bad import in saved_model.go

(As a sidenote, please follow the instructions on the Go README which differs slightly from the documentation on tensorflow.org. I'll work on updating the official docs): https://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.md

noj-richards

comment created time in 18 days

issue closedtensorflow/tensorflow

Installing tensorflow go failed

<em>Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template</em>

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): mac os 10.14.6
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
  • TensorFlow installed from (source or binary): n/a
  • TensorFlow version: n/a
  • Python version: n/a
  • Installed using virtualenv? pip? conda?: n/a
  • Bazel version (if compiling from source): n/a
  • GCC/Compiler version (if compiling from source): n/a
  • CUDA/cuDNN version: n/a
  • GPU model and memory: n/a

Describe the problem Followed https://www.tensorflow.org/install/lang_go but got this: go get github.com/tensorflow/tensorflow/tensorflow/go package github.com/tensorflow/tensorflow/tensorflow/go/genop/internal/proto/github.com/tensorflow/tensorflow/tensorflow/go/core: cannot find package "github.com/tensorflow/tensorflow/tensorflow/go/genop/internal/proto/github.com/tensorflow/tensorflow/tensorflow/go/core" in any of:

go get -d github.com/tensorflow/tensorflow/tensorflow/go yielded the same result.

Any other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

closed time in 18 days

freemanliu

issue commenttensorflow/tensorflow

Installing tensorflow go failed

Fixed as of https://github.com/tensorflow/tensorflow/commit/4221d1aa4d20ada495771528bb13ca786d0bdbe0

freemanliu

comment created time in 18 days

issue closedtensorflow/tensorflow

Golang tensorflow no longer builds due to bad import in saved_model.go

A recent change (Change-Id: Iefdf75ed88f54d97a0a7d210f5a42f3123205bf2) has broken an import in file tensorflow/blob/master/tensorflow/go/saved_model.go

tfpb "github.com/tensorflow/tensorflow/tensorflow/go/genop/internal/proto/github.com/tensorflow/tensorflow/tensorflow/go/core/framework"

closed time in 18 days

noj-richards

issue commenttensorflow/tensorflow

Golang tensorflow no longer builds due to bad import in saved_model.go

Fixed as of https://github.com/tensorflow/tensorflow/commit/4221d1aa4d20ada495771528bb13ca786d0bdbe0

noj-richards

comment created time in 18 days

pull request commenttensorflow/tensorflow

Fix go proto handling

Fixed in https://github.com/tensorflow/tensorflow/commit/4221d1aa4d20ada495771528bb13ca786d0bdbe0

jhseu

comment created time in 18 days

pull request commenttensorflow/tensorflow

Fix go proto handling

I have a change out to fix the import cycle. Will submit today.

jhseu

comment created time in 18 days

IssuesEvent

issue commenttensorflow/tensorflow

Installing tensorflow go failed

Reopening since there's still an issue. Will resolve tomorrow.

freemanliu

comment created time in 19 days

PR opened tensorflow/tensorflow

Reviewers
Fix go proto handling

Fixes #35133 and #34580

+58 -25

0 comment

19 changed files

pr created time in 23 days

create barnchjhseu/tensorflow

branch : go_fix

created branch time in 23 days

push eventjhseu/tensorflow

Anna R

commit sha b62fd4718b28b02b9a491b9c084f7d468711009e

Split out dependencies on :lib in tensorflow/core:util. PiperOrigin-RevId: 290179502 Change-Id: I7e3364f52d409236d3289a69e16a7b927bc4b42d

view details

TensorFlower Gardener

commit sha f0cf1dc02a6c1a21e1559f2644161571a2046a59

Merge pull request #34624 from noble-ai:dense_sparse_matmul PiperOrigin-RevId: 290179646 Change-Id: I1bd945d88215e3900005c3c70b4525ad3707cf91

view details

Jiho Choi

commit sha 806d8c8c7dd5ec8d8503478c657c15df1af28273

Make XPlaneBuilder to use reserved metadata ids for known stats in XPlaneSchema. Also, remove more dependencies to MetadataMatcher. PiperOrigin-RevId: 290183127 Change-Id: I5807c0d855134f9ee7ad4bcee2aa37c3cb0d1d9f

view details

Yunxing Dai

commit sha 6a6261c0a0e803891af95f5e754180739df1897d

Use xla update slice as gradient of slice. This change removes the constant requirement of slice position. PiperOrigin-RevId: 290185145 Change-Id: Id61aadf4d7ec3c869c17cbeb660516a4b4103a66

view details

A. Unique TensorFlower

commit sha c6fa2dc9e4c3330aa3e21014efdba3a18108d51b

Fix a bug that kernel launch events are over calculated. PiperOrigin-RevId: 290185357 Change-Id: I87ae6f8b43caf4678273e4d6e2f1b4c3c1d53a6f

view details

Allen Lavoie

commit sha 80f0540bc83d55a5e33407f38e1c370f5853814d

Fix TPU initialization for local servers Requires an identity in the TPU initialization function to avoid placement errors. I believe this only comes up when using local servers (i.e. affects mostly testing; we do have plenty of tests for TPUs on remote jobs). Also exposes a mapping from job name to TPU topology. I have a use for it: we need to look up the topology corresponding to the correct job when replicating a function. PiperOrigin-RevId: 290188617 Change-Id: I24e1e5995f6f55b565a1aac05909698ac3ee49d8

view details

wyzhao

commit sha c7f606f1de9f693e07706423f0c04364d3ca0587

loose shape check in PointwiseToLinalgConverter PointwiseToLinalgConverter only needs static rank not static shape, change accordingly to loose the restriction.

view details

Ian Langmore

commit sha 18645e7a3c63f539d3c7746792e4554ec2a7b6cf

BUGFIX: Properly set input_output_dtype on Circulant.inverse(). PiperOrigin-RevId: 290189697 Change-Id: I3acc7eb170cd0f14acc3f9729489d3b07afd0e70

view details

Tong Shen

commit sha 6b525249b8be9db9fd58a6e22696229fac538047

Add some XLA frontend attribute names. PiperOrigin-RevId: 290190699 Change-Id: I421510149dbc759fbe3e06a4990502d4772962b5

view details

A. Unique TensorFlower

commit sha db8a74a737cc735bb2a4800731d21f2de6d04961

[TFLRT] Reorganize tf_runtime directory. PiperOrigin-RevId: 290193702 Change-Id: I6d7b8895d660d9d59d81ecb50f58e3e4da40c25f

view details

A. Unique TensorFlower

commit sha 2ca0e2e810257334adf6138f5668674892067cec

Make XPlaneBuilder to use reserved metadata ids for known stats in XPlaneSchema. Also, remove more dependencies to MetadataMatcher. PiperOrigin-RevId: 290198703 Change-Id: I46cdbfe42d0a4306ff6e14544a5aa239989ccaf0

view details

A. Unique TensorFlower

commit sha 17d889c3d4a4adc416de7d7c3f159de6e4587a64

Print fused_instructions_computation() only when the instruction is fusion. Also, mark instructions as fused before they are potentially removed via unique pointers. PiperOrigin-RevId: 290204832 Change-Id: I6eaf68b81685ff8eb03458045ef440da5040740e

view details

Scott Zhu

commit sha 714e90aef6528d3d083a2ada53ec3c2d86320607

Update docstring for keras.layers.InputLayer. PiperOrigin-RevId: 290212523 Change-Id: Ie59ff7d410355e7ce69585fdea123812c86db2ad

view details

Reed Wanderman-Milne

commit sha 81b21433716e4fad1d51c85219406ace2b6176c0

Correctly handle Variable.assign*(...).assign*(...) in graph mode. For example, now `session.run(var.assign_add(1).assign_add(1))` will correctly increase the variable's value by 2. Also update variable assignment docstrings to indicate they return the variable. PiperOrigin-RevId: 290213217 Change-Id: I2a4c35edaaf722e21a41af9f85fd446afbec6995

view details

Scott Zhu

commit sha c61a87fd02f52d27045a5ebc4268076179c2d7fc

Update docstring for keras.layers.Activation PiperOrigin-RevId: 290213223 Change-Id: I464962c82f291a290e966c3cf3f26dba7804fd45

view details

A. Unique TensorFlower

commit sha a5b089c00039424cc74bb156577936a67b0e8437

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 290213270 Change-Id: I690bfaa9afbc91e7e7d49330d558bff6df90d36b

view details

fsx950223

commit sha 2f5fd5185aa5e2d55332b256c47b602797633eb6

use a safer way for numpy

view details

Scott Zhu

commit sha 9361cd6fe21c78fea9260935d5121c9c9cd76f93

Update docstring for keras.layers.ReLU. PiperOrigin-RevId: 290215220 Change-Id: Iff1321a25f7ee3c7a25c9725de8bbfcb7b65434c

view details

fsx950223

commit sha c24ea21d5173366344a656706c545366b950ba6a

add test case

view details

fsx950223

commit sha a712ade3d00e429d4fa10191fff47dbadef2fa00

format code

view details

push time in 23 days

startedgoogle-research/flax

started time in a month

startedamirgholami/PyHessian

started time in a month

issue commenttensorflow/tensorflow

[go] Include MetaGraphDef in SavedModel for go library

Do you need this for the Signature field? That has been added since this was filed.

elaPa

comment created time in a month

issue commenttensorflow/tensorflow

TPUStrategy broken in TF2 Keras

That error indicates that the backend is old relative to the Python version. You'll have to wait until the TF 2.1 release for this to work, or use a nightly TPU backend.

kpe

comment created time in 3 months

startedgoogle-research/seed_rl

started time in 3 months

push eventtensorflow/ecosystem

WeichenXu

commit sha 1b3a0529bfa7e8eca6419cb5bef4d6f85b86c81e

Fix flaky test "LocalWriteSuite" (#144) * fix flaky test and update pom artifact * init pr

view details

push time in 4 months

PR merged tensorflow/ecosystem

Fix flaky test "LocalWriteSuite" cla: yes

Fix flaky test "LocalWriteSuite"

The issue is in:

The test first create a temporary path by java.nio.Files.createTempDirectory, then delete it, then use the allocated temp path to be the saving path for dataframe. This is risky. because when we delete a directory, the path is released and can be allocated as new temp dir in other place, which cause the next line df.save (errorIfExisting mode) failed.

So I update the code. Do not delete the created temp dir, but create a sub-dir inside it as the df saving destination.

+5 -4

0 comment

1 changed file

WeichenXu123

pr closed time in 4 months

push eventWeichenXu123/ecosystem

mathon

commit sha bf540fe53ad61d41a230ae724305057c3f2ec8ed

fix Array[String] for SequenceExample (#140)

view details

Jonathan Hseu

commit sha a33db1102205e5ce95f14ca054de9d9ea707a63f

Merge branch 'master' into fix_flaky_test

view details

push time in 4 months

issue commenttensorflow/tensorflow

TPU, model.fit : 2GB of RAM limit

Yes, you'll likely have to switch to tf.data inputs if you have that large of datasets.

agsci2017

comment created time in 4 months

pull request commenttensorflow/ecosystem

Change spark-tensorflow-connector dependency to be spark 3.0.0 snapshot

Yeah, I don't mind keeping this open.

WeichenXu123

comment created time in 4 months

push eventtensorflow/ecosystem

mathon

commit sha bf540fe53ad61d41a230ae724305057c3f2ec8ed

fix Array[String] for SequenceExample (#140)

view details

push time in 4 months

PR merged tensorflow/ecosystem

fix Array[String] for SequenceExample cla: yes

ArrayType(String) should be a Feature not a FeatureList.

If row element is ArrayType(StringType, _), it will go to https://github.com/tensorflow/ecosystem/blob/master/spark/spark-tensorflow-connector/src/main/scala/org/tensorflow/spark/datasources/tfrecords/serde/DefaultTfRecordRowEncoder.scala#L194, triger a RuntimeException.

case _ => throw new RuntimeException(s"Cannot convert row element ${row.get(index)} to FeatureList.")
+7 -3

8 comments

2 changed files

doldre

pr closed time in 4 months

push eventdoldre/ecosystem

Fabian Höring

commit sha 4d1369c2972c6fdfa739dda105b3fc86bc911a62

Add udf DataFrameTfrConverter (#132) * Add udf DataFrameTfrConverter - allows to create serialized tf records from all columns of a DataFrame - this is useful to do Tensorflow inference on tf records exactly the same way as tf records creation - added Python inference example to README.md * Use SharedSparkSessionSuite for creating SparkContext * Provide example for inference use case on dataframe with tf records, estimator and vectorized udf - Requires Spark 2.4, Tested compatibility with the jar that is in spark 2.3

view details

Jonathan Hseu

commit sha fc557574f082a10a48632d34b16c756589257f9b

Merge branch 'master' into fix-array-string-sequence-example

view details

push time in 4 months

pull request commenttensorflow/ecosystem

Change spark-tensorflow-connector dependency to be spark 3.0.0 snapshot

I'm not opposed to this, but wouldn't it be better to wait until Spark 3.0.0 is released?

WeichenXu123

comment created time in 4 months

issue commenttensorflow/tensorflow

When importing TensorFlow, error loading Hadoop

Yep, sending out a change to lazy load.

wchargin

comment created time in 5 months

push eventtensorflow/ecosystem

Fabian Höring

commit sha 4d1369c2972c6fdfa739dda105b3fc86bc911a62

Add udf DataFrameTfrConverter (#132) * Add udf DataFrameTfrConverter - allows to create serialized tf records from all columns of a DataFrame - this is useful to do Tensorflow inference on tf records exactly the same way as tf records creation - added Python inference example to README.md * Use SharedSparkSessionSuite for creating SparkContext * Provide example for inference use case on dataframe with tf records, estimator and vectorized udf - Requires Spark 2.4, Tested compatibility with the jar that is in spark 2.3

view details

push time in 5 months

PR merged tensorflow/ecosystem

Add udf DataFrameTfrConverter cla: yes
  • allows to create serialized tf records from all columns of a DataFrame
  • this is useful to do Tensorflow inference on tf records exactly the same way as tf records creation
  • added Python example to README.md

Sidenotes:

  • the test is mostly inspired by TfRecordRowEncoderTest
  • I explicitly didn't include tests for Decimal types, this is indeed not working as described here https://github.com/tensorflow/ecosystem/issues/107 but is not failing in the other tests because it is only working on Row objects instead on a real Spark session, if fixed needs to be fixed in another review
  • udf created as described here: https://stackoverflow.com/questions/41780141/how-to-use-scala-udf-in-pyspark
+194 -0

6 comments

3 changed files

fhoering

pr closed time in 5 months

pull request commenttensorflow/ecosystem

fix Array[String] for SequenceExample

Mind adding a test?

doldre

comment created time in 5 months

issue commenttensorflow/tensorflow

tf.keras model.evaluate (and fit with validation) possible leak when running on TPU

Yeah, this is a known issue with the interaction with Keras + tf.distribute.Strategy in the TF 1.x. This is currently fixed in TF 2.x.

raulmt

comment created time in 6 months

more