profile
viewpoint
Jonathan Hseu jhseu @tensorflow @google San Francisco, CA Google Brain

tensorflow/rust 2714

Rust language bindings for TensorFlow

tensorflow/ecosystem 1144

Integration of TensorFlow with other open-source frameworks

jhseu/nca 37

Neighborhood Components Analysis in C++

vladfi1/DeepSmash 15

Attempt at phillip rewrite in rllib. Also attempts at training from slippi replays.

jhseu/pycassa 2

Simple python library for Cassandra

jhseu/tensorflow 1

Computation using data flow graphs for scalable machine learning

guptapriya/community 0

Stores documents used by the TensorFlow developer community

jhseu/buildifier 0

A bazel BUILD file formatter and editor

jhseu/dolphin 0

Dolphin is a GameCube / Wii emulator, allowing you to play games for these two platforms on PC with improvements.

jhseu/ecosystem 0

Integration of TensorFlow with other Open-source Frameworks

starteddeepmind/dqn_zoo

started time in an hour

issue commenttensorflow/tensorflow

Golang Tensorflow v2.2.0 fails install

@silverark How are you installing protoc? It looks like you're missing the default protos from your installation. Typically they're in /usr/local/include after installation, like so:

jhseu0:~$ locate any.proto
/usr/include/google/protobuf/any.proto
JacobSMoller

comment created time in 2 days

startedmit-pdos/xv6-riscv

started time in 2 days

issue commenttensorflow/tensorflow

Golang Tensorflow v2.3.0 installation fails

This is actually fixed at master following the instructions at: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.md

The fix didn't make it into TensorFlow 2.3. (The go get -d command will warn about package not found, but you can ignore it since it's built with go generate)

Note that you must build an updated version of libtensorflow.so as indicated in the instructions, otherwise you'll get errors.

jobale

comment created time in 4 days

push eventtensorflow/ecosystem

Greg Roodt

commit sha 77abfa1fcef2aacb8dba99838521bad8a567fca8

Spark 3.0.0 with Scala 2.12 (#169)

view details

push time in 4 days

PR merged tensorflow/ecosystem

Spark 3.0.0 with Scala 2.12 cla: yes

Spark 3.0.0 was officially released in June. The project is providing Scala 2.12 binaries.

I think the Tensorflow connector should now be targeting this newer version of Spark by default, and perhaps Spark 2.x in a branch. Spark 2.4.7 and other maintenance releases are likely in future, but I expect they will be bug fixes and backports or new features.

+5 -5

2 comments

1 changed file

groodt

pr closed time in 4 days

push eventgroodt/ecosystem

Andrew Audibert

commit sha 15f423571c5381355e12081f68ca6d58e97fe252

Update tf_std_data_server to use new server config argument (#170)

view details

Jonathan Hseu

commit sha b4688fe32ebcdfb47206ef5a3d272eb6c2f184ea

Merge branch 'master' into groodt-spark-3.0.0

view details

push time in 4 days

PullRequestReviewEvent
PullRequestReviewEvent

pull request commenttensorflow/tensorflow

Adding HDFS connection cache on tensorflow side can improve performance by 20+ times #43187

Can you verify whether this is still an issue in recent versions of Hadoop before we merge? If it still is an issue, I don't mind merging this.

yuanbopeng

comment created time in 7 days

Pull request review commenttensorflow/ecosystem

Spark 3.0.0 with Scala 2.12

          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">     <modelVersion>4.0.0</modelVersion>     <groupId>org.tensorflow</groupId>-    <artifactId>spark-tensorflow-connector_${scala.binary.version}</artifactId>+    <artifactId>spark-tensorflow-connector_2.12</artifactId>

Can this continue to use ${scala.binary.version}?

groodt

comment created time in 17 days

PullRequestReviewEvent

push eventtensorflow/ecosystem

Andrew Audibert

commit sha 15f423571c5381355e12081f68ca6d58e97fe252

Update tf_std_data_server to use new server config argument (#170)

view details

push time in 17 days

PR merged tensorflow/ecosystem

Update tf_std_data_server to use new server config argument cla: yes

This is needed after https://github.com/tensorflow/tensorflow/commit/b129d3dc1cfc1944b874c514724146df971ea935

+8 -6

0 comment

1 changed file

aaudiber

pr closed time in 17 days

PullRequestReviewEvent

PR closed tensorflow/tensorflow

Reviewers
Check in generated protobufs for Go cla: yes pending merge internally prtype:bugfix ready to pull size:XL

To make it more idiomatic and easier to install. See #35133

Gunhan, any objections to doing this? It adds 1.3 MB of generated files to the repo.

+25762 -0

9 comments

59 changed files

jhseu

pr closed time in a month

pull request commenttensorflow/tensorflow

Check in generated protobufs for Go

We had a discussion with the Go team who discouraged checking this in because other projects had faced issues with conflicts between generated protobufs and the protobuf files generated by bazel. Closing for now.

Users should run go generate as mentioned in the README.

jhseu

comment created time in a month

startedfastai/fastbook

started time in a month

starteddeepmind/optax

started time in a month

starteddeepmind/chex

started time in a month

issue commenttensorflow/tensorflow

Golang Tensorflow v2.2.0 fails install

@JacobSMoller Yeah, go get doesn't work for now. I made a pull request to fix it, but there were some objections to supporting both the bazel and generated protobufs to make it work.

For that error: it occurs when there's a version mismatch between libtensorflow.so and the TF ops in the source code. See my command above for building libtensorflow.so.

JacobSMoller

comment created time in 2 months

issue closedtensorflow/tensorflow

Golang Tensorflow v2.2.0 fails install

System information

  • OS Platform and Distribution (Macos catalina 10.15.4):
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
  • TensorFlow installed from (source or binary):
  • TensorFlow version:
  • Python version:
  • Installed using virtualenv? pip? conda?:
  • Bazel version (if compiling from source):
  • GCC/Compiler version (if compiling from source):
  • CUDA/cuDNN version:
  • GPU model and memory:

Describe the problem

Provide the exact sequence of commands / steps that you executed before running into the problem

Following https://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.mdhttps://github.com/tensorflow/tensorflow/blob/master/tensorflow/go/README.md

go generate github.com/tensorflow/tensorflow/tensorflow/go/op fails with failed to process "api_def_VarHandleOp.pbtxt": Attribute allowed_devices not defined in base api for VarHandleOp exit status 1 go/src/github.com/tensorflow/tensorflow/tensorflow/go/op/generate.go:18: running "go": exit status 1

go test github.com/tensorflow/tensorflow/tensorflow/go passes

building or running go code using tensorflow then proceeds to fail with the following output go: finding github.com/tensorflow/tensorflow/tensorflow/go/core/core_protos_go_proto latest go: finding github.com/tensorflow/tensorflow/tensorflow/go/core latest go: finding github.com/tensorflow/tensorflow/tensorflow/go latest go: finding github.com/tensorflow/tensorflow/tensorflow latest build command-line-arguments: cannot load github.com/tensorflow/tensorflow/tensorflow/go/core/core_protos_go_proto: cannot find module providing package github.com/tensorflow/tensorflow/tensorflow/go/core/core_protos_go_proto

the code run is the example that prints tf library version from https://www.tensorflow.org/install/lang_go

Any other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

closed time in 2 months

JacobSMoller

issue commenttensorflow/tensorflow

Golang Tensorflow v2.2.0 fails install

Fixed as part of: https://github.com/tensorflow/tensorflow/commit/e5e495db7bee77cd0fd5dda3b06bd743cbcf1ef8

Note that it requires from master: bazel build -c opt //tensorflow/tools/lib_package:libtensorflow tar -xzvf the package in /usr/local go generate github.com/tensorflow/tensorflow/tensorflow/go/op

JacobSMoller

comment created time in 2 months

issue commenttensorflow/tensorflow

Training error on Cloud TPUs

We don't currently support loading local files this way, although it'll work at some point in the near future. For now, you can load them outside of tf.data as numpy arrays and feed them instead.

caffeine-lab

comment created time in 2 months

push eventtensorflow/ecosystem

Andrew Audibert

commit sha 49845e4f6f06921cea679adf941a802019ff36ac

Rename master to dispatcher in tf.data service example. (#167) master was renamed to dispatcher in https://github.com/tensorflow/tensorflow/commit/c3f2d3d5710caa789eb54a4d24280a818b9375ba

view details

push time in 2 months

PR merged tensorflow/ecosystem

Rename master to dispatcher in tf.data service example. cla: yes

master was renamed to dispatcher in https://github.com/tensorflow/tensorflow/commit/c3f2d3d5710caa789eb54a4d24280a818b9375ba

+14 -14

0 comment

4 changed files

aaudiber

pr closed time in 2 months

PR closed tensorflow/tensorflow

Fix imports of generated go proto bindings cla: yes ready to pull size:XS

Fixes

#39744

Summary

After running the following in accordance with the installation instructions for go:

$ go get -d github.com/tensorflow/tensorflow/tensorflow/go
$ go generate github.com/tensorflow/tensorflow/tensorflow/go/op
$ go test github.com/tensorflow/tensorflow/tensorflow/go

The first command generates error message unrelated to this PR, a fix for which is ready to pull. That error message is relatively low impact and should not impact the functionality of the library after the go generate command is run. However, the third command generates the following error:

$ go test github.com/tensorflow/tensorflow/tensorflow/go
../../../go/src/github.com/tensorflow/tensorflow/tensorflow/go/saved_model.go:25:2: cannot find package "github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto" in any of:
	/home/abls/go/src/github.com/tensorflow/tensorflow/tensorflow/go/vendor/github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto (vendor tree)
	/usr/local/go/src/github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto (from $GOROOT)
	/home/abls/go/src/github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto (from $GOPATH)

This is caused by a bad import that seemed to be from an accident during a refactor. The problematic import that occurs in several files is:

import corepb "github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto"

This line is supposed to point to the generated files, however the generated files are actually placed in github.com/tensorflow/tensorflow/tensorflow/go/core/core_protos_go_proto due to the following line in several proto files in tensorflow/core/protobuf read by protoc during generation:

option go_package = "github.com/tensorflow/tensorflow/tensorflow/go/core/core_protos_go_proto";

The fix is simply to correct the import lines to point to the actual location of the files.

+3 -3

1 comment

3 changed files

abls

pr closed time in 2 months

pull request commenttensorflow/tensorflow

Fix imports of generated go proto bindings

Needed to change this a bit to match the BUILD rule, which is required for bazel compatibility: https://github.com/tensorflow/tensorflow/commit/e5e495db7bee77cd0fd5dda3b06bd743cbcf1ef8

abls

comment created time in 2 months

issue closedtensorflow/tensorflow

tensorflow for go install failed

MacOS version(10.15.4)

  1. I followed the instruction of https://www.tensorflow.org/install/lang_c ,I installed the c Library .but it doesn't word.the error is > hello_tf.c:2:10: fatal error: 'tensorflow/c/c_api.h' file not found

  2. github.com/tensorflow/tensorflow/tensorflow/go/saved_model.go:25:2: cannot find package "github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto" in any of: /usr/local/go/src/github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto (from $GOROOT) /Users/moses/workspace/godemo/src/github.com/tensorflow/tensorflow/tensorflow/go/core/protobuf/for_core_protos_go_proto (from $GOPATH)

how to solve this problem

closed time in 2 months

moseshu

issue commenttensorflow/tensorflow

tensorflow for go install failed

Fixed with https://github.com/tensorflow/tensorflow/commit/e5e495db7bee77cd0fd5dda3b06bd743cbcf1ef8

moseshu

comment created time in 2 months

push eventtensorflow/ecosystem

Liang Zhang

commit sha 8d96a9f9bcbd6e0404db03c6b06bb5b6cb72513f

Update the example for spark-tensorflow-distributor (#166) * fix example * fix example

view details

push time in 2 months

PR merged tensorflow/ecosystem

Update the example for spark-tensorflow-distributor cla: yes

This PR fixes the data downloading issue in the example code.

Reproduce: On a cluster with multiple GPUs per worker node, with spark.resources.tasks.gpu.amount set to 1, running the original example will trigger an error related to data downloading.

Cause: There will be multiple tasks running on the same worker and each task will try to write the data to the same path, which will corrupt the data.

Fix: Randomize the file path.

+28 -36

0 comment

2 changed files

liangz1

pr closed time in 2 months

startedfoambubble/foam

started time in 3 months

more