profile
viewpoint
Yong Tang yongtang San Francisco Bay Area, CA https://www.linkedin.com/in/yong-tang/ Open source contributor, CoreDNS, Docker, TensorFlow committer.

coredns/coredns 5595

CoreDNS is a DNS server that chains plugins

grpc/grpc-web 3815

gRPC for Web Clients

docker/swarmkit 2295

A toolkit for orchestrating distributed systems at any scale. It includes primitives for node discovery, raft-based consensus, task scheduling and more.

tensorflow/io 266

Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO

docker/go-connections 125

Utility package to work with network connections

docker/go-units 91

Parse and print size and time units in human-readable format

yongtang/clamav.js 21

A node.js library for ClamAV

yongtang/hadoop-xz 11

XZ (LZMA/LZMA2) Codec for Apache Hadoop

yongtang/clamav4j 5

Java library for ClamAV

push eventyongtang/io

Yong Tang

commit sha d5ce3f4ffd6a6f85d2d689cc82f1017048b5e0fc

Update to build Windows for STDIN_FILENO and read Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 20 minutes

push eventyongtang/io

Yong Tang

commit sha ce91cbd355e716ed0391800533e66653a24b9e2d

Update to build Windows for STDIN_FILENO and read Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 37 minutes

push eventyongtang/io

Yong Tang

commit sha ebb1c2b5506ef9a16d8acdc40db58b908e53b6c9

Update to build Windows for STDIN_FILENO and read Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 43 minutes

push eventyongtang/io

Yong Tang

commit sha 041ec840c4248ec69fb4f515a27ed2da3c52c4be

Update to build Windows for STDIN_FILENO and read Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in an hour

push eventyongtang/tensorflow

Lamar

commit sha 71dd20a99530f22c86a987088484db8f4f227e52

fixed static sized arrays with variable length using const int or int for the size of an array implies that it has variable length (ill-formed, https://en.cppreference.com/w/cpp/language/ub), static arrays' lengths should be constexpr or a macro constant

view details

A. Unique TensorFlower

commit sha 80acd88cd43f09a1a2980792e3955f2ce5147bfd

Automated rollback of commit 65bad48a424d76e943b1d5c24b868f77c0130650 PiperOrigin-RevId: 295346786 Change-Id: I089897b1b02373c02049b03bc3b986a2442a5127

view details

A. Unique TensorFlower

commit sha 1b8c5220b373d1fb05cf05748e4bf04c665a8959

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295348678 Change-Id: Ic4da02a2b6304a413b6d620ff6559fbc32667293

view details

A. Unique TensorFlower

commit sha 6d00b470f51a62536b3b56c8facc80d871214df5

Allow IndexLookup to load vocab from a file. PiperOrigin-RevId: 295348705 Change-Id: I4f2bb8e51bbeb94a916c57a0fc3423ea80b96956

view details

A. Unique TensorFlower

commit sha 7e53afc1611fc6031df77134ee4115405ec21e41

Fix RBE config for CUDA 10.1. PiperOrigin-RevId: 295356882 Change-Id: I2f0fefa29b827beab420c2e2575a18af1d75c5c2

view details

A. Unique TensorFlower

commit sha 721b5ac4bdb4389fdd454fcc49e8eb33c14ad590

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295357369 Change-Id: Iba7b1806b4102681239c673ed86b8b16b2e049e5

view details

A. Unique TensorFlower

commit sha 246944ffa9d0e73ad034f858eb2b231ca3adadc6

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295365277 Change-Id: I195ff98dde0cca9abe66975f8c95201b3674af25

view details

T.J. Alumbaugh

commit sha a86cf76c883dc27e326c3043ef3962dbf7490f5e

Enable Ruy-based GEMM for hybrid convolutions, and cache on narrow GEMMs PiperOrigin-RevId: 295368863 Change-Id: If660cb2f1acee8bf4f7e19785a2d0f13171b400a

view details

A. Unique TensorFlower

commit sha a515b1f9b7c9caf18e574fde0ab876e75acd5e20

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295382032 Change-Id: I96b59404c563e2bd171ac585e2c1bea25c44d852

view details

A. Unique TensorFlower

commit sha 992b5eb9facc724592661bfdf22e6f5765a0c63c

Automated rollback of commit a86cf76c883dc27e326c3043ef3962dbf7490f5e PiperOrigin-RevId: 295384253 Change-Id: I823ff194edf3f836c29958b043e39ec506a2470d

view details

Jose Baiocchi

commit sha 9f86c8c5a42e51b42880b69cde7f43f60d7276cc

Break dependency of profiler_backends on core:core_cpu_lib PiperOrigin-RevId: 295392448 Change-Id: I9f6f92cad23918aed15aac8f7bcca255752a2eec

view details

A. Unique TensorFlower

commit sha 00cb358ab2e67d0b06a21901ded13c57fd47e673

Fix typo in tf.stack PiperOrigin-RevId: 295395911 Change-Id: I776359b731a38d0f6f98f497fcc487adc552da2c

view details

A. Unique TensorFlower

commit sha 2d94eb1258e26662f8891cab56f7979890aed11e

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295399445 Change-Id: I2af3deed372a9068071145437fed4c65ef8af2df

view details

A. Unique TensorFlower

commit sha 05e92858200c852994f4aa7d27c691ddedaf2903

Disable broken test. PiperOrigin-RevId: 295403974 Change-Id: Iebd8b64d6137f53528e1674148acfefd6f530559

view details

A. Unique TensorFlower

commit sha 829ea85ef4198783494ecd918b40d660daaeaa51

compat: Update forward compatibility horizon to 2020-02-16 PiperOrigin-RevId: 295409487 Change-Id: I0e65dacc957e3dea02f2b519bc85b75513719be8

view details

A. Unique TensorFlower

commit sha d9f836a188bb534048bffaccf3e5b3c428da2fe7

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295417545 Change-Id: If315e11027cc9f3f6ed8c81fe99047bc07a08dfa

view details

A. Unique TensorFlower

commit sha be940c6d059557e8757391a8d73554d54796139d

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295444320 Change-Id: Ia931402f0e9d7a005e710862d3e631bba83add36

view details

A. Unique TensorFlower

commit sha 5589d47843656fad7a84d9ed156006e60ffab649

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295462068 Change-Id: I52f9eaf8900317fe643419c811c0d9c489486d26

view details

A. Unique TensorFlower

commit sha d6e8d078c9a9cd3d05f8d008673db6878d76a812

Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 295471666 Change-Id: Ib971e46e2b7958734af536447ea1fad2548d2092

view details

Juho Ha

commit sha 04903ab265d0c066179d2a5a1caff42c384f8007

Add missing exported files required to build tensorflow-lite(-gpu).aar PiperOrigin-RevId: 295472743 Change-Id: Idb7219338ee087f2544a8af821c138e738913370

view details

push time in an hour

push eventyongtang/io

Yong Tang

commit sha ee65d241eefaa27d29728b7912921356417f3cf6

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in an hour

create barnchyongtang/io

branch : text_ops

created branch time in an hour

push eventyongtang/io

Yong Tang

commit sha 69794b98375d42c1ca5a5dd9d5d85b952b5ef8dd

Enable Avro ops build on Windows (#790) * Enable Avro ops build on Windows Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable Snappy for Avro Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Consolidate boost BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Consolidate thrift BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add missing dependency Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable XZ Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 hours

push eventcoredns/coredns

dependabot-preview[bot]

commit sha 4f9a819c464ff733b48b1b5f3449fd78d1349ce7

build(deps): bump gopkg.in/DataDog/dd-trace-go.v1 from 1.20.1 to 1.21.0 (#3675) Bumps [gopkg.in/DataDog/dd-trace-go.v1](https://github.com/DataDog/dd-trace-go) from 1.20.1 to 1.21.0. - [Release notes](https://github.com/DataDog/dd-trace-go/releases) - [Commits](https://github.com/DataDog/dd-trace-go/compare/v1.20.1...v1.21.0) Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

view details

push time in 3 hours

delete branch coredns/coredns

delete branch : dependabot/go_modules/gopkg.in/DataDog/dd-trace-go.v1-1.21.0

delete time in 3 hours

PR merged coredns/coredns

Reviewers
build(deps): bump gopkg.in/DataDog/dd-trace-go.v1 from 1.20.1 to 1.21.0 dep

Bumps gopkg.in/DataDog/dd-trace-go.v1 from 1.20.1 to 1.21.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/DataDog/dd-trace-go/releases">gopkg.in/DataDog/dd-trace-go.v1's releases</a>.</em></p> <blockquote> <h2>1.21.0</h2> <p>This release includes new rule-based trace sampling, asynchronous payload delivery, as well as other improvements and bug fixes. 🐶</p> <h3>Features</h3> <ul> <li>ddtrace/tracer: add rule-based sampler (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/546">#546</a>)</li> <li>ddtrace/tracer: send payloads asynchronously (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/549">#549</a>)</li> <li>contrib/gomodule/redigo: Support ConnWithTimeout interface (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/554">#554</a>) (Thanks <a href="https://github.com/abustany">@abustany</a>)</li> <li>contrib/google.golang.org/grpc: add WithIgnoredMethods (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/577">#577</a>) (Thanks <a href="https://github.com/mathetake">@mathetake</a>)</li> </ul> <h3>Fixes and Improvements</h3> <ul> <li>contrib/net/http: cause the default config to set the analytics rate in spanOpts (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/564">#564</a>)</li> <li>contrib/database/sql: implement driver.NamedValueChecker (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/568">#568</a>)</li> <li>ddtrace/tracer: fix regression when flushing based on size (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/570">#570</a>)</li> <li>ddtrace/mocktracer: make (*mockspan).Finish idempotent (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/573">#573</a>) (Thanks <a href="https://github.com/phoenix2x">@phoenix2x</a>)</li> </ul> <p>To view all changes check out <a href="https://github.com/DataDog/dd-trace-go/compare/v1.20.1...v1.21.0">new commits</a> and the <a href="https://github.com/DataDog/dd-trace-go/milestone/37?closed=1">1.21.0 milestone</a>.</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/DataDog/dd-trace-go/commit/476eb07a59528ada834033720b91d62287301fa2"><code>476eb07</code></a> Revert "contrib/gin-gonic/gin: correct resource name" (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/587">#587</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/77043620bb1b0ffe53613d117a48e144cc266747"><code>7704362</code></a> contrib/google.golang.org/grpc: add WithIgnoredMethods (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/577">#577</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/e235f115c50900d6db7af8cb4885289b981e6562"><code>e235f11</code></a> contrib/gomodule/redigo: Support ConnWithTimeout interface (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/554">#554</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/254fd3b34eb4afa56b4ba5dfb37d7470c35ca3ae"><code>254fd3b</code></a> ddtrace/opentracer: remove old TODO (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/581">#581</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/3eef4276a6f84efe96d3adee4d4dc6da412c366d"><code>3eef427</code></a> ddtrace/tracer: log a warning when an unrecognized propagator is encountered....</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/55b601f5b136994d3e446795b4865e1fd9ef01b2"><code>55b601f</code></a> ddtrace/tracer: remove syncPush and flushChan (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/575">#575</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/087c9a7e1ec3eb1cbbd23bb340391902ed3a6924"><code>087c9a7</code></a> ddtrace/mocktracer: make (*mockspan).Finish idempotent (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/573">#573</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/e7bc00834b03990b49887dc94a59bb6c9bc90a3f"><code>e7bc008</code></a> Add dual license (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/571">#571</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/b12f0feaff5773fde59bc44a73951a3c69c3e85f"><code>b12f0fe</code></a> contrib/gin-gonic/gin: correct resource name (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/574">#574</a>)</li> <li><a href="https://github.com/DataDog/dd-trace-go/commit/5942c57d0caf089a3eaacc9e3366c5c77c7e633d"><code>5942c57</code></a> ddtrace/tracer: fix regression when flushing based on size (<a href="https://github-redirect.dependabot.com/DataDog/dd-trace-go/issues/570">#570</a>)</li> <li>Additional commits viewable in <a href="https://github.com/DataDog/dd-trace-go/compare/v1.20.1...v1.21.0">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
  • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
  • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
  • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language
  • @dependabot badge me will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in your Dependabot dashboard:

  • Update frequency (including time of day and day of week)
  • Pull request limits (per update run and/or open at any time)
  • Out-of-range updates (receive only lockfile updates, if desired)
  • Security updates (receive only security updates, if desired)

</details>

+3 -1

0 comment

2 changed files

dependabot-preview[bot]

pr closed time in 3 hours

issue commenttensorflow/io

Get a DICOM tag that is a sequence

@EstebanGuillen I am not able to see the attachment. Looks like GitHub filters out the attachment from the email. Can you attach the file to this issue page?

EstebanGuillen

comment created time in 3 hours

delete branch yongtang/io

delete branch : avro3

delete time in 3 hours

push eventtensorflow/io

Yong Tang

commit sha 69794b98375d42c1ca5a5dd9d5d85b952b5ef8dd

Enable Avro ops build on Windows (#790) * Enable Avro ops build on Windows Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable Snappy for Avro Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Consolidate boost BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Consolidate thrift BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Add missing dependency Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable XZ Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 hours

PR merged tensorflow/io

Enable Avro ops build on Windows

Enable Avro ops build on Windows.

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+165 -169

0 comment

6 changed files

yongtang

pr closed time in 3 hours

PR opened tensorflow/io

Enable Avro ops build on Windows

Enable Avro ops build on Windows.

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+165 -169

0 comment

6 changed files

pr created time in 12 hours

push eventyongtang/io

Yong Tang

commit sha 135820f38f0a2b94b2b31bec4d64e4ecdd698c05

Enable Avro ops build on Windows Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha d9f572246c183ddf120b5a619b7cbb13c0674b50

Enable Snappy for Avro Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 0f65eda6878a3eba8978e7a174dbfb411ae7924b

Consolidate boost BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 63665ca43bd91aece28b5c7830f3beb15b324077

Consolidate thrift BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 34eb214e48dd5a0dd3c97183d61a7d3aad866ea2

Add missing dependency Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 7ae016daf4bd968bc6fcb5d01a13ecb5f523d4d5

Enable XZ Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 12 hours

push eventyongtang/io

Yong Tang

commit sha 743910ec2339ec9fe30347c72cbfadbc653c886c

Enable XZ Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 14 hours

push eventyongtang/io

Yong Tang

commit sha f88cf692d6fe201770fe12dfaca514dc5adf65c4

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 14 hours

push eventyongtang/io

Yong Tang

commit sha 1975b44b4fd1c85a7d32f2a9fe965e98d2ee4235

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 14 hours

push eventyongtang/io

Yong Tang

commit sha 791eace37227fa2ad17154940da31bb54224a8c0

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 14 hours

push eventyongtang/io

Yong Tang

commit sha 1a8ecad7eafb6dd2af88322f563af314a50a64f7

Add missing dependency Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 995ddaade9db6eec02e5d85c53648d63ade3de78

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha ce5183382c27689c38b504a3c5c95a4e204a32ce

Enable XZ Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 15 hours

create barnchyongtang/io

branch : avro3

created branch time in 19 hours

push eventyongtang/io

Yong Tang

commit sha de490ba28e6b0551b905a4966613a8c45f1583dd

Add additional patch on grpc to allow using python3 only (#789) * Add additional patch on grpc to allow using python3 only Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update README.md to only install python3, and add sudo Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to include CentoS 8 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update macOS section Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to add python-devel Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 19 hours

delete branch yongtang/io

delete branch : python3

delete time in 19 hours

push eventyongtang/io

Yong Tang

commit sha 0d2e8e6f493347d8b467e392d8efe1d3ff38fd26

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha b7273c5b345966ba69813bebc2154aee491d250a

Enable Avro ops build on Windows Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha a7efe21496764a80209475f5d5f56310d7d44271

Enable Snappy for Avro Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 864274a279b8b7916fa25d0339598f735e3e3fb8

Consolidate boost BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 7d35bd08ceb6e492de0c63bb227e30c8fd7beecf

Consolidate thrift BUILD file Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 19 hours

push eventyongtang/io

Yong Tang

commit sha 75abc25aacbf7e880f7f1a387f43fc0a1e811d79

Enable TIFF Windows Build and Tests (#778) * Update TIFF tif_config.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF tiffconf.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable TIFF Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF to enable Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable PubSub Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Suyash Kumar

commit sha 09cb67c2633fe1306f7c799059d7f87f6662b6ab

Add tfio.genome tutorial (#775) * Initial Genome tutorial * Update _toc.yaml * Updates * Update genome tutorial * Update genome tutorial * Link fixes + print shapes. * Update docs/tutorials/genome.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Address reviewer comments * Update genome.ipynb * Update genome.ipynb Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

Yong Tang

commit sha 8567e158f7b54ff0b484ac5a435254744150c7c6

Update AWS SDK to 1.7.270 and enable Kinesis ops and tests for Windows (#780) * Enable Kineiss ops Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS to 1.7.270 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS SDK Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to avoid asm build on Linux (GCC version) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Mark Daoust

commit sha c1f0d00d57da6412c3038dbdc7d7561bd22bc03a

Remove tf_inspect patching. (#781) This is no longer necessary, and no longer works.

view details

Mark Daoust

commit sha d4687193c46193d516bf40d3f9c17f079afa99a6

Fix azure notebook (#782) * try: %ten sorflow_version * Update README.md * clear outputs * Set private outputs.

view details

Yong Tang

commit sha 8513fd0a6853391ccf4052bbb91dbdafa133f235

Add SQL version of Air Quality Data set, to be used by tutorial (#784) * Add SQL version of Air Quality Data set, to be used by tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove aapl as it is not open license Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Mark Daoust

commit sha 01c73c8eb5b8446fd9ca7567685144f736615141

Maybe fix npm install. (#785) * npm install azurite (no "-g") * set private_outputs

view details

Yong Tang

commit sha 609af2f9a2d2bc7f246b99e233809485824efb8f

Fix prometheus endpoint issue (#786) This PR fixes prometheus endpoint issue where `%d` should be '%s' Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 37ea2d64430e743b5623aadadece3d9a3b7cd2b8

Add a tutorial to load metrics from prometheus server (#774) * Add a tutorial to load metrics from prometheus server This PR adds tutorial to load metrics from prometheus server into tf.data.Dataset, so that it could be used for tf.keras and model training. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * sudo apt-get + sudo apt + split dig commands. * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Add sudo apt-get warning. * set private_outputs * Fix extra comma (`,`) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Fix typo Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move all imports into one block, as was suggested in review comment, also split shell and python block Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move the server setup in early section to avoid `Unreachable` failure (server may take a second or two to start up) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

Mark Daoust

commit sha 33a0474a4c0b009cec6a83c7f72d935ac7aacbc6

Prometheus - add sleep(10) for server startup. (#787) * sleep(10) to let the server startup * Move sleep line

view details

Yong Tang

commit sha 4fb30abe2597125cec973b22029b752619790d79

Update tf_configure.bzl to allow centos 7 build, and add README.md section (#788) * Update tf_configure.bzl to allow centos 7 build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update README.md to include details instructions for CentOS 7 and Ubuntu 18.04 build Also setup GitHub CI to validate the README.md Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 36b1f217ac6d9a4ac4ebaa76815a0b8ae8aefe5b

Add additional patch on grpc to allow using python3 only Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 9cb672512d960c826833519acf62c704894bcfff

Update README.md to only install python3, and add sudo Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 95054a88d05ca1ed7f4a778181bed3c83c8f68c1

Update to include CentoS 8 Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 2347dfb2312b87ff7c4e2979865f8dea82362d0a

Update macOS section Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 0c23c13111e6b8073e6d734521618d2c64f6b486

Update to add python-devel Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 59caf101f4b91f798946b8978e7b9c38edd3d79b

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 3d8b4b3acd96f6b7a5d154e17a7df53b83ec01fc

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha cd500b8e8ba30504460db49872d87d18d9cb690e

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 20 hours

PR opened tensorflow/io

Add additional patch on grpc to allow using python3 only

This PR add additional patch on grpc to allow using python3 without python2 dependency. Previously, due to the dependency of grpc, both python2 and python3 has to be available in the system to build tensorflow-io.

This PR patches grpc source code to remove the need for python2.

This PR also adds an additional section in README with instructions to build on CentOS 8.

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+81 -30

0 comment

4 changed files

pr created time in a day

push eventyongtang/io

Yong Tang

commit sha 0c23c13111e6b8073e6d734521618d2c64f6b486

Update to add python-devel Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in a day

push eventyongtang/io

Yong Tang

commit sha abb506eafb08e440b951f11c3f6b739b2d4f6f5e

Update to add python-devel Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in a day

issue commentedenhill/librdkafka

Segmentation Fault since v1.0.0 when EventCb is deleted after Consumer is already closed and deleted

@edenhill I checked the code, the SIGSEGV happens after consumer has been closed and deleted. It seems that there might still be thread calling logging after that (from the stack trace that looks to be rd_kafka_cgrp_max_poll_interval_check_tmr_cb? I am not sure though). Wondering if I need to call additional APIs to clean up?

Also SIGSEGV only happens after we upgrade librdkafka to a version later than v1.0.0+. We used to have v0.11.6 and never encountered the issue before. Wondering if there is any related change after v0.11.6?

yongtang

comment created time in a day

push eventyongtang/io

Yong Tang

commit sha 2347dfb2312b87ff7c4e2979865f8dea82362d0a

Update macOS section Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha f669f0366ac8c1714088d355c54a2cc725cad405

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/io

Yong Tang

commit sha 9022a398daf029abe7c7a7bc2bf7b2477a478351

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/io

Yong Tang

commit sha 412cf286907230d623c4d2d952104dafe77c09e9

Update macOS section Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha f4383190c8f7a25046a975ec4b97c9e735c06d39

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/io

Yong Tang

commit sha a10db270d9f315ed4db158f8abf89f4dc3ecba83

Update macOS section Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 5a7a82b53eda1ae534cda05cfd4572ebe74ad777

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/io

Yong Tang

commit sha 95054a88d05ca1ed7f4a778181bed3c83c8f68c1

Update to include CentoS 8 Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/io

Yong Tang

commit sha 6b844dc69e891a41d77111b02b2c47a6f24721e9

Update to include CentoS 8 Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

create barnchyongtang/io

branch : python3

created branch time in 2 days

push eventyongtang/io

Yong Tang

commit sha 4fb30abe2597125cec973b22029b752619790d79

Update tf_configure.bzl to allow centos 7 build, and add README.md section (#788) * Update tf_configure.bzl to allow centos 7 build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update README.md to include details instructions for CentOS 7 and Ubuntu 18.04 build Also setup GitHub CI to validate the README.md Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

issue commenttensorflow/io

Compiler errors while building on centos7 image

@ashahab PR #788 has been merged. You can check https://github.com/tensorflow/io#centos-7 for building instructions on CentOS 7.

ashahab

comment created time in 2 days

delete branch yongtang/io

delete branch : centos

delete time in 2 days

create barnchyongtang/io

branch : kafka-range

created branch time in 2 days

push eventyongtang/demo

Yong Tang

commit sha 36ac68e19a5855b52c4d18bfd0b8f8add0ba9dea

Update to split mac Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/demo

Yong Tang

commit sha 13bca5a390b48589a02be4da571e4cc198aa84c5

Update to v1.4.5 Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/tensorflow

frreiss

commit sha b6c5a9fb8e12e42681f4317f72ce52872653497c

Add soft-link to pylintrc to project root

view details

ANSHUMAN TRIPATHY

commit sha 7375586f215385d1c29a6a9538ae658f5ae7b936

Contradicting comments removed

view details

ANSHUMAN TRIPATHY

commit sha 447c1420b8c2a7d9baf19248c97880b8ff832e0f

[1] Review comments handled

view details

Koan-Sin Tan

commit sha ae2e9865a1ddfc782e9a41d89b59d4a7783c3f30

[tflite] bump SPLIT op ver from 1 to 3 in NNAPI delegate I need SPLIT op version 3. Since it's supported by TFLite and NNAPI 1.2. It's should be safe to bump the op version so that I can delegate SPLIT ops to accelerators.

view details

Arvind Sundararajan

commit sha d28af41cf90fb85c91e09cddbb08b7ad43bf30d9

Handle indexed slice empty shapes in IndexedSlices gradients correctly.

view details

Eugene Kuznetsov

commit sha ae0e325a9fd53f2981bc569a2e3f8699c72a2ddc

Fixing ROCm LSTM and GRU v2 test

view details

ekuznetsov139

commit sha 451487e515d978aeec3c7e3c8f42ad11e51ac0ab

Merge branch 'master' into rocm-gru-lstm

view details

Fei Hu

commit sha 1c924a1e67690dfa215ca7c2daa233cf6118c665

Fix a warning about the comparison of integer expressions of different signedness

view details

mbhuiyan

commit sha d7dae381702471276dd4d8bdc00f96f87ab493c7

Addng support of Conv backward for DNN 1.0

view details

Deven Desai

commit sha d26ee9801c8117f7fd6297a05a82eab98023a2c3

bug fix in the ROCm python implementation for gpu_lstm op

view details

Deven Desai

commit sha eb713b7448f61c610850ded6113ee7eace764fd3

addressing code-review comments

view details

Deven Desai

commit sha f026e707d34db8c8f06e48eba699a2cb6bf2ecde

Skipping unsupported subtests within the tests lstm_test and gru_test

view details

Deven Desai

commit sha 5f8828b16ed35dfdc448975443175829eb599ca3

fixing pylint errors

view details

Deven Desai

commit sha b5a3eda2dc4b16b61467ef4afd5f8211681133b3

[ROCm] Fix for a bug in the ROCm implementation for matrix_triangular_solve op

view details

mbhuiyan

commit sha 9a53a987145bee6906ba7f2035e4fbe2f28c5836

Added a method for refactoring

view details

mbhuiyan

commit sha 507d607c694310c9ea2b9849e9ed7dbe54c73e6c

Merge branch 'master' into dnn10_conv_bwd

view details

mbhuiyan

commit sha b6407acdd52c78d931caffdd646ca8631402bdc1

Fixing the clang format

view details

mbhuiyan

commit sha aba36d0286152da96d8cc3e4e8939b60374d7554

Calling refactored method

view details

Stephan Herhut

commit sha 77deb9292ccac6f9b58dac3c0640ca1b917b3d01

Add xla_lhlo.dynamic_broadcast_in_dim operation. Also change the type of the dynamic dimensions operand to vector of Integer, as index type is not supported in vectors. PiperOrigin-RevId: 295141631 Change-Id: Ie8b6d5adec65d70243a3b132ffc807cafd212b42

view details

A. Unique TensorFlower

commit sha 1d2b4f58a13ac8d53814885ddbf54c655ab7bd97

Fix a duplicate symbol linking error. PiperOrigin-RevId: 295150209 Change-Id: I1e9b256c51d4c9d7a8f589d588e295a8d137bbf8

view details

push time in 2 days

issue commenttensorflow/io

Get a DICOM tag that is a sequence

@EstebanGuillen Do you have a sample dicom file I can take a look?

EstebanGuillen

comment created time in 2 days

issue openededenhill/librdkafka

Segmentation Fault since v1.0.0 when EventCb is deleted after Consumer is already closed and deleted

Read the FAQ first: https://github.com/edenhill/librdkafka/wiki/FAQ

Description

We used to build with librdkafka v0.11.6 and the clean up is as follows:

KafkaEventCb kafka_event_cb = new KafkaEventCb(...)
...
conf->set("event_cb", &kafka_event_cb_, errstr).
.....
.....
.....
consumer_->unassign();
consumer_->close();
delete  consumer_;

The above code works fine with v0.11.6.

However, When we tried to update librdkafka to newer version (we tried 1.0.0-1.4.0RC), the above code will result in:


Thread 22 "python3" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff37fff700 (LWP 9070)]
0x00007fff8b5eb35f in RdKafka::log_cb_trampoline(rd_kafka_s const*, int, char const*, char const*) ()
   from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
(gdb) where
#0  0x00007fff8b5eb35f in RdKafka::log_cb_trampoline(rd_kafka_s const*, int, char const*, char const*) ()
   from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
#1  0x00007fff8b5f6e60 in rd_kafka_log_buf () from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
#2  0x00007fff8b5f709f in rd_kafka_log0 () from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
#3  0x00007fff8b6acd43 in rd_kafka_cgrp_max_poll_interval_check_tmr_cb ()
   from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
#4  0x00007fff8b621b03 in rd_kafka_timers_run () from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
#5  0x00007fff8b5fcb91 in rd_kafka_thread_main () from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
#6  0x00007fff8b70a530 in _thrd_wrapper_function ()
   from bazel-bin/tensorflow_io/core/python/ops/libtensorflow_io.so
#7  0x00007ffff77cc6db in start_thread (arg=0x7fff37fff700) at pthread_create.c:463
#8  0x00007ffff7b0588f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) 

From the above stack trace, it looks like Kafka log is still in usage after consumer has been deleted.

Wondering if there is any operation that should be called before EventCb is deleted? Or EventCb has to be a static variable that will live through the whole program?

How to reproduce

Please see above section.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • [x] librdkafka version (release number or git tag): v1.0.0 - v1.4.0RC
  • [x] Apache Kafka version: confluent-community-5.3.1-2.12
  • [x] librdkafka client configuration: N/a See comment about event_cb above
  • [x] Operating system: Ubuntu 18.04 and macOS
  • [x] Provide logs (with debug=.. as necessary) from librdkafka See stack trace above
  • [ ] Provide broker log excerpts
  • [x] Critical issue

created time in 2 days

issue commenttensorflow/io

Compiler errors while building on centos7 image

@ashahab The issue has been resolved. A newer version of gcc (devtoolset) in CentOS 7, libstdc++ also has to be linked statically by passing the following at the beginning of the build on CentOS 7:

BAZEL_LINKOPTS="-static-libstdc++ -static-libgcc" \
   BAZEL_LINKLIBS="-lm -l%:libstdc++.a"

Finally, a tf_configure.bzl update is needed to capture the lib64 vs. lib discrepancy in CentOS.

The PR is in #788. Once merged you can following the instructions on README.md to build with CentOS 7.

ashahab

comment created time in 2 days

PR opened tensorflow/io

Update tf_configure.bzl to allow centos 7 build, and add README.md section

This PR fixes the issue raised in #722 where tensorflow-io could not be compiled on CentOS 7.

The issue was that, CentOS 7's gcc version is too old, the libstdc++ version is also too old. In addition, CentOS's tensorflow header and libs are both /lib64/ and /lib/, which is not captured in tf_configure.bzl.

This PR updates several places:

  1. Add special handling in tf_configure.bzl to capture /lib64/ and /lib/
  2. Use devtoolset-9 to build .so on CentOS
  3. Use static libs linkage for libstdc++ through:
    BAZEL_LINKOPTS="-static-libstdc++ -static-libgcc" \
       BAZEL_LINKLIBS="-lm -l%:libstdc++.a"
    

Additional sections in README.md have been added to show the complete shell to build on CentOS 7 and Ubuntu 18.04. Two GitHub CI workflow have been added as well to validate the build on CentOS 7 and Ubuntu 18.04.

This PR fixes #722

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+126 -0

0 comment

4 changed files

pr created time in 2 days

push eventyongtang/io

Yong Tang

commit sha 299f94e93491b741bb23c9c9eef140bc62017ad5

Update tf_configure.bzl to allow centos 7 build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha add17d7fb3e87f68b7002122a81b562507e2aa5f

Update README.md to include details instructions for CentOS 7 and Ubuntu 18.04 build Also setup GitHub CI to validate the README.md Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

push eventyongtang/io

Yong Tang

commit sha 6898c8bef91202895946c27c77b2df88ad7a5aeb

Update README.md to include details instructions for CentOS 7 and Ubuntu 18.04 build Also setup GitHub CI to validate the README.md Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

push eventyongtang/io

Yong Tang

commit sha 417febb942e449498aec02c21c897f4e2d20fdd3

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 3ddffb1ed2e0fccc52887fe688426efba00699a8

Update tf_configure.bzl to allow centos 7 build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha fabb485e4eaef53b785028ccd3cd323e2abffbba

Update README.md to include details instructions for CentOS 7 and Ubuntu 18.04 build Also setup GitHub CI to validate the README.md Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

push eventyongtang/io

Yong Tang

commit sha 8ac6ad170b65458dc08c8a726abb769e879f3df2

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

push eventyongtang/io

Yong Tang

commit sha bd384a31978773ee9fef64f7ede80db073f48a8f

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

push eventyongtang/io

Yong Tang

commit sha 090b8ff21d53962b7bed1b3c001300f153ee6265

Show content Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

push eventyongtang/io

Yong Tang

commit sha 13524f998be2ab601690bd2be9d932fff28482ae

Update tf_configure.bzl to allow centos 7 build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 9b5c9f08f9644696edea9445b5b18275dc6ba3ef

Update README.md to include details instructions for CentOS 7 and Ubuntu 18.04 build Also setup GitHub CI to validate the README.md Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 955ced70ecb937cf718bf8bfdf3174f83c6e26ce

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

create barnchyongtang/io

branch : kafka-1.4

created branch time in 3 days

push eventyongtang/tensorflow

TensorFlower Gardener

commit sha b83cc0ac8eb83c85b78778bbe8a0c96f323d747e

Merge pull request #22231 from MichaelKonobeev:sparse-xent-op-hessian PiperOrigin-RevId: 260802377

view details

MichaelKonobeev

commit sha ea809e3ad7c0d8a1fc1170dec6c782c7feac299b

Implement IsZero in eager mode

view details

Koan-Sin Tan

commit sha e305ac4b75a9523bf047fdaef75159f13bd04b86

[tflite] add int8 input/output to label_image More and more models, such as MobilenetV3's EdgeTPU ones, are using post-training full integer quantization. With this patch, I can get reasonable results. ./label_image_int8 -m mobilenet_edgetpu_224_1.0_int8.tflite Loaded model mobilenet_edgetpu_224_1.0_int8.tflite resolved reporter INFO: Initialized TensorFlow Lite runtime. invoked average time: 15.363 ms 0.867188: 653 military uniform 0.0390625: 835 suit 0.015625: 458 bow tie 0.0078125: 907 Windsor tie 0.00390625: 716 pickelhaube

view details

Koan-Sin Tan

commit sha b698e34e97ba49aa2d562a42804476ab5a024ab0

clean up

view details

Koan-Sin Tan

commit sha ec72fed7066f44d09172b3cfa358a299b7e5ec12

address review comments 1. add explicit cast back 2. change int to TfLiteType

view details

Duncan Riach

commit sha ed955df9438a4e13f33a439338c92cbc029a713d

Change bias_op tests to work in eager mode (as well as graph mode)

view details

Duncan Riach

commit sha c227f00a33de7ed10ade9bfb0ddce2833110a0e6

Fix Ubuntu Sanity CI fail due to pylint error

view details

Anuj Rawat

commit sha 5670f0f29c50dc2427f8cb4386aaeab6094083f1

Fixing test that fails on AVX512 The operation categorical_crossentropy requires taking log as an intermediate step. Due to the rank (2) and shape (3, 3) of the tensors used in this example, on AVX2 and older builds, the log operation uses plog, Eigen's packet log method, whereas on AVX512 build, the log operation is not vectorized and ends up using std::log. Due to the precision mismatch between std::log and Eigen's plog, the results do not match exactly. The loss values comes out to be equal to [0.10536055 0.8046685 0.06187541], instead of [0.10536055 0.8046684 0.06187541]. This is an expected mismatch and should not fail the test. The absolutely correct way to test would be to compare hex values and make sure that the results are within the expected range of the ULP error. An easier fix would be to reduce the precision of the test to account for such mismatches between the implementation of operators in the underlying math libraries. We are taking the second approach and will compare results after rounding to 5 decimal places.

view details

MichaelKonobeev

commit sha 6fe6391ea937a3c20308b3986f7232967e6f0268

Unconditionally tag zero tensors

view details

MichaelKonobeev

commit sha b187faf53c68ff9b0c711b246116fb81660ad4c7

Remove expired forward compatibility check

view details

MichaelKonobeev

commit sha cb9ce8a40c41d35725900f0f0e12a934e28ba837

Merge branch 'master' into sparse-xent-op-hessian

view details

RichardXiao13

commit sha f8a15ce2b6f48523effe2dd42e7844ea7ef1d97a

Add usage example to math.poly_val

view details

RichardXiao13

commit sha 37b8d190b935d128c260c8a6acb871cd64748736

Update math_ops.py

view details

Richard Xiao

commit sha 3a63696e3b417603830131f989865a6f5b141482

Update math_ops.py

view details

William-Yin123

commit sha 691b55ff4d66e27f5a669d6955268f86627454af

committing updated docs

view details

Richard Xiao

commit sha 5bb949619601f1b9d49a2d73b7d918103a1d421d

Update math_ops.py

view details

William-Yin123

commit sha 5fa409d7fb78d7ee91699808591f2532eac4bdbd

test

view details

William-Yin123

commit sha bf4433f8c784a29c091dbc23bc45c7f74c47e6c9

committing updated docs

view details

William-Yin123

commit sha eed98d69ac410750fd6789557544e27344af4638

removed ids and shortened lines

view details

William-Yin123

commit sha 8ceebbaa72df62b43201597a3bde6d1d4ca2077f

fixed string_hash_bucket usage example

view details

push time in 3 days

push eventyongtang/io

Mark Daoust

commit sha c1f0d00d57da6412c3038dbdc7d7561bd22bc03a

Remove tf_inspect patching. (#781) This is no longer necessary, and no longer works.

view details

Mark Daoust

commit sha d4687193c46193d516bf40d3f9c17f079afa99a6

Fix azure notebook (#782) * try: %ten sorflow_version * Update README.md * clear outputs * Set private outputs.

view details

Yong Tang

commit sha 8513fd0a6853391ccf4052bbb91dbdafa133f235

Add SQL version of Air Quality Data set, to be used by tutorial (#784) * Add SQL version of Air Quality Data set, to be used by tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove aapl as it is not open license Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Mark Daoust

commit sha 01c73c8eb5b8446fd9ca7567685144f736615141

Maybe fix npm install. (#785) * npm install azurite (no "-g") * set private_outputs

view details

Yong Tang

commit sha 609af2f9a2d2bc7f246b99e233809485824efb8f

Fix prometheus endpoint issue (#786) This PR fixes prometheus endpoint issue where `%d` should be '%s' Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 37ea2d64430e743b5623aadadece3d9a3b7cd2b8

Add a tutorial to load metrics from prometheus server (#774) * Add a tutorial to load metrics from prometheus server This PR adds tutorial to load metrics from prometheus server into tf.data.Dataset, so that it could be used for tf.keras and model training. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * sudo apt-get + sudo apt + split dig commands. * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Add sudo apt-get warning. * set private_outputs * Fix extra comma (`,`) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Fix typo Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move all imports into one block, as was suggested in review comment, also split shell and python block Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move the server setup in early section to avoid `Unreachable` failure (server may take a second or two to start up) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

Mark Daoust

commit sha 33a0474a4c0b009cec6a83c7f72d935ac7aacbc6

Prometheus - add sleep(10) for server startup. (#787) * sleep(10) to let the server startup * Move sleep line

view details

Yong Tang

commit sha 3b492923a2f38e01afbef2e8ef0857c18357362c

Update tf_configure.bzl to allow centos 7 build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

push eventyongtang/io

Yong Tang

commit sha 609af2f9a2d2bc7f246b99e233809485824efb8f

Fix prometheus endpoint issue (#786) This PR fixes prometheus endpoint issue where `%d` should be '%s' Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 37ea2d64430e743b5623aadadece3d9a3b7cd2b8

Add a tutorial to load metrics from prometheus server (#774) * Add a tutorial to load metrics from prometheus server This PR adds tutorial to load metrics from prometheus server into tf.data.Dataset, so that it could be used for tf.keras and model training. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * sudo apt-get + sudo apt + split dig commands. * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Add sudo apt-get warning. * set private_outputs * Fix extra comma (`,`) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Fix typo Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move all imports into one block, as was suggested in review comment, also split shell and python block Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move the server setup in early section to avoid `Unreachable` failure (server may take a second or two to start up) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

Mark Daoust

commit sha 33a0474a4c0b009cec6a83c7f72d935ac7aacbc6

Prometheus - add sleep(10) for server startup. (#787) * sleep(10) to let the server startup * Move sleep line

view details

push time in 3 days

push eventtensorflow/io

Mark Daoust

commit sha 33a0474a4c0b009cec6a83c7f72d935ac7aacbc6

Prometheus - add sleep(10) for server startup. (#787) * sleep(10) to let the server startup * Move sleep line

view details

push time in 4 days

delete branch yongtang/io

delete branch : prometheus

delete time in 4 days

push eventtensorflow/io

Yong Tang

commit sha 37ea2d64430e743b5623aadadece3d9a3b7cd2b8

Add a tutorial to load metrics from prometheus server (#774) * Add a tutorial to load metrics from prometheus server This PR adds tutorial to load metrics from prometheus server into tf.data.Dataset, so that it could be used for tf.keras and model training. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * sudo apt-get + sudo apt + split dig commands. * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Add sudo apt-get warning. * set private_outputs * Fix extra comma (`,`) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Fix typo Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move all imports into one block, as was suggested in review comment, also split shell and python block Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Move the server setup in early section to avoid `Unreachable` failure (server may take a second or two to start up) Signed-off-by: Yong Tang <yong.tang.github@outlook.com> Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

push time in 4 days

PR merged tensorflow/io

Add a tutorial to load metrics from prometheus server

This PR adds tutorial to load metrics from prometheus server into tf.data.Dataset, so that it could be used for tf.keras and model training.

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+528 -0

4 comments

2 changed files

yongtang

pr closed time in 4 days

pull request commenttensorflow/io

Add a tutorial to load metrics from prometheus server

Thanks @lamberta @MarkDaoust for the review. I have updated the PR to place import modules into one section, and split the python and shell code, as was suggested in the review.

I also move the server setup into an early section, that should solve the potential AssertionError: Unreachable issue. The issue is likely caused by server not ready yet (may take a second or two to setup.)

Please take a look and let me know if there are any other issues.

yongtang

comment created time in 4 days

push eventyongtang/io

Yong Tang

commit sha 0aa3a709edd61cfc3a18e2c8a65471a8725cf767

Move the server setup in early section to avoid `Unreachable` failure (server may take a second or two to start up) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

PR opened tensorflow/io

Fix prometheus endpoint issue

This PR fixes prometheus endpoint issue where %d should be '%s'

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+5 -3

0 comment

2 changed files

pr created time in 4 days

push eventyongtang/io

push time in 4 days

create barnchyongtang/io

branch : prometheus-fix

created branch time in 4 days

push eventyongtang/io

Mark Daoust

commit sha 01c73c8eb5b8446fd9ca7567685144f736615141

Maybe fix npm install. (#785) * npm install azurite (no "-g") * set private_outputs

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha e98ffe91fe79d933bcc0f74804ec55a6c8e382d8

Fix typo Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha d4266c5920815204c3d4a2e6753559d8873a70e4

Move all imports into one block, as was suggested in review comment, also split shell and python block Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 1b0d70d82ded5a412bdd2d106060fddf8d55b599

Fix extra comma (`,`) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 0bc9c52dac61265ec6a9b77eb88875a299b9cfc3

Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha e9047c0e4c83cd0be49c80e1c5bc9ac88ee2f27c

Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha fa3a8db0b86304367b5956540e911a099c61be96

Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 31d76916d286c44b47dba5414e1554e71f8f35cf

Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha c88d5228d375b46c5589a8890c347b72ae02121b

Update docs/tutorials/prometheus.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 48d728bf8d758fbe25525a0f17a1e009e73cad50

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 82f3e5353e1044f0eb5602a1fdee8a4c43dedddd

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 063bc50d41dd18511386d6319e42c2e18d8d3600

Enable Avro Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 610a30bb5e851520d40cce6a7d70a1d1ca817fb6

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

issue commentcoredns/coredns

CoreDNS Projects for Summer of Code 2020

@WJayesh just keep in mind while this external health check is targeting for CoreDNS deployment in Kubernetes clusters, the health check program itself does not necessarily deployed in Kubernetes. Also, CoreDNS deployment on Kubernetes does not necessarily follow the "recommended" way mentioned in the link.

yongtang

comment created time in 4 days

push eventyongtang/io

Yong Tang

commit sha d8c228d1b137d81102eeb6b6c1926aac0704ab53

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 14c4356bacd63200f0a7ba4833b3e9812a146df5

Enable BigQuery Windows Build and Tests (#770) * Enable BigQuery Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable tests for Bigquery on Windows Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha c31de505c9ff67a8d5646e8bd6dbef4be23ee232

Enable GRPC Windows Build (#769) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 08e87817967cc34df7cacaa8df536a8c0f29a6ee

Add Additional Data to be used by tutorial (#771) This PR adds additional data for AAPL Stock price of 2019 (from Yahoo Finance) so that it could be used by an upcoming tutorial for Sql Dataset. (The data needs to be in place for Colab to run). The tutorial will be created in the next PR. Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 97acfce1445c0ccd172deab4f540c4d970573aca

Add necessary files to be used in tutorial (#773) This PR adds a couple of files that will be needed for the upcoming CoreDNS/Prometheus tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha ed839faffedf119d1af33f647958d1cbb4b2be52

Update HDF5 (#776) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha a033cf201ad6de21192d57ec1d111adc7afada16

Update DCMTK to 3.6.5 and enables DICOM Windows Build (#777) * Update dcmtk to 3.6.5 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable DICOM Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable DICOM Windows Tests Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 75abc25aacbf7e880f7f1a387f43fc0a1e811d79

Enable TIFF Windows Build and Tests (#778) * Update TIFF tif_config.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF tiffconf.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable TIFF Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF to enable Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable PubSub Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Suyash Kumar

commit sha 09cb67c2633fe1306f7c799059d7f87f6662b6ab

Add tfio.genome tutorial (#775) * Initial Genome tutorial * Update _toc.yaml * Updates * Update genome tutorial * Update genome tutorial * Link fixes + print shapes. * Update docs/tutorials/genome.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Address reviewer comments * Update genome.ipynb * Update genome.ipynb Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

Yong Tang

commit sha 8567e158f7b54ff0b484ac5a435254744150c7c6

Update AWS SDK to 1.7.270 and enable Kinesis ops and tests for Windows (#780) * Enable Kineiss ops Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS to 1.7.270 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS SDK Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to avoid asm build on Linux (GCC version) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Mark Daoust

commit sha c1f0d00d57da6412c3038dbdc7d7561bd22bc03a

Remove tf_inspect patching. (#781) This is no longer necessary, and no longer works.

view details

Mark Daoust

commit sha d4687193c46193d516bf40d3f9c17f079afa99a6

Fix azure notebook (#782) * try: %ten sorflow_version * Update README.md * clear outputs * Set private outputs.

view details

Yong Tang

commit sha 8513fd0a6853391ccf4052bbb91dbdafa133f235

Add SQL version of Air Quality Data set, to be used by tutorial (#784) * Add SQL version of Air Quality Data set, to be used by tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove aapl as it is not open license Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 1b7c68586c2b03a3352cbe09539c5a0c2e8fcc3f

Try Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

push eventyongtang/io

Yong Tang

commit sha 8513fd0a6853391ccf4052bbb91dbdafa133f235

Add SQL version of Air Quality Data set, to be used by tutorial (#784) * Add SQL version of Air Quality Data set, to be used by tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove aapl as it is not open license Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

pull request commenttensorflow/io

Add ArrowIOTensor with input from a pyarrow.Table

@BryanCutler Let me know if there are any questions or I can be of any help.

BryanCutler

comment created time in 4 days

pull request commenttensorflow/io

Add ArrowIOTensor with input from a pyarrow.Table

@BryanCutler The graph mode support could be added later in follow up PRs I think. Though it would be good to add a TODO, or add a stub test in io_tensor_eager_test.py, and mark the test as TODO, e.g.,

        pytest.param(
             "arrow", None,
             marks=[		
                 pytest.mark.skip(reason="TODO"),		
             ],		
         ),
BryanCutler

comment created time in 4 days

pull request commenttensorflow/io

Add ArrowIOTensor with input from a pyarrow.Table

@BryanCutler Also recently I saw quite some user request to ask for support in graph mode (for both dataset and iotensor). The need for graph mode support is mostly to combine with tf.data itself to form a full pipeline. In other words, user would prefer to use:

dataset = tf.data.Dataset.from_tensor_slices([filename1, filename2, filename3])
dataset = dataset.map(lambda f: tfio.IOTensor.from_...())

This will constitutes a complete data pipeline that could be deployed in production in a infinitely continuous way (after converting from_tensor_slices to a steam source of filename entries).

I think the above is a useful scenario that will help in production usage.

While IOTensor starts with eager mode in focus, it is not very far away to support graph mode. The only requirement for graph mode, is the dtype which has to be known at the graph construction time (REGISTER_OP).

We could ask user to provide the dtype (or tf.TensorSpec) information as an additional arg when graph mode is used. HDF5IOTensor is an example where dtype/spec is provided in graph mode.

Also, inside REGISTER_OP, it will help shape inference if the shape information is provided explicitly. However, shape is not absolutely needed. In most cases, a rank would be suffice in graph construction time. The shape information could be passed as a tensor in graph execution time later on.

Based on the above, it is possible to only ask user for dtype in graph mode. And convert shape in REGISTER_OP("IO>ArrowReadableRead") from attribute to tensor input. In this way, shape is not needed by user input. The SetShapeFn([](shape_inference::InferenceContext* c) could use Unknown shape.

BryanCutler

comment created time in 4 days

pull request commenttensorflow/io

Add ArrowIOTensor with input from a pyarrow.Table

@BryanCutler Thanks for the PR! Overall it looks good.

I would suggest to add a several simple tests in test_io_tensor_eager.py as well (in addition to the already added tests in test_arrow_eager.py. test_arrow_eager.py is a much more comprehensive tests suites, though test_arrow_eager.py is more like a feature comparision for each different op types.

For example, in the past hdf5 was not possible in graph mode so we skipped the test with:

        pytest.param(
             "hdf5", None,
             marks=[		
                 pytest.mark.skip(reason="TODO"),		
             ],		
         ),

until PR #730

(Note there are still some issues as reported by #745, though at least we have some coverage and knows which "feature" is available).

For Arrow, I think we could add a simple test in test_io_tensor_eager.py with one or two column only, just to show if some features are supported or not. If not supported we could just use pytest.marks to skip and add TODO.

The added tests in test_arrow_eager.py are used to serve a more through coverage.

BryanCutler

comment created time in 4 days

pull request commenttensorflow/io

Add a tutorial to load metrics from prometheus server

/cc @MarkDaoust @lamberta to take a look.

yongtang

comment created time in 4 days

pull request commenttensorflow/io

Add a tutorial to load metrics from prometheus server

This is another tutorial to load metrics data from prometheus server. prometheus is almost a common setup in kubernetes, and is widely seem in cloud-native devops/infrastructure. this tutorial could help devops/sre to use tensorflow/tensorflow-io to automate their operations.

yongtang

comment created time in 4 days

pull request commenttensorflow/io

Add a tutorial to create tf.data.Dataset from PostgreSQL database

Thanks @MarkDaoust for the review. The PR has been updated.

Now the installation and setup of PostgreSQL is in an (optional) section, and could be completely skipped if user provide a (already existing) postgresql's host, port, name, user, pass in environmental variable. That will allow not making dangerous changes on user's locally system (a warning is also in place).

The data set has also been changed to use a air quality dataset from UCI Machine Learning Repository (open license with citation requirement). A reference section has been added to the end of the notebook to meet the citation requirement.

Please take a look and let me know if there are any issues.

yongtang

comment created time in 4 days

push eventyongtang/io

Yong Tang

commit sha 97acfce1445c0ccd172deab4f540c4d970573aca

Add necessary files to be used in tutorial (#773) This PR adds a couple of files that will be needed for the upcoming CoreDNS/Prometheus tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha ed839faffedf119d1af33f647958d1cbb4b2be52

Update HDF5 (#776) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha a033cf201ad6de21192d57ec1d111adc7afada16

Update DCMTK to 3.6.5 and enables DICOM Windows Build (#777) * Update dcmtk to 3.6.5 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable DICOM Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable DICOM Windows Tests Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 75abc25aacbf7e880f7f1a387f43fc0a1e811d79

Enable TIFF Windows Build and Tests (#778) * Update TIFF tif_config.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF tiffconf.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable TIFF Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF to enable Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable PubSub Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Suyash Kumar

commit sha 09cb67c2633fe1306f7c799059d7f87f6662b6ab

Add tfio.genome tutorial (#775) * Initial Genome tutorial * Update _toc.yaml * Updates * Update genome tutorial * Update genome tutorial * Link fixes + print shapes. * Update docs/tutorials/genome.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Address reviewer comments * Update genome.ipynb * Update genome.ipynb Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

Yong Tang

commit sha 8567e158f7b54ff0b484ac5a435254744150c7c6

Update AWS SDK to 1.7.270 and enable Kinesis ops and tests for Windows (#780) * Enable Kineiss ops Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS to 1.7.270 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS SDK Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to avoid asm build on Linux (GCC version) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Mark Daoust

commit sha c1f0d00d57da6412c3038dbdc7d7561bd22bc03a

Remove tf_inspect patching. (#781) This is no longer necessary, and no longer works.

view details

Mark Daoust

commit sha d4687193c46193d516bf40d3f9c17f079afa99a6

Fix azure notebook (#782) * try: %ten sorflow_version * Update README.md * clear outputs * Set private outputs.

view details

Yong Tang

commit sha 8513fd0a6853391ccf4052bbb91dbdafa133f235

Add SQL version of Air Quality Data set, to be used by tutorial (#784) * Add SQL version of Air Quality Data set, to be used by tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove aapl as it is not open license Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 4eb0517fd24bd7d39c576195ae9ff0130404eaaa

Add a tutorial to create tf.data.Dataset from PostgreSQL database This PR adds a tutorial to create tf.data.Dataset from PostgreSQL database Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 4 days

pull request commenttensorflow/io

Maybe fix npm install.

@MarkDaoust When I try the notebook on Google Colab, it give the following error:

!npm --user install -g azurite@2.7.0

Error: setuid user id does not exist
    at /tools/node/lib/node_modules/npm/node_modules/uid-number/uid-number.js:49:16
    at ChildProcess.exithandler (child_process.js:267:7)
    at emitTwo (events.js:126:13)

On the other hand, if I try with the following (remove -g from the original, also removes --user):

!npm install azurite@2.7.0

Then the install is successful.

Don't know about your local setup, though I guess the error you encountered locally before (with !npm install -g azurite@2.7.0) might be due to the usage of -g?

Would you mind give !npm install azurite@2.7.0 a try locally? That might clear both the local execution error and Colab error?

MarkDaoust

comment created time in 4 days

push eventyongtang/io

Yong Tang

commit sha ed839faffedf119d1af33f647958d1cbb4b2be52

Update HDF5 (#776) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha a033cf201ad6de21192d57ec1d111adc7afada16

Update DCMTK to 3.6.5 and enables DICOM Windows Build (#777) * Update dcmtk to 3.6.5 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable DICOM Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable DICOM Windows Tests Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 75abc25aacbf7e880f7f1a387f43fc0a1e811d79

Enable TIFF Windows Build and Tests (#778) * Update TIFF tif_config.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF tiffconf.h Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable TIFF Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update TIFF to enable Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable PubSub Windows Build Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Suyash Kumar

commit sha 09cb67c2633fe1306f7c799059d7f87f6662b6ab

Add tfio.genome tutorial (#775) * Initial Genome tutorial * Update _toc.yaml * Updates * Update genome tutorial * Update genome tutorial * Link fixes + print shapes. * Update docs/tutorials/genome.ipynb Co-Authored-By: Billy Lamberta <blamb@google.com> * Address reviewer comments * Update genome.ipynb * Update genome.ipynb Co-authored-by: Mark Daoust <markdaoust@google.com> Co-authored-by: Billy Lamberta <blamb@google.com>

view details

Yong Tang

commit sha 8567e158f7b54ff0b484ac5a435254744150c7c6

Update AWS SDK to 1.7.270 and enable Kinesis ops and tests for Windows (#780) * Enable Kineiss ops Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS to 1.7.270 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update AWS SDK Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to avoid asm build on Linux (GCC version) Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Mark Daoust

commit sha c1f0d00d57da6412c3038dbdc7d7561bd22bc03a

Remove tf_inspect patching. (#781) This is no longer necessary, and no longer works.

view details

Mark Daoust

commit sha d4687193c46193d516bf40d3f9c17f079afa99a6

Fix azure notebook (#782) * try: %ten sorflow_version * Update README.md * clear outputs * Set private outputs.

view details

Yong Tang

commit sha 8513fd0a6853391ccf4052bbb91dbdafa133f235

Add SQL version of Air Quality Data set, to be used by tutorial (#784) * Add SQL version of Air Quality Data set, to be used by tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove aapl as it is not open license Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Yong Tang

commit sha 63d495c220cf5f8594ab8e0ee6cb83105bfe3f54

Add a tutorial to load metrics from prometheus server This PR adds tutorial to load metrics from prometheus server into tf.data.Dataset, so that it could be used for tf.keras and model training. Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 5 days

delete branch yongtang/io

delete branch : AirQualityUCI

delete time in 5 days

pull request commenttensorflow/io

Add SQL version of Air Quality Data set, to be used by tutorial

@terrytangyuan By the way, the license itself for the dataset is open, except for citation request. I will update the tutorial to include citation as requested.

yongtang

comment created time in 5 days

pull request commenttensorflow/io

Add SQL version of Air Quality Data set, to be used by tutorial

@terrytangyuan The dataset is in original CSV format. However, since the tutorial is for PostgreSQL, this PR adds a sql file (see the first added file docs/tutorials/postgresql/AirQualityUCI.sql) to ease the tutorial usage.

yongtang

comment created time in 5 days

push eventyongtang/io

Yong Tang

commit sha 27c499d51ca32424abeb5d50a902b12033121d09

Update Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 5 days

PR opened tensorflow/io

Add SQL version of Air Quality Data set, to be used by tutorial

Add SQL version of Air Quality Data set, to be used by tutorial

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+9386 -521

0 comment

3 changed files

pr created time in 5 days

more