profile
viewpoint
Yuan Tang terrytangyuan Ant Financial West Lafayette, IN https://terrytangyuan.github.com/about PMC of XGBoost and Apache MXNet | Committer of TensorFlow, Kubeflow, and ElasticDL | Author of numerous R, Python, and Go open source projects

sql-machine-learning/sqlflow 3582

Brings SQL and AI together.

tensorflow/skflow 3209

Simplified interface for TensorFlow (mimicking Scikit Learn) for Deep Learning

sinhrks/ggfortify 423

Define fortify and autoplot functions to allow ggplot2 to handle some popular R packages.

sql-machine-learning/elasticdl 419

Kubernetes-native Deep Learning Framework

kiosk-sh/kiosk 121

kiosk 🏢 Multi-Tenancy Extension For Kubernetes - Secure Cluster Sharing & Self-Service Namespace Provisioning

kubeflow/mpi-operator 101

Kubernetes Operator for Allreduce-style Distributed Training

terrytangyuan/dml 56

R package for Distance Metric Learning

terrytangyuan/autoplotly 52

Automatic Generation of Interactive Visualizations for Statistical Results

kubeflow/xgboost-operator 35

Incubating project for xgboost operator

terrytangyuan/ctv-databases 27

CRAN Task View: Databases with R

push eventtensorflow/io

Yong Tang

commit sha b3461d1735d86502ae416f51d7f940f17ef2c9b0

Add GitHub CI to test build instructions on macOS (#804) This PR adds GitHub CI to test build instructions on macOS, so that the content in README.md's development sections are verified. This helps us in improving the documentation on README.md so that we know the README.md is correct in steps. Note this one is different from the steps we take in building the wheel binary (building wheel use optimized gcc compiler flags for performance, while the README.md only guaranteed that build instruction with minimal configuration works correctly). Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 20 hours

PR merged tensorflow/io

Add GitHub CI to test build instructions on macOS

This PR adds GitHub CI to test build instructions on macOS, so that the content in README.md's development sections are verified.

This helps us in improving the documentation on README.md so that we know the README.md is correct in steps.

Note this one is different from the steps we take in building the wheel binary (building wheel use optimized gcc compiler flags for performance, while the README.md only guaranteed that build instruction with minimal configuration works correctly).

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+21 -3

0 comment

3 changed files

yongtang

pr closed time in 20 hours

starteddevspace-cloud/devspace

started time in a day

push eventtensorflow/io

Yong Tang

commit sha c2e67f8a13f355d6bfa0bea580e70654be5cb468

Add MP3 support (with minimp3) for Audio Dataset and IOTensor (#801) * Add MP3 support (with minimp3) for Audio Dataset and IOTensor This PR adds MP3 support (with minimp3) for Audio Dataset and IOTensor. The difference between this PR and FFmpeg is that FFmpeg is only supported on specific Ubuntu 16.04 and 18.04, though external .so. This PR builds minimp3 as part of the program and support Windows/Linux/macOS. Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update minimp3 to carry patch on Windows Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 2 days

PR merged tensorflow/io

Add MP3 support (with minimp3) for Audio Dataset and IOTensor

This PR adds MP3 support (with minimp3) for Audio Dataset and IOTensor. The difference between this PR and FFmpeg is that FFmpeg is only supported on specific Ubuntu 16.04 and 18.04, though external .so. This PR builds minimp3 as part of the program and support Windows/Linux/macOS.

/cc @faroit in case you might be interested.

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+218 -1

0 comment

9 changed files

yongtang

pr closed time in 2 days

startedmikefarah/yq

started time in 2 days

starteddeepmind/haiku

started time in 2 days

pull request commentkubeflow/mpi-operator

Changed to Distroless image

Sounds good then. Could you address the comments above?

jtfogarty

comment created time in 2 days

push eventtensorflow/io

Yong Tang

commit sha 60627f7eb3bbe7e5df35bc320451b236aff0d3fc

Enable pcap build on Windows, and remove unneeded headers (#796) * Remove unneeded header file This PR removes unneeded header file Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Enable pcap Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 3 days

PR merged tensorflow/io

Enable pcap build on Windows, and remove unneeded headers

Enable pcap build on Windows, and remove unneeded headers

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+104 -1268

1 comment

3 changed files

yongtang

pr closed time in 3 days

Pull request review commenttensorflow/io

Add multiple dimension slicing/indexing for HDF5

 def test_io_tensor_slice(fixture_lookup, io_tensor_fixture):   for start, stop in list(zip(indices, indices[1:] + [len(expected)])):     assert np.array_equal(io_tensor[start:stop], expected[start:stop]) +# full slicing/index across multiple dimensions+@pytest.mark.parametrize(+    ("io_tensor_fixture"),+    [+        pytest.param("hdf5_multiple_dimension"),+    ],+    ids=[+        "hdf5",+    ],+)+def test_io_tensor_slicd_multiple_dimension(fixture_lookup, io_tensor_fixture):

slicd -> slice

yongtang

comment created time in 3 days

delete branch terrytangyuan/tensorflow-1

delete branch : patch-1

delete time in 3 days

PR closed rstudio/tensorflow

Add setuptools to apt packages to fix Travis build

I observed the same error in my other project and this fixed the issue (hopefully fix the build here as well).

+1 -0

1 comment

1 changed file

terrytangyuan

pr closed time in 3 days

Pull request review commentscikit-learn-contrib/metric-learn

[WIP] SCML : Sparse Compositional Metric Learning

+"""+Sparse Compositional Metric Learning (SCML)+"""++from __future__ import print_function, absolute_import, division+import numpy as np+from .base_metric import _TripletsClassifierMixin, MahalanobisMixin+from sklearn.base import TransformerMixin+from .constraints import Constraints+from sklearn.preprocessing import normalize+from sklearn.neighbors import NearestNeighbors+from sklearn.cluster import KMeans+from sklearn.discriminant_analysis import LinearDiscriminantAnalysis+from sklearn.utils import check_array+++class _BaseSCML_global(MahalanobisMixin):++  _tuple_size = 3   # constraints are triplets++  def __init__(self, beta=1e-5, basis=None, n_basis=None,+               max_iter=100000, verbose=False,  preprocessor=None,+               random_state=None):+    self.beta = beta+    self.basis = basis+    self.n_basis = n_basis+    self.max_iter = max_iter+    self.verbose = verbose+    self.preprocessor = preprocessor+    self.random_state = random_state+    super(_BaseSCML_global, self).__init__(preprocessor)++  def _fit(self, triplets):+    """+    Optimization procedure to find a sparse vector of weights to+    construct the metric from the basis set. This is based on the+    dual averaging method.+    """++    if self.preprocessor is not None:+      n_features = self.preprocessor.shape[1]+    else:+      n_features = self.triplets.shape[1]++    self._initialize_basis(n_features)++    triplets = self._prepare_inputs(triplets, type_of_inputs='tuples')++    # TODO:+    # This algorithm is build to work with indeces, but in order to be+    # compliant with the current handling of inputs it is converted+    # back to indices by the following function. This should be improved+    # in the future.+    triplets, X = self._to_index_points(triplets)++    # TODO: should be given access to gamma?+    gamma = 5e-3+    dist_diff = self._compute_dist_diff(triplets, X)++    sizeT = triplets.shape[0]++    w = np.zeros((1, self.n_basis))+    avg_grad_w = np.zeros((1, self.n_basis))++    output_iter = 5000     # output every output_iter iterations

This should probably be exposed as well

grudloff

comment created time in 3 days

Pull request review commentscikit-learn-contrib/metric-learn

[WIP] SCML : Sparse Compositional Metric Learning

+"""+Sparse Compositional Metric Learning (SCML)+"""++from __future__ import print_function, absolute_import, division+import numpy as np+from .base_metric import _TripletsClassifierMixin, MahalanobisMixin+from sklearn.base import TransformerMixin+from .constraints import Constraints+from sklearn.preprocessing import normalize+from sklearn.neighbors import NearestNeighbors+from sklearn.cluster import KMeans+from sklearn.discriminant_analysis import LinearDiscriminantAnalysis+from sklearn.utils import check_array+++class _BaseSCML_global(MahalanobisMixin):++  _tuple_size = 3   # constraints are triplets++  def __init__(self, beta=1e-5, basis=None, n_basis=None,+               max_iter=100000, verbose=False,  preprocessor=None,+               random_state=None):+    self.beta = beta+    self.basis = basis+    self.n_basis = n_basis+    self.max_iter = max_iter+    self.verbose = verbose+    self.preprocessor = preprocessor+    self.random_state = random_state+    super(_BaseSCML_global, self).__init__(preprocessor)++  def _fit(self, triplets):+    """+    Optimization procedure to find a sparse vector of weights to+    construct the metric from the basis set. This is based on the+    dual averaging method.+    """++    if self.preprocessor is not None:+      n_features = self.preprocessor.shape[1]+    else:+      n_features = self.triplets.shape[1]++    self._initialize_basis(n_features)++    triplets = self._prepare_inputs(triplets, type_of_inputs='tuples')++    # TODO:+    # This algorithm is build to work with indeces, but in order to be

build -> built

grudloff

comment created time in 3 days

Pull request review commentscikit-learn-contrib/metric-learn

[WIP] SCML : Sparse Compositional Metric Learning

+"""+Sparse Compositional Metric Learning (SCML)+"""++from __future__ import print_function, absolute_import, division+import numpy as np+from .base_metric import _TripletsClassifierMixin, MahalanobisMixin+from sklearn.base import TransformerMixin+from .constraints import Constraints+from sklearn.preprocessing import normalize+from sklearn.neighbors import NearestNeighbors+from sklearn.cluster import KMeans+from sklearn.discriminant_analysis import LinearDiscriminantAnalysis+from sklearn.utils import check_array+++class _BaseSCML_global(MahalanobisMixin):++  _tuple_size = 3   # constraints are triplets++  def __init__(self, beta=1e-5, basis=None, n_basis=None,+               max_iter=100000, verbose=False,  preprocessor=None,+               random_state=None):+    self.beta = beta+    self.basis = basis+    self.n_basis = n_basis+    self.max_iter = max_iter+    self.verbose = verbose+    self.preprocessor = preprocessor+    self.random_state = random_state+    super(_BaseSCML_global, self).__init__(preprocessor)++  def _fit(self, triplets):+    """+    Optimization procedure to find a sparse vector of weights to+    construct the metric from the basis set. This is based on the+    dual averaging method.+    """++    if self.preprocessor is not None:+      n_features = self.preprocessor.shape[1]+    else:+      n_features = self.triplets.shape[1]++    self._initialize_basis(n_features)++    triplets = self._prepare_inputs(triplets, type_of_inputs='tuples')++    # TODO:+    # This algorithm is build to work with indeces, but in order to be+    # compliant with the current handling of inputs it is converted+    # back to indices by the following function. This should be improved+    # in the future.+    triplets, X = self._to_index_points(triplets)++    # TODO: should be given access to gamma?+    gamma = 5e-3

Yes would be better to expose this than hard-coding it here

grudloff

comment created time in 3 days

issue commentkubeflow/mpi-operator

Does MPI Operator have a plan about Python SDK?

Also cc @gaocegege @johnugeorge @richardsliu to chime in here as well.

aheeruru

comment created time in 3 days

issue commentkubeflow/mpi-operator

Does MPI Operator have a plan about Python SDK?

Isn't k8s Python SDK already sufficient for this? cc @jinchihe what was the original motivation for tf-operator's Python SDK?

There's no plan yet but let's use this issue to track people's interest first so we can prioritize later. Please 👍 this issue if you are interested in this.

aheeruru

comment created time in 3 days

pull request commentrstudio/tensorflow

Add setuptools to apt packages to fix Travis build

@dfalbel Seems like this is not needed anymore since the CI has been migrated to use GitHub Workflows, right?

terrytangyuan

comment created time in 4 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha e517ee8dabd4dce87feaa2e72b40572132a18349

Update about.html

view details

push time in 5 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha 8d0c6745ac7e5beb153f1c7b2a8f22a9c52c3d27

e.g. -> namely

view details

push time in 5 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha d9f4b6251cb1dd7971c9b1abb49592fb0198b3ae

Spell out PMC in about.html

view details

push time in 5 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha c4ee8973dfc799ea89d5fc75fbd6399106c93d7f

Spell out PMC

view details

push time in 5 days

Pull request review commentkubeflow/mpi-operator

set NVIDIA_VISIBLE_DEVICES to none

 func (c *MPIJobController) newLauncher(mpiJob *kubeflow.MPIJob, kubectlDeliveryI 		// issues with scheduler/container technologies. 		corev1.EnvVar{ 			Name:  "NVIDIA_VISIBLE_DEVICES",-			Value: "",+			Value: "none",

Seems like this is different than “”. See https://github.com/NVIDIA/nvidia-container-runtime/blob/master/README.md. Could you double check?

carmark

comment created time in 5 days

PR opened rstudio/tensorflow

Add setuptools to apt packages to fix Travis build

I observed the same error in my other project and this fixed the issue (hopefully fix the build here as well).

+1 -0

0 comment

1 changed file

pr created time in 5 days

push eventterrytangyuan/tensorflow-1

Yuan Tang

commit sha 29ce93fa965fa3540d091114634165733b4b0b03

Add setuptools to apt packages to fix Travis build

view details

push time in 5 days

push eventterrytangyuan/scaffolder

Yuan Tang

commit sha cec15d5530c443877eabc1fe899b998f6a5c7c0b

Update .travis.yml

view details

push time in 5 days

pull request commenttensorflow/io

Update README.md

Looks like you need to sign the CLA before we can merge this.

DylanTallchiefGit

comment created time in 5 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha 99255ab95e359b097cdd64858a57d6c9fcc77935

Update cv.html

view details

push time in 5 days

starteddwmkerr/hacker-laws

started time in 5 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha 22d281d56c8f8b8f0f44d49819554fb445680b0a

Change Read more link to GitHub Sponsors summary

view details

push time in 5 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha 38ebfbbdc70d984e0d045d857f69cae9c1937eba

Update cv.html

view details

push time in 5 days

push eventterrytangyuan/terrytangyuan.github.com

Yuan Tang

commit sha a86b5d16422ec349cea41ec4e7e29fcbab948ceb

Update about.html

view details

push time in 5 days

issue commentkubeflow/mpi-operator

kubectl-delivery container port

It’s a init container as part of launcher pod if that’s what you are looking for. Is there any context of why you want to specify the port of that container?

hyoseon-agile

comment created time in 5 days

push eventsql-machine-learning/elasticdl

HT

commit sha 32b649d20d8c8dc7e47293a3194c7c0b6c4bbac3

fix worker relaunch (#1748)

view details

push time in 5 days

delete branch sql-machine-learning/elasticdl

delete branch : fix_relaunch

delete time in 5 days

PR merged sql-machine-learning/elasticdl

fix worker relaunch
  1. In our worker relaunch logic, we assume that when a worker pod is deleted by preemption, it will trigger a k8s event with type == "DELETED" and phase != "Succeeded". Sometimes it does, but other times the event is with type == "DELETED" and phase == "Succeeded". All preempted pods will generate an event with type == "MODIFIED" and phase == "Failed" and exit_code==137. Deleted pods or OOMkilled pods also would generate such an event. So, we will add this event signature to decide if we need to relaunch the worker pod.

  2. Fix a bug that _update_addr does not update self._worker_addrs.

+26 -11

0 comment

1 changed file

skydoorkai

pr closed time in 5 days

push eventtensorflow/io

Yong Tang

commit sha de490ba28e6b0551b905a4966613a8c45f1583dd

Add additional patch on grpc to allow using python3 only (#789) * Add additional patch on grpc to allow using python3 only Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update README.md to only install python3, and add sudo Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to include CentoS 8 Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update macOS section Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update to add python-devel Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 6 days

PR merged tensorflow/io

Add additional patch on grpc to allow using python3 only

This PR add additional patch on grpc to allow using python3 without python2 dependency. Previously, due to the dependency of grpc, both python2 and python3 has to be available in the system to build tensorflow-io.

This PR patches grpc source code to remove the need for python2.

This PR also adds an additional section in README with instructions to build on CentOS 8.

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+81 -30

0 comment

4 changed files

yongtang

pr closed time in 6 days

PR opened r-lib/lobstr

Fix typo in NEWS.md
+1 -1

0 comment

1 changed file

pr created time in 6 days

push eventterrytangyuan/lobstr

Yuan Tang

commit sha 20359943baed57ef3c70dab8b8161dc6735503af

Fix typo in NEWS.md

view details

push time in 6 days

fork terrytangyuan/lobstr

Understanding complex R objects with tools similar to str()

https://lobstr.r-lib.org/

fork in 6 days

push eventtensorflow/io

Yong Tang

commit sha 4fb30abe2597125cec973b22029b752619790d79

Update tf_configure.bzl to allow centos 7 build, and add README.md section (#788) * Update tf_configure.bzl to allow centos 7 build Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Update README.md to include details instructions for CentOS 7 and Ubuntu 18.04 build Also setup GitHub CI to validate the README.md Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 7 days

PR merged tensorflow/io

Update tf_configure.bzl to allow centos 7 build, and add README.md section

This PR fixes the issue raised in #722 where tensorflow-io could not be compiled on CentOS 7.

The issue was that, CentOS 7's gcc version is too old, the libstdc++ version is also too old. In addition, CentOS's tensorflow header and libs are both /lib64/ and /lib/, which is not captured in tf_configure.bzl.

This PR updates several places:

  1. Add special handling in tf_configure.bzl to capture /lib64/ and /lib/
  2. Use devtoolset-9 to build .so on CentOS
  3. Use static libs linkage for libstdc++ through:
    BAZEL_LINKOPTS="-static-libstdc++ -static-libgcc" \
       BAZEL_LINKLIBS="-lm -l%:libstdc++.a"
    

Additional sections in README.md have been added to show the complete shell to build on CentOS 7 and Ubuntu 18.04. Two GitHub CI workflow have been added as well to validate the build on CentOS 7 and Ubuntu 18.04.

This PR fixes #722

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+126 -0

0 comment

4 changed files

yongtang

pr closed time in 7 days

issue closedtensorflow/io

Compiler errors while building on centos7 image

Hi, We are trying to build tensonflow/io v0.10.0. We are using a centos7 base instead of an ubuntu base, but otherwise our build process is similar:

# Build and run the Docker image
docker build -f tools/dev/Dockerfile -t tfio-dev .
# Run the docker image
docker run --rm --net=host -v ${PWD}:/v -w /v tfio-dev
# Inside the container configure install
./configure.sh
# Build TensorFlow I/O. More info on flags https://www.tensorflow.org/install/source#configuration_options
bazel build -c opt --copt=-fPIC -s --verbose_failures //tensorflow_io/core/...
# Run c++ tests
bazel test --verbose_failures //tensorflow_io/avro/... --action_env=LD_LIBRARY_PATH=$LD_LIBRARY_PATH
# Build the TensorFlow I/O package
python setup.py bdist_wheel
# TODO Install the wheel
pip install --upgrade --force-reinstall dist/tensorflow_io-*.whl
# Run tests
pytest -s -v tensorflow_io/avro/python/tests

Here's docker file:

FROM ${private-registry}/tensorflow/tensorflow-builder:0.0.1
RUN yum install -y \
    git \
    gcc \
    g++ \
    gdb \
    make \
    patch \
    curl \
    nano \
    unzip \
    ffmpeg \
    dnsutils

ARG BAZEL_VERSION=0.29.0
ARG BAZEL_OS=linux
RUN curl -sL https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-${BAZEL_OS}-x86_64.sh -o bazel-install.sh && \
    bash -x bazel-install.sh && \
    rm bazel-install.sh
ARG CONDA_OS=Linux
# Miniconda - Python 3.6, 64-bit, x86, latest
RUN curl -sL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o mconda-install.sh && \
    bash -x mconda-install.sh -b -p miniconda && \
    rm mconda-install.sh
ENV PATH="/miniconda/bin:$PATH"
ARG CONDA_ADD_PACKAGES=""
RUN conda create -y -q -n tfio-dev python=3.6 ${CONDA_ADD_PACKAGES}

ARG PIP_ADD_PACKAGES=""
ARG ARROW_VERSION=0.14.1
RUN /bin/bash -c "source activate tfio-dev && python -m pip install \
    avro-python3 \
    pytest \
    pylint \
    boto3 \
    google-cloud-pubsub==0.39.1 \
    pyarrow==${ARROW_VERSION} \
    pandas \
    fastavro \
    gast==0.2.2 \
    ${PIP_ADD_PACKAGES} \
    "
ENV TFIO_DATAPATH=bazel-bin

The error we get is attached. kafka-err.txt

closed time in 7 days

ashahab

startedcli/cli

started time in 8 days

issue closedopenjournals/joss-reviews

[PRE REVIEW]: PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data

Submitting author: @briandesilva (<a href="http://orcid.org/0000-0003-0944-900X">Brian de Silva</a>) Repository: <a href="https://github.com/dynamicslab/pysindy" target ="_blank">https://github.com/dynamicslab/pysindy</a> Version: v0.12.0 Editor: @terrytangyuan Reviewers: @sixpearls, @dawbarton Managing EiC: Daniel S. Katz

Author instructions

Thanks for submitting your paper to JOSS @briandesilva. Currently, there isn't an JOSS editor assigned to your paper.

@briandesilva if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

closed time in 8 days

whedon
IssuesEvent

issue commentopenjournals/joss-reviews

[PRE REVIEW]: PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data

I'm happy to review but I've got a heavy teaching load for the next 3 weeks so it'll have to be after that.

Thanks @dawbarton this is fine. @sixpearls can probably review first and then you can start reviewing once you are available.

whedon

comment created time in 8 days

startedPyCQA/bandit

started time in 8 days

delete branch kubeflow/katib

delete branch : terrytangyuan-patch-1

delete time in 8 days

PR opened kubeflow/katib

MXNet -> Apache MXNet

<!-- Thanks for sending a pull request! Here are some tips for you:

  1. If this is your first time, read our contributor guidelines https://git.k8s.io/community/contributors/guide/pull-requests.md#the-pull-request-submit-process and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
  2. If you want faster PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
  3. Follow the instructions for writing a release note: https://git.k8s.io/community/contributors/guide/release-notes.md
  4. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
  5. If this PR changes image versions, please title this PR "Bump <image name> from x.x.x to y.y.y." -->

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #

Special notes for your reviewer:

  1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note: <!-- Write your release note:

  1. Enter your extended release note in the below block. If the PR requires additional action from users switching to the new release, include the string "action required".
  2. If no release note is required, just write "NONE". -->

+1 -1

0 comment

1 changed file

pr created time in 8 days

create barnchkubeflow/katib

branch : terrytangyuan-patch-1

created branch time in 8 days

Pull request review commentsql-machine-learning/elasticdl

Add ODPS UDF to normalize table with key-value column to wide table

+# coding:utf-8+# qinlong.wql

Remove these comments

workingloong

comment created time in 8 days

Pull request review commentsql-machine-learning/elasticdl

Add ODPS UDF to normalize table with key-value column to wide table

+import argparse+import time++import odps+from odps import options, ODPS++options.log_view_host = "http://logview.odps.aliyun-inc.com:8080"

This information can be exposed to public?

workingloong

comment created time in 9 days

push eventkatib-examples/evaluation

Ce Gao

commit sha c14050207504ef0d8bc2c2b911f812b039dc7d30

feat: Refactor Signed-off-by: Ce Gao <gaoce@caicloud.io>

view details

Yuan Tang

commit sha 65b6a11bec95b4260d5d117831b4ad66f4e93310

Merge pull request #4 from gaocegege/gaoce feat: Refactor

view details

push time in 9 days

PR merged katib-examples/evaluation

feat: Refactor

Signed-off-by: Ce Gao gaoce@caicloud.io

+74 -85673

0 comment

25 changed files

gaocegege

pr closed time in 9 days

push eventtensorflow/io

Yong Tang

commit sha 609af2f9a2d2bc7f246b99e233809485824efb8f

Fix prometheus endpoint issue (#786) This PR fixes prometheus endpoint issue where `%d` should be '%s' Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 9 days

PR merged tensorflow/io

Fix prometheus endpoint issue

This PR fixes prometheus endpoint issue where %d should be '%s'

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+5 -3

0 comment

2 changed files

yongtang

pr closed time in 9 days

delete branch terrytangyuan/evaluation

delete branch : patch-1

delete time in 10 days

push eventtensorflow/io

Yong Tang

commit sha 8513fd0a6853391ccf4052bbb91dbdafa133f235

Add SQL version of Air Quality Data set, to be used by tutorial (#784) * Add SQL version of Air Quality Data set, to be used by tutorial Signed-off-by: Yong Tang <yong.tang.github@outlook.com> * Remove aapl as it is not open license Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

push time in 10 days

PR merged tensorflow/io

Add SQL version of Air Quality Data set, to be used by tutorial

Add SQL version of Air Quality Data set, to be used by tutorial

Signed-off-by: Yong Tang yong.tang.github@outlook.com

+9386 -521

2 comments

3 changed files

yongtang

pr closed time in 10 days

startedpsampaz/go-mod-outdated

started time in 10 days

PR opened tensorflow/tensorflow

Remove unnecessary "T" before "tf.Tensor" in all_reduce.py

I think these are not necessary unless I am missing something obvious here.

+41 -41

0 comment

1 changed file

pr created time in 10 days

create barnchtensorflow/tensorflow

branch : terrytangyuan-patch-2

created branch time in 10 days

delete branch terrytangyuan/mpi-operator

delete branch : error-log-zero-container

delete time in 10 days

push eventkubeflow/mpi-operator

Yuan Tang

commit sha 8b4df2194996eb18d48fa7c34d293e94f6e2e3b7

Add error log when pod spec does not have any containers (#190) Signed-off-by: terrytangyuan <terrytangyuan@gmail.com>

view details

push time in 10 days

PR merged kubeflow/mpi-operator

Reviewers
Add error log when pod spec does not have any containers approved lgtm size/XS

Sometimes if MPIJob spec has certain field in the wrong indentation, kubectl apply still passes the validation check. However, the controller will crash with panic in such cases.

For example, if there are no containers specified in the pod specs, the controller will crash. We may need a better error propagation mechanism but for now it's good to print out the error message in this case. Otherwise it's hard for users to debug.

Example panic log (truncated):

I0212 15:10:57.092103       1 mpi_job_controller.go:352] Starting workers
I0212 15:10:57.092145       1 mpi_job_controller.go:358] Started workers
I0212 15:10:57.092327       1 mpi_job_controller.go:433] Finished syncing job "kubemaker/tensorflow-benchmarks" (148.32µs)
E0212 15:10:57.092452       1 runtime.go:69] Observed a panic: runtime.boundsError{x:0, y:0, signed:true, code:0x0} (runtime error: index out of range [0] with length 0)
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:76
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:65
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/panic.go:679
/usr/local/go/src/runtime/panic.go:75
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:1086
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:740
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:535
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:408
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:417
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:369
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/wait/wait.go:133
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/wait/wait.go:134
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/wait/wait.go:88
/usr/local/go/src/runtime/asm_amd64.s:1357
panic: runtime error: index out of range [0] with length 0 [recovered]
	panic: runtime error: index out of range [0] with length 0

goroutine 218 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:58 +0x105
panic(0x136c020, 0xc000b87aa0)
	/usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2.newWorker(0xc0004e0480, 0x1, 0x9)

Signed-off-by: terrytangyuan terrytangyuan@gmail.com

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+6 -0

2 comments

1 changed file

terrytangyuan

pr closed time in 10 days

PR opened kubeflow/mpi-operator

Reviewers
Add error log when pod spec does not have any containers

Sometimes if MPIJob spec has certain field in the wrong indentation, kubectl apply still passes the validation check. However, the controller will crash with panic in such cases.

For example, if there are no containers specified in the pod specs, the controller will crash. We may need a better error propagation mechanism but for now it's good to print out the error message in this case. Otherwise it's hard for users to debug.

Example panic log (truncated):

I0212 15:10:57.092103       1 mpi_job_controller.go:352] Starting workers
I0212 15:10:57.092145       1 mpi_job_controller.go:358] Started workers
I0212 15:10:57.092327       1 mpi_job_controller.go:433] Finished syncing job "kubemaker/tensorflow-benchmarks" (148.32µs)
E0212 15:10:57.092452       1 runtime.go:69] Observed a panic: runtime.boundsError{x:0, y:0, signed:true, code:0x0} (runtime error: index out of range [0] with length 0)
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:76
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:65
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/panic.go:679
/usr/local/go/src/runtime/panic.go:75
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:1086
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:740
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:535
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:408
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:417
/go/src/github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2/mpi_job_controller.go:369
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/wait/wait.go:133
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/wait/wait.go:134
/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/wait/wait.go:88
/usr/local/go/src/runtime/asm_amd64.s:1357
panic: runtime error: index out of range [0] with length 0 [recovered]
	panic: runtime error: index out of range [0] with length 0

goroutine 218 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20181127025237-2b1284ed4c93/pkg/util/runtime/runtime.go:58 +0x105
panic(0x136c020, 0xc000b87aa0)
	/usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/kubeflow/mpi-operator/pkg/controllers/v1alpha2.newWorker(0xc0004e0480, 0x1, 0x9)

Signed-off-by: terrytangyuan terrytangyuan@gmail.com

+6 -0

0 comment

1 changed file

pr created time in 10 days

create barnchterrytangyuan/mpi-operator

branch : error-log-zero-container

created branch time in 10 days

push eventkubeflow/mpi-operator

Lei Xue

commit sha 83687032a8597dfb040cd3db4491845ce4f1e3b9

add scheduling groups of kube-batch/volcano (#188) Signed-off-by: Lei Xue <carmark.dlut@gmail.com>

view details

push time in 10 days

PR merged kubeflow/mpi-operator

Reviewers
add scheduling groups of kube-batch/volcano for role approved lgtm size/XS

<!-- Reviewable:start --> This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+8 -0

2 comments

1 changed file

carmark

pr closed time in 10 days

PR opened katib-examples/evaluation

Update README.md

The O should be capitalized

+2 -2

0 comment

1 changed file

pr created time in 10 days

push eventterrytangyuan/evaluation

Yuan Tang

commit sha 727ae40976c0c1b547eb5f4897b3925b5f643b3c

Update README.md

view details

push time in 10 days

PR merged kubeflow/mpi-operator

Reviewers
add options for gang scheduler name lgtm size/M

<!-- Reviewable:start --> This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+37 -46

4 comments

5 changed files

carmark

pr closed time in 10 days

push eventkubeflow/mpi-operator

Lei Xue

commit sha 7760b3c19822122072477232aafc37504d1fa7c3

add options for gang scheduler name (#189) * add option for gang scheduler name Signed-off-by: Lei Xue <carmark.dlut@gmail.com> * fix travis-ci issue Signed-off-by: Lei Xue <carmark.dlut@gmail.com>

view details

push time in 10 days

pull request commentkubeflow/mpi-operator

add options for gang scheduler name

/approve

carmark

comment created time in 10 days

pull request commentkubeflow/mpi-operator

add options for gang scheduler name

/lgtm

carmark

comment created time in 10 days

pull request commentkubeflow/mpi-operator

add options for gang scheduler name

Thanks. Could you fix the tests?

carmark

comment created time in 11 days

Pull request review commentkubeflow/mpi-operator

add scheduling groups of kube-batch/volcano for role

 rules:   - mpijobs/status   verbs:   - "*"+- apiGroups:+  - scheduling.incubator.k8s.io

Okay sounds good. Are there any code changes required in order to support that?

carmark

comment created time in 11 days

Pull request review commentkubeflow/mpi-operator

add scheduling groups of kube-batch/volcano for role

 rules:   - mpijobs/status   verbs:   - "*"+- apiGroups:+  - scheduling.incubator.k8s.io

Thanks for the PR. Could you explain why this is needed?

carmark

comment created time in 11 days

issue commentopenjournals/joss-reviews

[PRE REVIEW]: PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data

👋 Hi @jmbr @dawbarton @sixpearls @MakrinaAgaoglou, if you would like to review for this submission, please let us know here! We need at least two reviewers.

whedon

comment created time in 11 days

startedkiosk-sh/kiosk

started time in 11 days

issue commentopenjournals/joss-reviews

[PRE REVIEW]: PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data

@briandesilva If you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission.

whedon

comment created time in 11 days

issue commentopenjournals/joss-reviews

[PRE REVIEW]: relped: Build Relatedness Pedigrees

I am now an editor of JOSS so likely won't have time to participate in reviews. Thanks for checking though. BTW, @arfon could you remove me from the reviewers spreadsheet?

whedon

comment created time in 11 days

push eventterrytangyuan/scaffolder

terrytangyuan

commit sha cf67f5ffa897d26ba77a9479cf6f51cf50b7a1c4

Add setuptools to pip install Signed-off-by: terrytangyuan <terrytangyuan@gmail.com>

view details

push time in 11 days

push eventterrytangyuan/scaffolder

terrytangyuan

commit sha 68ec6a143f7e7e7746dca00d5486c2a19359772d

Added missing export statement Signed-off-by: terrytangyuan <terrytangyuan@gmail.com>

view details

push time in 11 days

startedtinygo-org/tinygo

started time in 12 days

push eventkubeflow/mpi-operator

Yuan Tang

commit sha f0dd258d926ee78a16260ddb9f8ccf8b379f511d

Use operator's namespace for initializing informers and endpoints/leaderElection (#186) Signed-off-by: terrytangyuan <terrytangyuan@gmail.com>

view details

push time in 12 days

delete branch kubeflow/mpi-operator

delete branch : terrytangyuan-patch-1

delete time in 12 days

PR merged kubeflow/mpi-operator

Reviewers
Use operator's namespace for initializing informers and endpoints/leaderElection approved lgtm size/S

Right now namespace variable is the Kubeflow namespace passed from environment variable. We should re-assign it to be operator-level namespace so that endpoints meta and leader election could work correctly.

<!-- Reviewable:start -->

This change is <img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/> <!-- Reviewable:end -->

+10 -7

2 comments

1 changed file

terrytangyuan

pr closed time in 12 days

push eventkubeflow/mpi-operator

terrytangyuan

commit sha 2da82acca310cad80697097809bc4850c8f32661

Use operator's namespace for initializing informers and endpoints/leaderElection Signed-off-by: terrytangyuan <terrytangyuan@gmail.com>

view details

push time in 12 days

push eventkubeflow/mpi-operator

terrytangyuan

commit sha 95bd77b6cc82c6255f1bdab6288aec695428a847

Use namespace variable instead Signed-off-by: terrytangyuan <terrytangyuan@gmail.com>

view details

push time in 12 days

PR opened kubeflow/mpi-operator

Switch to use operator's namespace for endpoints/leaderElection

Right now namespace variable is the Kubeflow namespace passed from environment variable. However, we should use the operator-level namespace for endpoints meta and leader election.

+1 -1

0 comment

1 changed file

pr created time in 12 days

more