profile
viewpoint
Yorkie Liu yorkie Learning & exploring

alibaba/pipcook 1095

Front-End Algorithm Engineering Platform based on Tensorflow.js

Southern/node-x509 97

Simple X509 certificate parser.

cocos-creator/firedoc 70

API Doc generator based on YUIDoc

koajs/koa-range 41

range request implementation for koa, see http://tools.ietf.org/html/rfc7233

macacajs/macaca-puppeteer 38

Macaca puppeteer driver

rvagg/archived-servertest 31

A simple HTTP server testing tool

cocos-creator-packages/ui-kit 30

The UI kit used in Editor Framework

cocos-creator-packages/ui-grid 7

A grid scale widget implemented by pixi graphcis module (WebGL)

weflex/react-touchable-tabular 4

A React Component to used to view tabular data based on facebook/fixed-data-table

cocos-creator/firedoc-theme-notab 3

A new theme without tab in navigation sidebar

push eventalibaba/pipcook

WenheLI

commit sha 41dc824a53a5c6f669fc767c7b4d46d0d32eaf90

deploy: 6fbc76873af4b1e906bdc39b405694501800898b

view details

push time in 2 hours

issue commentalibaba/pipcook

enhancement: improve developing experience

Right now, if we want to modify the anything inside pipcook system, we need to rebuild to make them work and that is time-consuming

I think we could move some of these plugins to org imgcook.

WenheLI

comment created time in 16 hours

push eventalibaba/pipcook

rickycao-qy

commit sha 378f20b6d42951b7ecc6c1124c02944c0b07cc12

deploy: 729700fcf6478dbed60fc050296bb0126aa7ee46

view details

push time in 17 hours

push eventalibaba/pipcook

yorkie

commit sha 84b4f7a521044aa571c45f090b73393d1d29d5d4

fix nits

view details

push time in 17 hours

Pull request review commentalibaba/pipcook

daemon: add migration support

+'use strict';+import { QueryInterface, DataTypes } from 'sequelize';++module.exports = {

ES Modules is required, too

WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

+'use strict';+import {QueryInterface, DataTypes} from 'sequelize';++module.exports = {+  up: async (queryInterface: QueryInterface) => {+    return queryInterface.sequelize.transaction(t => {+      return Promise.all([+        queryInterface.addColumn('pipelines', 'datasetProcess', {+          type: DataTypes.STRING+        }, { transaction: t }),+        queryInterface.addColumn('pipelines', 'datasetProcessParams', {+          type: DataTypes.STRING+        }, { transaction: t })+      ])+    })+  },++  down: (queryInterface: QueryInterface) => {+    return queryInterface.sequelize.transaction(t => {+      return queryInterface.removeColumn('pipelines', 'datasetProcess', { transaction: t }).then(() => {

So shall we use async/await instead?

WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

   },   "scripts": {     "postinstall": "lerna bootstrap",-    "build": "lerna run compile",+    "build": "lerna run compile && rm ./packages/daemon/dist/migrations/*.d.ts",

I see

WenheLI

comment created time in a day

push eventalibaba/pipcook

yorkie

commit sha e60305722b042cb13c20040db2885d62ead462bb

deploy: ac00e872b34c308421a67601e84e11949f9c12d1

view details

push time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

+'use strict';+import {QueryInterface, DataTypes} from 'sequelize';++module.exports = {+  up: async (queryInterface: QueryInterface) => {+    return queryInterface.sequelize.transaction(t => {+      return Promise.all([+        queryInterface.addColumn('pipelines', 'datasetProcess', {+          type: DataTypes.STRING+        }, { transaction: t }),+        queryInterface.addColumn('pipelines', 'datasetProcessParams', {+          type: DataTypes.STRING+        }, { transaction: t })+      ])+    })+  },++  down: (queryInterface: QueryInterface) => {+    return queryInterface.sequelize.transaction(t => {+      return queryInterface.removeColumn('pipelines', 'datasetProcess', { transaction: t }).then(() => {

Will the removeColumn function return a Promise?

WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

+'use strict';+import {QueryInterface, DataTypes} from 'sequelize';
import { QueryInterface, DataTypes } from 'sequelize';
WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

+const os = require('os');++module.exports = {+  dialect: 'sqlite',+  storage: process.env.PIPCOOK_STORAGE || (`${os.homedir()}/.pipcook/db/pipcook.db`)+}

ES Modules is required.

WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

+const os = require('os');
import os = from 'os';
WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

   },   "scripts": {     "postinstall": "lerna bootstrap",-    "build": "lerna run compile",+    "build": "lerna run compile && rm ./packages/daemon/dist/migrations/*.d.ts",

Why remove the declaration files?

WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

 The `pipeline_name` is the name of the pipeline file under "test/pipelines", suc - "mnist-image-classification" - "databinding-image-classification" +### Database+If your modification involves `Database` structure, you need to write a migration script under `daemon/src/migrations`.
If your modification involves `Database` structure, you need to write a migration script under `packages/daemon/src/migrations`.
WenheLI

comment created time in a day

Pull request review commentalibaba/pipcook

daemon: add migration support

 The `pipeline_name` is the name of the pipeline file under "test/pipelines", suc - "mnist-image-classification" - "databinding-image-classification" +### Database+If your modification involves `Database` structure, you need to write a migration script under `daemon/src/migrations`.

An empty line would be better.

WenheLI

comment created time in a day

push eventalibaba/pipcook

yorkie

commit sha 8ed372256af1f68fb51a8a89f2e8dfc0281b2226

deploy: 7760fb6c2795c9dc0c11ed87ec0e79400ee32921

view details

push time in a day

pull request commentalibaba/pipcook

costa: add python runnable for machine-learning developers

@rickycao-qy @FeelyChau I have rebased this pull request :)

yorkie

comment created time in a day

push eventalibaba/pipcook

Ruikun Cao

commit sha ba65cd64a7a263f046806748b061d3bdffbde178

Publish v1.0.5 (#387) - @pipcook/app@1.0.5 - @pipcook/boa@1.0.5 - @pipcook/pipcook-cli@1.0.5 - @pipcook/pipcook-core@1.0.5 - @pipcook/costa@1.0.5 - @pipcook/daemon@1.0.5 - @pipcook/pipboard@1.0.5 - @pipcook/plugins-coco-data-access@1.0.5 - @pipcook/plugins-csv-data-access@1.0.5 - @pipcook/plugins-pascalvoc-data-access@1.0.5 - @pipcook/plugins-textline-data-access@1.0.5 - @pipcook/plugins-chinese-poem-data-collect@1.0.5 - @pipcook/plugins-csv-data-collect@1.0.5 - @pipcook/plugins-fasttext-data-collect@1.0.5 - @pipcook/plugins-image-classification-data-collect@1.0.5 - @pipcook/plugins-mnist-data-collect@1.0.5 - @pipcook/plugins-object-detection-coco-data-collect@1.0.5 - @pipcook/plugins-object-detection-pascalvoc-data-collect@1.0.5 - @pipcook/plugins-image-data-process@1.0.5 - @pipcook/plugins-bayesian-model-define@1.0.5 - @pipcook/plugins-detectron-fasterrcnn-model-define@1.0.5 - @pipcook/plugins-detectron-maskrcnn-model-define@1.0.5 - @pipcook/plugins-fasttext-model-define@1.0.5 - @pipcook/plugins-tensorflow-cycle-gan-model-define@1.0.5 - @pipcook/plugins-tensorflow-mobilenet-model-define@1.0.5 - @pipcook/plugins-tensorflow-resnet-model-define@1.0.5 - @pipcook/plugins-tfjs-mobilenet-model-define@1.0.5 - @pipcook/plugins-tfjs-simplecnn-model-define@1.0.5 - @pipcook/plugins-tfjs-text-lstm-model-define@1.0.5 - @pipcook/plugins-bayesian-model-evaluate@1.0.5 - @pipcook/plugins-detectron-model-evaluate@1.0.5 - @pipcook/plugins-fasttext-model-evaluate@1.0.5 - @pipcook/plugins-image-classification-tensorflow-model-evaluate@1.0.5 - @pipcook/plugins-image-classification-tfjs-model-evaluate@1.0.5 - @pipcook/plugins-image-generation-tensorflow-model-evaluate@1.0.5 - @pipcook/plugins-text-creation-tfjs-model-evaluate@1.0.5 - @pipcook/plugins-tensorflow-model-load@1.0.5 - @pipcook/plugins-bayesian-model-train@1.0.5 - @pipcook/plugins-detectron-model-train@1.0.5 - @pipcook/plugins-fasttext-model-train@1.0.5 - @pipcook/plugins-image-classification-tensorflow-model-train@1.0.5 - @pipcook/plugins-image-classification-tfjs-model-train@1.0.5 - @pipcook/plugins-image-generation-tensorflow-model-train@1.0.5 - @pipcook/plugins-text-creation-tfjs-model-train@1.0.5 - @pipcook/sdk@1.0.5

view details

feely

commit sha 470de29d58f5b7a5484c74252665fa2f48496ac4

daemon: reference plugin name from core (#385)

view details

feely

commit sha 1dd3f74602a6ac26a19fd1aa55369f4537a6c7fb

core: fix test error (#391)

view details

豆糜

commit sha 3271e0787fe597a2d0a41a98070b5bdb93b196e4

fix: installation guide url (#386)

view details

WenheLI

commit sha c0ac74ed9b2a5d2553293f38e7f3f3fc786c1296

CI/CD: fix beta release skipping (#393) * fix: add from-git argument * fix: change preid to beta

view details

Yorkie Liu

commit sha 2175ca05d7a790bab0f2e483f56f830a6b4c30e3

build: tweak the versions to decrease root packages' size (#392)

view details

feely

commit sha 4b15d60eae5fd4cc6c96cecfcc2309e709910e2b

core: replace shortid with nanoid (#394)

view details

Johannes

commit sha 231bbcc7cb13b1c1264c3bc4e7b65c82978038e0

fix: path of the pipeline examples (#396)

view details

feely

commit sha 37b859f06c77d5489aaf3a58553e194240176a24

boa: fix redownload pybind11 when execute `lerna add` (#395)

view details

Yorkie Liu

commit sha c9343cbff1f9b0c9f1fcefe8cb77d5688a7d4698

build, doc: remove deprecated typedoc-plugin-lerna (#400) And we will generate our API documentation separately: 1. Pipcook SDK API 2. Pipcook Core API

view details

Yorkie Liu

commit sha 7760fb6c2795c9dc0c11ed87ec0e79400ee32921

boa: generate object instead of `dict` to Python function (#389)

view details

WenheLI

commit sha ac00e872b34c308421a67601e84e11949f9c12d1

build: re-installation takes too long (#399)

view details

yorkie

commit sha 05894291503467c39cd6b17968a870c6b8fd3aa4

fix nits

view details

yorkie

commit sha c79f8d42065bf8f61fcab9fd217b6f35b5d03803

Merge branch 'master' of github.com:alibaba/pipcook into costa/add-py-runtime

view details

push time in a day

push eventalibaba/pipcook

WenheLI

commit sha ac00e872b34c308421a67601e84e11949f9c12d1

build: re-installation takes too long (#399)

view details

push time in a day

delete branch alibaba/pipcook

delete branch : fix/clean-before-install

delete time in a day

PR merged alibaba/pipcook

build: re-installation takes too long build

This PR adds clean ahead of install process to avoid extremely long prune hoisting time. For more detail, please refer to this issue https://github.com/lerna/lerna/issues/602

+1 -1

2 comments

1 changed file

WenheLI

pr closed time in a day

push eventalibaba/pipcook

Yorkie Liu

commit sha 7760fb6c2795c9dc0c11ed87ec0e79400ee32921

boa: generate object instead of `dict` to Python function (#389)

view details

push time in a day

delete branch alibaba/pipcook

delete branch : fix/boa-function-call

delete time in a day

PR merged alibaba/pipcook

boa: generate object instead of `dict` to Python function boa

It creates a new class and instantiates it with given props when passing to Python. For example:

pythonfunc({ foobar: 10 });

At Python, we have to write in this way:

def func(opt):
  opt.get('foobar') # 10, because opt is a dict

Within this Pull Request, the developer is able:

def func(opt):
  opt.foobar

Otherwise, I also removed an incorrect case:

const o = { foobar: 10 };
len(o); // not working
len(dict(boa.kwargs(o))); // working

Even though it's more complexed than before, but it's more correct way to be used.

+40 -11

4 comments

8 changed files

yorkie

pr closed time in a day

push eventalibaba/pipcook

yorkie

commit sha 9295e63fd361398707b1ec35d2e06d2b283dac37

deploy: c9343cbff1f9b0c9f1fcefe8cb77d5688a7d4698

view details

push time in a day

push eventalibaba/pipcook

Yorkie Liu

commit sha c9343cbff1f9b0c9f1fcefe8cb77d5688a7d4698

build, doc: remove deprecated typedoc-plugin-lerna (#400) And we will generate our API documentation separately: 1. Pipcook SDK API 2. Pipcook Core API

view details

push time in a day

delete branch alibaba/pipcook

delete branch : fix/gh-398

delete time in a day

PR merged alibaba/pipcook

build, doc: remove deprecated typedoc-plugin-lerna build documentation

And we will generate our API documentation separately:

  1. Pipcook SDK API
  2. Pipcook Core API
+0 -1

2 comments

1 changed file

yorkie

pr closed time in a day

issue closedalibaba/pipcook

bug: New Typescript version breaks tsdoc

We upgraded the Typescript version to 3.9.x and it seems that this upgrade breaks the tsdoc building process in Github Actions

closed time in a day

WenheLI

pull request commentalibaba/pipcook

build, doc: remove deprecated typedoc-plugin-lerna

/cc @WenheLI

yorkie

comment created time in 2 days

PR opened alibaba/pipcook

build, doc: remove deprecated typedoc-plugin-lerna build documentation

And we will generate our API documentation separately:

  1. Pipcook SDK API
  2. Pipcook Core API
+0 -1

0 comment

1 changed file

pr created time in 2 days

create barnchalibaba/pipcook

branch : fix/gh-398

created branch time in 2 days

pull request commentalibaba/pipcook

bug: Re-installation takes too long

LGTM and that's great if we can provide some benchmarks, and I removed the bug label because we usually use that to file an issue is a bug that is to be addressed later.

WenheLI

comment created time in 2 days

Pull request review commentalibaba/pipcook

daemon: add migration support

+'use strict';++module.exports = {+  up: async (queryInterface, Sequelize) => {+    return queryInterface.sequelize.transaction(t => {+      return Promise.all([+        queryInterface.addColumn('pipelines', 'datasetProcess', {+          type: Sequelize.DataTypes.STRING+        }, { transaction: t }),+        queryInterface.addColumn('pipelines', 'datasetProcessParams', {+          type: Sequelize.DataTypes.STRING+        }, { transaction: t })+      ])+    })+  },++  down: async (queryInterface, Sequelize) => {+    return queryInterface.sequelize.transaction(t => {

Is there async/await function for this?

WenheLI

comment created time in 2 days

Pull request review commentalibaba/pipcook

daemon: add migration support

+

And we need to use TypeScript here.

WenheLI

comment created time in 2 days

Pull request review commentalibaba/pipcook

daemon: add migration support

++const os = require('os');++module.exports = {+  dialect: "sqlite",
  dialect: 'sqlite',
WenheLI

comment created time in 2 days

Pull request review commentalibaba/pipcook

daemon: add migration support

     "ci": "npm run cov",     "compile": "midway-bin build -c --port=6927",     "clean": "rm -rf src/app/public",-    "test": "midway-bin test --ts --full-trace"+    "test": "midway-bin test --ts --full-trace",+    "migration": "npx sequelize-cli db:migrate",+    "undo-migration": "npx sequelize-cli db:migrate:undo"
    "migration:undo": "npx sequelize-cli db:migrate:undo"
WenheLI

comment created time in 2 days

Pull request review commentalibaba/pipcook

daemon: add migration support

+const path = require('path');

Add a shebang?

WenheLI

comment created time in 2 days

Pull request review commentalibaba/pipcook

daemon: add migration support

 And anyone could try your plugin via the following command: $ pipcook plugin install your-pipcook-plugin-name ``` +## Database++If your modification involves `Database` structure, you need to write a migration script under `daemon/src/migrations`.+For more detail about it, please refer to [Migration](https://sequelize.org/master/manual/migrations.html).

We may need to add documentation for daemon developers or https://github.com/alibaba/pipcook/blob/master/docs/contributing/guide-to-contributor.md?

WenheLI

comment created time in 2 days

push eventalibaba/pipcook

yorkie

commit sha dc9012a0dce9141690e093acf3bf2063bab9fd57

fix nits

view details

push time in 2 days

push eventalibaba/pipcook

Yorkie Liu

commit sha 1f3c84d8c420a74c19585ace1913028e9cf5325d

Update index.ts

view details

push time in 2 days

pull request commentalibaba/pipcook

boa: generate object instead of `dict` to Python function

Could you please describe what the problem is? And it is better to provide a test case to reproduce.

In this case, the class CycleGan constructor expects dict, it will throw error if we merge this PR, so we need to refactor them together.

Ok, I will wrap them with a dict().

yorkie

comment created time in 2 days

pull request commentalibaba/pipcook

daemon: refactor plugin api

The next work is that we need to reimplement cli based on the new api.

Will you work that in this pull request?

FeelyChau

comment created time in 2 days

pull request commentalibaba/pipcook

build: tweak the versions to decrease root packages' size

There are some conflicts, could you resolve them and commit again?

Fixed, and let's wait for CI builds.

yorkie

comment created time in 2 days

push eventalibaba/pipcook

feely

commit sha 1dd3f74602a6ac26a19fd1aa55369f4537a6c7fb

core: fix test error (#391)

view details

豆糜

commit sha 3271e0787fe597a2d0a41a98070b5bdb93b196e4

fix: installation guide url (#386)

view details

WenheLI

commit sha c0ac74ed9b2a5d2553293f38e7f3f3fc786c1296

CI/CD: fix beta release skipping (#393) * fix: add from-git argument * fix: change preid to beta

view details

Yorkie Liu

commit sha bf8f5cddbe5f32340b28bf4cada73ca8d23b2939

Merge branch 'master' into fix/gh-344

view details

push time in 2 days

Pull request review commentalibaba/pipcook

boa: generate object instead of `dict` to Python function

 function _internalWrap(T, src={}) {     invoke: {       enumerable: false,       writable: false,-      value: args => T.invoke.apply(T, args),+      value: args => {+        return T.invoke.apply(T, args);

No any differences, just add brackets to wrap the function.

yorkie

comment created time in 2 days

pull request commentalibaba/pipcook

boa: generate object instead of `dict` to Python function

Could you please describe what the problem is? And it is better to provide a test case to reproduce.

yorkie

comment created time in 2 days

push eventalibaba/pipcook

WenheLI

commit sha 5fcd47762669d95d0e6b9c708f449a436bab5940

deploy: c0ac74ed9b2a5d2553293f38e7f3f3fc786c1296

view details

push time in 2 days

push eventalibaba/pipcook

rickycao-qy

commit sha 9d7180fd6b1ba482d5b3bacc10e9e85f8aa66447

deploy: 3271e0787fe597a2d0a41a98070b5bdb93b196e4

view details

push time in 4 days

startedtc39/proposal-Number.range

started time in 5 days

issue commenttc39/proposal-slice-notation

Consider basing the semantics on a method call

@caub Thanks, I will open a new discussion, and in my use-case is here, I wrote this library(@pipcook/boa) to call Python function from Node.js, and that's great if both the slice expression and step argument are supported :)

littledan

comment created time in 5 days

PR opened alibaba/pipcook

Reviewers
build: tweak the versions to decrease root packages' size build

This fixes #344, and the node_modules result is:

before after
1.3G 826M
+509 -82

0 comment

7 changed files

pr created time in 5 days

create barnchalibaba/pipcook

branch : fix/gh-344

created branch time in 5 days

push eventalibaba/pipcook

yorkie

commit sha efc241ed6ed7cdd13db5875f3e2a01b840e24bb8

deploy: 1dd3f74602a6ac26a19fd1aa55369f4537a6c7fb

view details

push time in 5 days

push eventalibaba/pipcook

feely

commit sha 1dd3f74602a6ac26a19fd1aa55369f4537a6c7fb

core: fix test error (#391)

view details

push time in 5 days

delete branch alibaba/pipcook

delete branch : fix/core-test

delete time in 5 days

PR merged alibaba/pipcook

core: fix test error core

fs.rmdir fails if the target directory is not empty, using fs.remove instand.

+2 -2

0 comment

1 changed file

FeelyChau

pr closed time in 5 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

+import { listen } from './request';+import { LogCallback } from './interface';++export class BaseApi {+  route: string;++  constructor(uri: string) {+    this.route = uri;+  }++  /**+   * listen log+   * @param logId log id+   * @param logCallback log callback+   */+  async log(logId: string, logCallback: LogCallback) {+    return new Promise((resolve, reject) => {+      listen(`${this.route}/log/${logId}`, undefined, {+        'log': (e: MessageEvent) => {+          const logObj = JSON.parse(e.data);+          if (logCallback) {+            logCallback(logObj.level, logObj.data);+          }+        },+        'error': (e: MessageEvent) => {+            reject(e.data);+        },+        'close': () => {
        'close': resolve
FeelyChau

comment created time in 5 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

+import { listen } from './request';+import { LogCallback } from './interface';++export class BaseApi {+  route: string;++  constructor(uri: string) {+    this.route = uri;+  }++  /**+   * listen log+   * @param logId log id+   * @param logCallback log callback+   */+  async log(logId: string, logCallback: LogCallback) {+    return new Promise((resolve, reject) => {+      listen(`${this.route}/log/${logId}`, undefined, {+        'log': (e: MessageEvent) => {+          const logObj = JSON.parse(e.data);+          if (logCallback) {+            logCallback(logObj.level, logObj.data);+          }+        },+        'error': (e: MessageEvent) => {+            reject(e.data);+        },+        'close': () => {+          resolve();+        }+      });+    });+  }+}

EOF

FeelyChau

comment created time in 5 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

   "name": "@pipcook/sdk",   "version": "1.0.4",   "description": "JavaScript SDK for accessing Pipcook Daemon",-  "main": "dist/index.js",+  "main": "src/index.ts",

Why do we have this change?

FeelyChau

comment created time in 5 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

 export class PipelineService {     }   } -  stopJob(id: string): boolean {+  stopJob(id: string): void {     const runnable = runnableMap[id];     if (runnable) {       runnable.destroy();       delete runnableMap[id];-      return true;     } else {-      return false;+      this.ctx.throw('no runnable found');

Throw with an error, please

FeelyChau

comment created time in 5 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

 export interface PluginResp { export interface PluginInstallingResp extends PluginResp {   logId: string; }++/**+ * response of pipline+ */+export interface PipelineResp {+  id: string;+  name: string;+  dataCollect: string;+  dataCollectParams: string;+  dataAccess: string;+  dataAccessParams: string;+  dataProcess: string;+  dataProcessParams: string;+  modelDefine: string;+  modelDefineParams: string;+  modelLoad: string;+  modelLoadParams: string;+  modelTrain: string;+  modelTrainParams: string;+  modelEvaluate: string;+  modelEvaluateParams: string;+}++/**+ * response of pipeline install+ */+export interface PipelineInstallingResp extends PipelineResp {+  logId: string;+}

EOF

FeelyChau

comment created time in 5 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

 export class JobController {   @get('/:id')   public async get() {     const { id } = this.ctx.params;-    try {-      const job = await this.pipelineService.getJobById(id);-      if (!job) {-        throw new Error('job not found');-      }-      successRes(this.ctx, { data: job });-    } catch (err) {-      failRes(this.ctx, { message: err.message });+    const job = await this.pipelineService.getJobById(id);+    if (!job) {+      throw new Error('job not found');
      throw new TypeError('job not found');

Use TypeError as the default error.

FeelyChau

comment created time in 5 days

push eventalibaba/pipcook

yorkie

commit sha e4c3bb9db806e4c119c701ee995ecf2dcd2ba861

fix nits

view details

push time in 5 days

issue commenttc39/proposal-slice-notation

Consider basing the semantics on a method call

since Array/String.prototype.slice don't and probably never will have this step argument

@caub Why the step argument would never be added to *.slice functions? IMO it might be able to be back-compatible :)

littledan

comment created time in 5 days

startedengine262/engine262

started time in 5 days

pull request commentalibaba/pipcook

costa: add python runnable for machine-learning developers

@rickycao-qy I have supported access to the object attributes in dot notation(#389), I will switch to dot when it gets merged like:

def main(input):
  input.fn1()
yorkie

comment created time in 5 days

PR opened alibaba/pipcook

Reviewers
boa: generate object instead of `dict` to Python function boa

It creates a new class and instantiates it with given props when passing to Python. For example:

pythonfunc({ foobar: 10 });

At Python, we have to write in this way:

def func(opt):
  opt.get('foobar') # 10, because opt is a dict

Within this Pull Request, the developer is able:

def func(opt):
  opt.foobar

Otherwise, I also removed an incorrect case:

const o = { foobar: 10 };
len(o); // not working
len(dict(boa.kwargs(o))); // working

Even though it's more complexed than before, but it's more correct way to be used.

+33 -5

0 comment

5 changed files

pr created time in 5 days

create barnchalibaba/pipcook

branch : fix/boa-function-call

created branch time in 5 days

pull request commentalibaba/pipcook

fix: installation guide url

@EliYao Thanks for the contribution. Could you rebase from master branch and re-run the CI pls. Otherwise LGTM.

Just start a new build :)

EliYao

comment created time in 5 days

issue commentalibaba/pipcook

Daemon: the output format for api

We are not to bind the service and client, the developer is able to deploy what they prefer to choose, and using the corresponding SDK. And maintaining multiple versions in single branch is hard, too.

FeelyChau

comment created time in 5 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

-import { Context, controller, inject, provide, get, post } from 'midway';-import { successRes } from '../../utils/response';+import { controller, inject, provide, get, post, del, put } from 'midway';+import * as createHttpError from 'http-errors';+import { BaseController } from './base'; import { PluginManager } from '../../service/plugin'; import ServerSentEmitter from '../../utils/emitter';+import { PluginInstallingResp } from '../../interface'; import Debug from 'debug'; const debug = Debug('daemon.app.plugin');  @provide() @controller('/plugin')-export class PluginController {-  @inject()-  ctx: Context;+export class PluginController extends BaseController {    @inject('pluginManager')   pluginManager: PluginManager; -  @get('/install')+  // TODO(feely): check if the plugin has been installed+  /**+   * create plugin and install it+   */+  @post()   public async install() {+    const { name, pyIndex } = this.ctx.request.body;+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, false);+    this.success(response);+  }++  /**+   * reinstall plugin+   */+  @put()+  public async reinstall() {     const { name, pyIndex } = this.ctx.query;-    const sse = new ServerSentEmitter(this.ctx);-    try {-      debug(`checking info: ${name}.`);-      const pkg = await this.pluginManager.fetch(name);-      sse.emit('info', pkg);+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, true);+    this.success(response);+  } -      debug(`installing ${name}.`);-      const plugin = await this.pluginManager.findOrCreateByPkg(pkg);-      try {-        await this.pluginManager.install(pkg, pyIndex);-        sse.emit('installed', pkg);-      } catch (err) {-        await this.pluginManager.removeById(plugin.id);-        throw err;+  /**+   * delete plugin by name+   */+  @del('/:id')+  public async remove() {+    if (typeof this.ctx.params.id === 'string' && this.ctx.params.id) {+      const plugin = await this.pluginManager.findById(this.ctx.params.id);+      if (plugin) {+        await this.pluginManager.uninstall(plugin);+        this.success();+      } else {+        throw createHttpError(400, `no plugin found by id ${this.ctx.params.id}`);       }-    } catch (err) {-      sse.emit('error', err?.message);-    } finally {-      sse.finish();+    } else {+      throw createHttpError(400, 'no id value found');     }   }--  @get('/uninstall')-  public async uninstall() {-    await this.pluginManager.uninstall(this.ctx.query.name);-    successRes(this.ctx, {});+  /**+   * delete all plugins+   */+  @del()+  public async removeAll() {+    const plugins = await this.pluginManager.list();+    for (const plugin of plugins) {+      await this.pluginManager.uninstall(plugin);

npm uninstall allows multiple packages, it means npm will ensure the data consistency.

FeelyChau

comment created time in 5 days

issue commentalibaba/pipcook

Daemon: the output format for api

I don't think the API versioning should be included at Pipcook, it depends on users IMO.

FeelyChau

comment created time in 5 days

pull request commentalibaba/pipcook

costa: add python runnable for machine-learning developers

@rickycao-qy I have updated the tests for using JavaScript functions in Python env at packages/costa/test/plugins/python-simple/init.py, the only problem is that the Python objects from JavaScript are a dictionary so that:

def main(input):
  input.get('fn1')() # working
  input.fn1() # not working

I will open another PR to change to case 2.

yorkie

comment created time in 6 days

push eventalibaba/pipcook

yorkie

commit sha a43eacb6273b6ee3c748d49af3aa17c23a4de4f1

fix nits

view details

push time in 6 days

issue closedalibaba/pipcook

cli: incorrect cli output when daemon is crashed

The CLI outputs the following when the daemon is crashed:

daemon is not started, run "pipcook daemon restart"

The correct behavior should be:

daemon is crashed with an error:
[stacktrace of daemon process]

closed time in 6 days

yorkie

issue closedalibaba/pipcook

boa: from Y import X could not be imported by destructing Y

The Python code:

from PIL import Image, ImageDraw

And:

const { Image, ImageDraw } = boa.import('PIL'); // not working
const Image = boa.import('PIL.Image'); // working
const ImageDraw = boa.import('PIL.ImageDraw'); // working

That's great if the destructing is working for sub modules.

closed time in 6 days

yorkie

issue commentalibaba/pipcook

cli: incorrect cli output when daemon is crashed

This is fixed by #345.

yorkie

comment created time in 6 days

issue commentalibaba/pipcook

boa: from Y import X could not be imported by destructing Y

Closing because this could not be completed logically.

yorkie

comment created time in 6 days

push eventalibaba/pipcook

yorkie

commit sha 4e3c5f6001646a1540ea1ef13cd3915fd7d04456

deploy: 470de29d58f5b7a5484c74252665fa2f48496ac4

view details

push time in 6 days

push eventalibaba/pipcook

feely

commit sha 470de29d58f5b7a5484c74252665fa2f48496ac4

daemon: reference plugin name from core (#385)

view details

push time in 6 days

delete branch alibaba/pipcook

delete branch : fix/plugin-name

delete time in 6 days

PR merged alibaba/pipcook

daemon: reference plugin name from core pipcook daemon
+2 -9

0 comment

1 changed file

FeelyChau

pr closed time in 6 days

pull request commentalibaba/pipcook

costa: add python runnable for machine-learning developers

@rickycao-qy Which case do we need to pass a function/class to Python?

yorkie

comment created time in 6 days

push eventalibaba/pipcook

yorkie

commit sha 421af02a2e5cb5caf82c423ecb96b26c2c7d8357

fix nits

view details

push time in 6 days

push eventalibaba/pipcook

yorkie

commit sha ec6865f8da42ac83bb4e5dd5a04fdf1fa101d330

add docs

view details

push time in 6 days

push eventalibaba/pipcook

yorkie

commit sha a86d0a25a46e1c72e2f00f7fc7a7eed70b90ace4

fix nits

view details

push time in 6 days

push eventalibaba/pipcook

yorkie

commit sha 4fbed9aa9a2835941f927d5c1c5e362d404205cd

fix nits

view details

push time in 6 days

push eventalibaba/pipcook

yorkie

commit sha e48b539f8bd7fe3005d8013ae5c9ac23fdf4a73b

deploy: ba65cd64a7a263f046806748b061d3bdffbde178

view details

push time in 6 days

pull request commentalibaba/pipcook

costa: add python runnable for machine-learning developers

Tests and documentation are still working in progress :)

yorkie

comment created time in 6 days

PR opened alibaba/pipcook

costa: add python runnable for machine-learning developers costa

In order to better attract machine learning developers to contribute models to Pipcook, this PR provides developers with a native Python development environment. For users who do not use Node.js and JavaScript, they can use pure Python to write plugins.

+48 -17

0 comment

10 changed files

pr created time in 6 days

push eventalibaba/pipcook

Ruikun Cao

commit sha ba65cd64a7a263f046806748b061d3bdffbde178

Publish v1.0.5 (#387) - @pipcook/app@1.0.5 - @pipcook/boa@1.0.5 - @pipcook/pipcook-cli@1.0.5 - @pipcook/pipcook-core@1.0.5 - @pipcook/costa@1.0.5 - @pipcook/daemon@1.0.5 - @pipcook/pipboard@1.0.5 - @pipcook/plugins-coco-data-access@1.0.5 - @pipcook/plugins-csv-data-access@1.0.5 - @pipcook/plugins-pascalvoc-data-access@1.0.5 - @pipcook/plugins-textline-data-access@1.0.5 - @pipcook/plugins-chinese-poem-data-collect@1.0.5 - @pipcook/plugins-csv-data-collect@1.0.5 - @pipcook/plugins-fasttext-data-collect@1.0.5 - @pipcook/plugins-image-classification-data-collect@1.0.5 - @pipcook/plugins-mnist-data-collect@1.0.5 - @pipcook/plugins-object-detection-coco-data-collect@1.0.5 - @pipcook/plugins-object-detection-pascalvoc-data-collect@1.0.5 - @pipcook/plugins-image-data-process@1.0.5 - @pipcook/plugins-bayesian-model-define@1.0.5 - @pipcook/plugins-detectron-fasterrcnn-model-define@1.0.5 - @pipcook/plugins-detectron-maskrcnn-model-define@1.0.5 - @pipcook/plugins-fasttext-model-define@1.0.5 - @pipcook/plugins-tensorflow-cycle-gan-model-define@1.0.5 - @pipcook/plugins-tensorflow-mobilenet-model-define@1.0.5 - @pipcook/plugins-tensorflow-resnet-model-define@1.0.5 - @pipcook/plugins-tfjs-mobilenet-model-define@1.0.5 - @pipcook/plugins-tfjs-simplecnn-model-define@1.0.5 - @pipcook/plugins-tfjs-text-lstm-model-define@1.0.5 - @pipcook/plugins-bayesian-model-evaluate@1.0.5 - @pipcook/plugins-detectron-model-evaluate@1.0.5 - @pipcook/plugins-fasttext-model-evaluate@1.0.5 - @pipcook/plugins-image-classification-tensorflow-model-evaluate@1.0.5 - @pipcook/plugins-image-classification-tfjs-model-evaluate@1.0.5 - @pipcook/plugins-image-generation-tensorflow-model-evaluate@1.0.5 - @pipcook/plugins-text-creation-tfjs-model-evaluate@1.0.5 - @pipcook/plugins-tensorflow-model-load@1.0.5 - @pipcook/plugins-bayesian-model-train@1.0.5 - @pipcook/plugins-detectron-model-train@1.0.5 - @pipcook/plugins-fasttext-model-train@1.0.5 - @pipcook/plugins-image-classification-tensorflow-model-train@1.0.5 - @pipcook/plugins-image-classification-tfjs-model-train@1.0.5 - @pipcook/plugins-image-generation-tensorflow-model-train@1.0.5 - @pipcook/plugins-text-creation-tfjs-model-train@1.0.5 - @pipcook/sdk@1.0.5

view details

push time in 6 days

delete branch alibaba/pipcook

delete branch : daily/publish_1_0_5

delete time in 6 days

PR merged alibaba/pipcook

Bump version to 1.0.5
  • @pipcook/app@1.0.5
  • @pipcook/boa@1.0.5
  • @pipcook/pipcook-cli@1.0.5
  • @pipcook/pipcook-core@1.0.5
  • @pipcook/costa@1.0.5
  • @pipcook/daemon@1.0.5
  • @pipcook/pipboard@1.0.5
  • @pipcook/plugins-coco-data-access@1.0.5
  • @pipcook/plugins-csv-data-access@1.0.5
  • @pipcook/plugins-pascalvoc-data-access@1.0.5
  • @pipcook/plugins-textline-data-access@1.0.5
  • @pipcook/plugins-chinese-poem-data-collect@1.0.5
  • @pipcook/plugins-csv-data-collect@1.0.5
  • @pipcook/plugins-fasttext-data-collect@1.0.5
  • @pipcook/plugins-image-classification-data-collect@1.0.5
  • @pipcook/plugins-mnist-data-collect@1.0.5
  • @pipcook/plugins-object-detection-coco-data-collect@1.0.5
  • @pipcook/plugins-object-detection-pascalvoc-data-collect@1.0.5
  • @pipcook/plugins-image-data-process@1.0.5
  • @pipcook/plugins-bayesian-model-define@1.0.5
  • @pipcook/plugins-detectron-fasterrcnn-model-define@1.0.5
  • @pipcook/plugins-detectron-maskrcnn-model-define@1.0.5
  • @pipcook/plugins-fasttext-model-define@1.0.5
  • @pipcook/plugins-tensorflow-cycle-gan-model-define@1.0.5
  • @pipcook/plugins-tensorflow-mobilenet-model-define@1.0.5
  • @pipcook/plugins-tensorflow-resnet-model-define@1.0.5
  • @pipcook/plugins-tfjs-mobilenet-model-define@1.0.5
  • @pipcook/plugins-tfjs-simplecnn-model-define@1.0.5
  • @pipcook/plugins-tfjs-text-lstm-model-define@1.0.5
  • @pipcook/plugins-bayesian-model-evaluate@1.0.5
  • @pipcook/plugins-detectron-model-evaluate@1.0.5
  • @pipcook/plugins-fasttext-model-evaluate@1.0.5
  • @pipcook/plugins-image-classification-tensorflow-model-evaluate@1.0.5
  • @pipcook/plugins-image-classification-tfjs-model-evaluate@1.0.5
  • @pipcook/plugins-image-generation-tensorflow-model-evaluate@1.0.5
  • @pipcook/plugins-text-creation-tfjs-model-evaluate@1.0.5
  • @pipcook/plugins-tensorflow-model-load@1.0.5
  • @pipcook/plugins-bayesian-model-train@1.0.5
  • @pipcook/plugins-detectron-model-train@1.0.5
  • @pipcook/plugins-fasttext-model-train@1.0.5
  • @pipcook/plugins-image-classification-tensorflow-model-train@1.0.5
  • @pipcook/plugins-image-classification-tfjs-model-train@1.0.5
  • @pipcook/plugins-image-generation-tensorflow-model-train@1.0.5
  • @pipcook/plugins-text-creation-tfjs-model-train@1.0.5
  • @pipcook/sdk@1.0.5
+150 -150

0 comment

90 changed files

rickycao-qy

pr closed time in 6 days

push eventalibaba/pipcook

yorkie

commit sha f62c856cc6293dc125b308aba146d3b4380fdee4

fix nits

view details

push time in 6 days

create barnchalibaba/pipcook

branch : costa/add-py-runtime

created branch time in 6 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

 export class PluginManager {       throw err;     }   }+  async startInstall(pkg: PluginPackage, pyIndex?: string, force?: boolean): Promise<PluginInstallingResp> {

Why not just install()?

FeelyChau

comment created time in 6 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

-import { Context, controller, inject, provide, get, post } from 'midway';-import { successRes } from '../../utils/response';+import { controller, inject, provide, get, post, del, put } from 'midway';+import * as createHttpError from 'http-errors';+import { BaseController } from './base'; import { PluginManager } from '../../service/plugin'; import ServerSentEmitter from '../../utils/emitter';+import { PluginInstallingResp } from '../../interface'; import Debug from 'debug'; const debug = Debug('daemon.app.plugin');  @provide() @controller('/plugin')-export class PluginController {-  @inject()-  ctx: Context;+export class PluginController extends BaseController {    @inject('pluginManager')   pluginManager: PluginManager; -  @get('/install')+  // TODO(feely): check if the plugin has been installed+  /**+   * create plugin and install it+   */+  @post()   public async install() {+    const { name, pyIndex } = this.ctx.request.body;+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, false);+    this.success(response);+  }++  /**+   * reinstall plugin+   */+  @put()+  public async reinstall() {     const { name, pyIndex } = this.ctx.query;-    const sse = new ServerSentEmitter(this.ctx);-    try {-      debug(`checking info: ${name}.`);-      const pkg = await this.pluginManager.fetch(name);-      sse.emit('info', pkg);+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, true);+    this.success(response);+  } -      debug(`installing ${name}.`);-      const plugin = await this.pluginManager.findOrCreateByPkg(pkg);-      try {-        await this.pluginManager.install(pkg, pyIndex);-        sse.emit('installed', pkg);-      } catch (err) {-        await this.pluginManager.removeById(plugin.id);-        throw err;+  /**+   * delete plugin by name+   */+  @del('/:id')+  public async remove() {+    if (typeof this.ctx.params.id === 'string' && this.ctx.params.id) {+      const plugin = await this.pluginManager.findById(this.ctx.params.id);+      if (plugin) {+        await this.pluginManager.uninstall(plugin);+        this.success();+      } else {+        throw createHttpError(400, `no plugin found by id ${this.ctx.params.id}`);       }-    } catch (err) {-      sse.emit('error', err?.message);-    } finally {-      sse.finish();+    } else {+      throw createHttpError(400, 'no id value found');     }   }--  @get('/uninstall')-  public async uninstall() {-    await this.pluginManager.uninstall(this.ctx.query.name);-    successRes(this.ctx, {});+  /**+   * delete all plugins+   */+  @del()+  public async removeAll() {+    const plugins = await this.pluginManager.list();+    for (const plugin of plugins) {+      await this.pluginManager.uninstall(plugin);+    }+    this.success();+  }+  /**+   * find a plugin by id+   */+  @get('/:id')+  public async get() {+    const plugin = await this.pluginManager.findById(this.ctx.params.id);+    if (plugin) {+      this.success(plugin);+    } else {+      throw createHttpError(404, 'no plugin found');+    }   } -  @get('/list')+  /**+   * list plugins+   */+  @get()   public async list() {     const plugins = await this.pluginManager.list({       datatype: this.ctx.query.datatype,       category: this.ctx.query.category     });-    successRes(this.ctx, { data: plugins });+    this.success(plugins);   }-  @get('/:id')-  public async info() {-    const plugin = await this.pluginManager.findById(this.ctx.params.id);-    successRes(this.ctx, { data: plugin });-  }-  @post('/upload')-  public async upload() {++  /**+   * create a plugin by tarball stream+   */+  @post('/tarball')+  public async uploadPackage() {     const fs = await this.ctx.getFileStream();
    const fstream = await this.ctx.getFileStream();
FeelyChau

comment created time in 6 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

-import { Context, controller, inject, provide, get, post } from 'midway';-import { successRes } from '../../utils/response';+import { controller, inject, provide, get, post, del, put } from 'midway';+import * as createHttpError from 'http-errors';+import { BaseController } from './base'; import { PluginManager } from '../../service/plugin'; import ServerSentEmitter from '../../utils/emitter';+import { PluginInstallingResp } from '../../interface'; import Debug from 'debug'; const debug = Debug('daemon.app.plugin');  @provide() @controller('/plugin')-export class PluginController {-  @inject()-  ctx: Context;+export class PluginController extends BaseController {    @inject('pluginManager')   pluginManager: PluginManager; -  @get('/install')+  // TODO(feely): check if the plugin has been installed+  /**+   * create plugin and install it+   */+  @post()   public async install() {+    const { name, pyIndex } = this.ctx.request.body;+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, false);+    this.success(response);+  }++  /**+   * reinstall plugin+   */+  @put()+  public async reinstall() {     const { name, pyIndex } = this.ctx.query;-    const sse = new ServerSentEmitter(this.ctx);-    try {-      debug(`checking info: ${name}.`);-      const pkg = await this.pluginManager.fetch(name);-      sse.emit('info', pkg);+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, true);+    this.success(response);+  } -      debug(`installing ${name}.`);-      const plugin = await this.pluginManager.findOrCreateByPkg(pkg);-      try {-        await this.pluginManager.install(pkg, pyIndex);-        sse.emit('installed', pkg);-      } catch (err) {-        await this.pluginManager.removeById(plugin.id);-        throw err;+  /**+   * delete plugin by name+   */+  @del('/:id')+  public async remove() {+    if (typeof this.ctx.params.id === 'string' && this.ctx.params.id) {+      const plugin = await this.pluginManager.findById(this.ctx.params.id);+      if (plugin) {+        await this.pluginManager.uninstall(plugin);+        this.success();+      } else {+        throw createHttpError(400, `no plugin found by id ${this.ctx.params.id}`);       }-    } catch (err) {-      sse.emit('error', err?.message);-    } finally {-      sse.finish();+    } else {+      throw createHttpError(400, 'no id value found');     }   }--  @get('/uninstall')-  public async uninstall() {-    await this.pluginManager.uninstall(this.ctx.query.name);-    successRes(this.ctx, {});+  /**+   * delete all plugins+   */+  @del()+  public async removeAll() {+    const plugins = await this.pluginManager.list();+    for (const plugin of plugins) {+      await this.pluginManager.uninstall(plugin);

Shall we support uninstalling multiple plugins in one function/fork?

FeelyChau

comment created time in 6 days

Pull request review commentalibaba/pipcook

WIP: daemon: refactor plugin api

-import { Context, controller, inject, provide, get, post } from 'midway';-import { successRes } from '../../utils/response';+import { controller, inject, provide, get, post, del, put } from 'midway';+import * as createHttpError from 'http-errors';+import { BaseController } from './base'; import { PluginManager } from '../../service/plugin'; import ServerSentEmitter from '../../utils/emitter';+import { PluginInstallingResp } from '../../interface'; import Debug from 'debug'; const debug = Debug('daemon.app.plugin');  @provide() @controller('/plugin')-export class PluginController {-  @inject()-  ctx: Context;+export class PluginController extends BaseController {    @inject('pluginManager')   pluginManager: PluginManager; -  @get('/install')+  // TODO(feely): check if the plugin has been installed+  /**+   * create plugin and install it+   */+  @post()   public async install() {+    const { name, pyIndex } = this.ctx.request.body;+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, false);+    this.success(response);+  }++  /**+   * reinstall plugin+   */+  @put()+  public async reinstall() {     const { name, pyIndex } = this.ctx.query;-    const sse = new ServerSentEmitter(this.ctx);-    try {-      debug(`checking info: ${name}.`);-      const pkg = await this.pluginManager.fetch(name);-      sse.emit('info', pkg);+    let response: PluginInstallingResp;+    debug(`checking info: ${name}.`);+    response = await this.pluginManager.installByName(name, pyIndex, true);
    const response = await this.pluginManager.installByName(name, pyIndex, true);
FeelyChau

comment created time in 6 days

more