profile
viewpoint
Maxime Daniel maxux ThreeFold Tech Gentoo, Liège, Belgium https://www.maxux.net

maxux/dashboard-home 14

Home dashboard central server

Jumpscale/prefab9 1

was called cuisine before

maxux/boot 1

asm boot experiments

Jumpscale/jscockpit 0

private neutral management environment for g8 based clouds

push eventthreefoldtech/0-db

Maxime Daniel

commit sha 217b86e99c7c941a2131fafdb378f3215f6cfafb

Force index and data to flush before jump

view details

push time in 3 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha 43b03f65a20a3bff783ae131213653899863e2d9

Update debug level of data write

view details

push time in 3 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha 5fbbf18644e4600366ab2f8fa804d20c943a331a

Add time log to perror

view details

push time in 3 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha 2b229e472a36fc58670b24b0a01c9ccb43495761

Fix hex debug output with time log

view details

push time in 3 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha 73f7447b8e09a7cef6fc823d1b536185cc783d9e

Reduce float precision for timelog

view details

push time in 3 days

PR opened threefoldtech/0-db

Prefix all logs with boot time

Improve debugging output with boot time prefixed:

[      0.0000100][*] 0-db engine, v1.1.0 (commit fe2a3bf3-dirty)
[      0.0001990][*] 0-db server, v1.0.0 (commit fe2a3bf3-dirty)
[      0.0002130][+] system: running mode: mixed mode
[      0.0002180][+] system: maximum namespace size: 16384.00 GB
[      0.0002250][+] system: instance id: 854640987
[      0.0002310][+] system: setting up environments
[      0.0002580][+] namespaces: initializing
[      0.0002670][+] namespaces: pre-allocating index (16777216 lazy branches)
[      0.0002860][+] namespaces: loading 'default'
[      0.0002980][+] namespaces: checking index [/mnt/storage/tmp/zdb-master-index/default]
[      0.0003060][+] namespaces: checking data [/mnt/storage/tmp/zdb-master-data/default]
[      0.0003370][+] namespace: loaded: default
[      0.0003430][+] -> maxsize: 0 (0.00 MB)
[      0.0003480][+] -> password protection: no
[      0.0003520][+] -> public access: yes
[      0.0003560][+] -> worm mode: no
[      0.0003640][+] index: initializing
[      0.0003840][+] index: loading file: /mnt/storage/tmp/zdb-master-index/default/zdb-index-00000
[      0.0003950][+] index: running mode: default key-value
[      0.0004020][+] index: writing 27 bytes on fd 3
[      0.0004450][+] index: created at: 2020-07-11 02:33:05
[      0.0005100][+] index: last open: 2020-08-06 12:13:24
[      0.0005220][+] index: populating: /mnt/storage/tmp/zdb-master-index/default/zdb-index-00000
[      0.0005270][+] index: loading in memory file: 0.00 MB
[      0.0005340][+] index: get: lookup key: 0x68656c6c6f[      0.0005370]
[      0.0005480][-] index: get: key not found
[      0.0009100][+] index: last offset: 27
[      0.0009210][+] index: healthy
[      0.0009310][+] index: active file: /mnt/storage/tmp/zdb-master-index/default/zdb-index-00000
[      0.0009350][+] index: verifyfing populated keys
[      0.0907490][+] index: uses: 1 branches
[      0.0907590][+] index: memory overhead: 131072.02 KB (134217752 bytes)
[      0.0907620][+] index: load: 1 entries
[      0.0907640][+] index: datasize: 0.00 MB (5 bytes)
[      0.0907650][+] index: raw usage: 0.1 KB (61 bytes)
[      0.0907930][+] data: reading file, finding last entry
[      0.0907960][+] data: entries read: 1, last offset: 26
[      0.0907990][+] data: active file: /mnt/storage/tmp/zdb-master-data/default/zdb-data-00000
[      0.0908130][+] namespaces: extra found: maxux
[      0.0908140][+] namespaces: loading 'maxux'
[      0.0908170][+] namespaces: checking index [/mnt/storage/tmp/zdb-master-index/maxux]
[      0.0908200][+] namespaces: checking data [/mnt/storage/tmp/zdb-master-data/maxux]
[      0.0908240][+] namespace: loaded: maxux
[      0.0908260][+] -> maxsize: 0 (0.00 MB)
[      0.0908270][+] -> password protection: no
[      0.0908290][+] -> public access: yes
[      0.0908300][+] -> worm mode: no
+100 -61

0 comment

19 changed files

pr created time in 3 days

create barnchthreefoldtech/0-db

branch : development-timelog

created branch time in 3 days

push eventthreefoldtech/0-db

Maxime Daniel

commit sha fe2a3bf3bef752f8f69760b185f500108127a6c0

Update tests, block mode is not allowed for now

view details

push time in 3 days

PR opened threefoldtech/zos

Reviewers
Stick to 0-fs v2.0.2
+6 -11

0 comment

1 changed file

pr created time in 3 days

create barnchthreefoldtech/zos

branch : zufs-stick-version

created branch time in 3 days

delete branch threefoldtech/0-flist

delete branch : development-v2-ids

delete time in 3 days

PR merged threefoldtech/0-flist

Support uid/gid and stay retro-compatible

Improve flist format, including UID and GID on ACL.

When creating an flist, the uid/gid of directories and files are copied from host, directly, in addition to username/groupname. This ID can be used to foward real userid/groupid from host system and not a resolved name if not desired.

This include also the capnp model used, copied from 0-fs.

+209 -39

0 comment

15 changed files

maxux

pr closed time in 3 days

push eventthreefoldtech/0-flist

Maxime Daniel

commit sha 1240563b1d4d4e71f1cda9d254ba2f87e35f2e8d

Update capnp generated files

view details

Maxime Daniel

commit sha 60a4f5a930e70f846f7e74288cff0be3da0c60f2

Disable db requirement for prefetch

view details

Maxime Daniel

commit sha c370e838b6d58aa4902b8e4693ad746144349ac8

Update capnp model and code to support uid/gid

view details

Maxime Daniel

commit sha b06706c1ae027c1e9145093f4b5408da0bd64805

Implement uid/gid support (related to #28)

view details

Maxime Daniel

commit sha d48b53a3c1ebdec978fc8c7e276360483bbcee88

Fix root Makefile

view details

Maxime Daniel

commit sha 67b9c2fe3fefc17b98626017443096cf036bc026

Merge pull request #30 from threefoldtech/development-v2-ids Support uid/gid and stay retro-compatible

view details

push time in 3 days

push eventthreefoldtech/0-flist

Maxime Daniel

commit sha d48b53a3c1ebdec978fc8c7e276360483bbcee88

Fix root Makefile

view details

push time in 5 days

PR opened threefoldtech/0-flist

Support uid/gid and stay retro-compatible

Improve flist format, including UID and GID on ACL.

When creating an flist, the uid/gid of directories and files are copied from host, directly, in addition to username/groupname. This ID can be used to foward real userid/groupid from host system and not a resolved name if not desired.

This include also the capnp model used, copied from 0-fs.

+208 -38

0 comment

14 changed files

pr created time in 10 days

PR merged threefoldtech/0-flist

Support get command

Add get command to download a remote file

+81 -4

0 comment

3 changed files

maxux

pr closed time in 10 days

delete branch threefoldtech/0-flist

delete branch : development-v2-prefetch

delete time in 10 days

push eventthreefoldtech/0-flist

Maxime Daniel

commit sha 98e78a900c0054a3048040f6514edc3a276997e9

Fix segfault on bad dup pointers

view details

Maxime Daniel

commit sha 7555a5df60ebc87929712100a95db76f7fc768aa

Add get command

view details

Maxime Daniel

commit sha eef383e3b0b8ddd20e21fb8c7e9458b1f07fb8d6

Merge pull request #29 from threefoldtech/development-v2-prefetch Support get command

view details

push time in 10 days

PR opened threefoldtech/0-flist

Support get command

Add get command to download a remote file

+81 -4

0 comment

3 changed files

pr created time in 10 days

create barnchthreefoldtech/0-flist

branch : development-v2-ids

created branch time in 10 days

issue commentthreefoldtech/0-flist

Extend flist to support file uid/gid in addition with username/groupname

Yep, new information have to be added when creating flist. But 0-fs and zflist will be retrocompatible :)

maxux

comment created time in 10 days

issue openedthreefoldtech/0-flist

Extend flist to support file uid/gid in addition with username/groupname

Current version of flist stores only username and groupname for files and directories permissions. This is based on a legacy but good intention to make a portable definition of group and user.

But on runtime, this lead to several issues, eg:

  • When doing an image of a stored filesystem, we can't use host mapping
  • When reading flist for container like 0-fs does, process runs on host, does the mapping with host and is not inline with expected container.

Storing uid/gid from source and forwarding to the end is the only portable solution to not depend on something on runtime.

Current flist format should be extended to support gid and uid inside ACL.

This is linked with https://github.com/threefoldtech/0-fs/issues/31

created time in 11 days

issue commentthreefoldtech/0-fs

user ids mapping issue on zos container

IMO, for sure, we have to keep UID and GID inside flist. I think it's better to keep mapped user/group for extended purpose and add UID and GID field on the ACL.

If the UID/GID is available, 0-fs directly forward this information and doesn't query anything on the host, this will fix lot of issues.

muhamadazmy

comment created time in 11 days

delete branch threefoldtech/0-initramfs

delete branch : development-zos-v2-zerotier

delete time in 11 days

pull request commentthreefoldtech/zos

flistd cleanup

The workflow to check if mount is ready and everything is okay seems quite hackish.

Why not implement in 0-fs to fork when everything is ready. So you can just start the process, wait for it to ends. When it ended, you check if it went all good or if an error occurred.

If 0-fs cleanup itself when closing it's stuff, why cleanup should happen in the back ? Except if 0-fs segfault, and even, we can umount gracefully on segfault. No ?

muhamadazmy

comment created time in 11 days

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha 0dea46ae7c7b1d03ed5bf3b2296a166b522bd089

Disable corex and dependencies build (shipped via flist)

view details

Maxime Daniel

commit sha 018a58f03fc30fed13ec9a13ef7cb1c2ca83b9ec

Do not save corex binary when optimizing

view details

Maxime Daniel

commit sha 8c8e3e7930627919e6f8113354868b6d72f97ca6

Merge pull request #47 from threefoldtech/development-zos-v2-no-corex Disable corex build

view details

Maxime Daniel

commit sha 6fe7cf8ad5ff173167b0ce8b251948b6878e330b

Merge branch 'development-zos-v2' into development-zos-v2-go-mod

view details

push time in 12 days

delete branch threefoldtech/0-initramfs

delete branch : development-zos-v2-no-corex

delete time in 12 days

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha 0dea46ae7c7b1d03ed5bf3b2296a166b522bd089

Disable corex and dependencies build (shipped via flist)

view details

Maxime Daniel

commit sha 018a58f03fc30fed13ec9a13ef7cb1c2ca83b9ec

Do not save corex binary when optimizing

view details

Maxime Daniel

commit sha 8c8e3e7930627919e6f8113354868b6d72f97ca6

Merge pull request #47 from threefoldtech/development-zos-v2-no-corex Disable corex build

view details

push time in 12 days

PR merged threefoldtech/0-initramfs

Disable corex build

Process corex is shipped by flist on runtime now

+8 -8

0 comment

1 changed file

maxux

pr closed time in 12 days

PR opened threefoldtech/0-initramfs

Disable corex build

Process corex is shipped by flist on runtime now

+8 -8

0 comment

1 changed file

pr created time in 12 days

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha 018a58f03fc30fed13ec9a13ef7cb1c2ca83b9ec

Do not save corex binary when optimizing

view details

push time in 13 days

create barnchthreefoldtech/0-initramfs

branch : development-zos-v2-no-corex

created branch time in 13 days

push eventthreefoldtech/0-initramfs

Maxime Daniel

commit sha 757b95df4d9c49fdcff6aa1cc65ac467b4d4ac19

Trigger new build

view details

push time in 24 days

issue commentthreefoldtech/0-db

Per namespace index mode

Implementation started by #77

maxux

comment created time in a month

PR opened threefoldtech/0-db

Enable single-instance index multi-mode

This pull request enable support of multiple mode running in a single zdb instance. A single instance of zdb can host multiple namespace each running in their own mode (user or seq).

Here are some commit messages explaining changes:

ad41af9

This commit implement changes needed to support runtime mixed index mode, which allow a single zdb instance to runs in multiple mode depending of the namespace (some namespace in userkey mode and some in sequential mode).

This commit only affect how internally you can change how mode is set and how current (not global) mode is handled to fetch and save data.

From this commit, if you don't provide any --mode argument to zdb on runtime, mixed mode will be enabled by default. Specifying --mode argument will set a specific mode internally.

26cf41f

Now, zdbd allows remote user to change namespace mode on runtime, after namespace creation. Using 'NSSET <namespace> mode <mode>' you can switch namespace mode to 'user' or 'seq' and run multiple mode on the same zdb instance.

Note: you can only change mode if the namespace is new, completly fresh without anything inside (no deleted keys or so).

If you run zdbd with a specific --mode flag, your zdb will be restricted to specified mode and you won't be able to change mode using NSSET command. You won't be able to load mixed index neither.

+172 -22

0 comment

12 changed files

pr created time in a month

create barnchthreefoldtech/0-db

branch : development-multi-mode

created branch time in a month

create barnchthreefoldfoundation/info_threefold

branch : maxux-techworks

created branch time in a month

issue commentthreefoldtech/home

GE: Track Usage

I updated the gist with a new script: https://gist.github.com/maxux/3df6b229902fa1401b3a505d45eec8a9#file-mksetup-py

Running this script as it will write a file (/tmp/nodes.json) which will host configuration generated. If will fetch your both reservation url and extract data from them. The script will produce a warning because your minio doesn't have the stats aggregator set.

It will also set all zdbs found on reservation.

When the script is done, you can run the new version of https://gist.github.com/maxux/3df6b229902fa1401b3a505d45eec8a9#file-minio-stats-py which will now read configuration from file /tmp/nodes.json. You will get a warning since no minio will be available. But all zdbs will be monitored correctly.

You have 125 zdbs, the script query each zdb one by one which can take some time, an improved version should query all of them in parallel but this is a proof-of-concept :)

joefoxton

comment created time in a month

issue commentthreefoldtech/home

GE: Track Usage

How do you send your reservation ? The property needs to contains an accessible redis server where to push periodic statistics. It can be in another container running redis. To make things easier for me, in a first step, this redis should be publicly accessible.

joefoxton

comment created time in a month

issue commentthreefoldtech/home

GE: Track Usage

Hello,

It's fine. I'm writing the script to build nodes descriptors based on your reservation. But you have something missing, on minio reservation, you have to provide the stats_aggregator: property to get container statistics pushed somewhere.

joefoxton

comment created time in a month

issue commentthreefoldtech/corex

segfault of corex process

Probably, maybe an upgrade of corex flist could confirm it.

On Wed, 8 Jul 2020 at 09:13, Christophe de Carvalho < notifications@github.com> wrote:

@maxux https://github.com/maxux the commit you reference is not yet deploy on mainnet and this crash we had was on mainnet so I think we can rule this out.

Do you think it could be already fixed by all the change recently made in corex ? ( new version of libwesocket etc? )

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/threefoldtech/corex/issues/22#issuecomment-655334488, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7TEECWS7OVFL5QPUK63E3R2QMDNANCNFSM4ORN7HEQ .

zaibon

comment created time in a month

issue commentthreefoldtech/corex

segfault of corex process

Could be related to https://github.com/threefoldtech/zos/commit/a6e3ab72a1c8086288cf86ecfd96e0447a23f1b8 maybe. The provided corex binary is stripped, informations attached can't help me to find out what happen, except by reproducing :/

zaibon

comment created time in a month

create barnchthreefoldtech/0-flist

branch : development-v2-prefetch

created branch time in a month

issue commentthreefoldtech/home

GE: Track Usage

Did you updated nodes specifications ? https://gist.github.com/maxux/3df6b229902fa1401b3a505d45eec8a9#file-minio-stats-py-L5-L29

You have to change host, post, channel and namespace for your minio/zdb storage. You can reach all theses information within the reservation. In addition, when doing the reservation, you have to include a statistics definition (a redis endpoint where to send statistics). Do you need more information on how to do that ?

I can simplify the script by auto-fillin data based on a reservation response, but I need to extract correct information :) Can you maybe give me your reservation link ?

joefoxton

comment created time in a month

issue closedthreefoldtech/home

corex improvements [development_corex]

goal

  • make sure we can use core-x as process manager & its stable enough

linked issues

  • [x] threefoldtech/jumpscaleX_core#300
  • [x] threefoldtech/jumpscaleX_core#299

closed time in a month

despiegk

issue closedthreefoldtech/jumpscaleX_core

Corex: issue related to stop

When I try to stop a process already running:

  • it tells me that it is stoped even if it is still running image

  • it also throws an exception

closed time in a month

RafyAmgadBenjamin

issue commentthreefoldtech/jumpscaleX_core

Corex: issue related to stop

I don't see how to reproduce, at least with latest version.

curl http://[2a02:2788:54:1096:b8c7:ccff:fe48:1eac]:7681/api/processes
{ "processes": [ { "pid": 9, "command": "\/bin\/bash", "state": "running", "id": 22518496, "status": 0 } ] }

curl http://[2a02:2788:54:1096:b8c7:ccff:fe48:1eac]:7681/api/process/stop?id=22518496
{ "status": "success" }

curl http://[2a02:2788:54:1096:b8c7:ccff:fe48:1eac]:7681/api/process/stop?id=22518496
{ "status": "error", "reason": "process already stopped" }

I can confirm process is stopped and not responding anymore. Starting a new bash, doing ps aux show only the running process.

Killing this process using kill -9 result to:

{
  "processes": [
    {
      "pid": 20,
      "command": "/bin/bash",
      "state": "stopped",
      "id": 22515360,
      "signal": 9,
      "status": 0
    },
    {
      "pid": 9,
      "command": "/bin/bash",
      "state": "stopped",
      "id": 22518496,
      "status": 0
    }
  ]
}

Which sound good.

RafyAmgadBenjamin

comment created time in a month

create barnchthreefoldfoundation/info_threefold

branch : development-maxime

created branch time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+STORCLI_VERSION="1.21.06"+ dependencies_storcli() {     apt-get update-    apt-get install wget      echo "[+] updating soruces list"     wget -O - http://archive.thomas-krenn.com/tk-archive.gpg.pub | apt-key add -

Did you tested your script ? Because it doesn't works on a fresh Ubuntu 18.04 docker. Which mean it won't work neither on the github workflow. The apt-key command fails, you need to install gnupg (apt-get install gnupg) first.

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+STORCLI_VERSION="1.21.06"+ dependencies_storcli() {     apt-get update-    apt-get install wget      echo "[+] updating soruces list"     wget -O - http://archive.thomas-krenn.com/tk-archive.gpg.pub | apt-key add --    cd /etc/apt/sources.list.d-    wget http://archive.thomas-krenn.com/tk-main-xenial.list-    wget http://archive.thomas-krenn.com/tk-optional-xenial.list++    wget -O /etc/apt/sources.list.d/tk-main-xenial.list http://archive.thomas-krenn.com/tk-main-xenial.list+    wget -O /etc/apt/sources.list.d/tk-optional-xenial.list http://archive.thomas-krenn.com/tk-optional-xenial.list     cd     apt-get update      echo "[+] installing storcli"-    apt-get install storcli+    apt-get install storcli=${STORCLI_VERSION}

This could break if package is not available on the ubuntu repository anymore.

You should download it from: https://docs.broadcom.com/docs-and-downloads/raid-controllers/raid-controllers-common-files/007.1316.0000.0000_Unified_StorCLI_PUL.zip

This is the last version, more up-to-date than Ubuntu version (1.21.06 Nov 18, 2016 vs 007.1316.0000.0000 Mar 12, 2020)

Extract the zip, extract the Unified_storcli_all_os/Linux/storcli-007.1316.0000.0000-1.noarch.rpm and take the binary from there.

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+STORCLI_VERSION="1.21.06"+ dependencies_storcli() {     apt-get update-    apt-get install wget      echo "[+] updating soruces list"     wget -O - http://archive.thomas-krenn.com/tk-archive.gpg.pub | apt-key add --    cd /etc/apt/sources.list.d-    wget http://archive.thomas-krenn.com/tk-main-xenial.list-    wget http://archive.thomas-krenn.com/tk-optional-xenial.list++    wget -O /etc/apt/sources.list.d/tk-main-xenial.list http://archive.thomas-krenn.com/tk-main-xenial.list+    wget -O /etc/apt/sources.list.d/tk-optional-xenial.list http://archive.thomas-krenn.com/tk-optional-xenial.list     cd

You don't change directory before, this could even do weird stuff with the pushd from build_storcli

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+STORCLI_VERSION="1.21.06"+ dependencies_storcli() {     apt-get update

This one is not needed, you re-do it after wget later

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

 dependencies_hd-idle() {  download_hd-idle() {     echo "[+] downloading and installing hd-idle"-    go get github.com/adelolmo/hd-idle+    go get ${HD_IDLE_REPOSITORY}@${HD_IDLE_VERSION}+}++prepare_hd-idle() {+    echo "[+] prepare hd-idle"+    github_name "hd-idle-${HD_IDLE_VERSION}"

I would recommend to use ${HD_IDLE_VERSION:0:8} to get a short id (otherwise filename will be large)

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+dependencies_storcli() {+    apt-get update+    apt-get install wget++    echo "[+] updating soruces list"+    wget -O - http://archive.thomas-krenn.com/tk-archive.gpg.pub | apt-key add -+    cd /etc/apt/sources.list.d+    wget http://archive.thomas-krenn.com/tk-main-xenial.list+    wget http://archive.thomas-krenn.com/tk-optional-xenial.list+    cd+    apt-get update++    echo "[+] installing storcli"+    apt-get install storcli+}++install_storcli() {+    echo "[+] copying storcli"++    mkdir -p "${ROOTDIR}/sbin"+    cp -av "/usr/sbin/storcli" "${ROOTDIR}/sbin/"+}++build_storcli() {+    pushd "${DISTDIR}"++    dependencies_storcli+    install_storcli++    popd+}+

You also never use github_name function which is needed by workflow to know package name/version specified later for the flist (used to specify the version/change the version without changing the workflow)

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+dependencies_storcli() {+    apt-get update+    apt-get install wget++    echo "[+] updating soruces list"+    wget -O - http://archive.thomas-krenn.com/tk-archive.gpg.pub | apt-key add -+    cd /etc/apt/sources.list.d

Maybe use wget -O to avoid switching directory

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+dependencies_storcli() {+    apt-get update+    apt-get install wget

wget is already installed by basesystem, which is supposed to be installed first for every packages. You can skip this part.

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+dependencies_storcli() {+    apt-get update+    apt-get install wget++    echo "[+] updating soruces list"+    wget -O - http://archive.thomas-krenn.com/tk-archive.gpg.pub | apt-key add -+    cd /etc/apt/sources.list.d+    wget http://archive.thomas-krenn.com/tk-main-xenial.list+    wget http://archive.thomas-krenn.com/tk-optional-xenial.list+    cd+    apt-get update++    echo "[+] installing storcli"+    apt-get install storcli

Again, using this way, you don't specify a version, you just use latest version they provide.

DylanVerstraete

comment created time in a month

Pull request review commentthreefoldtech/zos

Disk shutdown

+HDIDLE_VERSION="master"+HDIDLE_REPOSITORY="https://github.com/adelolmo/hd-idle.git"++dependencies_hd-idle() {+    . ${PKGDIR}/../golang/golang.sh+    build_golang+    HD_IDLE_HOME="${GOPATH}/src/github.com/adelolmo"+}++download_hd-idle() {+    echo "[+] downloading and installing hd-idle"+    go get github.com/adelolmo/hd-idle

We are supposed to ship specific version and not latest version automatically, right @zaibon ? You should download a specific version from github, not using go get like this.

DylanVerstraete

comment created time in a month

issue commentthreefoldtech/home

GE: Track Usage

Here is a visual proof-of-concept on what you can get: https://asciinema.org/a/Ka13oZvQOcUX4HypkeYkgl6GH This is just by using stats provided by zos and zdb, nothing more :)

joefoxton

comment created time in a month

PR opened threefoldtech/tfexplorer

Disable disk-type selection on tfuser

Recent changes prevent container to be executed on HDD, only SSD are allowed as disk-type. Disable user selection and force disk to be set as SSD by default.

+1 -10

0 comment

2 changed files

pr created time in a month

create barnchthreefoldtech/tfexplorer

branch : tfuser-disk-type

created branch time in a month

push eventthreefoldtech/zos

Maxime Daniel

commit sha 21c4240744b951db94ba5aed12e3c9538e50fa3f

Use public peerlist, not wrong testing peerlist

view details

push time in a month

PR opened threefoldtech/zos

Reviewers
Fallback to backup peers list if public list is down

If public peer list https://publicpeers.neilalexander.dev/publicnodes.json is down, fallback to hardcoded list

+25 -3

0 comment

1 changed file

pr created time in a month

create barnchthreefoldtech/zos

branch : yggdrasil-fallback-peers

created branch time in a month

push eventthreefoldtech/zos

Dylan Verstraete

commit sha aea934201c6c69cfcf1821c658970023cee85763

Centralize environment dependent settings into environment package (#841) * move some variables to environment package * support ZOS_BIN_REPO variable enviroment Co-authored-by: Christophe de Carvalho <christophe.dcpm@gmail.com>

view details

Maxime Daniel

commit sha 1ef5c6083cb2554562c7dfccc54dc94a562d83cd

Merge branch 'master' into shim-logs-0.2

view details

push time in 2 months

issue closedthreefoldtech/corex

Lagging interface and missing lines

When executing some stuff like apt-get update or apt-get install or scripts like configure, interface is lagging and sometime even some lines are skipped.

There is probably an issue with pty reading or buffer.

closed time in 2 months

maxux

PR opened threefoldtech/zos

Reviewers
Update shim-logs to v0.2

Fix 100% CPU usage. #845

+2 -2

0 comment

1 changed file

pr created time in 2 months

created tagthreefoldtech/shim-logs

tagv0.2

External binary used with shim to handle containers logs

created time in 2 months

create barnchthreefoldtech/zos

branch : shim-logs-0.2

created branch time in 2 months

release threefoldtech/shim-logs

v0.2

released time in 2 months

issue closedthreefoldtech/zos

High CPU usage by shim-logs

Some nodes seems to get shim-logs process eating 100% CPU. It seems shim-logs doesn't close itself when underlaying process dies or so. It's probably because containerd doesn't kill shim-logs on error.

...
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
...

The EPOLLHUP event is not catched.

closed time in 2 months

maxux

issue commentthreefoldtech/zos

High CPU usage by shim-logs

Fixed by https://github.com/threefoldtech/shim-logs/commit/d2dadea65f8169cb52f19c7f914ee0cc2945dee4

maxux

comment created time in 2 months

issue openedthreefoldtech/zos

High CPU usage by shim-logs

Some nodes seems to get shim-logs process eating 100% CPU. It seems shim-logs doesn't close itself when underlaying process dies or so. It's probably because containerd doesn't kill shim-logs on error.

...
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
epoll_wait(10, [{EPOLLHUP, {u32=4, u64=4}}, {EPOLLHUP, {u32=3, u64=3}}], 64, -1) = 2
...

The EPOLLHUP event is not catched.

created time in 2 months

push eventthreefoldtech/shim-logs

Maxime Daniel

commit sha d2dadea65f8169cb52f19c7f914ee0cc2945dee4

Fix non-catched EPOLLHUP

view details

push time in 2 months

issue commentthreefoldtech/corex

Lagging interface and missing lines

Could be fixed by #21, checking.

maxux

comment created time in 2 months

PR opened threefoldtech/zos

Reviewers
Update corex to 2.1.3, potential lag fix
+2 -2

0 comment

1 changed file

pr created time in 2 months

create barnchthreefoldtech/zos

branch : corex-2.1.3

created branch time in 2 months

release threefoldtech/corex

2.1.3

released time in 2 months

created tagthreefoldtech/corex

tag2.1.3

Terminal multiplexer webui based

created time in 2 months

delete branch threefoldtech/corex

delete branch : lag-fix

delete time in 2 months

push eventthreefoldtech/corex

Maxime Daniel

commit sha 941a0eda17668e57c629ff73d7bd8d0f165e0235

Fix lag with lws 3.2, based on ttyd upstream fix

view details

Maxime Daniel

commit sha a68ba9afe9dde401f69a3d90905592c9dea1f094

Merge pull request #21 from threefoldtech/lag-fix Fix lag with lws 3.2, based on ttyd upstream fix

view details

push time in 2 months

PR merged threefoldtech/corex

Fix lag with lws 3.2, based on ttyd upstream fix

Fix lag from upstream patch modified

+15 -3

0 comment

3 changed files

maxux

pr closed time in 2 months

PR opened threefoldtech/corex

Fix lag with lws 3.2, based on ttyd upstream fix

Fix lag from upstream patch modified

+15 -3

0 comment

3 changed files

pr created time in 2 months

create barnchthreefoldtech/corex

branch : lag-fix

created branch time in 2 months

Pull request review commentthreefoldtech/zos

Centralize environment dependent settings into environment package

 func getEnvironmentFromParams(params kernel.Params) (Environment, error) { 		env.BcdbPassword = e 	} +	if e := os.Getenv("ZOS_FLIST_URL"); e != "" {+		env.FlistURL = e

We have others environment variable aswell defined. In the past, we talked about a way to define environment variable easily for the whole system, that was never implemented but still could be. It's easier IMO than editing service file to add custom flags (like debug)

DylanVerstraete

comment created time in 2 months

Pull request review commentthreefoldtech/zos

Centralize environment dependent settings into environment package

 func getEnvironmentFromParams(params kernel.Params) (Environment, error) { 		env.BcdbPassword = e 	} +	if e := os.Getenv("ZOS_FLIST_URL"); e != "" {+		env.FlistURL = e

That's nice ! But if adding an environment variable for flist, could add one for BinRepo too ? :)

DylanVerstraete

comment created time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha 4e01d7bd02a732a89db378fd20af0cc72a19b822

Umount old zdb container before restarting it

view details

push time in 2 months

push eventthreefoldtech/zos

Christophe de Carvalho

commit sha 3eb73f885b8ae8ebcf2134fc3a50fd8ae9db33c4

fix tracking of resource unit consumed by 0-DB namespaces fixes #839

view details

Maxime Daniel

commit sha 2ead5891ba9426c15cb9787a141f0796bcb23cb6

Merge branch 'master' into zdb-update

view details

push time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha 97dff0579df31e0b486e7c51c6e4c5c7a306dd63

Clean rootfs if zdb can't start when restarting

view details

push time in 2 months

push eventthreefoldtech/zos

Muhamad Azamy

commit sha 983441c709cc9ad9b0b1fb3c262dfa2ee7b5e1d9

move file to avoid "text file busy" error

view details

Christophe de Carvalho

commit sha 2590522bff832f1ec63e69fffdbf4ad71c83e0ad

make warning log level print in yellow

view details

Maxime Daniel

commit sha 00e35bb82edb235ed0d1ba7952d59ab781a2cf4a

Merge branch 'master' into zdb-update

view details

push time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha 3e3feca8352742f241a3a080744d9886fd06deb7

Change how zdb container are restarted when upgrading

view details

push time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha 71a940e134080884ddcf0fc8f875d2a4f9105f4c

Update workflow to support corex pre-release and release

view details

push time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha 9777bde2c845021dd4600be4f8f084ce1787ebd3

Fix shitty formatting of yaml development

view details

push time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha 0d2c9aabe3fdb9ae71c867626e2a5f5890d89d3d

Clean workflow from buggy merge

view details

push time in 2 months

push eventthreefoldtech/zos

Muhamad Azamy

commit sha 983441c709cc9ad9b0b1fb3c262dfa2ee7b5e1d9

move file to avoid "text file busy" error

view details

Christophe de Carvalho

commit sha 2590522bff832f1ec63e69fffdbf4ad71c83e0ad

make warning log level print in yellow

view details

Maxime Daniel

commit sha dc6a20dd13338f5864b78e3581a2ee397631ca18

Merge branch 'master' into extra-bins-corex

view details

push time in 2 months

push eventthreefoldtech/zos

Christophe de Carvalho

commit sha 8b2a795b68d44ec69673b1d75313f32576620d77

networkd: simplify the routing table of the network resources leave wireguard deal with routing and only set a single route for the full network range

view details

Christophe de Carvalho

commit sha 5167874f59eb6869df5df751579bc83313289fb1

networkd: do remove address from wg interface if needed

view details

Christophe de Carvalho

commit sha 68c2a67feecf06216e928009909fd0875baf849d

network: allow public namespace to also receives IPV6 configuration from router advertisement - enable accept_ra and accept_ra_defrtr in public namespace

view details

Maxime Daniel

commit sha 072a139e6f25138b281285266b4d84ff54f264bc

Support WithBinaryLogURI from containerd to restart logs shim on kill

view details

Christophe de Carvalho

commit sha 1dbb05bee442a1709f4475ce7b311f5c1e75edf9

cleanup unused code

view details

Christophe de Carvalho

commit sha 36dd193e2fd34182fed8fc374b74ec4b2c82403d

Merge pull request #825 from threefoldtech/nr_routes networkd: simplify the routing table of the network resources

view details

Maxime Daniel

commit sha ff54a39729f27d4f78ab169a3a0e04c5c76c5478

Merge branch 'master' into containerd-restart

view details

Maxime Daniel

commit sha cf3c2a98c51ca53b566674c1cd8951d65d6151a7

Update containerd dependencies

view details

Maxime Daniel

commit sha 5f2c5903070cf9d1c1eee74fe6c544dc065a7eda

Update containerd external binary

view details

Christophe de Carvalho

commit sha f5bca533c7fd617eced43fe14fbf6f2396525d44

Merge pull request #828 from threefoldtech/containerd-restart Support WithBinaryLogURI from containerd to restart logs shim on kill

view details

Christophe de Carvalho

commit sha 01745452d27449151d79846612bb547f14797c47

add build script for yggdrasil

view details

Christophe de Carvalho

commit sha 50e19353799075e49c6dad0b49cff06ad5fe8a1f

add build script for yggdrasil

view details

Christophe de Carvalho

commit sha 1b6dbba8645ed53b7e270a9cca27333e2b78844e

implement yggdrasil management

view details

Christophe de Carvalho

commit sha 5e005508dc392c826b527979f6ed56ec154df1da

start yggdrasil intergration specification

view details

Christophe de Carvalho

commit sha 9c5037054f6b6feeace9e156d163e2ae4c33b2b0

Try out to embed yggdrasil node directly into networkd

view details

Christophe de Carvalho

commit sha b83d84da15f10b3e18e2e3d8474444de52c62668

wip

view details

Christophe de Carvalho

commit sha a59ee53a882305dd5166855b1be824c9d7df67a2

allow to have an yggdrasil interface into 0-db containers

view details

Christophe de Carvalho

commit sha 44b0ed1d24e8d0de04b44271f8bd32f8b558a2ed

start yygdrasil in ndmz namespace through zinit

view details

Christophe de Carvalho

commit sha 8ff32fa5494ededf22bb5db046b56ddc080da5aa

clean up yggdrasil package

view details

Christophe de Carvalho

commit sha e557af94bd410bf3134b56a9054b821d89739ec7

networkd: make ndmz an interface and implement ipv4 only and dualstack version

view details

push time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha 28229c616cfff9d1e3a9a0f0d7eccd9e739f1299

Implement zdb upgrade restarter

view details

push time in 2 months

push eventthreefoldtech/zos

Maxime Daniel

commit sha b4939ac7f7e4b70586781e95bfe3e5b14fc1b4e6

Delete zdb container before upgrade it

view details

push time in 2 months

more