profile
viewpoint
Kirill Kolyshkin kolyshkin @docker East Renton Highlands, WA, USA

issue commentdocker/for-linux

dockerd: high memory usage

@ceecko can you please collect memory usage dumps and share it with us? The following article explains how to do that: https://success.docker.com/article/how-do-i-gather-engine-heap-information

ceecko

comment created time in a month

issue commentmoby/moby

dockerd consumes super high memory(35GB)

@TimeBye can you please take a memory profile and share? This describes how to do it: https://success.docker.com/article/how-do-i-gather-engine-heap-information

Also, do you have any steps to reproduce this behavior? In other words, what do I do on a clean machine with docker installed in order to see what you see?

TimeBye

comment created time in a month

pull request commentmoby/moby

35991- make `--device` works at privileged mode

As I mentioned in the original PR, a documentation update is needed to warn that device permissions are ignored in privileged mode. Can you work on this one @akhilerm ?

akhilerm

comment created time in a month

PR closed moby/moby

fix issue: make --device works at privileged mode status/2-code-review status/failing-ci

Signed-off-by: wenlxie wenlxie@ebay.com

Please provide the following information: -->

- What I did fixs #35991 - How I verify it Make the docker binary, test following cases

  • case 1
docker run -it --privileged=true --device /dev/vdc:/dev/vda busybox sh

Then I will get the error message

docker: Error response from daemon: linux runtime spec devices: In privileged mode,Path:/dev/vda In Container should be different with any path on host.
  • case 2
docker run -it --privileged=true --device /dev/vdc:/dev/vdx busybox sh

Then I enter the docker, I can see the device /dev/vdx I exit the docker, and there is no /dev/vdx at host

- Description for the changelog Make --device options works at privileged mode

+40 -0

19 comments

2 changed files

wenlxie

pr closed time in a month

pull request commentmoby/moby

fix issue: make --device works at privileged mode

Carried over in https://github.com/moby/moby/pull/40291

wenlxie

comment created time in a month

pull request commentmoby/moby

These guys are many great things, but I think none of them is boring

@duststorm please squash your commits into one

duststorm

comment created time in 2 months

push eventmoby/moby

Sascha Grunert

commit sha 4138cd22abeaa7d1c49a96fa4c0045feb32b847e

Fix possible runtime panic in Lgetxattr If `unix.Lgetxattr` returns an error, then `sz == -1` which will cause a runtime panic if `errno == unix.ERANGE`. Signed-off-by: Sascha Grunert <sgrunert@suse.com>

view details

Kirill Kolyshkin

commit sha d163fbba3c82a165247acf6913e5c68a718fec80

Merge pull request #40283 from saschagrunert/lgetxattr-panic Fix possible runtime panic in Lgetxattr

view details

push time in 2 months

PR merged moby/moby

Fix possible runtime panic in Lgetxattr area/images kind/bugfix process/cherry-pick status/4-merge

If unix.Lgetxattr returns an error, then sz == -1 which will cause a runtime panic if errno == unix.ERANGE.

+15 -6

4 comments

1 changed file

saschagrunert

pr closed time in 2 months

pull request commentmoby/moby

Fix possible runtime panic in Lgetxattr

The new code seems to be exactly the same as in runc (https://github.com/opencontainers/runc/blob/d8b5c1c/libcontainer/system/xattrs_linux.go)

saschagrunert

comment created time in 2 months

pull request commentmoby/moby

Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2

@ebalders this one was not merged; #40250 was. In there you will be able to see links to backports to 19.03 branch and monitor it

vikramhh

comment created time in 2 months

PR closed moby/moby

Reviewers
Improve rootless Docker overlay support detection area/rootless area/storage/overlay status/2-code-review

Signed-off-by: Brian Turek brian.turek@gmail.com

<!-- Please make sure you've read and understood our contributing guidelines; https://github.com/moby/moby/blob/master/CONTRIBUTING.md

** Make sure all your commits include a signature generated with git commit -s **

For additional information on our contributing process, read our contributing guide https://docs.docker.com/opensource/code/

If this is a bug fix, make sure your description includes "fixes #xxxx", or "closes #xxxx"

Please provide the following information: -->

- What I did Fixed overlay support detection for rootless Docker on CentOS (and other non-Ubuntu based distros). The current implementation causes runtime errors when running rootless Docker on CentOS due lack of mounting permissions. See docker/for-linux#836 and docker-library/docker#193 for related tickets

- How I did it Current overlay detection works by merely checking for filesystem support for overlay rather than overlay support for that particular user (i.e. Ubuntu allows non-root users to use overlay, CentOS does not). This patch changes the logic to attempt a real overlay mount and determine overlay support by checking whether than mount was successful.

Note that this is literally the first time I've written Go so constructive criticism is appreciated.

- How to verify it

  • Run the new dockerd binary as rootless on CentOS, the storage-driver falls back to vfs
  • Run the new dockerd binary as root on CentOS, the storage-driver is overlay2
  • Run the new dockerd binary as rootless on Ubuntu, the storage-driver is overlay2
  • Run the new dockerd binary as root on Ubuntu, the storage-driver is overlay2

- Description for the changelog <!-- Write a short (one line) summary that describes the changes in this pull request for inclusion in the changelog: --> Improve rootless Docker overlay support detection

- A picture of a cute animal (not mandatory but encouraged)

+70 -34

10 comments

2 changed files

Caligatio

pr closed time in 2 months

pull request commentmoby/moby

Improve rootless Docker overlay support detection

#40194 was merged, this one can be closed

Caligatio

comment created time in 2 months

push eventmoby/moby

vikrambirsingh

commit sha 83783af08ea321091e5f08d46ee0d06431c9eff2

Disable tests missed by PR 40155 The tests starts a new daemon, but attempts to run it with overlay2, and using a unix:// socket, which doesn't really work on Windows. 40155 tried to disable such tests but missed two of them. They are being disabled with this change. Signed-off-by: vikrambirsingh <vikrambir.singh@docker.com>

view details

Kirill Kolyshkin

commit sha b55a25a4257e1ce5852814e776d3b3f4c9b81532

Merge pull request #40199 from vikramhh/touchup_40155 Disable integration/system tests that are failing on Windows with error "protocol not available"

view details

push time in 2 months

PR merged moby/moby

Disable integration/system tests that are failing on Windows with error "protocol not available" area/testing platform/windows process/cherry-pick status/2-code-review

The tests starts a new daemon, but attempts to run it with overlay2, and using a unix:// socket, which doesn't really work on Windows.

https://github.com/moby/moby/pull/40155 tried to disable such tests but missed two of them. They are being disabled with this change.

relates to https://github.com/moby/moby/issues/40156 "Integration: fix TestInfoDebug and other tests that spin up daemons for Windows"

Signed-off-by: vikrambirsingh vikrambir.singh@docker.com

<!-- Please make sure you've read and understood our contributing guidelines; https://github.com/moby/moby/blob/master/CONTRIBUTING.md

** Make sure all your commits include a signature generated with git commit -s **

For additional information on our contributing process, read our contributing guide https://docs.docker.com/opensource/code/

If this is a bug fix, make sure your description includes "fixes #xxxx", or "closes #xxxx"

Please provide the following information: -->

- What I did Disabled two tests that were supposed to have been disabled by #40155 but got missed out

- How I did it Skipped tests if OS is Windows

- How to verify it Checks will verify that there are no failures on RS5

- Description for the changelog <!-- Write a short (one line) summary that describes the changes in this pull request for inclusion in the changelog: --> Skip all integration/system tests on Windows that are using unix:// socket

- A picture of a cute animal (not mandatory but encouraged)

+2 -0

2 comments

1 changed file

vikramhh

pr closed time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static int add_freezer_state(struct cg_controller *controller) 	return 0; } +static const char namestr[] = "name=";+static int __new_open_cgroupfs(struct cg_ctl *cc)+{+	int fsfd, fd;+	char *name;++	fsfd = sys_fsopen("cgroup", 0);+	if (fsfd < 0) {+		pr_perror("Unable to open the cgroup file system");+		return -1;+	}++	if (strstartswith(cc->name, namestr)) {+		if (sys_fsconfig(fsfd, FSCONFIG_SET_STRING,+				 "name", cc->name + strlen(namestr), 0) < 0) {+			pr_perror("Unable to configure the cgroup (%s) file system", cc->name);+			goto err;+		}+	} else {+		name = strdupa(cc->name);+		while (name) {+			char *c = strchr(name, ',');+			if (c)+				*c = 0;+			if (sys_fsconfig(fsfd, FSCONFIG_SET_FLAG, name, NULL, 0) <0) {+				pr_perror("Unable to configure the cgroup (%s) file system", name);+				goto err;+			}+			if (c)+				name = c + 1;+			else+				break;+		}+	}++	if (sys_fsconfig(fsfd, FSCONFIG_CMD_CREATE, NULL, NULL, 0) <0) {+		pr_perror("Unable to create the cgroup (%s) file system", cc->name);+		goto err;+	}++	fd = sys_fsmount(fsfd, 0, 0);+	if (fd < 0)+		pr_perror("Unable to mount the cgroup (%s) file system", cc->name);+	close(fsfd);++	return fd;+err:+	close(fsfd);+	return -1;+}++static int open_cgroupfs(struct cg_ctl *cc)+{+	char prefix[] = ".criu.cgmounts.XXXXXX";+		char mopts[1024];+	int fd;++	if (kdat.has_fsopen)+		return __new_open_cgroupfs(cc);++	if (strstartswith(cc->name, namestr))+		snprintf(mopts, sizeof(mopts), "none,%s", cc->name);+	else+		snprintf(mopts, sizeof(mopts), "%s", cc->name);++	if (mkdtemp(prefix) == NULL) {+		pr_perror("can't make dir for cg mounts");+		return -1;+	}++	if (mount("none", prefix, "cgroup", 0, mopts) < 0) {+		pr_perror("couldn't mount %s", mopts);

or unable since you're using it in the new code

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static int add_freezer_state(struct cg_controller *controller) 	return 0; } +static const char namestr[] = "name=";+static int __new_open_cgroupfs(struct cg_ctl *cc)+{+	int fsfd, fd;+	char *name;++	fsfd = sys_fsopen("cgroup", 0);+	if (fsfd < 0) {+		pr_perror("Unable to open the cgroup file system");+		return -1;+	}++	if (strstartswith(cc->name, namestr)) {+		if (sys_fsconfig(fsfd, FSCONFIG_SET_STRING,+				 "name", cc->name + strlen(namestr), 0) < 0) {+			pr_perror("Unable to configure the cgroup (%s) file system", cc->name);+			goto err;+		}+	} else {+		name = strdupa(cc->name);+		while (name) {+			char *c = strchr(name, ',');

Have you considered using strtok instead?

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static int add_freezer_state(struct cg_controller *controller) 	return 0; } +static const char namestr[] = "name=";+static int __new_open_cgroupfs(struct cg_ctl *cc)+{+	int fsfd, fd;+	char *name;++	fsfd = sys_fsopen("cgroup", 0);+	if (fsfd < 0) {+		pr_perror("Unable to open the cgroup file system");+		return -1;+	}++	if (strstartswith(cc->name, namestr)) {+		if (sys_fsconfig(fsfd, FSCONFIG_SET_STRING,+				 "name", cc->name + strlen(namestr), 0) < 0) {+			pr_perror("Unable to configure the cgroup (%s) file system", cc->name);+			goto err;+		}+	} else {+		name = strdupa(cc->name);+		while (name) {+			char *c = strchr(name, ',');+			if (c)+				*c = 0;+			if (sys_fsconfig(fsfd, FSCONFIG_SET_FLAG, name, NULL, 0) <0) {+				pr_perror("Unable to configure the cgroup (%s) file system", name);+				goto err;+			}+			if (c)+				name = c + 1;+			else+				break;+		}+	}++	if (sys_fsconfig(fsfd, FSCONFIG_CMD_CREATE, NULL, NULL, 0) <0) {+		pr_perror("Unable to create the cgroup (%s) file system", cc->name);+		goto err;+	}++	fd = sys_fsmount(fsfd, 0, 0);+	if (fd < 0)+		pr_perror("Unable to mount the cgroup (%s) file system", cc->name);+	close(fsfd);++	return fd;+err:+	close(fsfd);+	return -1;+}++static int open_cgroupfs(struct cg_ctl *cc)+{+	char prefix[] = ".criu.cgmounts.XXXXXX";+		char mopts[1024];+	int fd;++	if (kdat.has_fsopen)+		return __new_open_cgroupfs(cc);++	if (strstartswith(cc->name, namestr))+		snprintf(mopts, sizeof(mopts), "none,%s", cc->name);+	else+		snprintf(mopts, sizeof(mopts), "%s", cc->name);++	if (mkdtemp(prefix) == NULL) {+		pr_perror("can't make dir for cg mounts");+		return -1;+	}++	if (mount("none", prefix, "cgroup", 0, mopts) < 0) {+		pr_perror("couldn't mount %s", mopts);

maybe s/couldn't/can't/ while we're at it

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static int add_freezer_state(struct cg_controller *controller) 	return 0; } +static const char namestr[] = "name=";+static int __new_open_cgroupfs(struct cg_ctl *cc)+{+	int fsfd, fd;+	char *name;++	fsfd = sys_fsopen("cgroup", 0);+	if (fsfd < 0) {+		pr_perror("Unable to open the cgroup file system");+		return -1;+	}++	if (strstartswith(cc->name, namestr)) {+		if (sys_fsconfig(fsfd, FSCONFIG_SET_STRING,+				 "name", cc->name + strlen(namestr), 0) < 0) {+			pr_perror("Unable to configure the cgroup (%s) file system", cc->name);+			goto err;+		}+	} else {+		name = strdupa(cc->name);+		while (name) {+			char *c = strchr(name, ',');+			if (c)+				*c = 0;+			if (sys_fsconfig(fsfd, FSCONFIG_SET_FLAG, name, NULL, 0) <0) {+				pr_perror("Unable to configure the cgroup (%s) file system", name);+				goto err;+			}+			if (c)+				name = c + 1;+			else+				break;+		}+	}++	if (sys_fsconfig(fsfd, FSCONFIG_CMD_CREATE, NULL, NULL, 0) <0) {+		pr_perror("Unable to create the cgroup (%s) file system", cc->name);+		goto err;+	}++	fd = sys_fsmount(fsfd, 0, 0);+	if (fd < 0)+		pr_perror("Unable to mount the cgroup (%s) file system", cc->name);+	close(fsfd);++	return fd;+err:+	close(fsfd);+	return -1;+}++static int open_cgroupfs(struct cg_ctl *cc)+{+	char prefix[] = ".criu.cgmounts.XXXXXX";+		char mopts[1024];

possiible indentation issue

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static void restore_pgid(void) 		futex_set_and_wake(&rsti(current)->pgrp_set, 1); } +static int __old_mount_proc()+{+	char proc_mountpoint[] = "/tmp/crtools-proc.XXXXXX";+	int fd;++	if (mkdtemp(proc_mountpoint) == NULL) {+		pr_perror("mkdtemp failed %s", proc_mountpoint);+		return -1;+	}++	pr_info("Mount procfs in %s\n", proc_mountpoint);+	if (mount("proc", proc_mountpoint, "proc", MS_MGC_VAL | MS_NOSUID | MS_NOEXEC | MS_NODEV, NULL)) {+		pr_perror("mount failed");+		if (rmdir(proc_mountpoint))+			pr_err("Unable to remove %s", proc_mountpoint);+		return -1;+	}++	fd = open_detach_mount(proc_mountpoint);+	return fd;+}+ static int mount_proc(void) { 	int fd, ret;-	char proc_mountpoint[] = "/tmp/crtools-proc.XXXXXX";  	if (root_ns_mask == 0) 		fd = ret = open("/proc", O_DIRECTORY); 	else {-		if (mkdtemp(proc_mountpoint) == NULL) {-			pr_perror("mkdtemp failed %s", proc_mountpoint);-			return -1;-		}--		pr_info("Mount procfs in %s\n", proc_mountpoint);-		if (mount("proc", proc_mountpoint, "proc", MS_MGC_VAL | MS_NOSUID | MS_NOEXEC | MS_NODEV, NULL)) {-			pr_perror("mount failed");-			rmdir(proc_mountpoint);-			return -1;-		}--		ret = fd = open_detach_mount(proc_mountpoint);+		if (kdat.has_fsopen)+			fd = ret = mount_detached_fs("proc");

You have not introduced this function yet (at least according to what github shows me). This will break bisect.

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static void restore_pgid(void) 		futex_set_and_wake(&rsti(current)->pgrp_set, 1); } +static int __old_mount_proc()

Maybe s/__old/legacy/?

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static void restore_pgid(void) 		futex_set_and_wake(&rsti(current)->pgrp_set, 1); } +static int __old_mount_proc()+{+	char proc_mountpoint[] = "/tmp/crtools-proc.XXXXXX";+	int fd;++	if (mkdtemp(proc_mountpoint) == NULL) {+		pr_perror("mkdtemp failed %s", proc_mountpoint);+		return -1;+	}++	pr_info("Mount procfs in %s\n", proc_mountpoint);+	if (mount("proc", proc_mountpoint, "proc", MS_MGC_VAL | MS_NOSUID | MS_NOEXEC | MS_NODEV, NULL)) {+		pr_perror("mount failed");+		if (rmdir(proc_mountpoint))+			pr_err("Unable to remove %s", proc_mountpoint);

not related to this patch, but while we're at it, this should be pr_perror() as rmdir(2) sets errno.

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static void restore_pgid(void) 		futex_set_and_wake(&rsti(current)->pgrp_set, 1); } +static int __old_mount_proc()+{+	char proc_mountpoint[] = "/tmp/crtools-proc.XXXXXX";+	int fd;++	if (mkdtemp(proc_mountpoint) == NULL) {+		pr_perror("mkdtemp failed %s", proc_mountpoint);+		return -1;+	}++	pr_info("Mount procfs in %s\n", proc_mountpoint);

not related to this patch, but while we're at it, maybe s/in/at/?

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

Use the new mount API

 static int kerndat_has_inotify_setnextwd(void) 	return ret; } +static int kerndat_has_fsopen(void)+{+	if (syscall(__NR_fsopen, NULL, -1) != -1) {+		pr_err("fsopen doesn't fail\n");

s/doesn't/must not/

avagin

comment created time in 2 months

Pull request review commentcheckpoint-restore/criu

readme: github pull-requests is the preferred way to contribute

 Here are some useful hints to get involved. * CRIU does need [extensive testing](https://github.com/checkpoint-restore/criu/issues?q=is%3Aissue+is%3Aopen+label%3Atesting); * Documentation is always hard, we have [some information](https://criu.org/Category:Empty_articles) that is to be extracted from people's heads into wiki pages as well as [some texts](https://criu.org/Category:Editor_help_needed) that all need to be converted into useful articles; * Feedback is expected on the github issues page and on the [mailing list](https://lists.openvz.org/mailman/listinfo/criu);-* For historical reasons we do not accept PRs, instead [patches are welcome](http://criu.org/How_to_submit_patches);+* We accept github pull-requests and this is the preferred way to contribute to CRIU. If you prefer to send patches by email, you are welcome to send them to [the devel list](http://criu.org/How_to_submit_patches);

s/pull-requests/pull requests/

avagin

comment created time in 2 months

issue commentdocker/for-linux

experimental --platform doesn't seem to work with ubuntu

Well, I think that setarch i386 can be used:

$ docker run --platform=i386 -ti ubuntu:16.04 /bin/bash
root@3a80cb4c4eae:/# uname -a
Linux 3a80cb4c4eae 4.19.79+ #92 SMP Mon Oct 14 11:37:29 PDT 2019 x86_64 x86_64 x86_64 GNU/Linux
root@3a80cb4c4eae:/# setarch i386
# uname -a
Linux 3a80cb4c4eae 4.19.79+ #92 SMP Mon Oct 14 11:37:29 PDT 2019 i686 i686 i686 GNU/Linux
# 

In general, I think the problem is, the --platform is only for creating the container, not for running it (otherwise we could do personality() syscall).

pmatos

comment created time in 2 months

Pull request review commentmoby/moby

Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2

 func (s *DockerSuite) TestRunAttachFailedNoLeak(c *testing.T) { 	assert.Assert(c, err != nil, "Command should have failed but succeeded with: %s\nContainer 'test' [%+v]: %s\nContainer 'fail' [%+v]: %s", out, err1, out1, err2, out2) 	// check for windows error as well 	// TODO Windows Post TP5. Fix the error message string-	assert.Assert(c, strings.Contains(out, "port is already allocated") ||-		strings.Contains(out, "were not connected because a duplicate name exists") ||-		strings.Contains(out, "The specified port already exists") ||-		strings.Contains(out, "HNS failed with error : Failed to create endpoint") ||-		strings.Contains(out, "HNS failed with error : The object already exists"), fmt.Sprintf("Output: %s", out))+	assert.Assert(c, strings.Contains(strings.ToLower(out), "port is already allocated") ||

Also, test change would be better as a separate commit (preceding the one updating the vendor)

vikramhh

comment created time in 2 months

Pull request review commentmoby/moby

Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2

 func (s *DockerSuite) TestRunAttachFailedNoLeak(c *testing.T) { 	assert.Assert(c, err != nil, "Command should have failed but succeeded with: %s\nContainer 'test' [%+v]: %s\nContainer 'fail' [%+v]: %s", out, err1, out1, err2, out2) 	// check for windows error as well 	// TODO Windows Post TP5. Fix the error message string-	assert.Assert(c, strings.Contains(out, "port is already allocated") ||-		strings.Contains(out, "were not connected because a duplicate name exists") ||-		strings.Contains(out, "The specified port already exists") ||-		strings.Contains(out, "HNS failed with error : Failed to create endpoint") ||-		strings.Contains(out, "HNS failed with error : The object already exists"), fmt.Sprintf("Output: %s", out))+	assert.Assert(c, strings.Contains(strings.ToLower(out), "port is already allocated") ||

I would suggest to do

out = strings.ToLower(out)

instead. Otherwise, this code is doing it five times :-\

vikramhh

comment created time in 2 months

pull request commentmoby/moby

Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2

just lowercase out before doing comparisons, and make sure all comprarison strings are also lowercased

i.e. something like

diff --git a/integration-cli/docker_cli_run_test.go b/integration-cli/docker_cli_run_test.go
index af9044a92c..4271aba72e 100644
--- a/integration-cli/docker_cli_run_test.go
+++ b/integration-cli/docker_cli_run_test.go
@@ -3945,11 +3945,11 @@ func (s *DockerSuite) TestRunAttachFailedNoLeak(c *testing.T) {
        assert.Assert(c, err != nil, "Command should have failed but succeeded with: %s\nContainer 'test' [%+v]: %s\nContainer 'fail' [%+v]: %s", out, err1, out1, err2, out2)
        // check for windows error as well
        // TODO Windows Post TP5. Fix the error message string
-       assert.Assert(c, strings.Contains(out, "port is already allocated") ||
+       assert.Assert(c, strings.Contains(strings.ToLower(out), "port is already allocated") ||
                strings.Contains(out, "were not connected because a duplicate name exists") ||
-               strings.Contains(out, "The specified port already exists") ||
-               strings.Contains(out, "HNS failed with error : Failed to create endpoint") ||
-               strings.Contains(out, "HNS failed with error : The object already exists"), fmt.Sprintf("Output: %s", out))
+               strings.Contains(out, "the specified port already exists") ||
+               strings.Contains(out, "hns failed with error : failed to create endpoint") ||
+               strings.Contains(out, "hns failed with error : the object already exists"), fmt.Sprintf("Output: %s", out))
        dockerCmd(c, "rm", "-f", "test")
 
        // NGoroutines is not updated right away, so we need to wait before failing
vikramhh

comment created time in 2 months

pull request commentmoby/moby

Bump hcsshim to b3f49c06ffaeef24d09c6c08ec8ec8425a0303e2

I am seeing the following failure on Windows RS1

[2019-11-22T18:31:50.937Z] --- FAIL: TestDockerSuite/TestRunAttachFailedNoLeak (4.19s)

[2019-11-22T18:31:50.937Z] docker_cli_run_test.go:3948: assertion failed: expression is false: strings.Contains(out, "port is already allocated") || strings.Contains(out, "were not connected because a duplicate name exists") || strings.Contains(out, "The specified port already exists") || strings.Contains(out, "HNS failed with error : Failed to create endpoint") || strings.Contains(out, "HNS failed with error : The object already exists"): Output: d:\CI\PR-40128\20\binary\docker.exe: Error response from daemon: failed to create endpoint fail on network nat: hns failed with error : The object already exists.

and it looks like it's relevant.

@vikramhh I suggest you fix this test case to do case-insensitive comparison instead (just lowercase out before doing comparisons, and make sure all comprarison strings are also lowercased).

vikramhh

comment created time in 2 months

pull request commentdocker/engine

[18.06] backport "daemon.ContainerLogs(): fix resource leak on follow"

(closed as 18.06 is no more)

kolyshkin

comment created time in 2 months

fork kolyshkin/doodle

A Home for Docker Doodles

fork in 2 months

pull request commentmoby/moby

[WIP] Fix mount loop on "docker cp"

@rrebollo a workaround is to replace docker cp with docker exec $CT cat /path/to/container/file > host_file or, if you need many files, docker exec $CT tar cf - /some/container/dir | tar xf -

kolyshkin

comment created time in 2 months

Pull request review commentmoby/moby

daemon/listeners: use chgrp binary for docker.sock

 func Init(proto, addr, socketGroup string, tlsConfig *tls.Config) ([]net.Listene 		} 		ls = append(ls, l) 	case "unix":-		gid, err := lookupGID(socketGroup)+		gid := os.Getgid()+		l, err := sockets.NewUnixSocket(addr, gid) 		if err != nil {-			if socketGroup != "" {+			return nil, errors.Wrapf(err, "can't create unix socket %s", addr)+		}+		if socketGroup != "" {+			out, err := exec.Command("chgrp", socketGroup, addr).CombinedOutput()

@tonistiigi can you please elaborate why do you want to have both implementations? In case chgrp binary is not available, or as an optimization?

kolyshkin

comment created time in 2 months

pull request commentdocker/libnetwork

resolver: less debug

@arkodg I have modified this PR according to your comments; please take another look

kolyshkin

comment created time in 2 months

push eventkolyshkin/libnetwork

Sebastiaan van Stijn

commit sha f741dc9c305fea900b96b8a838f959395799cf78

Update Golang 1.12.12 (CVE-2019-17596) Golang 1.12.12 ------------------------------- full diff: https://github.com/golang/go/compare/go1.12.11...go1.12.12 go1.12.12 (released 2019/10/17) includes fixes to the go command, runtime, syscall and net packages. See the Go 1.12.12 milestone on our issue tracker for details. https://github.com/golang/go/issues?q=milestone%3AGo1.12.12 Golang 1.12.11 (CVE-2019-17596) ------------------------------- full diff: https://github.com/golang/go/compare/go1.12.10...go1.12.11 go1.12.11 (released 2019/10/17) includes security fixes to the crypto/dsa package. See the Go 1.12.11 milestone on our issue tracker for details. https://github.com/golang/go/issues?q=milestone%3AGo1.12.11 [security] Go 1.13.2 and Go 1.12.11 are released Hi gophers, We have just released Go 1.13.2 and Go 1.12.11 to address a recently reported security issue. We recommend that all affected users update to one of these releases (if you're not sure which, choose Go 1.13.2). Invalid DSA public keys can cause a panic in dsa.Verify. In particular, using crypto/x509.Verify on a crafted X.509 certificate chain can lead to a panic, even if the certificates don't chain to a trusted root. The chain can be delivered via a crypto/tls connection to a client, or to a server that accepts and verifies client certificates. net/http clients can be made to crash by an HTTPS server, while net/http servers that accept client certificates will recover the panic and are unaffected. Moreover, an application might crash invoking crypto/x509.(*CertificateRequest).CheckSignature on an X.509 certificate request, parsing a golang.org/x/crypto/openpgp Entity, or during a golang.org/x/crypto/otr conversation. Finally, a golang.org/x/crypto/ssh client can panic due to a malformed host key, while a server could panic if either PublicKeyCallback accepts a malformed public key, or if IsUserAuthority accepts a certificate with a malformed public key. The issue is CVE-2019-17596 and Go issue golang.org/issue/34960. Thanks to Daniel Mandragona for discovering and reporting this issue. We'd also like to thank regilero for a previous disclosure of CVE-2019-16276. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

elangovan sivanandam

commit sha 571783238bee54062ebc781c321f7833a23e38f7

Merge pull request #2472 from thaJeztah/bump_golang_1.12.12 Update Golang 1.12.12 (CVE-2019-17596)

view details

Arko Dasgupta

commit sha 4420ee92f5b3b951f98a36b2bc8144a19b560a22

Fix panic in drivers/overlay/encryption.go Issue - "index out of range" panic in drivers/overlay/encryption.go:539 due to a mismatch in indices between curKeys and spis due to case where updateKeys might bail out due to an error and not update the spis Fix - Reconfigure keys when there is a key update failure Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>

view details

elangovan sivanandam

commit sha 1680ce717394f8aa9ba6de26b851b7e02699d490

Merge pull request #2462 from arkodg/fix-key-spi-panic Fix panic in drivers/overlay/encryption.go

view details

Kir Kolyshkin

commit sha c5aec55b42f214dbf518ddd9a099d66b4dc27224

resolver: less debug Observed lots of debug messages from resolver (and associated high CPU usage because of lots of logging) when debug logging is turned on. Here is some data from a real system: > $ journalctl -u docker --since="2019-10-21 21:00:00" --until="2019-10-21 22:00:00" | wc -l > 188621 > $ journalctl -u docker --since="2019-10-21 21:00:00" --until="2019-10-21 22:00:00" | grep -E 'Name To resolve: |\[resolver\] ' | wc -l > 186319 So, it was about 200000 lines logger for just one hour, and about 99% of that are from resolver. While this might be the peculiarity of a particular setup, the number of such messages still seem way too excessive. Remove the ones that are not errors. In case those are needed, one have to recompile this package with debug_log build tag. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

Kir Kolyshkin

commit sha 0f20b88efc813ff7340ae345ea01d27f0dcd241e

More logs cleanup Use more `debugf` in places that look definitely like developer's debug which is most probably useless to an end user. Promote some debug errors that looks more like warnings to use Warnf rather than Debugf. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

push time in 2 months

pull request commentdocker/libnetwork

resolver: less debug

@kolyshkin I'm just very reluctant to delete these logs

@arkodg I don't see any value in logging every DNS request (and I guess most of this info is easy to obtain using tcpdump/wireshark).

Can we add a compile time parameter / global const enableDebugLogs = false to reduce logging for now

Yes but IMHO this will look ugly. OK let me try it.

kolyshkin

comment created time in 2 months

pull request commentdocker/docker.github.io

logging/gelf: recommend to disable compression

Looks ok but maybe there's a way to somehow emphasize this warning? https://deploy-preview-9881--docsdocker.netlify.com//config/containers/logging/gelf/

kolyshkin

comment created time in 2 months

pull request commentmoby/moby

logger/gelf: use compression level 0 by default

Seems like the only option left is to recommend disabling gelf compression in docs. Will follow up with a doc update PR

https://github.com/docker/docker.github.io/pull/9881

kolyshkin

comment created time in 2 months

pull request commentdocker/docker.github.io

logging/gelf: recommend to disable compression

@adrian-plata @lena-larionova PTAL

kolyshkin

comment created time in 2 months

PR opened docker/docker.github.io

logging/gelf: recommend to disable compression

It was discovered that compression in gelf logging driver is highly inefficient in terms of CPU. IOW, lots of CPU is used when compression is enabled.

The natural solution would be to change the default to none, i.e. no compression, but that would break some setups (in particular, using Logstash < 7.4).

The only option left is to warn the user about the consequences of leaving the compression enabled.

For more background details, please see https://github.com/moby/moby/pull/40101

PS it makes sense to backport this to older engine version docs, too.

+1 -1

0 comment

1 changed file

pr created time in 2 months

create barnchkolyshkin/docker.github.io

branch : gelf-nocom

created branch time in 2 months

PR closed docker/docker.github.io

[WIP] logging/gelf: document the new default

This documents the new compression default for docker engine gelf logging driver. For details, see https://github.com/moby/moby/pull/40101

+1 -1

3 comments

1 changed file

kolyshkin

pr closed time in 2 months

pull request commentdocker/docker.github.io

[WIP] logging/gelf: document the new default

Closing since we're not merging moby/moby#40101

kolyshkin

comment created time in 2 months

pull request commentmoby/moby

overlay[2]: rm extra checks in init

@snajpa would you help in testing this on some previously unsupported configurations (e.g. zfs)?

kolyshkin

comment created time in 2 months

PR opened moby/moby

Reviewers
overlay[2]: rm extra checks in init

This builds on top of (and currently includes) https://github.com/moby/moby/pull/40194

Now that we do check if overlay is working by performing an actual overlayfs mount, there's no need in extra checks for the kernel version or the filesystem type. Actual mount check is sufficient.

+50 -147

0 comment

4 changed files

pr created time in 2 months

create barnchkolyshkin/moby

branch : ovr-rm-checks

created branch time in 2 months

PR closed moby/moby

Reviewers
logger/gelf: use compression level 0 by default area/logging impact/changelog kind/performance status/2-code-review

go-gelf package used by dockerd uses gzip compression by default, which is costly in terms of CPU. Since the data is sent over the network, it's a trade-off between CPU and network bandwidth.

This comes from observing the internal infrastructure on which dockerd writes containers' logs to one of its containers (which itself does something else with those logs). With default gzip -1 compression, dockerd was using 150-220% CPU, which decreased to about 30% without compression.

In addition, there are multiple reports of gelf using too much CPU:

  • https://github.com/moby/moby/issues/19665
  • https://github.com/moby/moby/issues/19209

With this commit, we disable compression but the output produced is still gzip-compatible. This is done because there are reports that Logstash only got support for uncompressed log messages recently (see https://github.com/logstash-plugins/logstash-input-gelf/pull/48).

- Description for the changelog

  • log/gelf: default compression level set to 0 to prioritize saving CPU over network bandwidth
+1 -0

22 comments

1 changed file

kolyshkin

pr closed time in 2 months

pull request commentmoby/moby

logger/gelf: use compression level 0 by default

Closing as per previous comment. Will follow up with a doc update PR

kolyshkin

comment created time in 2 months

pull request commentmoby/moby

logger/gelf: use compression level 0 by default

OK, some more testing showed that using gelf-compression-level: 0 is not making things better, dockerd CPU usage stays about the same, and the CPU profiles collected with pprof does not differ much from each other. Changing gelf-compression-type to none changes things dramatically (as described in earlier comments).

Apparently that happens since most of the CPU time is spent not on compression itself, but rather go runtime (allocation and garbage collection). Looks like gzipping a lot of small data objects is inefficient.

Given the fact that the support for uncompressed input has only made its way to logstash very recently (see https://github.com/moby/moby/pull/40101#issuecomment-553047340), we can't change the default to be gelf-compression-type: none without the risk of breaking users.

Seems like the only option left is to recommend disabling gelf compression in docs.

kolyshkin

comment created time in 2 months

pull request commentmoby/moby

logger/gelf: use compression level 0 by default

Took a while to figure out how to test it. Here's how:

LOGSTASH_VERSION=7.3.1
docker run --rm -it -p 127.0.0.1:12201:12201/udp docker.elastic.co/logstash/logstash:${LOGSTASH_VERSION} bin/logstash -e 'input { gelf {} }'
# in another terminal window
docker run --rm --log-driver=gelf --log-opt=gelf-address=udp://127.0.0.1:12201 --log-opt=gelf-compression-type=gzip --log-opt=gelf-compression-level=0 alpine echo hahaha

Using the above test, found out that

  • gelf-compression-type=none is not working with logstash < 7.4.0 (7.3.1 was tested)
  • gelf-compression-type=none works with logstash >= 7.4.0 (7.4.2 was tested)
  • gelf-compression-type=gzip --log-opt=gelf-compression-level=0 works with all versions (7.3.1 and 7.4.2 were tested)
kolyshkin

comment created time in 2 months

pull request commentmoby/moby

logger/gelf: use compression level 0 by default

So, the "Compression: none" only works since logstash-input-gelf 3.3.0 (https://rubygems.org/gems/logstash-input-gelf/versions/3.3.0), which is included in logstash 7.4.0, released Oct 1, 2019 (https://www.elastic.co/guide/en/logstash/7.4/logstash-7-4-0.html).

This means we can't disable compression now since I'm afraid many users still haven't updated logstash to >= 7.4.

Now we need to check whether setting "compression-level: 0" works with older logstash

kolyshkin

comment created time in 2 months

pull request commentlogstash-plugins/logstash-input-gelf

Update gelfd dependecy to allow uncompressed input

This made its way to logstash 7.4, released Oct 1, 2019 https://www.elastic.co/guide/en/logstash/7.4/logstash-7-4-0.html

ptqa

comment created time in 2 months

issue commentlogstash-plugins/logstash-input-gelf

I suggest to append some note in Gelf input plugin documentation: only UDP + gzipped message support

Information about plugin only accepting gzipped input was added in https://github.com/logstash-plugins/logstash-input-gelf/pull/28

The fix to this input plugin to accept uncompressed input was merged in #48, so it is fixed in 3.3.0 release: https://github.com/logstash-plugins/logstash-input-gelf/commit/76c111f32160c5c92c6f440d7ac5f73af68e57f7

This issue can be closed now.

harobed

comment created time in 2 months

pull request commentmoby/moby

Adding the ability to configure default capabilities

@burnMyDread I think you need to rebase this on top of the current master first.

burnMyDread

comment created time in 2 months

pull request commentmoby/moby

Check for OS Type and skip within the test

Hey @vikramhh, can you please add some prefix to this PR (and maybe commit as well). The current subject ("Check for OS Type and skip within the test") looks way too generic.

I suggest volume plugin tests: or something like that

vikramhh

comment created time in 2 months

pull request commentmoby/moby

Move package versions to ARGs in Dockerfile

Left a few comments. The last commit is marked as WIP, so marking the whole PR as WIP for now.

thaJeztah

comment created time in 2 months

Pull request review commentmoby/moby

Move package versions to ARGs in Dockerfile

 RUN --mount=type=cache,target=/root/.cache/go-build \         && GOPATH="$GOPATH/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH" \            go build -buildmode=pie -o /build/registry-v2 github.com/docker/distribution/cmd/registry \         && case $(dpkg --print-architecture) in \-               amd64|ppc64*|s390x) \+               amd64|armhf|ppc64*|s390x) \

I can't understand why this is change in this commit

thaJeztah

comment created time in 2 months

Pull request review commentmoby/moby

Move package versions to ARGs in Dockerfile

 #!/bin/sh -: ${VNDR_COMMIT:=f5ab8fc5fb64d66b5c6e55a0bcb58b2e92362fa0}- install_vndr() {+	: "${VNDR_COMMIT?}"+	: "${PREFIX?}"+ 	echo "Install vndr version $VNDR_COMMIT" 	git clone https://github.com/LK4D4/vndr.git "$GOPATH/src/github.com/LK4D4/vndr"-	cd "$GOPATH/src/github.com/LK4D4/vndr"+	cd "$GOPATH/src/github.com/LK4D4/vndr" || return

There's no need to have this || return, since we do set -e in hack/dockerfile/install/install.sh.

Same in other places.

thaJeztah

comment created time in 2 months

pull request commentmoby/moby

logger/gelf: use no compression by default

@ingshtrom the alternative is to set compression level to 0. In this case, no compression is performed but the data is still wrapped in gzip format.

Is it possible that you test that change?

kolyshkin

comment created time in 2 months

push eventkolyshkin/moby

Hannes Ljungberg

commit sha 4d09fab232ed282d020afbe1e0935b53379df4ad

Update service networks documentation The previous description stated that an array of names / ids could be passed when the API in reality expects objects in the form of NetworkAttachmentConfig. This is fixed by updating the description and adding a definition for NetworkAttachmentConfig. Signed-off-by: Hannes Ljungberg <hannes@5monkeys.se>

view details

Daniel Black

commit sha 7b4b940470ee34c96bf434b810e4cd5ca2e68182

/containers/{id}/json missing Platform To match ContainerJSONBase api/types/types.go Signed-off-by: Daniel Black <daniel@linux.ibm.com>

view details

Sebastiaan van Stijn

commit sha 6756f5f378d0f4f9efbda50fabb5bfdef2e5c4a7

API: update docs that /session left experimental in V1.39 The `/session` endpoint left experimental in API V1.39 through 239047c2d36706f2826b0a9bc115e0a08b1c3d27 and 01c9e7082eba71cbe60ce2e47acb9aad2c83c7ef, but the API reference was not updated accordingly. This updates the API documentation to match the change. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Tibor Vass

commit sha fbdd437d295595e88466b33a550a8707b9ebb709

daemon/config: fix filter type in BuildKit GC config For backwards compatibility, the old incorrect object format for builder.GC.Rule.Filter still works but is deprecated in favor of array of strings akin to what needs to be passed on the CLI. Signed-off-by: Tibor Vass <tibor@docker.com>

view details

Tibor Vass

commit sha 85733620ebea3da75abe7d732043354aa0883f8a

daemon/config: add MarshalJSON for future proofing If anything marshals the daemon config now or in the future this commit ensures the correct canonical form for the builder GC policies' filters. Signed-off-by: Tibor Vass <tibor@docker.com>

view details

Justen Martin

commit sha 3b49bd1d840d64ec603333eae28655b9ff5edc0c

replaced call to deprecated grpc method WithDialer with WithContextDialer Signed-off-by: Justen Martin <jmart@the-coder.com>

view details

Brian Goff

commit sha bef73d8b0721ae60dfc6ab6875328ffa9adbda49

Wait for c8d process exit instead of polling API In the containerd supervisor, instead of polling the healthcheck API every 500 milliseconds we can just wait for the process to exit. Signed-off-by: Brian Goff <cpuguy83@gmail.com>

view details

Brian Goff

commit sha 1e000435e60da678f3cb44ce4e5153d70328742c

Merge pull request #40096 from cpuguy83/c8d_no_healthcheck_loop Wait for c8d process exit instead of polling API

view details

Sebastiaan van Stijn

commit sha 05469b5fa2b48cf20cd0137ca8c45645b63049ff

daemon: add "isWindows" const Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 301a2fbeca2a4285335946c4a914b05f71cbb978

builder/dockerfile/mockbackend_test.go: suppress SA9005 (staticcheck) ``` builder/dockerfile/mockbackend_test.go:107:21: SA9005: struct doesn't have any exported fields, nor custom marshaling (staticcheck) return json.Marshal(rawImage(*i)) ^ ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 94647b5d8609b28ba807ec41b3ed198838dcaecf

graphdriver/aufs: SA4021: x = append(y) is equivalent to x = y (staticcheck) ``` daemon/graphdriver/aufs/aufs_test.go:746:8: SA4021: x = append(y) is equivalent to x = y (staticcheck) ids = append(ids[2:]) ^ ``` Also pre-allocating the ids slice while we're at it. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha ec1fd4b1b0401fad3d03654c16057712aff34e29

distribution: SA4021: x = append(y) is equivalent to x = y (staticcheck) ``` distribution/push_v2_test.go:552:29: SA4021: x = append(y) is equivalent to x = y (staticcheck) return nil, errcode.Errors(append([]error{errcode.ErrorCodeUnauthorized.WithMessage("unauthorized")})) ^ ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 5f47cef514acba3d0fa0856064057d4c7f218c31

fix nolint comments for SA1019: filters.ToParamWithVersion is deprecated The old nolint comment didn't seem to work anymore; ``` client/container_list.go:39:22: SA1019: filters.ToParamWithVersion is deprecated: do not use in any new code; use ToJSON instead (staticcheck) client/events.go:94:22: SA1019: filters.ToParamWithVersion is deprecated: do not use in any new code; use ToJSON instead (staticcheck) client/image_list.go:28:22: SA1019: filters.ToParamWithVersion is deprecated: do not use in any new code; use ToJSON instead (staticcheck) ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 1f7beb85949c4c31b3b5874510531021d5a2b45b

daemon/events/testutils: remove redundant variable Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 04fcb6cfbf0d3419891af82e26d963f22b248fa4

pkg/jsonmessage: fix SA1006: printf-style function with no arguments Also fixed some incorrectly formatted comments ``` pkg/jsonmessage/jsonmessage.go:180:20: SA1006: printf-style function with dynamic format string and no further arguments should use print-style function instead (staticcheck) fmt.Fprintf(out, endl) ^ ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha cba180cac9de350dc5cf9ab9036401e6f44ff339

graphdriver/btrfs: SA4003: no value of type uint64 is less than 0 (staticcheck) ``` daemon/graphdriver/btrfs/btrfs.go:609:5: SA4003: no value of type uint64 is less than 0 (staticcheck) if driver.options.size <= 0 { ^ ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 6d9c219c549ba9e6b48e945dce3c9e4e64214850

daemon: S1033: unnecessary guard around call to delete (gosimple) ``` daemon/container_operations.go:787:2: S1033: unnecessary guard around call to delete (gosimple) if _, ok := container.NetworkSettings.Networks[n.ID()]; ok { ^ ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha af3bbcc00ceb0197bd22c46666affcf1a9724479

aufs: SA4011: did you mean to break out of the outer loop? (staticcheck) As caught by staticcheck (after disabling the default exclusion rules); Based on the comment, this break was indeed meant to break the loop and return the error. ``` daemon/graphdriver/aufs/mount.go:54:4: SA4011: ineffective break statement. Did you mean to break out of the outer loop? (staticcheck) break ^ ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 4840fd895328da757bd67b6be212f86cae8f93eb

pkg/mount: SA4011: ineffective break statement (staticcheck) ``` pkg/mount/mountinfo_linux.go:93:5: SA4011: ineffective break statement. Did you mean to break out of the outer loop? (staticcheck) break ^ ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 47502344b9aab6919f0b426e037b68e899735abd

golangci-lint: update exclusion rules for todo's Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

push time in 2 months

pull request commentmoby/moby

Update README.md

@SREELAKSHMI-NAYATH can you please sign your commit?

SREELAKSHMI-NAYATH

comment created time in 2 months

pull request commentmoby/moby

Bump Golang 1.13.4

Rerun CI since it failed earlier because the golang image was not available at the time

tao12345666333

comment created time in 2 months

issue commentmoby/moby

Flaky test: TestCreateServiceConfigFileMode

Got a different failure on s390 (https://github.com/moby/moby/pull/40194)

[2019-11-08T20:17:13.312Z] === Failed [2019-11-08T20:17:13.312Z] === FAIL: s390x.integration.service TestCreateServiceConfigFileMode (8.89s) [2019-11-08T20:17:13.312Z] create_test.go:329: assertion failed: 2 (int) != 1 (int)

kolyshkin

comment created time in 2 months

pull request commentmoby/moby

overlay[2] graphdriver: Fix/improve overlayfs support check for rootless

fixed bad import statement (my system still had old github.com/Sirupsen/logrus)

kolyshkin

comment created time in 2 months

push eventkolyshkin/moby

Yong Tang

commit sha f09dc2f4fc68c0e622797404763b757739b79aaa

Fix docker crash when creating namespaces with UID in /etc/subuid and /etc/subgid This fix tries to address the issue raised in 39353 where docker crash when creating namespaces with UID in /etc/subuid and /etc/subgid. The issue was that, mapping to `/etc/sub[u,g]id` in docker does not allow numeric ID. This fix fixes the issue by probing other combinations (uid:groupname, username:gid, uid:gid) when normal username:groupname fails. This fix fixes 39353. Signed-off-by: Yong Tang <yong.tang.github@outlook.com>

view details

Sebastiaan van Stijn

commit sha 9cf349d0f80d2399cdfad0321e0f181c2e7efa17

bump libnetwork 90afbb01e1d8acacb505a092744ea42b9f167377 full diff: https://github.com/docker/libnetwork/compare/0025177e3dabbe0de151be0957dcaff149d43536...90afbb01e1d8acacb505a092744ea42b9f167377 includes: - docker/libnetwork#/2459 Fix Error Check in NewNetwork - docker/libnetwork#/2466 Revert "Merge pull request #2339 from phyber/iptables-check" - reverts docker/libnetwork#/2339 controller: Check if IPTables is enabled for arrangeUserFilterRule - re-opens docker/libnetwork#2158 dockerd when run with --iptables=false modifies iptables by adding DOCKER-USER - re-opens moby/moby#35777 With iptables=false dockerd still creates DOCKER-USER chain and rules - re-opens docker/for-linux#136 dockerd --iptables=false adds DOCKER-USER chain and modify FORWARD chain anyway Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 1a88e0255496ca8a5ffa70e845da43381c7fc8ea

Merge pull request #39764 from yongtang/39353-subgid-subuid Fix docker crash when creating namespaces with UID in /etc/subuid and /etc/subgid

view details

Tibor Vass

commit sha 36ffe9edc2b37a5154633f3fbc260217114039d4

Merge pull request #40192 from thaJeztah/bump_libnetwork bump libnetwork 90afbb01e1d8acacb505a092744ea42b9f167377

view details

Kir Kolyshkin

commit sha d5687079ad8ad27c467ef5c8758a73c519b45d9b

overlay: move supportsMultipleLowerDir to utils This moves supportsMultipleLowerDir() to overlayutils so it can be used from both overlay and overlay2. The only changes made were: * replace logger with logrus * don't use workDirName mergedDirName constants * add mnt var to improve readability a bit This is a preparation for the next commit. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

Kir Kolyshkin

commit sha 649e4c88899878c9cdf9036f6bc7d62e2b39c04b

Fix/improve overlay support check Before this commit, overlay check was performed by looking for `overlay` in /proc/filesystem. This obviously might not work for rootless Docker (fs is there, but one can't use it as non-root). This commit changes the check to perform the actual mount, by reusing the code previously written to check for multiple lower dirs support. The old check is removed from both drivers, as well as the additional check for the multiple lower dirs support in overlay2 since it's now a part of the main check. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

push time in 2 months

pull request commentmoby/moby

Improve rootless Docker overlay support detection

There is already code like this in overlay2, please see #35527

I suggest to reuse that code instead. We can add an option to try multiple lower dirs, move the code into overlayutils, and use it from both overlay and overlay2.

Here it is: https://github.com/moby/moby/pull/40194

Caligatio

comment created time in 2 months

pull request commentmoby/moby

overlay[2] graphdriver: Fix/improve overlayfs support check for rootless

@Caligatio could you test this?

@thaJeztah @dmcgowan PTAL

kolyshkin

comment created time in 2 months

PR opened moby/moby

Reviewers
overlay[2] graphdriver: Fix/improve overlayfs support check for rootless

Inspired by https://github.com/moby/moby/pull/40131

Overlay check is performed by looking for overlay in /proc/filesystem. This obviously might not work for rootless Docker (fs is there, but one can't use it as non-root, for example, see https://github.com/docker-library/docker/issues/193).

This PR changes the check to perform the actual mount, by reusing the code previously written to check for multiple lower dirs support. The old check is removed from both drivers, as well as the additional check for the multiple lower dirs support in overlay2 since it's now a part of the main check.

The PR is split into two commits for the sake of easier review.

  • First commit moves the supportsMultipleLowerDir to overlayutils with minimal modifications
  • Second commit renames it to SupportsOverlay(), makes the multiple lower dir check optional, and makes both overlay and overlay2 use the new check.

PS nice LOC reduction:

 daemon/graphdriver/overlay/overlay.go           | 37 +++++--------------------------------
 daemon/graphdriver/overlay2/check.go            | 32 --------------------------------
 daemon/graphdriver/overlay2/overlay.go          | 47 +++++------------------------------------------
 daemon/graphdriver/overlayutils/overlayutils.go | 44 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 54 insertions(+), 106 deletions(-)

fixes https://github.com/docker/for-linux/issues/836

+54 -106

0 comment

4 changed files

pr created time in 2 months

push eventkolyshkin/moby

lzhfromustc

commit sha 49fbb9c9854ff18ad9304f435c7c6722b0b4cfdb

registry: add a critical section to protect authTransport.modReq Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>

view details

Ziheng Liu

commit sha 6233217a31395b69aa814c7d3db5cf844eb87437

integration/internal/container: fix a goroutine leak bug by adding 1 buffer Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>

view details

Kir Kolyshkin

commit sha 9d4e81e8bf0d52a063c46a3dc826f7e85068b07d

hack/validate/vendor: print diff for modified files In case some files were modified (rather than merely added or removed), we're curious to see the diff for those. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

Kir Kolyshkin

commit sha 4be12ad3d04aefe6d5822d426813b33d2d4f9a7e

hack/validate/vendor: shellcheck fixes The export statement is definitely not needed. The rest is obvious. > In hack/validate/vendor line 3: > export SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" > ^-- SC2155: Declare and assign separately to avoid masking return values. > > > In hack/validate/vendor line 43: > if ls -d vendor/$f > /dev/null 2>&1; then > ^-- SC2086: Double quote to prevent globbing and word splitting. > > > In hack/validate/vendor line 44: > found=$(find vendor/$f -iregex '.*LICENSE.*' -or -iregex '.*COPYRIGHT.*' -or -iregex '.*COPYING.*' | wc -l) > ^-- SC2086: Double quote to prevent globbing and word splitting. > > > In hack/validate/vendor line 45: > if [ $found -eq 0 ]; then > ^-- SC2086: Double quote to prevent globbing and word splitting. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

Kir Kolyshkin

commit sha 57910190288c71242d914c292930a496d05f30cb

hack/validate/vendor: simplify looking for license It was suggested that we use '.*\(COPYING\|LICENSE\|COPYRIGHT\).*' as an argument to `find -iregex`, and this is how it all started. Next thing, there is no COPYRIGHT in any of the vendored packages, so it can be removed for good. Next, we should not look too deep inside the package directory, as the license should be in its root directory, so add `-maxdepth 1` to `find`. This should also speed things up. Finally, since we're not using the recursion feature of `find`, it can be replaced with `echo | grep`. While at it, * avoid temporary $pkgs variable as it is only used once; * replace `ls -d "vendor/$f" > /dev/null 2>&1` with `test -d`. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

Sam Whited

commit sha b96a0c775400821d80972619fbfe6a2070f3e9ba

Add daemon options required by buildkit tests Signed-off-by: Sam Whited <sam@samwhited.com>

view details

Sebastiaan van Stijn

commit sha 31abc6c089eb5acc8161f480335b33b12564a565

Merge pull request #40177 from SamWhited/buildkit_test_options Add daemon options required by buildkit tests

view details

Sam Whited

commit sha 0c9b232bf5263ab896637b394308510c4cfbd45d

Remove unused GlobalFlags Signed-off-by: Sam Whited <sam@samwhited.com>

view details

Akihiro Suda

commit sha 65523469c7e6f100230ba500c1d28516ea6bd384

Merge pull request #40187 from SamWhited/remove_global_args Remove unused GlobalFlags

view details

Kirill Kolyshkin

commit sha 154cf042fdc35801d280ae2d67128cdcd561b6a2

Merge pull request #40144 from lzhfromustc/GL_outputDone integration/internal/container: fix a goroutine leak bug

view details

Tõnis Tiigi

commit sha fee149e723dff096cb77cfa28f0eabc7b3830990

Merge pull request #40143 from lzhfromustc/IFP_modReq registry: add a critical section to protect authTransport.modReq

view details

Sebastiaan van Stijn

commit sha e9bd017b680cf3eb05d8db21500979ac22969658

Merge pull request #40148 from kolyshkin/vendor-diff hack/validate/vendor: print diff for modified files

view details

Kir Kolyshkin

commit sha 6b40f7c2406260e189749c0201f1786dc59e80e5

overlay: move supportsMultipleLowerDir to utils This moves supportsMultipleLowerDir() to overlayutils so it can be used from both overlay and overlay2. The only changes made were: * replace logger with logrus * don't use workDirName mergedDirName constants * add mnt var to improve readability a bit This is a preparation for the next commit. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

Kir Kolyshkin

commit sha 8c3a366edc1412676a56e48f808b56ce8e64cb2f

Fix/improve overlay support check Before this commit, overlay check was performed by looking for `overlay` in /proc/filesystem. This obviously might not work for rootless Docker (fs is there, but one can't use it as non-root). This commit changes the check to perform the actual mount, by reusing the code previously written to check for multiple lower dirs support. The old check is removed from both drivers, as well as the additional check for the multiple lower dirs support in overlay2 since it's now a part of the main check. Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

view details

push time in 2 months

create barnchkolyshkin/moby

branch : rootless-overlay

created branch time in 2 months

pull request commentmoby/moby

Improve rootless Docker overlay support detection

@Caligatio do you want to work on that ^^^^^?

Caligatio

comment created time in 2 months

pull request commentmoby/moby

Improve rootless Docker overlay support detection

There is already code like this in overlay2, please see https://github.com/moby/moby/pull/35527

I suggest to reuse that code instead. We can add an option to try multiple lower dirs, move the code into overlayutils, and use it from both overlay and overlay2.

Caligatio

comment created time in 2 months

push eventmoby/moby

Ziheng Liu

commit sha 6233217a31395b69aa814c7d3db5cf844eb87437

integration/internal/container: fix a goroutine leak bug by adding 1 buffer Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>

view details

Kirill Kolyshkin

commit sha 154cf042fdc35801d280ae2d67128cdcd561b6a2

Merge pull request #40144 from lzhfromustc/GL_outputDone integration/internal/container: fix a goroutine leak bug

view details

push time in 2 months

PR merged moby/moby

integration/internal/container: fix a goroutine leak bug area/testing kind/bugfix status/2-code-review

- What I did Avoid a goroutine leak bug by changing an unbuffered channel into a channel with 1 buffer. In the code below, if select chooses case <-ctx.Done(), then the child goroutine will be blocked at outputDone <- err and leaked. https://github.com/moby/moby/blob/a09e6e323e55e1a9b21df9c2c555f5668df3ac9b/integration/internal/container/exec.go#L60-L77

- How to verify it The change in this patch is very safe: the case err := <-outputDone in the parent goroutine will still be blocked until outputDone <- err is executed, but outputDone <- err will never be blocked. This fix is similar to https://github.com/kubernetes/kubernetes/pull/5316

- Description for the changelog NONE

+1 -1

0 comment

1 changed file

lzhfromustc

pr closed time in 2 months

pull request commentmoby/moby

registry: add a critical section to protect authTransport.modReq

@tiborvass PTAL

lzhfromustc

comment created time in 2 months

pull request commentmoby/moby

refactored integration/service/instead_test.go to ues unique resource…

@jmartin84 can you please sign your commit? Something like

git commit --amend -s
git push -f jmartin84 unique-names-intergration-service-inspect-test

should do it

jmartin84

comment created time in 2 months

pull request commentmoby/moby

logger/gelf: use no compression by default

Hmm, actually, compress/flate is only used by gelf for compression level constants; for the actual compression, either compress/gzip or compress/zlib is used. Default is gzip, and gzip apparently uses flate, and gelf sets the level to 1 by default, so as far as I understand this fast flate algo is actually used.

Even with all this, it still takes lots of CPU.

So, I think the best course of action would be to disable compression.

kolyshkin

comment created time in 2 months

pull request commentmoby/moby

logger/gelf: use no compression by default

Well, default compression is level 6 and not level 1:

case level == DefaultCompression:
	level = 6

I also found out that pkg/deflate has a separate "fast" implementation that is used for level==1.

So, we should at least set the default to be level 1.

kolyshkin

comment created time in 2 months

pull request commentdocker/cli

Allow the external CAs to be removed entirely using the CLI

Perhaps --no-external-ca or --remove-external-ca?

cyli

comment created time in 2 months

push eventmoby/moby

Sebastiaan van Stijn

commit sha 3df4f86f21fbcae3535e2231828dce16a1940dbb

swagger: fix "generated code" comment not in correct format As described in https://golang.org/s/generatedcode, Go has a formalized format that should be used to indicate that a file is generated. Matching that format helps linters to skip generated files; From https://golang.org/s/generatedcode (https://github.com/golang/go/issues/13560#issuecomment-288457920); > Generated files are marked by a line of text that matches the regular expression, in Go syntax: > > ^// Code generated .* DO NOT EDIT\.$ > > The `.*` means the tool can put whatever folderol it wants in there, but the comment > must be a single line and must start with `Code generated` and end with `DO NOT EDIT.`, > with a period. > > The text may appear anywhere in the file. This patch updates the template used for our generated types to match that format. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha c511db70ed39f344f41ea8773cc9264a1eeddfda

api/types: re-generate with new template Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Sebastiaan van Stijn

commit sha 6186e9fe8794660d669f19a2e2ba7127321b817f

hack/make/.go-autogen: fix "generated code" comment not in correct format As described in https://golang.org/s/generatedcode, Go has a formalized format that should be used to indicate that a file is generated. Matching that format helps linters to skip generated files; From https://golang.org/s/generatedcode (https://github.com/golang/go/issues/13560#issuecomment-288457920); > Generated files are marked by a line of text that matches the regular expression, in Go syntax: > > ^// Code generated .* DO NOT EDIT\.$ > > The `.*` means the tool can put whatever folderol it wants in there, but the comment > must be a single line and must start with `Code generated` and end with `DO NOT EDIT.`, > with a period. > > The text may appear anywhere in the file. This patch updates the autogenerated code to match that format. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Kirill Kolyshkin

commit sha c36460c437c8c515c543dd31afcbb5c2a9f5dd48

Merge pull request #40077 from thaJeztah/fix_autogen_detection Update "auto-generate" comments to improve detection by linters

view details

push time in 3 months

PR merged moby/moby

Reviewers
Update "auto-generate" comments to improve detection by linters area/testing process/cherry-pick status/2-code-review

As described in https://golang.org/s/generatedcode, Go has a formalized format that should be used to indicate that a file is generated.

Matching that format helps linters to skip generated files;

From https://golang.org/s/generatedcode (https://github.com/golang/go/issues/13560#issuecomment-288457920);

Generated files are marked by a line of text that matches the regular expression, in Go syntax:

^// Code generated .* DO NOT EDIT.$

The .* means the tool can put whatever folderol it wants in there, but the comment must be a single line and must start with Code generated and end with DO NOT EDIT., with a period.

The text may appear anywhere in the file.

This patch updates the autogenerated code to match that format.

+11 -20

4 comments

10 changed files

thaJeztah

pr closed time in 3 months

push eventmoby/moby

Sebastiaan van Stijn

commit sha 9a7e96b5b7e97e034ce7bb0f1e7788d1bd881c7f

Rename "v1" to "statsV1" follow-up to 27552ceb15bca544820229e574427d4c1d6ef585, where this was left as a review comment, but the PR was already merged. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

view details

Kirill Kolyshkin

commit sha 76dbd884d3f1a02dc193305d2ac5824bcd3e4f0f

Merge pull request #40167 from thaJeztah/stats_alias Rename "v1" to "statsV1"

view details

push time in 3 months

PR merged moby/moby

Rename "v1" to "statsV1" area/runtime kind/refactor status/2-code-review

follow-up to 27552ceb15bca544820229e574427d4c1d6ef585 (https://github.com/moby/moby/pull/40154), where this was left as a review comment, but the PR was already merged.

+5 -5

3 comments

2 changed files

thaJeztah

pr closed time in 3 months

pull request commentmoby/moby

Rename "v1" to "statsV1"

hmm why not just stats? Mentioning repeatedly that we have indeed imported v1 of the package in question during every use of it seems kinda excessive (and reminds me of Hungarian notation in Win :cry:).

thaJeztah

comment created time in 3 months

push eventmoby/moby

Sam Whited

commit sha d6a91ca71c655f71c171e375b787c9c8b361c19e

Rename DCO check param in Jenkinsfile Previously it was a negative parameter for skiping the DCO check, but this is different from other checks. It was requested that I change this in #40023 but I'm factoring it out as an unrelated change. Signed-off-by: Sam Whited <sam@samwhited.com>

view details

Kirill Kolyshkin

commit sha 7cde98488c2cfd7c3bc5a4a9044047cdab596663

Merge pull request #40159 from SamWhited/jenkins_dco_var_name Rename DCO check param in Jenkinsfile

view details

push time in 3 months

PR merged moby/moby

Rename DCO check param in Jenkinsfile area/testing process/cherry-pick status/2-code-review

Previously it was a negative parameter for skiping the DCO check, but this is different from other checks. It was requested that I change this in #40023 by @thaJeztah but I'm factoring it out into a new PR to reduce the diff size since this is an unrelated change.

EDIT: left off the DCO to check that this works (and it does). Definitely not just because I forgot it as usual.

+4 -4

2 comments

1 changed file

SamWhited

pr closed time in 3 months

issue commentmoby/moby

Dockerd eats too much RAM

@dperny PTAL ^^^

goetas

comment created time in 3 months

issue openeddocker/docker.github.io

config/containers/runmetrics: rm lxc reference

File: config/containers/runmetrics.md

Looks like this doc is referring to LXC which is for quite some time is not part of docker. I think such references are to be removed.

created time in 3 months

issue commentmoby/moby

Dockerd eats too much RAM

Here's the graph (generated from the 002 profile, they look similar). Unfortunately I don't know this codebase good enough to figure out what is going on, but it is clear from the pic who is using the most memory. Yet, it's about 3.5GB rather than 11GB (I guess the rest is not garbage collected yet?)

image

goetas

comment created time in 3 months

issue closedmoby/moby

Abnormal RAM consumption by dockerd when using fluentd

Description

When you start a simple container that cycles logs and sends them to fluentd, dockerd starts consuming a lot of RAM. Consumption occurs before buffer overflow, as seen in the dockerd logs.

Based on the #1 and #2 sources, the default fluentd buffer size is 1 MB, but the 'dockerd' consumption grows to 900 MB. If you limit the buffer to, for example, 512 KB (by setting --log-opt fluentd-buffer-limit=512kb), then dockerd grows to 530 MB.

The fluentd buffer size may simply be specified in kilobytes rather than bytes. The unit of measure, unfortunately, is not specified in the logs:

error="fluent#appendBuffer: Buffer full, limit 1048576"

Steps to reproduce the issue:

  1. Simply run td-agent (fluentd) on localhost and next container:
id=$(
  sudo \
    docker run \
      --rm \
      --detach \
      --log-driver fluentd \
      --log-opt mode=non-blocking \
      --log-opt fluentd-address=localhost:24224 \
      --log-opt fluentd-async-connect=true \
      --log-opt fluentd-sub-second-precision=true \
      alpine \
      /usr/bin/yes crashme
)

sleep 30
sudo docker kill ${id}

Describe the results you received:

See at screencast:

Please note that dockerd does not release RAM after the container is stopped.

github

Describe the results you expected:

dockerd should not consume so much RAM

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:29:11 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:27:45 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 19.03.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.15.0-1051-aws
 Operating System: Ubuntu 18.04.1 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 1.91GiB
 Name: test
 ID: XYTA:N6SY:IYSU:ANM6:LQRH:AN75:MPS6:TKUK:LGB5:WQBU:NGCR:J775
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

t3.small instance on AWS.

closed time in 3 months

abra7134

issue commentmoby/moby

Abnormal RAM consumption by dockerd when using fluentd

Unable to repro, no reply from the reporter, closing.

@abra7134 feel free to reopen!

abra7134

comment created time in 3 months

pull request commentdocker/cli

Stack: Support cap_add, cap_drop and privileged on services

@olljanat this needs a rebase

olljanat

comment created time in 3 months

Pull request review commentmoby/moby

bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a

 package types // import "github.com/docker/docker/libcontainerd/types" import ( 	"time" -	"github.com/containerd/cgroups"+	v1 "github.com/containerd/cgroups/stats/v1"

ditto

thaJeztah

comment created time in 3 months

Pull request review commentmoby/moby

bump containerd/cgroups 5fbad35c2a7e855762d3c60f2e474ffcad0d470a

 import ( 	"strings" 	"time" -	containerd_cgroups "github.com/containerd/cgroups"+	v1 "github.com/containerd/cgroups/stats/v1"

this is kinda weird to call this v1. maybe just cgroups?

thaJeztah

comment created time in 3 months

Pull request review commentdocker/cli

kubernetes/conversion_test: use test-builders package

 func withPort(port swarm.PortConfig) func(*swarm.Service) { 	} } -func withStatus(running, desired uint64) func(*swarm.Service) {-	return func(service *swarm.Service) {-		service.ServiceStatus = &swarm.ServiceStatus{-			RunningTasks: running,-			DesiredTasks: desired,-		}-	}-}--func makeSwarmService(t *testing.T, service, id string, opts ...func(*swarm.Service)) swarm.Service {+func makeSwarmService(t *testing.T, name, id string, opts ...func(*swarm.Service)) swarm.Service { 	t.Helper()-	s := swarm.Service{-		ID: id,-		Spec: swarm.ServiceSpec{-			Annotations: swarm.Annotations{-				Name: service,-			},-			TaskTemplate: swarm.TaskSpec{-				ContainerSpec: &swarm.ContainerSpec{-					Image: "image",-				},-			},-		},-	}-	for _, o := range opts {-		o(&s)-	}-	return s+	options := append([]func(*swarm.Service){}, ServiceID(id), ServiceName(name), ServiceImage("image"))

Appending to an empty slice does not make sense to me. Easier to write as

options := []func(*swarm.Service){ServiceID(id), ServiceName(name), ServiceImage("image")}
thaJeztah

comment created time in 3 months

Pull request review commentdocker/cli

kubernetes/conversion_test: use test-builders package

 import ( // Any number of service builder functions can be passed to augment it. // Currently, only ServiceName is implemented func Service(builders ...func(*swarm.Service)) *swarm.Service {-	service := &swarm.Service{-		ID: "serviceID",-		Spec: swarm.ServiceSpec{-			Annotations: swarm.Annotations{-				Name: "defaultServiceName",-			},-			EndpointSpec: &swarm.EndpointSpec{},-		},-	}+	service := &swarm.Service{}+	ServiceID("serviceID")+	ServiceName("defaultServiceName")

I don't understand this part. You call a function that returns a function and then you discard its result, meaning you never call the function returned.

Should be something like

ServiceID("serviceID")(service)
ServiceName("defaultServiceName")(service)
thaJeztah

comment created time in 3 months

Pull request review commentdocker/cli

kubernetes/conversion_test: use test-builders package

 import ( // Any number of service builder functions can be passed to augment it. // Currently, only ServiceName is implemented

Not really in scope of this PR, but this comment line seems obsoleted. Since commit dea478b851 which added it (and the mentioned ServiceName()), we got 535af2d868 which adds ServicePort(), ServiceImage() etc.

thaJeztah

comment created time in 3 months

issue commentmoby/moby

Abnormal RAM consumption by dockerd when using fluentd

I was unable to repro using the above repro steps (with docker-ce 19.03 and td-agent 3.5.0 on a CentOS 7).

What I did is:

wget http://packages.treasuredata.com.s3.amazonaws.com/3/redhat/7/x86_64/td-agent-3.5.0-0.el7.x86_64.rpm
yum install td-agent-3.5.0-0.el7.x86_64.rpm
systemctl start td-agent
systemctl status td-agent # make sure it's running
ss -lnp4 # make sure fluentd port is 24224

and then ran the above repro. I can see dockerd, ruby, and yes in top output and they all use some considerable CPU time, but I don't see any noticeable RSS grow.

@abra7134 can you repro this on a clean instance? If yes, please provide detailed steps, maybe I'm missing something.

abra7134

comment created time in 3 months

pull request commentmoby/moby

hack/validate/vendor: print diff for modified files

Should we add NOTICE? IIUC, the notice is the actual copyright/licensing information, and having a license file is merely for convenience

  1. Currently, we don't have any packages that fail validation (i.e. they all either container COPYING or LICENSE in some way).

  2. Let's have a look,

$ grep -vEc '^$|^#' vendor.conf 
118
$ find vendor -name NOTICE\* | wc -l
13

...out of 118 packages that moby/moby currently vendors, only 13 packages contain NOTICE, and about half of those NOTICEs only contain a copyright statement but not any licensing information. I guess the aim of this check we are actually checking for licensing terms (to make sure it's compatible with moby's own licensing terms) not copyright.

kolyshkin

comment created time in 3 months

pull request commentmoby/moby

hack/validate/vendor: print diff for modified files

@thaJeztah first two commits remain the same (so you won't have to re-review), the only change is I added the third one with validate_vendor_used as you suggested.

kolyshkin

comment created time in 3 months

more