Hi,

I am using your API with Node.JS and I tried following the createContainer example:

// Trialling both dockerode-promise-wrapper and dockerode
const DockerCallback = require('dockerode');
const dockerOrigin = new DockerCallback();
dockerOrigin.createContainer({Image: 'ubuntu', Cmd: ['/bin/bash'], name: 'ubuntu-test'}, function (err, container) {
	console.error(err);
	console.log(container);
});

But I get the following error:

Error: (HTTP code 404) no such container - No such image: ubuntu:latest 
    at <project_dir>/node_modules/docker-modem/lib/modem.js:296:17
    at getCause (<project_dir>/node_modules/docker-modem/lib/modem.js:326:7)
    at Modem.buildPayload (<project_dir>/node_modules/docker-modem/lib/modem.js:295:5)
    at IncomingMessage.<anonymous> (<project_dir>/node_modules/docker-modem/lib/modem.js:270:14)
    at IncomingMessage.emit (events.js:208:15)
    at endReadableNT (_stream_readable.js:1168:12)
    at processTicksAndRejections (internal/process/task_queues.js:77:11) {
  reason: 'no such container',
  statusCode: 404,
  json: { message: 'No such image: ubuntu:latest' }
}

Docker Version:

Client: Docker Engine - Community
 Version:           19.03.4
 API version:       1.40
 Go version:        go1.12.10
 Git commit:        9013bf5
 Built:             Thu Oct 17 23:44:48 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.4
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.10
  Git commit:       9013bf5
  Built:            Thu Oct 17 23:50:38 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Node Version: v12.8.0 NPM Version: 6.13.0

And then I am using macOS Mojave, Version 10.14.5

Thanks, Danny

Do you have the ubuntu image?

Pull the image before running that code: docker pull ubuntu

Published in v2.0.4

Going to publish it today.

On Fri, 1 Nov 2019 at 14:33, Brandon Waterloo [MSFT] < [email protected]> wrote:

@apocas https://github.com/apocas We're getting close to releasing our next extension version of vscode-docker, probably soon after VSCode 1.40 is out. Can you release the next version of docker-modem?

Thanks in advance, I really appreciate it!

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/apocas/docker-modem/pull/109?email_source=notifications&email_token=AAER3JLFJS7XTJDIJLCSZXDQRQ44VA5CNFSM4JB6YQGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC3C4AI#issuecomment-548810241, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAER3JJFAFGHNHUIUOSPN4DQRQ44VANCNFSM4JB6YQGA .

-- Pedro Dias

There must be an error, how did you tried to catch the error? You need to catch it on the start.

You please print and check the start error? If no container id was returned, promise was rejected and there will an error.

Is it only happening on v3.x?

Could be related to this: https://github.com/apocas/dockerode/pull/490

Could you give more details about this? How so "too large"? Larger than what?

Yeah, I completely missed that too. 👍

Resolves #107

• Allow SSH DOCKER_HOST URLs
• Accept username in DOCKER_HOST, i.e. ssh://user@host
• Use SSH_AUTH_SOCK env var to determine auth agent (Docker CLI also requires ssh-agent to be set up)

Due to a combination of defaultOpts (https://github.com/apocas/docker-modem/blob/master/lib/modem.js#L16) and this line, any value passed in as the user in DOCKER_HOST would be ignored. I believe the behavior would be that the current user is used. There's a high chance that won't work, however--if the local username is not the same as the SSH username.

Ideally we should alter defaultOpts to permit things like ssh://user@host

👍

You should be able to pass an "agent" to the Modem. https://github.com/apocas/docker-modem/blob/master/lib/modem.js#L81

I simplified it, by using the already existing "host" and "port" from Modem's constructor. https://github.com/apocas/docker-modem/blob/master/lib/modem.js#L72

But we can't just remove this condition because the are endpoints expecting the data "stringifed". Like https://github.com/apocas/dockerode/blob/master/test/docker.js#L658

On the other hand there are parameters that are expected by Docker to be repeated like "t" in imageBuild. https://docs.docker.com/engine/api/v1.40/#operation/ImageBuild "A name and optional tag to apply to the image in the name:tag format. If you omit the tag the default latest value is assumed. You can provide several t parameters."

If "extrahosts" is one of this cases we can add it to the exception list, like "t".

I'm on Windows 10 (18950) running WSL 2 with the Docker Desktop WSL 2 Technical Preview. The TP separates the Windows and WSL contexts to prevent destructive operations and Dockerode only accesses the Windows context. For example, on Windows, with WSL 2 enabled I see this (notice that wsl is active)

Running $ docker ps and $ docker images shows containers and images that are currently running within the WSL 2 context while Dockerode continues to use the default context and shows containers and images in the Windows context.

Is there a way to specify the context or use whatever is active in Docker Desktop?

https://github.com/apocas/docker-modem/pull/106

Thanks for this!

Just noticed a small typo. Here is a pr fixing that

👍

Passes: https://github.com/apocas/dockerode/blob/master/test/docker.js#L648 Fails: https://github.com/apocas/dockerode/blob/master/test/docker.js#L658

So we can't just remove the stringify. Add that parameter as an exception?

Missed that. Can't remember why that was done. I suspect that endpoints didn't behave the same way, regarding input.

Will check if removing the stringify doesn't break anything.

Just published docker-modem v2.0.2. It's available now, just update dockerode dependencies via npm :)

In Windows, DOCKER_HOST can contain things like npipe:////./pipe/docker_engine or npipe:////./pipe/docker_wsl.

Currently it isn't possible to use DOCKER_HOST = npipe:////./pipe/docker_wsl to use the WSL2 Docker engine; rather, you must pass in socketPath to dockerode. If you try using the above for DOCKER_HOST you'll get an error.

👍 Will publish it tonight.

I can't replicated this, there's even a test for this.

https://github.com/apocas/docker-modem/blob/master/test/modem_test.js#L109

´´´ var opts = { "limit": 12, "filters": { "label": ["staging", "env=green"] }, "t": ["repo:latest", "repo:1.0.0"] }; ´´´ limit=12&filters={"label":["staging","env=green"]}&t=repo:latest&t=repo:1.0.0

The array "t" was properly converted.

The regex parsing the DOCKER_HOST URI does not support IPv6 IP addresses: https://github.com/apocas/docker-modem/blob/99fcf0d10521f974dbe335a47a09ecf30ca6c1a2/lib/modem.js#L28

E.g., 'tcp://[::1]:12345' would split at the second colon instead of the last. You could use Node.js' 'url' module instead of a regex.

Published in v2.0.1

Just pushed a fix to master. Will publish it later tonight :)

Node.js's url module doesn't handle strings without a protocol specified. (which is used in a few use cases in Docker) When a protocol isn't specified, docker-modem defaults to tcp:// then it used Node.js URL.

Hi guys, dockerode developer here.

@Chuxel Do you want to contribute your hack to docker-modem? :)

https://github.com/Chuxel/docker-modem/commit/47e7a97a53b4a915e9a01d01e83421a0074b1e59

have an issue when trying to copy a file from my node server which is created from a string using 'fs' in a tmp dir but keep getting the same error.

Error processing tar file(exit status 1): unexpected EOF 0, json: null } [Node] at \node_modules\docker-modem\lib\modem.js:257:17

here is the code im using to create the docker container and then create and copy the file into the container.

fs.writeFile('./tmp/attempt.java', code, (async (err) => { console.log(err); if (err) { return } let container = await docker.createContainer({ Image: 'openjdk', Tty: true, StdinOnce: true }); await container.start(); try { await container.putArchive('./tmp/attempt.java', { path: '/' }); } catch (error) { console.log(error); } }));

Docker's putArchive expects a .tar file :)

https://docs.docker.com/engine/api/v1.40/#operation/PutContainerArchive

"The input stream must be a tar archive compressed with one of the following algorithms: identity (no compression), gzip, bzip2, xz."

Title says it all I think... couldn't find anything about it in the docs. Thanks

See https://snyk.io/test/github/apocas/dockerode

The vulnerability: https://snyk.io/vuln/npm:chownr:20180731

Nothing being reported on snyk atm.

Problem

This is the same as the issue here, on a pull the docker daemon responds to with an error: context cancelled. The issues seems to be a couple of things:

• the Content-Length is calculated using a data payload that is never sent in the request. Perhaps the query string should be used https://github.com/apocas/docker-modem/blob/master/lib/modem.js#L167 ?
• undefined opts are passed in as 'undefined' strings, eg: tag, leading to empty query params
• the Content-Type is set as application/json though no json payload is sent, maybe application/x-www-form-urlencoded or text/plain?

Here is an example failing request options, changing the type to text/plain or removing the Content-length fixes the issue.

 { path: '/images/create?fromImage=coinstac%2Fcoinstac-base&tag=',
   method: 'POST',
   headers: { 'Content-Type': 'application/json', 'Content-Length': 38 },
   key: undefined,
   cert: undefined,
   ca: undefined,
   socketPath: '//./pipe/docker_engine' }

Will close this one for now. Will reopen if more info is provided.

Interesting, will check this one.

Could you give more details? Can't replicate this.

This comes from way back. At the time the Docker API was very young. There were endpoints sending/posting data in very different ways and different signatures everywhere. Question mark was used in order to easily identify which ones used query string :)

Demux now works using .on('data') which prevents read from being called while the stream is in flowing mode which can cause the internal buffer to already be drained while trying to read. This implementation borrows heavily from https://github.com/Nathan219/docker-stream-cleanser but is reduced to try to match this use-case as much as possible.

https://nodejs.org/api/stream.html#stream_readable_read_size

This method should only be called in paused mode. In flowing mode, this method is called automatically until the internal buffer is drained.

• gets the json for a given image

Revisiting this one. Hummm this will make the "Image" object behave in a non-standard way, because then some functions may refer to a different image.

Let's keep this under the radar for now and check if it affects more people.

For now you could also specify the image name string with whatever you want (containing the version) this every function refers to the same image.

👍

Closes #45

At the moment if Docker connection fails then the modem can hang requests. However when using timeout combine with Attach then the two seem to conflict. Having set my timeout to 1000, it will also still the on-going attach connection even if it has successfully connected.

The node.js documentation says: "Sets the socket to timeout after timeout milliseconds of inactivity on the socket" where as I want the timeout to be on the initial connection.

Proposed Solution: On Connect / Data / Error we clear the timeout.

Debug log: modem Sending: { path: '/containers/create?Image=', method: 'POST', port: '2376' } +0ms modem Received: {"Id":"a54e6dc2a17dd84493bf1a9335e499bb80a659be990d268cf2e9185cd99db28f","Warnings":null} +154ms modem Sending: { path: '/containers/a54e6dc2a17dd84493bf1a9335e499bb80a659be990d268cf2e9185cd99db28f/attach?stream=true&stdin=true&stdout=true&stderr=true', method: 'POST', port: '2376' } +3ms modem Sending: { path: '/containers/a54e6dc2a17dd84493bf1a9335e499bb80a659be990d268cf2e9185cd99db28f/start', port: '2376' } +29ms modem Received: +199ms modem Sending: { path: '/containers/a54e6dc2a17dd84493bf1a9335e499bb80a659be990d268cf2e9185cd99db28f/resize?w=100&h=25', method: 'POST', port: '2376' } +2ms modem Sending: { path: '/containers/a54e6dc2a17dd84493bf1a9335e499bb80a659be990d268cf2e9185cd99db28f/resize?w=118&h=25', method: 'POST', port: '2376' } +5ms modem Received: +47ms modem Received: +15ms modem Timeout of 1000 ms exceeded +971ms

Couldn't replicate this. Will reopen if an example is provided.

second approach to fix #488

I use this code the create a new Docker image:

    const imageOptions = { t: 'imagename' };
    const imageContent = {
      context: '/path/to/docker/content',
      src: ['Dockerfile'],
    };
    return dockerode.buildImage(imageContent, imageOptions)

I use the promise style calls and catch this call in case of an error. So I was testing my code when the needed files are missing and boom - I got this error (catched by the global uncaught exception handler):

UNCAUGHT_EXCEPTION {"error":"ENOENT: no such file or directory, lstat '/path/to/docker/content/Dockerfile'","stack":"Error: ENOENT: no such file or directory, lstat '/path/to/docker/content/Dockerfile'"}

It looks like dockerode does not handle the stream error when using tar-fs:

    pack = tar.pack(file.context, {
      entries: file.src
    });
    //next lines are missing
    pack.on('error', function(error) {
      //handle error
    });

👍

Sure, I can even make you contributor :)

changed result.licenseapi to result.whmcsapi

👍 Will a small tweak to this fix, in order to also support the licenses API. Will ping when published.