the description of "swarm service inspect" seems strange in: https://docker-py.readthedocs.io/en/1.10.0/api/

inspect_service
Create a service, similar to the docker service create command. See the services documentation for details.

Thanks

Thanks for the report. That's tied to the version of the code at time of release, which for 1.10.0 was 3 years ago, so we won't do anything to fix this.

Here is the full traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/local/lib/python3.6/dist-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 368, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/local/lib/python3.6/dist-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 78, in <module>
    main()
  File "main.py", line 32, in main
    docker_client.login(username=docker_username, password=docker_password)
  File "/usr/local/lib/python3.6/dist-packages/docker/client.py", line 183, in login
    return self.api.login(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/daemon.py", line 149, in login
    response = self._post_json(self._url('/auth'), data=req_data)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 289, in _post_json
    return self._post(url, data=json.dumps(data2), **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 226, in _post
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

So I am trying to log into docker hub using this package. The python script running this is running in a docker container. Below is the Dockerfile:

# Base image with docker and python3.6 installed
FROM mattgleich/python-and-docker

# Image metadata
LABEL maintainer="[email protected]"
LABEL description="Automatically build images anywhere that docker runs"

# Installing Git
RUN apt-get update
RUN apt-get -qq -y install git

# Installing requirements
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt 
RUN export GIT_PYTHON_REFRESH=quiet

COPY /src /src
WORKDIR /src
RUN rm -rf repos

# Fixing some stuff with docker
RUN eval "$(docker-machine env default)"

CMD [ "python3", "main.py" ]

The docker hub password is stored in a folder called configs that is then put into the container using a volume. Below is the error that I get when I start the container with docker compose:

version: "3"

services:
    auto-build-images:
        container_name: auto-build-images
        build: .
        network_mode: "host"
        volumes:
            - ~/auto-build-images-config:/src/config

As far as the script goes, below is a link to the GitHub repo for the project: https://github.com/Matt-Gleich/Auto-Build-Images-Anywhere

RUN eval "$(docker-machine env default)" will not persist these environment values at runtime. You'll need a startup script.

I'm going to close this as it's not related to the SDK.

You should be able to trigger an update operation to set the role from manager to worker with:

• client.update_node if you're using the low-level API (APIClient).
• client.nodes.get(node_id).update if you're using the high-level API (DockerClient).

That's an error at the server level (in the Docker daemon) - the SDK is behaving appropriately here by raising.

We have encountered a following issue in following scenario:

Scenario:

• Have docker on a node
• Have docker-service with one replica
• Scale service down to zero replicas
• List containers in a loop (waiting for the container of a service to disappear)

Expected results: Eventually created list of the containers has one less element.

Actual results:

Nondeterministically the code fails with following backtrace:

...
07:40:01    File ..., line 108, in getAllContainerNames
07:40:01      return [container.name for client in self._swarm.clients for container in client.containers.list(all=True)]
07:40:01    File "/tmp/tmp.xrnVzrxn9J/lib/python2.7/site-packages/docker/models/containers.py", line 943, in list
07:40:01      containers.append(self.get(r['Id']))
07:40:01    File "/tmp/tmp.xrnVzrxn9J/lib/python2.7/site-packages/docker/models/containers.py", line 880, in get
07:40:01      resp = self.client.api.inspect_container(container_id)
07:40:01    File "/tmp/tmp.xrnVzrxn9J/lib/python2.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
07:40:01      return f(self, resource_id, *args, **kwargs)
07:40:01    File "/tmp/tmp.xrnVzrxn9J/lib/python2.7/site-packages/docker/api/container.py", line 756, in inspect_container
07:40:01      self._get(self._url("/containers/{0}/json", container)), True
07:40:01    File "/tmp/tmp.xrnVzrxn9J/lib/python2.7/site-packages/docker/api/client.py", line 267, in _result
07:40:01      self._raise_for_status(response)
07:40:01    File "/tmp/tmp.xrnVzrxn9J/lib/python2.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
07:40:01      raise create_api_error_from_http_exception(e)
07:40:01    File "/tmp/tmp.xrnVzrxn9J/lib/python2.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
07:40:01      raise cls(e, response=response, explanation=explanation)
07:40:01  NotFound: 404 Client Error: Not Found ("No such container: 3701121bed79b3925404c76b5f66a89c0d20fd86eab541fdd85690cecd07aafa")

Reproduction: I was able to reproduce this issue with a minimalisic test:

• In first terminal run

import docker
cli = docker.DockerClient()
while True:
   a = cli.containers.list(all=True)

• In second terminal run: for i in `seq 10`; do docker rm -f `docker run -d busybox tail -F /dev/null`; done (in my environment this issue occurs quickly and 10 loop iterations are enough) Output obtained:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/home/bzak/persistentVenv/local/lib/python2.7/site-packages/docker/models/containers.py", line 943, in list
    containers.append(self.get(r['Id']))
  File "/home/bzak/persistentVenv/local/lib/python2.7/site-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/home/bzak/persistentVenv/local/lib/python2.7/site-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/home/bzak/persistentVenv/local/lib/python2.7/site-packages/docker/api/container.py", line 756, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/home/bzak/persistentVenv/local/lib/python2.7/site-packages/docker/api/client.py", line 267, in _result
    self._raise_for_status(response)
  File "/home/bzak/persistentVenv/local/lib/python2.7/site-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/home/bzak/persistentVenv/local/lib/python2.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.NotFound: 404 Client Error: Not Found ("No such container: edf87eb0d1906fd2c567f62846922e3b537552fe28f689d761baece250b21145")

Suspicions: I believe this is a race condition in which docker-py tries to list containers first and then inspect them. Since those commands are "non-atomic" the list can contain container that is removed immediately thereafter and tries to inspect a non-existing container.

Additional information: Python: Python 2.7.5 Docker-py: 4.0.2 Docker:

Client:
 Version:           18.09.3
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        774a1f4
 Built:             Thu Feb 28 06:33:21 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.3
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       774a1f4
  Built:            Thu Feb 28 06:02:24 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Use sparse=True or ignore_removed=True: https://docker-py.readthedocs.io/en/stable/containers.html#docker.models.containers.ContainerCollection.list

@shin- I see there's one (unresolved) issue on the 4.1.0 milestone; #2355 - is that a blocker for 4.1.0, or was it just put on the milestone "tentatively" ?

Not a blocker, more of a wish 😅

To check what the next tag should be (4.1.0 or 4.0.3); looking through the changes, most changes are fixes;

• diff 4.0.2...master (may be deceiving, as we're comparing between branches, and some commits may not match)
• raw diff: https://github.com/docker/docker-py/compare/4.0.2..master

I think (may want to double-check) that #2333 is the only feature that was added; if that's indeed a feature, then it should be 4.1.0

@ulyssessouza @shin- ^^

I'd go ahead and release 4.1.0 without #2355 (which we can move into the next milestone)

@kolyshkin I'm going to be offline for a couple of weeks starting tomorrow, so if @ulyssessouza or @rumpl can take the lead on that that'd be ideal! Thanks guys :)

Versions are already pinned on our end: https://github.com/docker/docker-py/blob/master/setup.py#L29-L30

See https://github.com/docker/docker-py/pull/2300

I'd recommend switching to Python 3 if possible. Python 2 is reaching EOL in a few months, and it doesn't make a lot of sense for us to keep investing in supporting older versions of Python.

Alternatively, you can use the binary builds of Compose which are already compiled with Python 3.7.

The BUSYBOX variable was renamed to TEST_IMG in 54b48a9b7ab59b4dcf49acf49ddf52035ba3ea08 (https://github.com/docker/docker-py/pull/2407), however 0ddf428b6ce7accdac3506b45047df2cb72941ec (https://github.com/docker/docker-py/pull/2333) got merged after that change, but was out of date, and therefore caused the tests to fail:

=================================== FAILURES ===================================
________ ServiceTest.test_create_service_with_network_attachment_config ________
tests/integration/api_service_test.py:379: in test_create_service_with_network_attachment_config
    container_spec = docker.types.ContainerSpec(BUSYBOX, ['true'])
E   NameError: global name 'BUSYBOX' is not defined

Fix the test by using the correct variable name.

@hnine999 No, that's an error on our end - we'll fix it shortly. Feel free to submit your PR in the meantime!

Taken from the root README:

import docker
client = docker.from_env()

produces:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/csmith/Dev/repo/venv/lib/python3.7/site-packages/docker/client.py", line 85, in from_env
    timeout=timeout, version=version, **kwargs_from_env(**kwargs)
  File "/Users/csmith/Dev/repo/venv/lib/python3.7/site-packages/docker/client.py", line 40, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/Users/csmith/Dev/repo/venv/lib/python3.7/site-packages/docker/api/client.py", line 128, in __init__
    config_dict=self._general_configs, credstore_env=credstore_env,
TypeError: load_config() got an unexpected keyword argument 'config_dict'

The same error is seen when creating a client using docker.DockerClient(...)

docker==4.0.2
docker-py==1.10.6
docker-pycreds==0.4.0
dockerpty==0.4.1
Python 3.7.0
Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:18:17 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:17:52 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

https://github.com/docker/docker-py/issues/1916#issuecomment-386077775

The current documentation makes it appear like the only way to attach networks to a service is to provide them as a list of strings of ids/names.

Attaching networks also support setting aliases and driver options. I've added the new type docker.types.NetworkAttachmentConfig which is used to document these features.

I also found that a documentation link to ConfigReference did not work. It should be referenced as docker.types.ConfigReference, this is now fixed.

docker.types.ConfigReference was also referenced by its absolute path. I changed this to make the documentation a bit easier to read.

Not sure if this becomes active on the PR itself (because I'm not a maintainer on this repo, so Jenkinsfile changes may only become active after merged?)

I'm sure it'll be fine :) Thanks!

i hope i didn't miss other appropriate places.

With this change applied, the default debian package repository can be replaced with a mirror;

make APT_MIRROR=cdn-fastly.deb.debian.org build-py3

...

Step 5/19 : RUN apt-get update && apt-get -y install     gnupg2     pass     curl
 ---> Running in 01c1101a0bd0
Get:1 http://cdn-fastly.deb.debian.org/debian buster InRelease [118 kB]
Get:2 http://cdn-fastly.deb.debian.org/debian-security buster/updates InRelease [39.1 kB]
Get:3 http://cdn-fastly.deb.debian.org/debian buster-updates InRelease [46.8 kB]
Get:4 http://cdn-fastly.deb.debian.org/debian buster/main amd64 Packages [7897 kB]
Get:5 http://cdn-fastly.deb.debian.org/debian-security buster/updates/main amd64 Packages [22.8 kB]

A typo in conditional logic can prevent auth.INDEX_URL from being added to auth_data in cases where auth.INDEX_URL is missing but an auth.INDEX_NAME is available.

Signed-off-by: Matt Fluet [email protected]

Sorry this took a while - LGTM. Thanks!

• set junitxml suit name to docker-py may help when sending multiple junit files to jenkins
• pytest: update to v4.2.1 - use xunit2 for compatibility with Jenkins pytest-dev/pytest#3547: --junitxml can emit XML compatible with Jenkins xUnit. junit_family INI option accepts legacy|xunit1, which produces old style output, and xunit2 that conforms more strictly to https://github.com/jenkinsci/xunit-plugin/blob/xunit-2.3.2/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd

LGTM - Thank you!

Thank you!

Hi!

You can already provide a custom password through the base_url parameter, e.g. ssh://[email protected]:22. For more advanced uses, you should be able to write your own Adapter subclass and pass it to the APIClient through mount() (see the requests doc on that topic). For example,

ssh_adapter = MySSHAdapter(url, key_path)
client = APIClient(base_url='ssh://bogus:22')
client.mount('http+docker://ssh', ssh_adapter)

I think eventually we may want to have the option of passing a private key to the Client instantiation in some form (TBD - maybe something similar to the TLSConfig), but anything beyond that, like first-class adapter modularity, would probably be beyond the scope of what we're trying to do with the library.

Hope that helps!

docker-compose builds that pull from private registries are broken on macs that use the OS keychain (ie. osxkeychain credsStore) due to a change in the way the auth information is processed in auth.py.

The osxkeychain maintains an auths section with host keys but no values in .docker/config.json, so this needs to be accounted for:

  "auths": {
    "some.registry.host": {},
    },

This change will resolve any keys from the auths that have no values defined as a first step. Confirmed that this fixed the build for me locally.

Example return before change:

{'some.registry.host': {}, 'https://some.registry.host': {'ServerAddress': 'https://some.registry.host', 'Username': 'userid1', 'Password': 'encodedPassword'}}

Example return after change:

{'some.registry.host': {'ServerAddress': 'https://some.registry.host', 'Username': 'userid1', 'Password': 'encodedPassword'}, 'https://some.registry.host': {'ServerAddress': 'https://some.registry.host', 'Username': 'userid1', 'Password': 'encodedPassword'}}

Affects docker-py 3.7.0+ and docker-compose 1.24.0+

LGTM, thanks! cc @ulyssessouza to sort out integration of that fix in Compose.

You're passing the 'image' string to search() and getting results for the image term. You probably mean to pass the image variable.

Hello I want to use the code below to retrieve nginx images from docker hub but it displayed different images from docker hub instead of nginx images.

import click
import docker
import subprocess

client = docker.from_env()   

@click.command()
@click.argument('image') 
def search(image):
      click.echo(client.images.search('image'))   

if __name__  ==  '__main__':
         search()

I executed the program as:

python client.py nginx

Output:

 {u'is_automated': False, u'is_official': False, u'name': u'iron/images', u'star_count': 7, u'description': u'docker images'}, {u'is_automated': True, u'is_official': False, u'name': u'ebonharme/images', u'star_count': 1, u'description': u'Docker images built automatically'}, {u'is_automated': False, u'is_official': False, u'name': u'ckassen/imageservice', u'star_count': 0, u'description': u'Workshop Imageservice'}, {u'is_automated': False, u'is_official': False, u'name': u'bugswarm/images', u'star_count': 0, u'description': u''}, {u'is_automated': False, u'is_official': False, u'name': u'wwdemo/images', u'star_count': 0, u'description': u''}, {u'is_automated': False, u'is_official': True, u'name': u'registry', u'star_count': 2657, u'description': u'The Docker Registry 2.0 implementation for storing and distributing Docker images'}