As for top-level key, any 3rd-level key which starts with `x-` will be
ignored by compose. This allows for users to:
* include additional metadata in their compose files
* create YAML anchor objects that can be re-used in other parts of the config
This matches a similar feature in the swagger spec definition:
https://swagger.io/specification/#specificationExtensions
This means a composefile like the following is valid
```
verison: "3.7"
services:
foo:
image: foo/bar
x-foo: bar
network:
bar:
x-bar: baz
```
It concerns services, volumes, networks, configs and secrets.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Add validation for stack names to prevent an empty name resulting in _all_
stacks to be returned after filtering, which can result in removal of services
for all stacks if `--prune`, or `docker stack rm` is used.
Before this change;
docker stack deploy -c docker-compose.yml one
docker stack deploy -c docker-compose.yml two
docker stack deploy -c docker-compose.yml three
docker stack deploy -c docker-compose.yml --prune ''
Removing service one_web
Removing service two_web
Removing service three_web
After this change:
docker stack deploy -c docker-compose.yml one
docker stack deploy -c docker-compose.yml two
docker stack deploy -c docker-compose.yml three
docker stack deploy -c docker-compose.yml --prune ''
invalid stack name: ""
Other stack commands were updated as well:
Before this change;
docker stack deploy -c docker-compose.yml ''
Creating network _default
failed to create network _default: Error response from daemon: rpc error: code = InvalidArgument desc = name must be valid as a DNS name component
docker stack ps ''
nothing found in stack:
docker stack rm ''
Removing service one_web
Removing service three_web
Removing service two_web
After this change:
docker stack deploy -c docker-compose.yml ''
invalid stack name: ""
docker stack ps ''
invalid stack name: ""
docker stack rm ''
invalid stack name: ""
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Don't wrap the code to be slightly more readable
- Rename `getServiceFilter()` to `getStackServiceFilter()` to be
consistent with other helper functions.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Add "kubernetes" struct in config file with "allNamespaces" option, to opt-out this behavior when set as "disabled"
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
`docker stack services --filter=label=foo=bar --filter=label=foo=baz my-stack` with Swarm gets handled as `filter on (a label named foo with value bar) AND (a label named foo with value baz).
This obviously yields an empty result set every time, but if and how this should be changed is out of scope here, so simply align Kubernetes with Swarm for now.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
Before this change:
----------------------------------------------------
Create a service with reservations and limits for memory and cpu:
docker service create --name test \
--limit-memory=100M --limit-cpu=1 \
--reserve-memory=100M --reserve-cpu=1 \
nginx:alpine
Verify the configuration
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
}
}
Update just CPU limit and reservation:
docker service update --limit-cpu=2 --reserve-cpu=2 test
Notice that the memory limit and reservation is not preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000
},
"Reservations": {
"NanoCPUs": 2000000000
}
}
Update just Memory limit and reservation:
docker service update --limit-memory=200M --reserve-memory=200M test
Notice that the CPU limit and reservation is not preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"MemoryBytes": 209715200
},
"Reservations": {
"MemoryBytes": 209715200
}
}
After this change:
----------------------------------------------------
Create a service with reservations and limits for memory and cpu:
docker service create --name test \
--limit-memory=100M --limit-cpu=1 \
--reserve-memory=100M --reserve-cpu=1 \
nginx:alpine
Verify the configuration
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 1000000000,
"MemoryBytes": 104857600
}
}
Update just CPU limit and reservation:
docker service update --limit-cpu=2 --reserve-cpu=2 test
Confirm that the CPU limits/reservations are updated, but memory limit and reservation are preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000,
"MemoryBytes": 104857600
},
"Reservations": {
"NanoCPUs": 2000000000,
"MemoryBytes": 104857600
}
}
Update just Memory limit and reservation:
docker service update --limit-memory=200M --reserve-memory=200M test
Confirm that the Mempry limits/reservations are updated, but CPU limit and reservation are preserved:
docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' test
{
"Limits": {
"NanoCPUs": 2000000000,
"MemoryBytes": 209715200
},
"Reservations": {
"NanoCPUs": 2000000000,
"MemoryBytes": 209715200
}
}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch hides the [flags] in the usage output of commands, using the
new `.DisableFlagsInUseLine` option, instead of the temporary workaround
added in 8e600e10f7
Before this change:
docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...] [flags]
Run a command in a new container
After this change:
docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `docker version` output now uses a tabwriter, so use single
tabs to print the output.
Before this change:
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.10.1
Git commit: f150324
Built: Wed May 9 22:20:16 2018
OS/Arch: linux/amd64
Experimental: true
Kubernetes:
Version: v1.9.6
StackAPI: v1beta2
After this change:
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.10.1
Git commit: f150324
Built: Wed May 9 22:20:16 2018
OS/Arch: linux/amd64
Experimental: true
Kubernetes:
Version: v1.9.6
StackAPI: v1beta2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Switch from x/net/context to context made "go vet" see the previously
unseen errors:
> cli/command/container/start.go:57::error: the cancelFun function is
> not used on all paths (possible context leak) (vet)
> cli/command/container/start.go:63::error: this return statement may be
> reached without using the cancelFun var defined on line 57 (vet)
> cli/command/container/run.go:159::error: the cancelFun function is not
> used on all paths (possible context leak) (vet)
> cli/command/container/run.go:164::error: this return statement may be
> reached without using the cancelFun var defined on line 159 (vet)
Do call the cancel function.
Note we might end up calling it twice which is fine as long as I can see
from the Go 1.10 source code.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Since go 1.7, "context" is a standard package. Since go 1.9,
x/net/context merely provides some types aliased to those in
the standard context package.
The changes were performed by the following script:
for f in $(git ls-files \*.go | grep -v ^vendor/); do
sed -i 's|golang.org/x/net/context|context|' $f
goimports -w $f
for i in 1 2; do
awk '/^$/ {e=1; next;}
/\t"context"$/ {e=0;}
{if (e) {print ""; e=0}; print;}' < $f > $f.new && \
mv $f.new $f
goimports -w $f
done
done
[v2: do awk/goimports fixup twice]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Removing a host by `<host>:<ip>` should only remove occurences of the host with
a matching IP-address, instead of removing all entries for that host.
In addition, combining `--host-rm` and `--host-add` for the same host should
result in the new host being added.
This patch fixes the way the diff is calculated to allow combining
removing/adding, and to support entries having both a canonical, and aliases.
Aliases cannot be added by the CLI, but are supported in the Service spec, thus
should be taken into account:
Entries can be removed by either a specific `<host-name>:<ip-address>`
mapping, or by `<host>` alone:
- If both IP-address and host-name is provided, only remove the hostname
from entries that match the given IP-address.
- If only a host-name is provided, remove the hostname from any entry it
is part of (either as _canonical_ host-name, or as _alias_).
- If, after removing the host-name from an entry, no host-names remain in
the entry, the entry itself should be removed.
For example, the list of host-entries before processing could look like this:
hosts = &[]string{
"127.0.0.2 host3 host1 host2 host4",
"127.0.0.1 host1 host4",
"127.0.0.3 host1",
"127.0.0.1 host1",
}
Removing `host1` removes every occurrence:
hosts = &[]string{
"127.0.0.2 host3 host2 host4",
"127.0.0.1 host4",
}
Whereas removing `host1:127.0.0.1` only remove the host if the IP-address matches:
hosts = &[]string{
"127.0.0.2 host3 host1 host2 host4",
"127.0.0.1 host4",
"127.0.0.3 host1",
}
Before this patch:
$ docker service create --name my-service --host foo:127.0.0.1 --host foo:127.0.0.2 --host foo:127.0.0.3 nginx:alpine
$ docker service update --host-rm foo:127.0.0.1 --host-add foo:127.0.0.4 my-service
$ docker service inspect --format '{{.Spec.TaskTemplate.ContainerSpec.Hosts}}' my-service
[]
After this patch is applied:
$ docker service create --name my-service --host foo:127.0.0.1 --host foo:127.0.0.2 --host foo:127.0.0.3 nginx:alpine
$ docker service update --host-rm foo:127.0.0.1 --host-add foo:127.0.0.5 my-service
$ docker service inspect --format '{{.Spec.TaskTemplate.ContainerSpec.Hosts}}' my-service
[127.0.0.2 foo 127.0.0.3 foo 127.0.0.4 foo]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "update" and "rollback" configurations were cross-wired, as a result, setting
`--rollback-*` options would override the service's update-options.
Creating a service with both update, and rollback configuration:
docker service create \
--name=test \
--update-failure-action=pause \
--update-max-failure-ratio=0.6 \
--update-monitor=3s \
--update-order=stop-first \
--update-parallelism=3 \
--rollback-failure-action=continue \
--rollback-max-failure-ratio=0.5 \
--rollback-monitor=4s \
--rollback-order=start-first \
--rollback-parallelism=2 \
--tty \
busybox
Before this change:
docker service inspect --format '{{json .Spec.UpdateConfig}}' test \
&& docker service inspect --format '{{json .Spec.RollbackConfig}}' test
Produces:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
After this change:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":2,"FailureAction":"continue","Monitor":4000000000,"MaxFailureRatio":0.5,"Order":"start-first"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This ensures Windows paths are handled correctly as explained in the path package documentation.
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
It will be helpful to expose the pull implementation which supports
pulling private images for other CLI commands that rely on helper images.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
When creating manifest lists, don't use "*" as the permission when
creating the token handler. This causes problems with gitlab's repos.
Fixes https://github.com/docker/cli/issues/1010
Signed-off-by: Christy Norman <christy@linux.vnet.ibm.com>
- Some of them don't make sense anymore
- Some are deprecated and removed from the engine since a few versions
already.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
… and other cases too. Updating mergo fixes the bugs (but introduced a
slight behaviour change that had to be fixed too)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
When updating a service with the `--force` option, the `ForceUpdate`
property of the taskspec is incremented.
Stack deploy did not take this into account, and reset this
field to its default value (0), causing the service to be
re-deployed.
This patch copies the existing value before updating the service.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
> HasAvailableFlags checks if the command contains any flags (local
> plus persistent from the entire structure) which are not hidden or
> deprecated.
This fix the `--help` display when the `Options` is empty (but
showing), like on `docker trust key`
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
- `replaceDockerfileForContentTrust` is only used when content trust is
enabled, so remove the boolean.
- rename `isContentTrustEnabled` to `contentTrustEnabled`
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Some of them are skipped for now (because the feature is not supported
or needs more work), some of them are fixed.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Fix tests that failed when using cmp.Compare()
internal/test/testutil/assert
InDelta
Fix DeepEqual with kube metav1.Time
Convert some ErrorContains to assert
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Overriding is the incorrect part of speech for this sentence. It is more common to state that the values are overridden instead.
Other options would include:
```golang
// ldflags is overriding these values
```
```golang
// These values will be overridden by ldflags
```
etc.
Signed-off-by: Kevin Kirsche <kevin.kirsche@verizon.com>
Commit 2b17f4c8a8 fixed the way empty labels
are taken into account (i.e. not interpolated from environment variable),
but it created a regression.
`ValidateLabel` functions doesn't allow empty label value, but it has
always been possible to pass an empty label via the cli (`docker run --label foo`).
This fixes that by not validating the label flag.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Includes:
- [client] Remove duplicate NewClient functions
- Add API support for templated secrets and configs
- Adjust minimum API version for templated configs/secrets
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Add `Version` to `types.Config`
- Add a new `Services` types (that is just `[]ServiceConfig`) and add
`MarshalYAML` method on it.
- Clean other top-level custom marshaling as `Services` is the only one
required.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Historically, the Dockerfile had to be insde the build-context, because it was
sent as part of the build-context.
3f6dc81e10
added support for passing the Dockerfile through stdin, in which case the
contents of the Dockerfile is injected into the build-context.
This patch uses the same mechanism for situations where the location of the
Dockerfile is passed, and its path is outside of the build-context.
Before this change:
$ mkdir -p myproject/context myproject/dockerfiles && cd myproject
$ echo "hello" > context/hello
$ echo -e "FROM busybox\nCOPY /hello /\nRUN cat /hello" > dockerfiles/Dockerfile
$ docker build --no-cache -f $PWD/dockerfiles/Dockerfile $PWD/context
unable to prepare context: the Dockerfile (/Users/sebastiaan/projects/test/dockerfile-outside/myproject/dockerfiles/Dockerfile) must be within the build context
After this change:
$ mkdir -p myproject/context myproject/dockerfiles && cd myproject
$ echo "hello" > context/hello
$ echo -e "FROM busybox\nCOPY /hello /\nRUN cat /hello" > dockerfiles/Dockerfile
$ docker build --no-cache -f $PWD/dockerfiles/Dockerfile $PWD/context
Sending build context to Docker daemon 2.607kB
Step 1/3 : FROM busybox
---> 6ad733544a63
Step 2/3 : COPY /hello /
---> 9a5ae1c7be9e
Step 3/3 : RUN cat /hello
---> Running in 20dfef2d180f
hello
Removing intermediate container 20dfef2d180f
---> ce1748f91bb2
Successfully built ce1748f91bb2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds the Engine version to `docker node ls`, and `.EngineVersion` as a
template option.
With this patch applied:
docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
wp9231itoqsh4rqceojqo01vp * linuxkit-025000000001 Ready Active Leader 18.01.0-ce
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
backend.
Currently, the timeout is set to 0, which means no timeout. Set it to a
sane default timeout of 30 seconds.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
To ensure we are loading the composefile the same wether we are pointing
to swarm or kubernetes, we need to share the loading code between both.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
`--label-file` has the exact same behavior as `--env-file`, meaning any
placeholder (i.e. a simple key, no `=` sign, no value), it will get the
value from the environment variable.
For `--label-file` it should just add an empty label.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Prefer "strict" values for orchestrator, as it's
easier to add aliases (if we think it's needed) than
to remove them later.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When adding a network using `docker service update --network-add`,
the new network was added by _name_.
Existing entries in a service spec are listed by network ID, which
resulted in the CLI not detecting duplicate entries for the same
network.
This patch changes the behavior to always use the network-ID,
so that duplicate entries are correctly caught.
Before this change;
$ docker network create -d overlay foo
$ docker service create --name=test --network=foo nginx:alpine
$ docker service update --network-add foo test
$ docker service inspect --format '{{ json .Spec.TaskTemplate.Networks}}' test
[
{
"Target": "9ot0ieagg5xv1gxd85m7y33eq"
},
{
"Target": "9ot0ieagg5xv1gxd85m7y33eq"
}
]
After this change:
$ docker network create -d overlay foo
$ docker service create --name=test --network=foo nginx:alpine
$ docker service update --network-add foo test
service is already attached to network foo
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Enable inspection (aka "shallow pull") of images' manifest info, and
also the creation of manifest lists (aka "fat manifests").
The workflow for creating a manifest list will be:
`docker manifest create new-list-ref-name image-ref [image-ref...]`
`docker manifest annotate new-list-ref-name image-ref --os linux --arch
arm`
`docker manifest push new-list-ref-name`
The annotate step is optional. Most architectures are fine by default.
There is also a `manifest inspect` command to allow for a "shallow pull"
of an image's manifest: `docker manifest inspect
manifest-or-manifest_list`.
To be more in line with the existing external manifest tool, there is
also a `-v` option for inspect that will show information depending on
what the reference maps to (list or single manifest).
Signed-off-by: Christy Perez <christy@linux.vnet.ibm.com>
Signed-off-by: Daniel Nephin <dnephin@docker.com>
- More strict on orchestrator flag
- Make orchestrator flag more explicit as experimental
- Add experimentalCLI annotation on kubernetes flags
- Better kubeconfig error message
- Prefix service name with stackname in ps and services stack subcommands
- Fix yaml documentation
- Fix code coverage ignoring generated code
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
* Refactor tests on version and kubernetes switch
* Fix rebase errors
* Refactor for gocyclo linter
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Services do not support custom "pid"-modes (e.g. `--pid=host`), but this
option was ignored silently when deploying a stack.
This patch adds `pid` to the list of unsupported options so that a warning
is printed;
With this patch applied:
$ docker stack deploy -c docker-compose.yml foobar
Ignoring unsupported options: pid
Creating network foobar_default
Creating service foobar_test
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Define command and subcommands only once
- Use annotations for k8s or swarm specific flags or subcommands
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
- Add support for kubernetes for docker stack command
- Update to go 1.9
- Add kubernetes to vendors
- Print orchestrator in docker version command
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
Allow to mark some commands and flags experimental on cli (i.e. not
depending to the state of the daemon). This will allow more flexibility
on experimentation with the cli.
Marking `docker trust` as cli experimental as it is documented so.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Vincent Demeester
Sebastiaan van Stijn
Silvin Lubecki
Sebastiaan van Stijn
Zachary Romero
Vincent Demeester
Mathieu Champlon
Simon Ferquel
Victor Vieux
Anda Xu
Brian Goff
Kir Kolyshkin
Essam A. Hassan
Justyn Temme
Daniel Hiltgen
Christy Norman
Nassim 'Nass' Eddequiouaq
Tibor Vass
Bogdan Anton
Tonis Tiigi
Daniel Nephin
Kevin Kirsche
Arash Deshmeh
shhsu@microsoft.com
Joffrey F
Daniel Nephin
Wataru Ishida
Anusha Ragunathan
Ivan Markin
Jake Sanders
Sungwon Han
WTFKr0
Ethan Haynes
Akim Demaille
Michael Friis
Paweł Szczekutowicz