While performance will be worse, we can safely ignore the --stream
option when used, and print a deprecation warning instead of failing
the build.
With this patch:
echo -e "FROM scratch\nLABEL foo=bar" | docker build --stream -
DEPRECATED: The experimental --stream flag has been removed and the build context
will be sent non-streaming. Enable BuildKit instead with DOCKER_BUILDKIT=1
to stream build context, see https://docs.docker.com/go/buildkit/
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM scratch
--->
Step 2/2 : LABEL foo=bar
---> Running in 99e4021085b6
Removing intermediate container 99e4021085b6
---> 1a7a41be241f
Successfully built 1a7a41be241f
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The CLI disabled experimental features by default, requiring users
to set a configuration option to enable them.
Disabling experimental features was a request from Enterprise users
that did not want experimental features to be accessible.
We are changing this policy, and now enable experimental features
by default. Experimental features may still change and/or removed,
and will be highlighted in the documentation and "usage" output.
For example, the `docker manifest inspect --help` output now shows:
EXPERIMENTAL:
docker manifest inspect is an experimental feature.
Experimental features provide early access to product functionality. These features
may change between releases without warning or can be removed entirely from a future
release. Learn more about experimental features: https://docs.docker.com/go/experimental/
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
All output of the usage / --help output uses spaces, and having a tab
in the output can be somewhat cumbersome (e.g. our YAML docs generator
doesn't like them, and copy/pasing the output in iTerm produces a warning).
This patch changes the output to use two spaces instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When initializing the API client, the User-Agent was added to any custom
HTTPHeaders that were configured. However, because the map was not properly
dereferenced, the original map was modified, causing the User-Agent to also
be saved to config.json after `docker login` and `docker logout`:
Before this change;
$ cat ~/.docker/config.json
cat: can't open '/root/.docker/config.json': No such file or directory
$ docker login -u myusername
Password:
...
Login Succeeded
$ cat ~/.docker/config.json
{
"auths": {
"https://index.docker.io/v1/": {
"auth": "<base64 auth>"
}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.12 (linux)"
}
}
$ docker logout
{
"auths": {},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.12 (linux)"
}
}
After this change:
$ cat ~/.docker/config.json
cat: can't open '/root/.docker/config.json': No such file or directory
$ docker login -u myusername
Password:
...
Login Succeeded
$ cat ~/.docker/config.json
{
"auths": {
"https://index.docker.io/v1/": {
"auth": "<base64 auth>"
}
}
}
$ docker logout
Removing login credentials for https://index.docker.io/v1/
$ cat ~/.docker/config.json
{
"auths": {}
}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This implements a special "RESET" value that can be used to reset the
list of capabilities to add/drop when updating a service.
Given the following service;
| CapDrop | CapAdd |
| -------------- | ------------- |
| CAP_SOME_CAP | |
When updating the service, and applying `--cap-drop RESET`, the "drop" list
is reset to its default:
| CapDrop | CapAdd |
| -------------- | ------------- |
| | |
When updating the service, and applying `--cap-drop RESET`, combined with
`--cap-add CAP_SOME_CAP` and `--cap-drop CAP_SOME_OTHER_CAP`:
| CapDrop | CapAdd |
| -------------- | ------------- |
| CAP_FOO_CAP | CAP_SOME_CAP |
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adding/removing capabilities when updating a service is considered a tri-state;
- if the capability was previously "dropped", then remove it from "CapabilityDrop",
but do NOT add it to "CapabilityAdd". However, if the capability was not yet in
the service's "CapabilityDrop", then simply add it to the service's "CapabilityAdd"
- likewise, if the capability was previously "added", then remove it from
"CapabilityAdd", but do NOT add it to "CapabilityDrop". If the capability was
not yet in the service's "CapabilityAdd", then simply add it to the service's
"CapabilityDrop".
In other words, given a service with the following:
| CapDrop | CapAdd |
| -------------- | ------------- |
| CAP_SOME_CAP | |
When updating the service, and applying `--cap-add CAP_SOME_CAP`, the previously
dropped capability is removed:
| CapDrop | CapAdd |
| -------------- | ------------- |
| | |
When updating the service a second time, applying `--cap-add CAP_SOME_CAP`,
capability is now added:
| CapDrop | CapAdd |
| -------------- | ------------- |
| | CAP_SOME_CAP |
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When creating and updating services, we need to avoid unneeded service churn.
The interaction of separate lists to "add" and "drop" capabilities, a special
("ALL") capability, as well as a "relaxed" format for accepted capabilities
(case-insensitive, `CAP_` prefix optional) make this rather involved.
This patch updates how we handle `--cap-add` / `--cap-drop` when _creating_ as
well as _updating_, with the following rules/assumptions applied:
- both existing (service spec) and new (values passed through flags or in
the compose-file) are normalized and de-duplicated before use.
- the special "ALL" capability is equivalent to "all capabilities" and taken
into account when normalizing capabilities. Combining "ALL" capabilities
and other capabilities is therefore equivalent to just specifying "ALL".
- adding capabilities takes precedence over dropping, which means that if
a capability is both set to be "dropped" and to be "added", it is removed
from the list to "drop".
- the final lists should be sorted and normalized to reduce service churn
- no validation of capabilities is handled by the client. Validation is
delegated to the daemon/server.
When deploying a service using a docker-compose file, the docker-compose file
is *mostly* handled as being "declarative". However, many of the issues outlined
above also apply to compose-files, so similar handling is applied to compose
files as well to prevent service churn.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The tabwriter was configured to have a min-width for columns of 20 positions.
This seemed quite wide, and caused smaller columns to be printed with a large
gap between.
Before:
docker container stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
29184b3ae391 amazing_shirley 0.00% 800KiB / 1.944GiB 0.04% 1.44kB / 0B 0B / 0B 1
403c101bad56 agitated_swartz 0.15% 34.31MiB / 1.944GiB 1.72% 10.2MB / 206kB 0B / 0B 51
0dc4b7f6c6be container2 0.00% 1.012MiB / 1.944GiB 0.05% 12.9kB / 0B 0B / 0B 5
2d99abcc6f62 container99 0.00% 972KiB / 1.944GiB 0.05% 13kB / 0B 0B / 0B 5
9f9aa90173ac foo 0.00% 820KiB / 1.944GiB 0.04% 13kB / 0B 0B / 0B 5
docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
29184b3ae391 docker-cli-dev "ash" 4 hours ago Up 4 hours amazing_shirley
403c101bad56 docker-dev:master "hack/dind bash" 3 days ago Up 3 days agitated_swartz
0dc4b7f6c6be nginx:alpine "/docker-entrypoint.…" 4 days ago Up 4 days 80/tcp container2
2d99abcc6f62 nginx:alpine "/docker-entrypoint.…" 4 days ago Up 4 days 80/tcp container99
9f9aa90173ac nginx:alpine "/docker-entrypoint.…" 4 days ago Up 4 days 80/tcp foo
docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
docker-cli-dev latest 5f603caa04aa 4 hours ago 610MB
docker-cli-native latest 9dd29f8d387b 4 hours ago 519MB
docker-dev master 8132bf7a199e 3 days ago 2.02GB
docker-dev improve-build-errors 69e208994b3f 11 days ago 2.01GB
docker-dev refactor-idtools 69e208994b3f 11 days ago 2.01GB
After:
docker container stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
29184b3ae391 amazing_shirley 0.14% 5.703MiB / 1.944GiB 0.29% 1.44kB / 0B 0B / 0B 10
403c101bad56 agitated_swartz 0.15% 56.97MiB / 1.944GiB 2.86% 10.2MB / 206kB 0B / 0B 51
0dc4b7f6c6be container2 0.00% 1016KiB / 1.944GiB 0.05% 12.9kB / 0B 0B / 0B 5
2d99abcc6f62 container99 0.00% 956KiB / 1.944GiB 0.05% 13kB / 0B 0B / 0B 5
9f9aa90173ac foo 0.00% 980KiB / 1.944GiB 0.05% 13kB / 0B 0B / 0B 5
docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
29184b3ae391 docker-cli-dev "ash" 12 minutes ago Up 12 minutes amazing_shirley
403c101bad56 docker-dev:master "hack/dind bash" 3 days ago Up 3 days agitated_swartz
0dc4b7f6c6be nginx:alpine "/docker-entrypoint.…" 4 days ago Up 4 days 80/tcp container2
2d99abcc6f62 nginx:alpine "/docker-entrypoint.…" 4 days ago Up 4 days 80/tcp container99
9f9aa90173ac nginx:alpine "/docker-entrypoint.…" 4 days ago Up 4 days 80/tcp foo
docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
docker-cli-dev latest 5f603caa04aa 4 hours ago 610MB
docker-cli-native latest 9dd29f8d387b 4 hours ago 519MB
docker-dev master 8132bf7a199e 3 days ago 2.02GB
docker-dev improve-build-errors 69e208994b3f 11 days ago 2.01GB
docker-dev refactor-idtools 69e208994b3f 11 days ago 2.01GB
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The vanity domain is down, and the project has moved
to a new location.
vendor check started failing because of this:
Collecting initial packages
Download dependencies
unrecognized import path "vbom.ml/util" (https fetch: Get https://vbom.ml/util?go-get=1: dial tcp: lookup vbom.ml on 169.254.169.254:53: no such host)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When using `docker rm` / `docker container rm` with the `-f` / `--force` option, attempts to remove non-existing containers should print a warning, but should return a zero exit code ("successful").
Currently, a non-zero exit code is returned, marking the removal as "failed";
$ docker rm -fv 798c9471b695
Error: No such container: 798c9471b695
$ echo $?
1
The command should match the behavior of `rm` / `rm -f`, with the exception that
a warning is printed (instead of silently ignored):
Running `rm` with `-f` silences output and returns a zero exit code:
touch some-file && rm -f no-such-file some-file; echo exit code: $?; ls -la
# exit code: 0
# total 0
# drwxr-xr-x 2 sebastiaan staff 64 Aug 14 12:17 .
# drwxr-xr-x 199 sebastiaan staff 6368 Aug 14 12:13 ..
mkdir some-directory && rm -rf no-such-directory some-directory; echo exit code: $?; ls -la
# exit code: 0
# total 0
# drwxr-xr-x 2 sebastiaan staff 64 Aug 14 12:17 .
# drwxr-xr-x 199 sebastiaan staff 6368 Aug 14 12:13 ..
Note that other reasons for a delete to fail should still result in a non-zero
exit code, matching the behavior of `rm`. For instance, in the example below,
the `rm` failed because directories can only be removed if the `-r` option is used;
touch some-file && mkdir some-directory && rm -f some-directory no-such-file some-file; echo exit code: $?; ls -la
# rm: some-directory: is a directory
# exit code: 1
# total 0
# drwxr-xr-x 3 sebastiaan staff 96 Aug 14 14:15 .
# drwxr-xr-x 199 sebastiaan staff 6368 Aug 14 12:13 ..
# drwxr-xr-x 2 sebastiaan staff 64 Aug 14 14:15 some-directory
This patch updates the `docker rm` / `docker container rm` command to not produce
an error when attempting to remove a missing containers, and instead only print
the error, but return a zero (0) exit code.
With this patch applied:
docker create --name mycontainer busybox \
&& docker rm nosuchcontainer mycontainer; \
echo exit code: $?; \
docker ps -a --filter name=mycontainer
# df23cc8573f00e97d6e948b48d9ea7d75ce3b4faaab4fe1d3458d3bfa451f39d
# mycontainer
# Error: No such container: nosuchcontainer
# exit code: 0
# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Combining `-add` and `-rm` flags on `docker service update` should
be usable to explicitly replace existing options. The current order
of processing did not allow this, causing the `-rm` flag to remove
properties that were specified in `-add`. This behavior was inconsistent
with (for example) `--host-add` and `--host-rm`.
This patch updates the behavior to first remove properties, then
add new properties.
Note that there's still some improvements to make, to make the removal
more granulas (e.g. to make `--label-rm label=some-value` only remove
the label if value matches `some-value`); these changes are left for
a follow-up.
Before this change:
-----------------------------
Create a service with two env-vars
```bash
docker service create --env FOO=bar --env BAR=baz --name=test nginx:alpine
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Env }}' test | jq .
[
"FOO=bar",
"BAR=baz"
]
```
Update the service, with the intent to replace the value of `FOO` for a new value
```bash
docker service update --env-rm FOO --env-add FOO=updated-foo test
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Env }}' test | jq .
[
"BAR=baz"
]
```
Create a service with two labels
```bash
docker service create --label FOO=bar --label BAR=baz --name=test nginx:alpine
docker service inspect --format '{{json .Spec.Labels }}' test | jq .
{
"BAR": "baz",
"FOO": "bar"
}
```
Update the service, with the intent to replace the value of `FOO` for a new value
```bash
docker service update --label-rm FOO --label-add FOO=updated-foo test
docker service inspect --format '{{json .Spec.Labels }}' test | jq .
{
"BAR": "baz"
}
```
Create a service with two container labels
```bash
docker service create --container-label FOO=bar --container-label BAR=baz --name=test nginx:alpine
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Labels }}' test | jq .
{
"BAR": "baz",
"FOO": "bar"
}
```
Update the service, with the intent to replace the value of `FOO` for a new value
```bash
docker service update --container-label-rm FOO --container-label-add FOO=updated-foo test
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Labels }}' test | jq .
{
"BAR": "baz",
}
```
With this patch applied:
--------------------------------
Create a service with two env-vars
```bash
docker service create --env FOO=bar --env BAR=baz --name=test nginx:alpine
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Env }}' test | jq .
[
"FOO=bar",
"BAR=baz"
]
```
Update the service, and replace the value of `FOO` for a new value
```bash
docker service update --env-rm FOO --env-add FOO=updated-foo test
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Env }}' test | jq .
[
"BAR=baz",
"FOO=updated-foo"
]
```
Create a service with two labels
```bash
docker service create --label FOO=bar --label BAR=baz --name=test nginx:alpine
docker service inspect --format '{{json .Spec.Labels }}' test | jq .
{
"BAR": "baz",
"FOO": "bar"
}
```
Update the service, and replace the value of `FOO` for a new value
```bash
docker service update --label-rm FOO --label-add FOO=updated-foo test
docker service inspect --format '{{json .Spec.Labels }}' test | jq .
{
"BAR": "baz",
"FOO": "updated-foo"
}
```
Create a service with two container labels
```bash
docker service create --container-label FOO=bar --container-label BAR=baz --name=test nginx:alpine
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Labels }}' test | jq .
{
"BAR": "baz",
"FOO": "bar"
}
```
Update the service, and replace the value of `FOO` for a new value
```bash
docker service update --container-label-rm FOO --container-label-add FOO=updated-foo test
docker service inspect --format '{{json .Spec.TaskTemplate.ContainerSpec.Labels }}' test | jq .
{
"BAR": "baz",
"FOO": "updated-foo"
}
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When doing `docker service inspect --pretty` on services without
`TaskTemplate.Resources` or `TaskTemplate.Resources.Limits`, the command
fails. This is due to a missing check on ResourceLimitPids().
This bug has been introduced by 395a6d560d
and produces following error message:
```
Template parsing error: template: :139:10: executing "" at <.ResourceLimitPids>: error calling ResourceLimitPids: runtime error: invalid memory address or nil pointer dereference
```
Signed-off-by: Albin Kerouanton <albin@akerouanton.name>
Both libaries provide similar functionality. We're currently using
Google Shlex in more places, so prefering that one for now, but we
could decide to switch to mattn/go-shellwords in future if that
library is considered better (it looks to be more actively maintained,
but that may be related to it providing "more features").
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In situations where `~/.docker/config.json` was a symlink, saving
the file would replace the symlink with a file, instead of updating
the target file location;
mkdir -p ~/.docker
touch ~/real-config.json
ln -s ~/real-config.json ~/.docker/config.json
ls -la ~/.docker/config.json
# lrwxrwxrwx 1 root root 22 Jun 23 12:34 /root/.docker/config.json -> /root/real-config.json
docker login
# Username: thajeztah
# Password:
# Login Succeeded
ls -la ~/.docker/config.json
-rw-r--r-- 1 root root 229 Jun 23 12:36 /root/.docker/config.json
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The wrapping made the code harder to read (and in some cases destracted
from the actual code flow).
Some of these functions take too many arguments; instead of hiding that,
it probably better to make it apparent that something needs to be done
(and fix it :-)).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- change `validateResolveImageFlag()` to only perform _validation_,
and not combine it with modifying the option.
- use a `switch` instead of `if` in `validateResolveImageFlag()`
- `deployServices()`: break up some `switch` cases to make them
easier to read/understand the logic.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- TestParseRunAttach: use subtests to reduce cyclomatic complexity
- TestParseRunWithInvalidArgs: use subtests, and check if the expected
error is returned.
- Removed parseMustError() as it was mostly redundant
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Tibor Vass
Sebastiaan van Stijn
Sebastiaan van Stijn
Jorge Vallecillo
Jennings Zhang
Albin Kerouanton
Olli Janatuinen
Brian Goff
horpto
Tonis Tiigi
Wang Yumu
Silvin Lubecki
Brian Wieder