When creating and updating services, we need to avoid unneeded service churn.
The interaction of separate lists to "add" and "drop" capabilities, a special
("ALL") capability, as well as a "relaxed" format for accepted capabilities
(case-insensitive, `CAP_` prefix optional) make this rather involved.
This patch updates how we handle `--cap-add` / `--cap-drop` when _creating_ as
well as _updating_, with the following rules/assumptions applied:
- both existing (service spec) and new (values passed through flags or in
the compose-file) are normalized and de-duplicated before use.
- the special "ALL" capability is equivalent to "all capabilities" and taken
into account when normalizing capabilities. Combining "ALL" capabilities
and other capabilities is therefore equivalent to just specifying "ALL".
- adding capabilities takes precedence over dropping, which means that if
a capability is both set to be "dropped" and to be "added", it is removed
from the list to "drop".
- the final lists should be sorted and normalized to reduce service churn
- no validation of capabilities is handled by the client. Validation is
delegated to the daemon/server.
When deploying a service using a docker-compose file, the docker-compose file
is *mostly* handled as being "declarative". However, many of the issues outlined
above also apply to compose-files, so similar handling is applied to compose
files as well to prevent service churn.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
* Added two new modes accepted by the `--mode` flag
* `replicated-job` creates a replicated job
* `global-job` creates a global job.
* When using `replicated-job` mode, the `replicas` flag sets the
`TotalCompletions` parameter of the job. This is the total number of
tasks that will run
* Added a new flag, `max-concurrent`, for use with `replicated-job`
mode. This flag sets the `MaxConcurrent` parameter of the job, which
is the maximum number of replicas the job will run simultaneously.
* When using `replicated-job` or `global-job` mode, using any of the
update parameter flags will result in an error, as jobs cannot be
updated in the traditional sense.
* Updated the `docker service ls` UI to include the completion status
(completed vs total tasks) if the service is a job.
* Updated the progress bars UI for service creation and update to
support jobs. For jobs, there is displayed a bar covering the overall
progress of the job (the number of tasks completed over the total
number of tasks to complete).
* Added documentation explaining the use of the new flags, and of jobs
in general.
Signed-off-by: Drew Erny <derny@mirantis.com>
The github.com/flynn-archive/go-shlex package is a fork of Google/shlex,
and the repository is now archived, so let's switch to the maintained
version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Updates the CredentialSpec handling code for services to allow using
swarm Configs.
Additionally, fixes a bug where the `--credential-spec` flag would not
be respected on service updates.
Signed-off-by: Drew Erny <drew.erny@docker.com>
This patch fixes a bug where labels use the same behavior as `--env`, resulting
in a value to be copied from environment variables with the same name as the
label if no value is set (i.e. a simple key, no `=` sign, no value).
An earlier pull request addressed similar cases for `docker run`;
2b17f4c8a8, but this did not address the
same situation for (e.g.) `docker service create`.
Digging in history for this bug, I found that use of the `ValidateEnv`
function for labels was added in the original implementation of the labels feature in
abb5e9a077 (diff-ae476143d40e21ac0918630f7365ed3cR34)
However, the design never intended it to expand environment variables,
and use of this function was either due to either a "copy/paste" of the
equivalent `--env` flags, or a misunderstanding (the name `ValidateEnv` does
not communicate that it also expands environment variables), and the existing
`ValidateLabel` was designed for _engine_ labels (which required a value to
be set).
Following the initial implementation, other parts of the code followed
the same (incorrect) approach, therefore leading the bug to be introduced
in services as well.
This patch:
- updates the `ValidateLabel` to match the expected validation
rules (this function is no longer used since 31dc5c0a9a),
and the daemon has its own implementation)
- corrects various locations in the code where `ValidateEnv` was used instead of `ValidateLabel`.
Before this patch:
```bash
export SOME_ENV_VAR=I_AM_SOME_ENV_VAR
docker service create --label SOME_ENV_VAR --tty --name test busybox
docker service inspect --format '{{json .Spec.Labels}}' test
{"SOME_ENV_VAR":"I_AM_SOME_ENV_VAR"}
```
After this patch:
```bash
export SOME_ENV_VAR=I_AM_SOME_ENV_VAR
docker service create --label SOME_ENV_VAR --tty --name test busybox
docker container inspect --format '{{json .Config.Labels}}' test
{"SOME_ENV_VAR":""}
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since go 1.7, "context" is a standard package. Since go 1.9,
x/net/context merely provides some types aliased to those in
the standard context package.
The changes were performed by the following script:
for f in $(git ls-files \*.go | grep -v ^vendor/); do
sed -i 's|golang.org/x/net/context|context|' $f
goimports -w $f
for i in 1 2; do
awk '/^$/ {e=1; next;}
/\t"context"$/ {e=0;}
{if (e) {print ""; e=0}; print;}' < $f > $f.new && \
mv $f.new $f
goimports -w $f
done
done
[v2: do awk/goimports fixup twice]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The "update" and "rollback" configurations were cross-wired, as a result, setting
`--rollback-*` options would override the service's update-options.
Creating a service with both update, and rollback configuration:
docker service create \
--name=test \
--update-failure-action=pause \
--update-max-failure-ratio=0.6 \
--update-monitor=3s \
--update-order=stop-first \
--update-parallelism=3 \
--rollback-failure-action=continue \
--rollback-max-failure-ratio=0.5 \
--rollback-monitor=4s \
--rollback-order=start-first \
--rollback-parallelism=2 \
--tty \
busybox
Before this change:
docker service inspect --format '{{json .Spec.UpdateConfig}}' test \
&& docker service inspect --format '{{json .Spec.RollbackConfig}}' test
Produces:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
After this change:
{"Parallelism":3,"FailureAction":"pause","Monitor":3000000000,"MaxFailureRatio":0.6,"Order":"stop-first"}
{"Parallelism":2,"FailureAction":"continue","Monitor":4000000000,"MaxFailureRatio":0.5,"Order":"start-first"}
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
`--label-file` has the exact same behavior as `--env-file`, meaning any
placeholder (i.e. a simple key, no `=` sign, no value), it will get the
value from the environment variable.
For `--label-file` it should just add an empty label.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
When adding a network using `docker service update --network-add`,
the new network was added by _name_.
Existing entries in a service spec are listed by network ID, which
resulted in the CLI not detecting duplicate entries for the same
network.
This patch changes the behavior to always use the network-ID,
so that duplicate entries are correctly caught.
Before this change;
$ docker network create -d overlay foo
$ docker service create --name=test --network=foo nginx:alpine
$ docker service update --network-add foo test
$ docker service inspect --format '{{ json .Spec.TaskTemplate.Networks}}' test
[
{
"Target": "9ot0ieagg5xv1gxd85m7y33eq"
},
{
"Target": "9ot0ieagg5xv1gxd85m7y33eq"
}
]
After this change:
$ docker network create -d overlay foo
$ docker service create --name=test --network=foo nginx:alpine
$ docker service update --network-add foo test
service is already attached to network foo
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
and enable the new WarnUnmatchedDirective to warn if a nolint is unnecessary.
remove some unnecessary nolint
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Commit 330a0035334871d92207b583c1c36d52a244753f added a `--detach=false` option
to various service-related commands, with the intent to make this the default in
a future version (17.09).
This patch changes the default to use "interactive" (non-detached), allowing
users to override this by setting the `--detach` option.
To prevent problems when connecting to older daemon versions (17.05 and below,
see commit db60f25561), the detach option is
ignored for those versions, and detach is always true.
Before this change, a warning was printed to announce the upcoming default:
$ docker service create nginx:alpine
saxiyn3pe559d753730zr0xer
Since --detach=false was not specified, tasks will be created in the background.
In a future release, --detach=false will become the default.
After this change, no warning is printed, but `--detach` is disabled;
$ docker service create nginx:alpine
y9jujwzozi0hwgj5yaadzliq6
overall progress: 1 out of 1 tasks
1/1: running [==================================================>]
verify: Service converged
Setting the `--detach` flag makes the cli use the pre-17.06 behavior:
$ docker service create --detach nginx:alpine
280hjnzy0wzje5o56gr22a46n
Running against a 17.03 daemon, without specifying the `--detach` flag;
$ docker service create nginx:alpine
kqheg7ogj0kszoa34g4p73i8q
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fix use `scope=swarm` for service related network inspect.
The purpose is that, in case multiple networks with the same
name exist in different scopes, it is still possible to obtain
the network for services.
This fix is related to moby/moby#33630 and docker/cli#167
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Sebastiaan van Stijn
Jim Lin
Albin Kerouanton
Drew Erny
Drew Erny
Olli Janatuinen
Akihiro Suda
Arash Deshmeh
Vincent Demeester
Vincent Demeester
Kir Kolyshkin
Renaud Gaubert
Simon Ferquel
Daniel Nephin
Victor Vieux
Yong Tang
Sebastiaan van Stijn
Abhinandan Prativadi
Nishant Totla
Aaron Lehmann
Gaetan de Villele
Tibor Vass
Dong Chen
Daniel Nephin