network is `nil` if the following case:
```
services:
foo:
image: nginx
networks:
mynetwork:
```
It's a valid compose so we should not panic.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
the service definition uses the format as defined
in http://man7.org/linux/man-pages/man5/hosts.5.html
(IP_address canonical_hostname [aliases...])
This format is the _reverse_ of the format used in
the container API.
Commit f32869d956eb175f88fd0b16992d2377d8eae79c
inadvertently used the incorrect order.
This fixes the order, and correctly sets it to;
IP-Address hostname
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Modify the service update and create APIs to return optional warning
messages as part of the response. Populate these messages with an
informative reason when digest resolution fails.
This is a small API change, but significantly improves the UX. The user
can now get immediate feedback when they've specified a nonexistent
image or unreachable registry.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
use secret store interface instead of embedded secret data into container
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
For some reason, `go vet` and `go fmt` validate does not capture
several issues.
The following was the output of `go vet`:
```
ubuntu@ubuntu:~/docker$ go vet ./... 2>&1 | grep -v ^vendor | grep -v '^exit status 1$'
cli/command/formatter/container_test.go:393: possible formatting directive in Log call
volume/volume_test.go:257: arg mp.RW for printf verb %s of wrong type: bool
```
The following was the output of `go fmt -s`:
```
ubuntu@ubuntu:~/docker$ gofmt -s -l . | grep -v ^vendor
cli/command/stack/list.go
daemon/commit.go
```
Fixed above issues with `go vet` and `go fmt -s`
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to fix the crash caused by `docker service inspect --pretty`,
by performing necessary nil pointer check.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is an attempt to address the issue raised in 28339. In
`docker ps`, the formatter needs to expose all fields of `types.Container`
to `preProcessor` so that template could be executed.
This direct exposing is unreliable and could cause issues as user may incorrectly
assume all fields in `types.Container` will be available for templating.
However, the purpose of `preProcessor` is to only find out if `.Size`
is defined (so that opts.size could be set accordingly).
This fix defines `preProcessor` as `map[string]bool` with a func `Size()`.
In this way, any unknown fields will be ignored.
This fix adds several test cases to the existing `TestBuildContainerListOptions`.
This fix fixes 28339.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
A recent PR added `(ns|us|ms|s|m|h)` to the descriptions of some
duration options, but not all. Add it to the remaining options for
consistency.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
As is specified in 28018, it would be useful to know the manager's addresses
even in a worker node. This is especially useful when there are many
worker nodes in a big cluster.
The information is available in `info.Swarm.RemoteManagers`.
This fix add the information of `Manager Addresses` to the output
of `docker info`, to explicitly show it.
A test has been added for this fix.
This fix fixes 28018.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
… for `docker images`.
This deprecates the `filter` param for the `/images` endpoint and make a
new filter called `reference` to replace it. It does change the CLI
side (still possible to do `docker images busybox:musl`) but changes the
cli code to use the filter instead (so that `docker images --filter
busybox:musl` and `docker images busybox:musl` act the same).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
This change incorporates feedback from @thaJeztah in the PR that added
the autolock flag. It changes the descriptions to be different for
"swarm init" and "swarm update" so that the boolean nature so that the
purpose of the flag in both contexts is clearer.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix tries to address 27902 by adding a flag `--host`
to `docker service create` and `--host-add/--host-rm` to
`docker service update`, so that it is possible to
specify extra `host:ip` settings in `/etc/hosts`.
This fix adds `Hosts` in swarmkit's `ContainerSpec` so that it
is possible to specify extra hosts during service creation.
Related docs has been updated.
An integration test has been added.
This fix fixes 27902.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
- Neither swarm init or swarm update should take an unlock key
- Add an autolock flag to turn on autolock
- Make the necessary docker api changes
- Add SwarmGetUnlockKey API call and use it when turning on autolock
- Add swarm unlock-key subcommand
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix is a follow up to #27567 based on:
https://github.com/docker/docker/pull/27567#issuecomment-259295055
In #27567, `--dns-options` has been added to `service create/update`,
together with `--dns` and `--dns-search`. The `--dns-opt` was used
in `docker run`.
This fix add `--dns-option` (not `--dns-options`) to `docker run/create`, and hide
`--dns-opt`. It is still possible to use `--dns-opt` with
`docker run/create`, though it will not show up in help output.
This fix change `--dns-options`to --dns-option` for `docker service create`
and `docker service update`.
This fix also updates the docs and bash/zsh completion scripts.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
- use Filters instead of Filter for secret list
- UID, GID -> string
- getSecrets -> getSecretsByName
- updated test case for secrets with better source
- use golang.org/x/context instead of context
- for grpc conversion allocate with make
- check for nil with task.Spec.GetContainer()
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
- use /secrets for swarm secret create route
- do not specify omitempty for secret and secret reference
- simplify lookup for secret ids
- do not use pointer for secret grpc conversion
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
- fix lint issues
- use errors pkg for wrapping errors
- cleanup on error when setting up secrets mount
- fix erroneous import
- remove unneeded switch for secret reference mode
- return single mount for secrets instead of slice
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
This fix is based on the comment:
https://github.com/docker/docker/pull/28147#discussion_r86996347
Previously the output string of the `DurationOpt` is `duration-ptr`
and `Uint64Opt` is `uint64-ptr`. While it is clear to developers,
for a normal user `-ptr` might not be very informative.
On the other hand, the default value of `DurationOpt` and `Uint64Opt`
has already been quite informative: `none`. That means if no flag
provided, the value will be treated as none.
(like a ptr with nil as the default)
For that reason this fix removes the `-ptr`.
Also, the output in the docs of `service create` has been quite
out-of-sync with the true output. So this fix updates the docs
to have the most up-to-date help output of `service create --help`.
This fix is related to #28147.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
- Refactored to move resolution code into the idresolver
- Made `ps` output more bearable by shortening service IDs in task names
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
This fix adds `--dns-add`, `--dns-rm`, `--dns-opt-add`, `--dns-opt-rm`,
`--dns-search-add` and `--dns-search-rm` to `service update`.
An integration test and a unit test have been added to cover the changes in this fix.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to fix the issue raised in 24391 about allowing
custom DNS settings to service definition.
This fix adds `DNSConfig` (`Nameservers`, `Options`, `Search`) to
service definition, as well as `--dns`, `--dns-opt`, and `dns-search`
to `service create`.
An integration test has been added to cover the changes in this fix.
This fix fixes 24391.
A PR in swarmkit will be created separately.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Add the following options to "swarm init" and "swarm update":
- --max-snapshots: Retain this many old Raft snapshots in addition
to the latest one
- --snapshot-interval: Number of log entries between Raft snapshots
These options already existed in SwarmKit and the Docker API but were
never exposed in the CLI. I'm adding them here to fix this oversight.
--max-snapshots may be useful for debugging purposes and more
conservative users who want to store rolling backups of old versions of
the Raft state.
--snapshot-interval is most useful for performance tuning. The default
value of 10000 may not be ideal for some setups.
There is also a LogEntriesForSlowFollowers option that is not exposed. I
decided not to expose it along with these others because I don't think
it's generally useful (and I'm not sure what I would call the CLI flag).
But if people want, I can expose it for the sake of completeness.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix is based on the comment in
https://github.com/docker/docker/pull/27567#discussion_r86910604
Basically, in the help output of `docker service update`, the `--xxx-add`
flags typically have plural forms while `--xxx-rm` flags have singular
forms.
This fix updates the help output for consistency.
This fix also updates the related docs in `service_update.md`.
The help output in `service_update.md` has been quite out-of-sync
with the actual output so this fix replaces the output with the
most up-to-date output.
This fix is related to #27567.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Signed-off-by: Victor Vieux <vieux@docker.com>
update cobra and use Tags
Signed-off-by: Victor Vieux <vieux@docker.com>
allow client to talk to an older server
Signed-off-by: Victor Vieux <vieux@docker.com>
This fix tries to add `--tty` to `docker service create/update`. As was
specified in 25644, `TTY` flag has been added to SwarmKit and is
already vendored.
This fix add `--tty` to `docker service create/update`.
Related document has been updated.
Additional integration tests has been added.
This fix fixes 25644.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the proposal raised in 27921 and add
`--cpus` flag for `docker run/create`.
Basically, `--cpus` will allow user to specify a number (possibly partial)
about how many CPUs the container will use. For example, on a 2-CPU system
`--cpus 1.5` means the container will take 75% (1.5/2) of the CPU share.
This fix adds a `NanoCPUs` field to `HostConfig` since swarmkit alreay
have a concept of NanoCPUs for tasks. The `--cpus` flag will translate
the number into reused `NanoCPUs` to be consistent.
This fix adds integration tests to cover the changes.
Related docs (`docker run` and Remote APIs) have been updated.
This fix fixes 27921.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
The short description should be kept short. Spanning on several lines is
a bit ugly. A user can still get more information in the manual or we
can expand the long description instead if we want (there is currently
none).
This reverts a bit of #26683.
Signed-off-by: Vincent Bernat <vincent@bernat.im>
Manager now auto-detects the address that an agent connects to the cluster
from and stores it. This is useful for many kinds of internal cluster
management tools.
Signed-off-by: Drew Erny <drew.erny@docker.com>
This adds support to display names or id of container instead of what
was provided in the request.
This keeps the default behavior (`docker stats byname` will display
`byname` in the `CONTAINER` colmun and `docker stats byid` will display
the id in the `CONTAINER` column) but adds two new format directive.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Allow built images to be squash to scratch.
Squashing does not destroy any images or layers, and preserves the
build cache.
Introduce a new CLI argument --squash to docker build
Introduce a new param to the build API endpoint `squash`
Once the build is complete, docker creates a new image loading the diffs
from each layer into a single new layer and references all the parent's
layers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
These features were originally scheduled
for removal in docker 1.13, but we changed
our deprecation policy to keep features
for three releases instead of two.
This updates the deprecation version
to match the deprecation policy.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In file `api/types/client.go`, some of the "*Options{}" structs own a
`Filters` field while some else have the name of `Filter`, this commit
will rename all `Filter` to `Filters` for consistency. Also `Filters`
is consistent with API with format `/xxx?filters=xxx`, that's why
`Filters` is the right name.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
To load an image from a tar file, you can specify
the tar file in the -i/--input option:
docker load -i image_1.tar
or using stdin:
docker load < image_1.tar
cat image_1.tat | docker load
If the image file isn't given the `docker load`
command gets stuck.
To avoid that, the load makes sure the CLI input is
not a terminal or the `--input` option was set.
If not then an error message is shown.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
--group-add was used for specifying groups for both service create
and service update. For create it was confusing since we don't have
an existing set of groups. Instead I added --group to create, and
moved --group-add to service update only, like --group-rm
This deals with issue 27646
Signed-off-by: Lily Guo <lily.guo@docker.com>
Update flag documentation
Specify that --group, --group-add and --groupd-rm refers to
supplementary user groups
Signed-off-by: Lily Guo <lily.guo@docker.com>
Fix docs for groups and update completion scripts
Signed-off-by: Lily Guo <lily.guo@docker.com>
As for `ps`, `images`, `network ls` and `volume ls`, this makes it
possible to define a custom default format.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
A HealthConfig entry was added to the ContainerSpec associated with the
service being created or updated.
Signed-off-by: Cezar Sa Espinola <cezarsa@gmail.com>
This fix tries to address the issue in 24712 and add
`--env-file` file to `docker create service`.
Related documentation has been updated.
An additional integration has been added.
This fix fixes 24712.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
The --name flag was inadvertently added to
docker service update, but is not supported,
as it has various side-effects (e.g., existing
tasks are not renamed).
This removes the flag from the service update
command.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
containers may specify these cgroup values at runtime. This will allow
processes to change their priority to real-time within the container
when CONFIG_RT_GROUP_SCHED is enabled in the kernel. See #22380.
Also added sanity checks for the new --cpu-rt-runtime and --cpu-rt-period
flags to ensure that that the kernel supports these features and that
runtime is not greater than period.
Daemon will support a --cpu-rt-runtime flag to initialize the parent
cgroup on startup, this prevents the administrator from alotting runtime
to docker after each restart.
There are additional checks that could be added but maybe too far? Check
parent cgroups to ensure values are <= parent, inspecting rtprio ulimit
and issuing a warning.
Signed-off-by: Erik St. Martin <alakriti@gmail.com>
`docker network prune` prunes unused networks, including overlay ones.
`docker system prune` also prunes unused networks.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Currently, there's no way to restart the tasks of a service without
making an actual change to the service. This leads to us giving awkward
workarounds as in
https://github.com/docker/docker.github.io/pull/178/files, where we tell
people to scale a service up and down to restore balance, or make
unnecessary changes to trigger a restart.
This change adds a --force option to "docker service update", which
forces the service to be updated even if no changes require that.
Since rolling update parameters are respected, the user can use
"docker service --force" to do a rolling restart. For example, the
following is supported:
docker service update --force --update-parallelism 2 \
--update-delay 5s myservice
Since the default value of --update-parallelism is 1, the default
behavior is to restart the service one task at a time.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Passing {{.Networks}} to the format parameter will prompt ps to
display all the networks the container is connected to.
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Keeping the current behavior for exec, i.e., inheriting
variables from main process. New variables will be added
to current ones. If there's already a variable with that
name it will be overwritten.
Example of usage: docker exec -it -e TERM=vt100 <container> top
Closes#24355.
Signed-off-by: Jonh Wendell <jonh.wendell@redhat.com>
This adds support for two enhancements to swarm service rolling updates:
- Failure thresholds: In Docker 1.12, a service update could be set up
to either pause or continue after a single failure occurs. This adds
an --update-max-failure-ratio flag that controls how many tasks need to
fail to update for the update as a whole to be considered a failure. A
counterpart flag, --update-monitor, controls how long to monitor each
task for a failure after starting it during the update.
- Rollback flag: service update --rollback reverts the service to its
previous version. If a service update encounters task failures, or
fails to function properly for some other reason, the user can roll back
the update.
SwarmKit also has the ability to roll back updates automatically after
hitting the failure thresholds, but we've decided not to expose this in
the Docker API/CLI for now, favoring a workflow where the decision to
roll back is always made by an admin. Depending on user feedback, we may
add a "rollback" option to --update-failure-action in the future.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix tries to address the issue raised in 24392 where
labels with duplicate keys exist in `docker info`, which
contradicts with the specifications in the docs.
The reason for duplicate keys is that labels are stored as
slice of strings in the format of `A=B` (and the input/output).
This fix tries to address this issue by checking conflict
labels when daemon started, and remove duplicate labels (K-V).
The existing `/info` API has not been changed.
An additional integration test has been added to cover the
changes in this fix.
This fix fixes 24392.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
- some function signatures have changed - use the new ones
- re-generate the notary delegation key certs, since notary doesn't allow SHA1
- fix some error message mapping because now if a root rotation fails to validate trusted operations will fail
Signed-off-by: cyli <cyli@twistedmatrix.com>
This fix tries to address the issue raised in 25855 where the command
`docker deploy` can only accept a STACK without extension of `.dab`.
In other words, `docker deploy hellojavaee.dab` gives an error:
```
Bundle hellojavaee.dab.dab not found. Specify the path with --file
```
This fix updates the way namespace STACK is taken so that in case
`STACK.dab` is provided with `docker deploy`:
```
$ docker deploy STACK.dab
```
The `STACK` is used as namespace (instead of `STACK.dab`).
NOTE: This fix will only allows `.dab` extension in namespace, because
it is not possible to have a namespace with `.` in the middle. In other
words, a namespace `hello.java.ee` will not work anyway (whether the file
`hello.java.ee` exists or not).
An additional integration test has been added to cover the changes.
This fix fixes 25855.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is related to 27049 and 27047. For `--label` flag, if string slice is
used (like 27047), then quote can not be used in command and will result in
an error :
```
line 1, column 14: bare " in non-quoted-field
```
The issue 27047 has been fixed by 27049.
Recently I found out that both `docker network create --label` and `docker volume create --label`
still use string slice and will return the same error when quotes are used.
This fix fixes `docker network create --label` and `docker volume create --label`
by using `ListOpt` (as 27049) as well.
This fix has been tested and verified manually.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
The formatter.ContainerStats struct exposes its Mutex.
This is a bad design and should be fixed.
To fix that, I separated the statistics
attributes from ContainerStats to StatsEntry and
hid the mutex. Notice that the mutex protects both
the `err` field and the statistics attributes.
Then, implemented SetStatistics, SetError, GetStatistics
and GetError to avoid races.
Moreover, to make this less granular, I decided to
replace the read-write mutex with the regular mutex and
to pass a StatsEntry slice to formatter.ContainerStatsWrite
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
When sending a build context to a remote server it may be
(significantly) advantageous to compress the build context. This commit
adds support for gz compression when constructing a build context
using a command like "docker build --compress ."
Signed-off-by: Paul Kehrer <paul.l.kehrer@gmail.com>
Those data include:
- size of data shared with other images
- size of data unique to a given image
- how many containers are using a given image
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
This fix tries to address the issue raised in 24958 where previously
`docker swarm init` will automatically fill in all the default value
(instead of letting swarmkit to handle the default).
This fix update the `swarm init` so that initial value are passed only
when a flag change has been detected.
This fix fixes 24958.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue related to 24108 and 24790, and
also the case from 24620#issuecomment-233715656
The reason for the failure case in the above mentioned issues is that
currently Task names are actually indexed by Service Name
(`e.ServiceAnnotations.Name`)
To fix it, a pull request in swarmkit (swarmkit/pull/1193) has been
opened separately.
This fix adds the integration tests for the above mentioned issues.
Swarmkit revendoring is needed to completely fix the issues.
This fix fixes 24108.
This fix fixes 24790.
This fix is related to 24620.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Currently, if the service mode is not "global", this code assumes that
Replicated is non-nil. This assumption may not be true in the future.
Instead of making the assumption, explicitly check that Replicated is
non-nil before using it.
Similarly, for limits and reservations, enclose methods that read from
Limits and Reservations within checks that those fields are non-nil.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Allows the user to use `pretty` as the format string.
This enables users to put custom format options into their CLI config
just like is supported for `docker ps` and `docker images`
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Better formatting for usage template.
Group commands in usage to management/operation commands.
Remove the word Docker from the description of management commands.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
This fix tries to address the issue raised in 26453 where bad syntax
on dockerfile is not checked before building, thus user has to wait
before seeing error in dockerfile.
This fix fixes the issue by evaluating all the instructions and check
syntax before dockerfile is invoked actually.
All existing tests pass.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
When `docker inspect` is invoked, it is possible to pass a flag of
`-s` for container types to display size information. If `-s` is used
for non-container types then a warning `WARNING: --size ignored for volume`
will show up.
However, currently `WARNING: --size ignored for volume` will show up even
when `-s` is not passed to `docker inspect` for non-container types.
This fix fixes this issue by checking if `-s` has been passed or not (`getSize`).
Also, since image inspect does not support `-s`, `IsSizeSupported` has been changed
to false for images.
This fix is tested manually.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 26300. Previously
`docker images` will use `HumanSize()` to display the size which
has a fixed precision of 4 (thus 3 fractional digits). This
could be problematic in certain languages (e.g. , German, see
26300) as `.` may be interpreted as thousands-separator in number.
This fix use `CustomSize()` instead and limit the precision to 3
(thus 2 fractional digits).
This fix has been tested manually.
This fix fixes 26300.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Anusha Ragunathan
Vincent Demeester
Daniel Nephin
Antonio Murdaca
Sebastiaan van Stijn
Nishant Totla
Aaron Lehmann
Boaz Shuster
Michael Crosby
Tõnis Tiigi
Evan Hazlett
Brian Goff
Victor Vieux
Yong Tang
Sebastiaan van Stijn
John Howard
yupeng
Jana Radhakrishnan
Lei Jitang
Derek McGowan
Victor Vieux
Andrea Luzzardi
Tibor Vass
Aanand Prasad
Andrew Hsu
yuexiao-wang
Kenfe-Mickael Laventure
milindchawre
allencloud
Akihiro Suda
WangPing
Misty Stanley-Jones
Alicia Lauerman
Josh Horwitz
Justin Cormack
Kunal Kushwaha
Vincent Bernat
Drew Erny
Nikolay Milovanov
Zhang Wei
Doug Davis
Qiang Huang
Lily Guo
Cezar Sa Espinola
boucher
Ce Gao
Erik St. Martin
sandyskies
Amit Krishnan
Jonh Wendell
cyli
Alexander Morozov
Arnaud Porterie
Paul Kehrer
Misty Stanley-Jones
Daniel Nephin
John Howard
Antonio Murdaca
sakeven
Phil Estes