This change adds the ability to do --tail and --since on docker service
logs. It wires up the API endpoints to each other and fixes some older
bugs. It adds integration tests for these new features.
Signed-off-by: Drew Erny <drew.erny@docker.com>
In https://github.com/torvalds/linux/commit/5ca3726 (released in v4.7-rc1) the
content of the `cpuacct.usage_percpu` file in sysfs was changed to include both
online and offline cpus. This broke the arithmetic in the stats helpers used by
`docker stats`, since it was using the length of the PerCPUUsage array as a
proxy for the number of online CPUs.
Add current number of online CPUs to types.StatsJSON and use it in the
calculation.
Keep a fallback to `len(v.CPUStats.CPUUsage.PercpuUsage)` so this code
continues to work when talking to an older daemon. An old client talking to a
new daemon will ignore the new field and behave as before.
Fixes#28941.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
- Align output. Previously, output would end up unaligned because of
longer task names (e.g. web.1 vs web.10)
- Truncate task IDs and add a --no-trunc option
- Added a --no-ids option to remove IDs altogether
- Got rid of the generic ID Resolver as we need more customization.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Server-side rollback can take advantage of the rollback-specific update
parameters, instead of being treated as a normal update that happens to
go back to a previous version of the spec.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Without this fix the error the client might see is:
target is unknown
which wasn't helpful to me when I saw this today. With this fix I
now see:
MediaType is unknown: 'text/html'
which helped me track down the issue to the registry I was talking to.
Signed-off-by: Doug Davis <dug@us.ibm.com>
This adds support for placement preferences in Swarm services.
- Convert PlacementPreferences between GRPC API and HTTP API
- Add --placement-pref, --placement-pref-add and --placement-pref-rm to CLI
- Add support for placement preferences in service inspect --pretty
- Add integration test
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The overlay(2) drivers were moved up in the list of storage drivers,
and are known to have problems if the backing filesystem does not
support d_type.
Commit 2e20e63da2a8a0ffbbb3f2146f87559e17f43046 added a warning,
which is logged in the daemon logs, however, many users do not
check those logs, and may overlook this warning.
This patch adds the same warning to the output of `docker info`
so that the warning is more easily found.
In addition, the output of warnings printed by `docker info` is
optimized, by;
- moving all warnings to the _end_ of the output, instead of
mixing them with the regular output
- wrapping the storage-driver warnings, so that they are more
easily readable
Example output with this patch applied
============================================
devicemapper using loopback devices:
...
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
overlay2 on xfs without d_type support;
...
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: overlay2: the backing xfs filesystem is formatted without d_type support, which leads to incorrect behavior.
Reformat the filesystem with ftype=1 to enable d_type support.
Running without d_type support will not be supported in future releases.
WARNING: bridge-nf-call-iptables is disabled
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fix tries to address the issue raised in 25696 where
it was not possible to specify `--stop-signal` for `docker service create`
and `docker service update`, in order to use special signal to stop
the container.
This fix adds `--stop-signal` and update the `StopSignal` in `Config`
through `service create` and `service update`.
Related docs has been updated.
Integration test has been added.
This fix fixes 25696.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix ignore some functions in the Go template when header is
redendered, so that `--format "{{truncate .ID 1}}"` will still
be able to redener the header correctly.
Additional test cases have been added to the unit test.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is an attempt to address
https://github.com/docker/docker/pull/28213#issuecomment-273840405
Currently when specify table format with table `--format "table {{.ID}}..."`,
the delimiter in the header section of the table is always `"\t"`.
That is actually different from the content of the table as the delimiter
could be anything (or even contatenated with `.`, for example):
```
$ docker service ps web --format 'table {{.Name}}.{{.ID}}' --no-trunc
NAME ID
web.1.inyhxhvjcijl0hdbu8lgrwwh7
\_ web.1.p9m4kx2srjqmfms4igam0uqlb
```
This fix is an attampt to address the skewness of the table when delimiter
is not `"\t"`.
The basic idea is that, when header consists of `table` key, the header section
will be redendered the same way as content section. A map mapping each
placeholder name to the HEADER entry name is used for the context of the header.
Unit tests have been updated and added to cover the changes.
This fix is related to #28313.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 27189 where
it is not possible to support configured formatting stored in
config.json.
Since `--format` was not supported in `docker service ps`,
the flag `--format` has also been added in this fix.
This fix
1. Add `--format` to `docker service ps`
2. Add `tasksFormat` to config.json
3. Add `--format` to `docker stack ps`
4. Add `--format` to `docker node ps`
The related docs has been updated.
An integration test has been added.
This fix fixes 27189.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This commit fixes panic when execute stats command:
* use --format {{.Name}} with --all when there're exited containers.
* use --format {{.Name}} while stating exited container.
The root cause is when stating an exited container, the result from the
api didn't contain the Name and ID field, which will make format
process panic.
Panic log is like this:
```
panic: runtime error: slice bounds out of range [recovered]
panic: runtime error: slice bounds out of range
goroutine 1 [running]:
panic(0xb20f80, 0xc420014110)
/usr/local/go/src/runtime/panic.go:500 +0x1a1
text/template.errRecover(0xc4201773e8)
/usr/local/go/src/text/template/exec.go:140 +0x2ad
panic(0xb20f80, 0xc420014110)
/usr/local/go/src/runtime/panic.go:458 +0x243
github.com/docker/docker/cli/command/formatter.(*containerStatsContext).Name(0xc420430160,
0x0, 0x0)
/go/src/github.com/docker/docker/cli/command/formatter/stats.go:148
+0x86
reflect.Value.call(0xb9a3a0, 0xc420430160, 0x2213, 0xbe3657, 0x4,
0x11bc9f8, 0x0, 0x0, 0x4d75b3, 0x1198940, ...)
/usr/local/go/src/reflect/value.go:434 +0x5c8
reflect.Value.Call(0xb9a3a0, 0xc420430160, 0x2213, 0x11bc9f8, 0x0, 0x0,
0xc420424028, 0xb, 0xb)
/usr/local/go/src/reflect/value.go:302 +0xa4
text/template.(*state).evalCall(0xc420177368, 0xb9a3a0, 0xc420430160,
0x16, 0xb9a3a0, 0xc420430160, 0x2213, 0x1178fa0, 0xc4203ea330,
0xc4203de283, ...)
/usr/local/go/src/text/template/exec.go:658 +0x530
```
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.
Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
We ignored errors for simple syntax in `PortOpt` (missed that in the
previous migration of this code). This make sure we don't ignore
`nat.Parse` errors.
Test has been migrate too (errors are not exactly the same as before
though -_-)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
This adjusts `command.PromptForConfirmation` in `cli/command/utils.go` to use `bufio`'s `ReadLine` rather than using `fmt.Fscan` for reading input, which makes `<Enter>` properly accept the default value of "No" as one would expect.
This new code actually came from `cli/command/plugin/install.go`'s `acceptPrivileges` function, which I've also refactored here to use `command.PromptForConfirmation` as it should.
Additionally, this updates `cli/command/plugin/upgrade.go`'s `runUpgrade` function to use `command.PromptForConfirmation` for further consistency.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
This fix is an attempt to fix issue raised in #28005 where
`docker stats` on Windows shows Linux headers if there is
no containers in stats.
The reason for the issue is that, in case there is no container,
a header is faked in:
https://github.com/docker/docker/blob/v1.13.0/cli/command/formatter/formatter.go#L74-L78
which does not know OS type information (as OS was stored with container stat entries)
This fix tries to fix the issue by moving OS type information
to stats context (instead of individual container stats entry).
Additional unit tests have been added.
This fix fixes#28005.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This allows a plugin to be upgraded without requiring to
uninstall/reinstall a plugin.
Since plugin resources (e.g. volumes) are tied to a plugin ID, this is
important to ensure resources aren't lost.
The plugin must be disabled while upgrading (errors out if enabled).
This does not add any convenience flags for automatically
disabling/re-enabling the plugin during before/after upgrade.
Since an upgrade may change requested permissions, the user is required
to accept permissions just like `docker plugin install`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix tries to improve the display of `docker service ls`
and adds `--format` flag to `docker service ls`.
In addition to `--format` flag, several other improvement:
1. Updates `docker stacks service`.
2. Adds `servicesFormat` to config file.
Related docs has been updated.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 29972 where
it was not possible to specify `--read-only` for `docker service create`
and `docker service update`, in order to have the container's root file
system to be read only.
This fix adds `--read-only` and update the `ReadonlyRootfs` in `HostConfig`
through `service create` and `service update`.
Related docs has been updated.
Integration test has been added.
This fix fixes 29972.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix made several updates:
1. Update opts.MemBytes so that default value will not show up.
The reason is that in case a default value is decided by daemon,
instead of client, we actually want to not show default value.
2. Move `docker run/create/build` to use opts.MemBytes for `--shm-size`
This is to bring consistency between daemon and docker run
3. docs updates.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix fixes issue raised in 29492 where it was not
possible to specify a default `--default-shm-size` in daemon
configuration for each `docker run``.
The flag `--default-shm-size` which is reloadable, has been
added to the daemon configuation.
Related docs has been updated.
This fix fixes 29492.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is a follow up for comment:
https://github.com/docker/docker/pull/28896#issuecomment-265392703
Currently secret name or ID prefix resolving is done at the client
side, which means different behavior of API and CMD.
This fix moves the resolving from client to daemon, with exactly the
same rule:
- Full ID
- Full Name
- Partial ID (prefix)
All existing tests should pass.
This fix is related to #288896, #28884 and may be related to #29125.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix adds `--filter enabled=true` to `docker plugin ls`,
as was specified in 28624.
The related API and docs has been updated.
An integration test has been added.
This fix fixes 28624.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This introduce a new `--device-cgroup-rule` flag that allow a user to
add one or more entry to the container cgroup device `devices.allow`
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Docker special-cases build-contexts starting with `github.com`, and
treats them as remote URLs.
Because of this special treatment, local build contexts in a directory
named "github.com" are ignored by `docker build`.
This patch changes the way the build-context is detected and first
checks if a local path with the given name exists before considering
it to be a remote URL.
Before this change;
$ mkdir -p github.com/foo/bar && echo -e "FROM scratch\nLABEL iam=local" > github.com/foo/bar/Dockerfile
$ docker build -t dont-ignore-me github.com/foo/bar
Username for 'https://github.com':
After this change;
$ mkdir -p github.com/foo/bar && echo -e "FROM scratch\nLABEL iam=local" > github.com/foo/bar/Dockerfile
$ docker build -t dont-ignore-me github.com/foo/bar
Sending build context to Docker daemon 2.048 kB
Step 1/2 : FROM scratch
--->
Step 2/2 : LABEL iam local
---> Using cache
---> ae2c603fe970
Successfully built ae2c603fe970
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When Size was reverted to be equal to VirtualSize, the df command
formatter was not correctly updated to account for the change.
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
This fix tries to fix the issue in 30279 where `docker ps --format`
fails if `.Label` has args. For example:
```
docker ps --format '{{.ID}}\t{{.Names}}\t{{.Label "some.label"}}'
```
The reason for the failure is that during the preprocessing phase
to detect the existance of `.Size`, the `listOptionsProcessor`
does not has a method of `Label(name string) string`.
This results in the failure of
```
template: :1:24: executing "" at <.Label>: Label is not a method but has arguments
```
This fix fixes the issue by adding needed method of `Label(name string) string`.
This fix fixes 30279.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Commit ed13c3abfb242905ec012e8255dc6f26dcf122f6 added flags
for Docker Content Trust. Depending on the `verify` boolean,
the message is "Skip image verification", or "Skip image signing".
"Signing" is intended for `docker push` / `docker plugin push`.
During the migration to Cobra, this boolean got flipped for
`docker push` (9640e3a4514f96a890310757a09fd77a3c70e931),
causing `docker push` to show the incorrect flag description.
This patch changes the flags to use the correct description
for `docker push`, and `docker plugin push`.
To prevent this confusion in future, the boolean argument
is removed, and a `AddTrustSigningFlags()` function is added.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fix tries to address the enhancement discussed in 28735 to add
`--format` for the output of `docker plugin ls`.
This fix
1. Add `--format` and `--quiet` flags to `docker plugin ls`
2. Convert the current implementation to use `formatter`, consistent with
other docker list commands.
3. Add `pluginsFormat` for config.json.
Related docs has been updated.
Several unit tests have been added to cover the changes.
This fix is related to 28708 and 28735.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Use resolving to repo info as the split point between the
legitimate reference package and forked reference package.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
How to test (it should not print `{}`, and just returns JSON with the actual data):
$ docker images --format '{{json .}}'
$ docker container stats --format '{{json .}}'
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Docker 1.13 moves the `--rm` flag to the daemon,
through an AutoRemove option in HostConfig.
When using API 1.24 and under, AutoRemove should not be
used, even if the daemon is version 1.13 or above and
"supports" this feature.
This patch fixes a situation where an 1.13 client,
talking to an 1.13 daemon, but using the 1.24 API
version, still set the AutoRemove property.
As a result, both the client _and_ the daemon
were attempting to remove the container, resulting
in an error:
ERRO[0000] error removing container: Error response from daemon:
removal of container ce0976ad22495c7cbe9487752ea32721a282164862db036b2f3377bd07461c3a
is already in progress
In addition, the validation of conflicting options
is moved from `docker run` to `opts.parse()`, so
that conflicting options are also detected when
running `docker create` and `docker start` separately.
To resolve the issue, the `AutoRemove` option is now
always set to `false` both by the client and the
daemon, if API version 1.24 or under is used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch fixed below 4 types of code line
1. Remove unnecessary variable assignment
2. Use variables declaration instead of explicit initial zero value
3. Change variable name to underbar when variable not used
4. Add erro check and return for ignored error
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
The prune PR changed the meaning of the file to mean "space on disk
only unique to this image", this PR revert this change.
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
This fix adds a new flag `--availability` to `swarm join`.
Related documentation has been updated.
An integration test has been added.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 24596 where it was not
possible to join as manager only (`--availability=drain`).
This fix adds a new flag `--availability` to `swarm join`.
Related documentation has been updated.
An integration test has been added.
NOTE: Additional pull request for swarmkit and engine-api will
be created separately.
This fix fixes 24596.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
- Make sure we use the correct network name for external ones.
- Make the default network overridable and only creates networks that
are used by services — so that default network is only created if a
service doesn't declare a network.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Start work on adding unit tests to our cli code in order to have to
write less costly integration test.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This fix tries to add a placeholder `.CreatedAt` for Go
format template in `docker network ls --format`.
While working on 29226, I noticed that it is not possible to
display network's creation time in `docker network ls`, with or
without `--format`.
We are able to find the timestamp through `docker network inspect` though.
However, as we allows networks to be pruned based on the timestamp
(see 29226), showing the timestamp in `docker network ls --format`
would be much useful now.
This fix adds the `.CreatedAt` placeholder for `docker network ls --format`.
The default output was not changed for `docker network ls --format`.
A test case for unit tests has been added.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is a follow up for comment
https://github.com/docker/docker/pull/28535#issuecomment-263215225
This fix provides `--filter until=<timestamp>` for `docker container/image prune`.
This fix adds `--filter until=<timestamp>` to `docker container/image prune`
so that it is possible to specify a timestamp and prune those containers/images
that are earlier than the timestamp.
Related docs has been updated
Several integration tests have been added to cover changes.
This fix fixes#28497.
This fix is related to #28535.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
When building a Dockerfile from a Windows client on a Linux daemon, a
"security warning" is printed
on stderr. Having this warning printed on stderr makes it difficult to
distinguish a failed build from one that's succeeding, and the only way to
suppress the warning is through the -q option, which also suppresses every
output. This change prints the warning on stdout, instead of stderr, to
resolve this situation.
When using both `--secret-rm` and `--secret-add` on `docker service update`,
`--secret-rm` was always performed last. This made it impossible to update
a secret that was already in use on a service (for example, to change
it's permissions, or mount-location inside the container).
This patch changes the order in which `rm` and `add` are performed,
allowing updating a secret in a single `docker service update`.
Before this change, the `rm` was always performed "last", so the secret
was always removed:
$ echo "foo" | docker secret create foo -f -
foo
$ docker service create --name myservice --secret foo nginx:alpine
62xjcr9sr0c2hvepdzqrn3ssn
$ docker service update --secret-rm foo --secret-add source=foo,target=foo2 myservice
myservice
$ docker service inspect --format '{{ json .Spec.TaskTemplate.ContainerSpec.Secrets }}' myservice | jq .
null
After this change, the `rm` is performed _first_, allowing users to
update a secret without updating the service _twice_;
$ echo "foo" | docker secret create foo -f -
1bllmvw3a1yaq3eixqw3f7bjl
$ docker service create --name myservice --secret foo nginx:alpine
lr6s3uoggli1x0hab78glpcxo
$ docker service update --secret-rm foo --secret-add source=foo,target=foo2 myservice
myservice
$ docker service inspect --format '{{ json .Spec.TaskTemplate.ContainerSpec.Secrets }}' myservice | jq .
[
{
"File": {
"Name": "foo2",
"UID": "0",
"GID": "0",
"Mode": 292
},
"SecretID": "tn9qiblgnuuut11eufquw5dev",
"SecretName": "foo"
}
]
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Brian Goff
Daniel Nephin
Justin Cormack
Vincent Demeester
Sebastiaan van Stijn
Boaz Shuster
Tibor Vass
Drew Erny
Ian Campbell
allencloud
Ying Li
Harald Albers
Andrea Luzzardi
Aaron Lehmann
Yong Tang
Kenfe-Mickaël Laventure
yuexiao-wang
Doug Davis
Akihiro Suda
Genki Takiuchi
Sebastiaan van Stijn
yupengzte
fate-grand-order
Victor Vieux
Krasi Georgiev
Reficul
Tony Abboud
Anusha Ragunathan
Alexander Morozov
Nishant Totla
Aaron.L.Xu
Zhang Wei
Derek McGowan
Tianon Gravi
John Howard
kaiwentan
Josh Hawn
Daehyeok Mun
Dong Chen
ttronicum
Stephen J Day
Victor Vieux
Tõnis Tiigi