This partially mitigates #1739 ("Docker commands take 1 minute to timeout if
context endpoint is unreachable") and is a simpler alternative to #1747 (which
completely defers the client connection until an actual call is attempted).
Note that the previous 60s delay was the culmination of two separate 30s
timeouts since the ping is tried twice. This with this patch the overall
timeout is 20s. https://github.com/moby/moby/pull/39206 will remove the second
ping and once that propagates to this tree the timeout will be 10s.
Signed-off-by: Ian Campbell <ijc@docker.com>
- Follows the proposal on issue [#34394](https://github.com/moby/moby/issues/34394)
- Maintains current behavior as default (Pull image if missing)
- Adds tristate flag allowing modification (PullMissing, PullAlways, PullNever)
Signed-off-by: Zander Mackie <zmackie@gmail.com>
With this patch it is possible to alias an existing allowed command.
At the moment only builder allows an alias.
This also properly puts the build command under builder, instead of image
where it was for historical reasons.
Signed-off-by: Tibor Vass <tibor@docker.com>
This is to make it easier to implement support for exporting contexts in
3rd party code, or to create mocks in tests.
2 exemples where it simplify things:
- docker-app desktop-specific context decorator (which rewrites parts of
the docker context to simplify UX when using on Docker Desktop contexts)
- ucp for including a context in the connection bundle
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
This change refines the warning message returned during docker login to
only warn for unencrypted storage when the users password is being stored.
If the remote registry supports identity tokens, omit the warning,
since those tokens can be independently managed and revoked.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
The github.com/flynn-archive/go-shlex package is a fork of Google/shlex,
and the repository is now archived, so let's switch to the maintained
version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Adds tests for setting and updating swarm service CredentialSpecs,
especially when using a Config as a credential spec.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Updates the CredentialSpec handling code for services to allow using
swarm Configs.
Additionally, fixes a bug where the `--credential-spec` flag would not
be respected on service updates.
Signed-off-by: Drew Erny <drew.erny@docker.com>
This regroup all Kubernetes extra fields for compose-on-kubernetes
v1alpha3 in a single x-kubernetes object.
Also use the same naming scheme as cap_add etc. for fiels inside this
object.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
On the server v0.4.21 has introduced a better way of dealing with
intra-stack networking: if the user can specify a list of endpoints
exposed internally, we now can setup a ClusterIP for this to avoid the
pitfalls of DNS-based load balancing.
This exposes the feature using the "Expose" compose field, and adds an
extra x-internal-service-type field to explicitly define how intra-stack
networking is handled on a service.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
For clarity, on `docker context use` or `docker context ls`, this adds a
warning if the DOCKER_HOST variable is set because it overrides the
active context.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
This refactors the way networking options are parsed, and makes the
client able to pass options for multiple networks. Currently, the
daemon does not yet accept multiple networks when creating a container,
and will produce an error.
For backward-compatibility, the following global networking-related
options are associated with the first network (in case multiple
networks are set);
- `--ip`
- `--ip6`
- `--link`
- `--link-local-ip`
- `--network-alias`
Not all of these options are supported yet in the advanced notation,
but for options that are supported, setting both the per-network option
and the global option will produce a "conflicting options" error.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The commit contains cli changes to support driver options for a network in
docker run and docker network connect cli's. The driver-opt, aliases is now
supported in the form of csv as per network option in service commands in
swarm mode since docker/cli#62 . This commit extends this support to docker
run command as well.
For docker connect command `--driver-opt` is added to pass driver specific
options for the network the container is connecting to.
Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
--from creates a context from a named context.
By default `context create` will create a context from the current context.
Replaced "from-current=" docker/kubernetes option with "from=" to allow specifying which context to copy the settings from.
Signed-off-by: Nick Adcock <nick.adcock@docker.com>
Old selector was wrong (it watched for the label we applied to child
resources when reconciling the stack, instead of the stack itself)
This should be back-ported to older version of the CLI
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
bump required:
- replacing vendor ghodss/yaml with sigs.k8s.io/yaml
- adding vendor k8s.io/klog and github.com/evanphx
- compose-on-kubernetes
removed 'IncludeUninitialized' from watch as it have been removed from k8s
Signed-off-by: Nick Adcock <nick.adcock@docker.com>
Instead of using an `if else if else`, switch to a sequence of independent
`if` blocks containing a `return`.
Instead of defining a return variable and updating it in the `if` blocks
and returning at the end, make each `if` block return the desired value
independenly.
Signed-off-by: David Scott <dave.scott@docker.com>
Previously if the Docker engine was not running the behaviour of
commands would vary depending on whether the --debug flag was provided.
For example, consider `docker logout`:
$ docker logout
Not logged in to
-- note the missing server URL
$ docker --debug logout
Warning: failed to get default registry endpoint from daemon (Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?). Using system default: https://index.docker.io/v1/
Not logged in to https://index.docker.io/v1/
-- note the server URL is present
This patch makes only the debug printing conditional on the `--debug` flag,
not the return value.
Signed-off-by: David Scott <dave.scott@docker.com>
In the initial implementation I thought it would be good to not pass on the
deprecation to plugins (since they are new). However it turns out this causes
`docker helloworld -h` to print a spurious "pflag: help requested" line:
$ docker helloworld -h
pflag: help requested
See 'docker helloworld --help'.
Usage: docker helloworld [OPTIONS] COMMAND
A basic Hello World plugin for tests
...
Compared with:
$ docker ps -h
Flag shorthand -h has been deprecated, please use --help
Usage: docker ps [OPTIONS]
This is in essence because having the flag undefined hits a different path
within cobra, causing `c.execute()` to return early due to getting an error
(`flag.ErrHelp`) from `c.ParseFlags`, which launders the error through our
`FlagErrorFunc` which wraps it in a `StatusError` which in turn defeats an `if
err == flag.ErrHelp` check further up the call chain. If the flag is defined we
instead hit a path which returns a bare `flag.ErrHelp` without wrapping it.
I considered updating our `FlagErrorFunc` to not wrap `flag.ErrHelp` (and then
following the chain to the next thing) however while doing that I realised that
the code for `-h` (and `--help`) is deeply embedded into cobra (and its flags
library) such that actually using `-h` as a plugin argument meaning something
other than `help` is basically impossible/impractical. Therefore we may as well
have plugins behave identically to the monolithic CLI and support (deprecated)
the `-h` argument.
With this changed the help related blocks of `SetupRootCommand` and
`SetupPluginRootCommand` are now identical, so consolidate into
`setupCommonRootCommand`.
Tests are updated to check `-h` in a variety of scenarios, including the happy
case here.
Signed-off-by: Ian Campbell <ijc@docker.com>
As pointed out in #1459, docker cli fails to detect that the input is a tarball,
in case it is generated by `git archive --format=tgz`.
This happens because `git archive` adds some metadata to the initial tar header,
and so it is more than 1 block (of 512 bytes) long, while we only provide 1 block
to archive/tar.Next() and it fails.
To fix, give it 2 blocks :)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This adds validation to `docker container run` / `docker container create`;
Validation of labels provided through flags was removed in 31dc5c0a9a,
after the validation was changed to fix labels without values, and to prevent
labels from being expanded with environment variables in 2b17f4c8a8
However, now empty label names from _files_ (`--label-file`) followed different
validation rules than labels passed through `--label`.
This patch adds back minimal validation for labels passed through the command-line
Before this patch:
```bash
docker container create \
--name label \
--label==with-leading-equal-sign \
--label=without-value \
--label=somelabel=somevalue \
--label " = " \
--label=with-quotes-in-value='{"foo"}' \
--label='with"quotes"in-key=test' \
busybox
docker container inspect --format '{{json .Config.Labels}}' label
```
```json
{
"": "with-leading-equal-sign",
" ": " ",
"somelabel": "somevalue",
"with\"quotes\"in-key": "test",
"with-quotes-in-value": "{\"foo\"}",
"without-value": ""
}
```
After this patch:
```bash
docker container create \
--name label \
--label==with-leading-equal-sign \
--label=without-value \
--label=somelabel=somevalue \
--label " = " \
--label=with-quotes-in-value='{"foo"}' \
--label='with"quotes"in-key=test' \
busybox
invalid argument "=with-leading-equal-sign" for "-l, --label" flag: invalid label format: "=with-leading-equal-sign"
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch fixes a bug where labels use the same behavior as `--env`, resulting
in a value to be copied from environment variables with the same name as the
label if no value is set (i.e. a simple key, no `=` sign, no value).
An earlier pull request addressed similar cases for `docker run`;
2b17f4c8a8, but this did not address the
same situation for (e.g.) `docker service create`.
Digging in history for this bug, I found that use of the `ValidateEnv`
function for labels was added in the original implementation of the labels feature in
abb5e9a077 (diff-ae476143d40e21ac0918630f7365ed3cR34)
However, the design never intended it to expand environment variables,
and use of this function was either due to either a "copy/paste" of the
equivalent `--env` flags, or a misunderstanding (the name `ValidateEnv` does
not communicate that it also expands environment variables), and the existing
`ValidateLabel` was designed for _engine_ labels (which required a value to
be set).
Following the initial implementation, other parts of the code followed
the same (incorrect) approach, therefore leading the bug to be introduced
in services as well.
This patch:
- updates the `ValidateLabel` to match the expected validation
rules (this function is no longer used since 31dc5c0a9a),
and the daemon has its own implementation)
- corrects various locations in the code where `ValidateEnv` was used instead of `ValidateLabel`.
Before this patch:
```bash
export SOME_ENV_VAR=I_AM_SOME_ENV_VAR
docker service create --label SOME_ENV_VAR --tty --name test busybox
docker service inspect --format '{{json .Spec.Labels}}' test
{"SOME_ENV_VAR":"I_AM_SOME_ENV_VAR"}
```
After this patch:
```bash
export SOME_ENV_VAR=I_AM_SOME_ENV_VAR
docker service create --label SOME_ENV_VAR --tty --name test busybox
docker container inspect --format '{{json .Config.Labels}}' test
{"SOME_ENV_VAR":""}
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This maps the `--template-driver` flag on secret and config creation.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The warning, printed before running `docker system prune` was printing the
filters in JSON format.
This patch attempts to make the output human readable;
- updating the code, and template to print filters individually
- reducing the indentation (which was quite deep)
Before this patch was applied;
```
docker system prune --filter until=24h --filter label=hello-world --filter label!=foo=bar --filter label=bar=baz
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
- Elements to be pruned will be filtered with:
- label={"label":{"bar=baz":true,"hello-world":true},"label!":{"foo=bar":true},"until":{"24h":true}}
Are you sure you want to continue? [y/N]
```
With this patch applied;
```
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
Items to be pruned will be filtered with:
- label!=foo=bar
- label!=never=remove-me
- label=bar=baz
- label=hello-world
- label=remove=me
- until=24h
Are you sure you want to continue? [y/N]
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The warning, printed before runing docker system prune was missing any filter
that was set in the configuration file. In addition, the warning prefixes the
filters with `label=`, which is no longer accurate, now that the prune command
also supports "until" as a filter.
Before this change, only the filters set on the command-line were shown,
and any filter set in the configuration file was missing;
```
mkdir -p ./test-config
echo '{"pruneFilters": ["label!=never=remove-me", "label=remove=me"]}' > test-config/config.json
docker --config=./test-config system prune --filter until=24h --filter label=hello-world --filter label!=foo=bar --filter label=bar=baz
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
- Elements to be pruned will be filtered with:
- label={"label":{"bar=baz":true,"hello-world":true},"label!":{"foo=bar":true},"until":{"24h":true}}
Are you sure you want to continue? [y/N]
```
With this patch applied, both options from the commandline and options set
in the configuration file are shown;
```
mkdir -p ./test-config
echo '{"pruneFilters": ["label!=never=remove-me", "label=remove=me"]}' > test-config/config.json
docker --config=./test-config system prune --filter until=24h --filter label=hello-world --filter label!=foo=bar --filter label=bar=baz
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
- Elements to be pruned will be filtered with:
- filter={"label":{"bar=baz":true,"hello-world":true,"remove=me":true},"label!":{"foo=bar":true,"never=remove-me":true},"until":{"24h":true}}
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- when using "--context default" parameter
- when printing the list of contexts
- when exporting the default context to a tarball
Signed-off-by: Jean-Christophe Sirot <jean-christophe.sirot@docker.com>
(+1 squashed commit)
Squashed commits:
[20670495] Fix CLI initialization for the `docker stack deploy --help` command and ensure that the dockerCli.CurrentContext() always returns a non empty context name (default as a fallback)
Remove now obsolete code handling empty string context name
Minor code cleanup
Signed-off-by: Jean-Christophe Sirot <jean-christophe.sirot@docker.com>
This makes things more idempotent, rather than relying on undoing the
interspersed settings.
Note that the underlying `Flag`s remain shared, it's just the `FlagSet` which
is duplicated.
Signed-off-by: Ian Campbell <ijc@docker.com>
The issue with plugin options clashing with globals is that when cobra is
parsing the command line and it comes across an argument which doesn't start
with a `-` it (in the absence of plugins) distinguishes between "argument to
current command" and "new subcommand" based on the list of registered sub
commands.
Plugins breaks that model. When presented with `docker -D plugin -c foo` cobra
parses up to the `plugin`, sees it isn't a registered sub-command of the
top-level docker (because it isn't, it's a plugin) so it accumulates it as an
argument to the top-level `docker` command. Then it sees the `-c`, and thinks
it is the global `-c` (for AKA `--context`) option and tries to treat it as
that, which fails.
In the specific case of the top-level `docker` subcommand we know that it has
no arguments which aren't `--flags` (or `-f` short flags) and so anything which
doesn't start with a `-` must either be a (known) subcommand or an attempt to
execute a plugin.
We could simply scan for and register all installed plugins at start of day, so
that cobra can do the right thing, but we want to avoid that since it would
involve executing each plugin to fetch the metadata, even if the command wasn't
going to end up hitting a plugin.
Instead we can parse the initial set of global arguments separately before
hitting the main cobra `Execute` path, which works here exactly because we know
that the top-level has no non-flag arguments.
One slight wrinkle is that the top-level `PersistentPreRunE` is no longer
called on the plugins path (since it no longer goes via `Execute`), so we
arrange for the initialisation done there (which has to be done after global
flags are parsed to handle e.g. `--config`) to happen explictly after the
global flags are parsed. Rather than make `newDockerCommand` return the
complicated set of results needed to make this happen, instead return a closure
which achieves this.
The new functionality is introduced via a common `TopLevelCommand` abstraction
which lets us adjust the plugin entrypoint to use the same strategy for parsing
the global arguments. This isn't strictly required (in this case the stuff in
cobra's `Execute` works fine) but doing it this way avoids the possibility of
subtle differences in behaviour.
Fixes#1699, and also, as a side-effect, the first item in #1661.
Signed-off-by: Ian Campbell <ijc@docker.com>
This happens on Windows when dialing a named pipe (a path which is used by CLI
plugins), in that case some debugging shows:
DEBU[0000] conn is a *winio.win32MessageBytePipe
DEBU[0000] conn is a halfReadCloser: false
DEBU[0000] conn is a halfWriteCloser: true
the raw stream connection does not implement halfCloser
In such cases we can simply wrap with a nop function since closing for read
isn't too critical.
Signed-off-by: Ian Campbell <ijc@docker.com>
Setting `Setsid` is needed for SSH connection helper with `ProxyCommand`
config, so as to detach TTY.
e.g.
$ cat ~/.ssh/config
Host foo
Hostname foo
ProxyCommand ssh -W %h:%p bastion
$ DOCKER_HOST=ssh://foo docker run -it --rm alpine
/ #
Fix#1707
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
After switching to Go 1.12, the format-string causes an error;
```
=== Errors
cli/config/config_test.go:154:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:217:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:253:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:288:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:435:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
cli/config/config_test.go:448:3: Fatalf format %q has arg config of wrong type *github.com/docker/cli/cli/config/configfile.ConfigFile
DONE 1115 tests, 2 skipped, 6 errors in 215.984s
make: *** [Makefile:22: test-coverage] Error 2
Exited with code 2
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `conn` here is `*winio.win32MessageBytePipe` which does not have a
`CloseRead` method (it does have `CloseWrite`) resulting in:
docker@WIN-NUC0 C:\Users\docker>.\docker-windows-amd64.exe system dial-stdio
the raw stream connection does not implement halfCloser
Also disable the path which uses this for cli-plugins on Windows.
Signed-off-by: Ian Campbell <ijc@docker.com>
Plugins are expected to be management commands ("docker <object> <verb>").
This patch modified the usage output to shown plugins in the "Management commands"
section.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a bit manual (as the unit test attests) so we may find we want to add
some helpers/accessors, but this is enough to let plugins use it and to
preserve the information through round-trips.
Signed-off-by: Ian Campbell <ijc@docker.com>
- The placement of the vendor is now in the end of the line.
- A '*' is now added as suffix of plugins' top level commands.
Signed-off-by: Ulysses Souza <ulysses.souza@docker.com>
This means that plugins can use whatever methods the monolithic CLI supports,
which is good for consistency.
This relies on `os.Args[0]` being something which can be executed again to
reach the same binary, since it is propagated (via an envvar) to the plugin for
this purpose. This essentially requires that the current working directory and
path are not modified by the monolithic CLI before it launches the plugin nor
by the plugin before it initializes the client. This should be the case.
Previously the fake apiclient used by `TestExperimentalCLI` was not being used,
since `cli.Initialize` was unconditionally overwriting it with a real one
(talking to a real daemon during unit testing, it seems). This wasn't expected
nor desirable and no longer happens with the new arrangements, exposing the
fact that no `pingFunc` is provided, leading to a panic. Add a `pingFunc` to
the fake client to avoid this.
Signed-off-by: Ian Campbell <ijc@docker.com>
Fairly straight forward. It became necessary to wrap `Plugin.Err` with a type
which implements `encoding.MarshalText` in order to have that field rendered
properly in the `docker info -f '{{json}}'` output.
Since I changed the type somewhat I also added a unit test for `formatInfo`.
Signed-off-by: Ian Campbell <ijc@docker.com>
To do this we add a stub `cobra.Command` for each installed plugin (only when
invoking `help`, not for normal running).
This requires a function to list all available plugins so that is added here.
Signed-off-by: Ian Campbell <ijc@docker.com>
Also includes the scaffolding for finding a validating plugin candidates.
Argument validation is moved to RunE to support this, so `noArgs` is removed.
Signed-off-by: Ian Campbell <ijc@docker.com>
- The `/build/prune` endpoint was added in API v1.31
- The `/network` endpoints were added in API v1.21
This patch hides these commands on older API versions
Before this change:
```
DOCKER_API_VERSION=1.0 docker
...
Management Commands:
builder Manage builds
container Manage containers
image Manage images
manifest Manage Docker image manifests and manifest lists
network Manage networks
system Manage Docker
trust Manage trust on Docker images
```
After this change
```
DOCKER_API_VERSION=1.0 docker
...
Management Commands:
container Manage containers
image Manage images
manifest Manage Docker image manifests and manifest lists
system Manage Docker
trust Manage trust on Docker images
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
That is, the helper to be used from the plugin's `main`.
Also add a `helloworld` plugin example and build integration.
Signed-off-by: Ian Campbell <ijc@docker.com>
I authored this for `contentTrustEnabled` prior to 7f207f3f95, so this now
tests the funcation argument version.
Signed-off-by: Ian Campbell <ijc@docker.com>
This helps to avoid circular includes, by separating the pure data out from the
actual functionality in the cli subpackage, allowing other code which is
imported to access the data.
Signed-off-by: Ian Campbell <ijc@docker.com>
Ian Campbell
Tibor Vass
Zander Mackie
Ravi Shekhar Jethani
Tõnis Tiigi
Sebastiaan van Stijn
Simon Ferquel
Tibor Vass
Daniel Hiltgen
Sebastiaan van Stijn
Drew Erny
Mårten Cassel
Sune Keller
Silvin Lubecki
Abhinandan Prativadi
Nick Adcock
David Scott
Nao YONASHIRO
Akihiro Suda
Lifubang
Kir Kolyshkin
Sunny
Vincent Demeester
Stephane Jeandeaux
Jean-Christophe Sirot
Elliot Luo
Ryan Zhang
Olli Janatuinen
Ulysses Souza
Philipp Schmied
John Howard