The Docker CLI supports storing/managing credentials without a
credential-helper, in which case credentials are fetched from/saved to
the CLI config file (`~/.docker/config.json`). This is all managed
entirely by the CLI itself, without resort to a separate binary.
There are a few issues with this approach – for one, saving the
credentials together with all the configurations make it impossible to
share one without the other, so one can't for example bind mount the
config file into a container without also including all configured
credentials.
Another issue is that this has made it so that any other clients
accessing registry credentials (such as
https://github.com/google/go-containerregistry) all have to both:
- read/parse the CLI `config.json`, to check for credentials there,
which also means they're dependent on this type and might break if the
type changes/we need to be careful not to break other codebases parsing
this file, and can't change the location where plaintext credentials
are stored.
- support the credential helper protocol, so that they can access
credentials when users do have configured credential helpers.
This means that if we want to do something like support oauth
credentials by having credential-helpers refresh oauth tokens before
returning them, we have to both implement that in each credential-helper
and in the CLI itself, and any client directly reading `config.json`
will also need to implement this logic.
This commit turns the Docker CLI binary into a multicall binary, acting
as a standalone credentials helper when invoked as
`docker-credential-file`, while still storing/fetching credentials from
the configuration file (`~/.docker/config.json`), and without any
further changes.
This represents a first step into aligning the "no credhelper"/plaintext
flow with the "credhelper" flow, meaning that instead of this being an
exception where credentials must be read directly from the config file,
credentials can now be accessed in the exact same way as with other
credential helpers – by invoking `docker-credential-[credhelper name]`,
such as `docker-credential-pass`, `docker-credential-osxkeychain` or
`docker-credential-wincred`.
This would also make it possible for any other clients accessing
credentials to untangle themselves from things like the location of the
credentials, parsing credentials from `config.json`, etc. and instead
simply support the credential-helper protocol, and call the
`docker-credential-file` binary as they do others.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
In 4b5a196fee, we changed the CLI global
meter provider shutdown in order to handle any error returned by the
metric export.
Unfortunately, we dropped a `defer` during the fix, which
causes the meter provider to be immediately shutdown after being created
and metrics to not be collected/exporter.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
registerCompletionFuncForGlobalFlags was called from newDockerCommand,
at which time no context-store is initialized yet, so it would return
a nil value, probably resulting in `store.Names` to panic, but these
errors are not shown when running the completion. As a result, the flag
completion would fall back to completing from filenames.
This patch changes the function to dynamically get the context-store;
this fixes the problem mentioned above, because at the time the completion
function is _invoked_, the CLI is fully initialized, and does have a
context-store available.
A (non-exported) interface is defined to allow the function to accept
alternative implementations (not requiring a full command.DockerCLI).
Before this patch:
docker context create one
docker context create two
docker --context <TAB>
.DS_Store .idea/ Makefile
.dockerignore .mailmap build/
...
With this patch:
docker context create one
docker context create two
docker --context <TAB>
default one two
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's an alias for cobra.FixedCompletions but takes a variadic list
of strings, so that it's not needed to construct an array for this.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows dockerMain() to return an error "as usual", and puts the
responsibility for turning that into an appropriate exit-code in
main() (which also sets the exit-code when terminating).
We could consider putting this utility in the cli package and exporting
it if would be useful for doing a similar handling in plugins.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Improve the output for these validation errors:
- Removes the short command description from the output. This information
does not provide much useful help, and distracts from the error message.
- Reduces punctuation, and
- Prefixes the error message with the binary / root-command name
(usually `docker:`) to be consistent with other similar errors.
- Adds an empty line between the error-message and the "call to action"
(`Run 'docker volume --help'...` in the example below). This helps
separating the error message and "usage" from the call-to-action.
Before this patch:
$ docker volume ls one two three
"docker volume ls" accepts no arguments.
See 'docker volume ls --help'.
Usage: docker volume ls [OPTIONS]
List volumes
$ docker volume create one two three
"docker volume create" requires at most 1 argument.
See 'docker volume create --help'.
Usage: docker volume create [OPTIONS] [VOLUME]
Create a volume
With this patch:
$ docker volume ls one two three
docker: 'docker volume ls' accepts no arguments
Usage: docker volume ls [OPTIONS]
Run 'docker volume ls --help' for more information
$ docker voludocker volume create one two three
docker: 'docker volume create' requires at most 1 argument
Usage: docker volume create [OPTIONS] [VOLUME]
SRun 'docker volume create --help' for more information
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this patch, output for invalid top-level and sub-commands differed.
For top-level commands, the CLI would print an error-message and a suggestion
to use `--help`. For missing *subcommands*, we would hit a different code-path,
and different output, which includes full "usage" / "help" output.
While it is a common convention to show usage output, and may have been
a nice gesture when docker was still young and only had a few commands
and options ("you did something wrong; here's an overview of what you
can use"), that's no longer the case, and many commands have a _very_
long output.
The result of this is that the error message, which is the relevant
information in this case - "You mis-typed something" - is lost in the
output, and hard to find (sometimes even requiring scrolling back).
The output is also confusing, because it _looks_ like something ran
successfully (most of the output is not about the error!).
Even further; the suggested resolution (try `--help` to see the correct
options) is rather redundant, because running teh command with `--help`
produces _exactly_ the same output as was just showh, baring the error
message. As a fun fact, due to the usage output being printed, the
output even contains not one, but _two_ "call to actions";
- `See 'docker volume --help'.` (under the erro message)
- `Run 'docker volume COMMAND --help' for more information on a command.`
(under the usage output)
In short; the output is too verbose, confusing, and doesn't provide
a good UX. Let's reduce the output produced so that the focus is on the
important information.
This patch:
- Changes the usage to the short-usage.
- Changes the error-message to mention the _full_ command instead of only
the command after `docker` (so `docker no-such-command` instead of
`no-such-command`).
- Prefixes the error message with the binary / root-command name
(usually `docker:`); this is something we can still decide on, but
it's a pattern we already use in some places. The motivation for this
is that `docker` commands can often produce output that's a combination
of output from the CLI itself, output from the daemon, and even output
from the container. The `docker:` prefix helps to distinguish where
the message originated from (the `docker` CLI in this case).
- Adds an empty line between the error-message and the "call to action"
(`Run 'docker volume --help'...` in the example below). This helps
separating the error message ("unkown flag") from the call-to-action.
Before this patch:
Unknown top-level command:
docker nosuchcommand foo
docker: 'nosuchcommand' is not a docker command.
See 'docker --help'
Unknown sub-command:
docker volume nosuchcommand foo
Usage: docker volume COMMAND
Manage volumes
Commands:
create Create a volume
inspect Display detailed information on one or more volumes
ls List volumes
prune Remove unused local volumes
rm Remove one or more volumes
update Update a volume (cluster volumes only)
Run 'docker volume COMMAND --help' for more information on a command.
After this patch:
Unknown top-level command:
docker nosuchcommand foo
docker: unknown command: docker nosuchcommand
Run 'docker --help' for more information
Unknown sub-command:
docker volume nosuchcommand foo
docker: unknown command: 'docker volume nosuchcommand'
Usage: docker volume COMMAND
Run 'docker volume --help' for more information
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 6fef143dbc switched the CLI to use
BuildKit by default, but as part of that removed the use of the
BuildkitVersion field as returned by Ping.
Some follow-up changes in commits e38e6c51ff and
e7a8748b93 updated the logic for detecting whether
BuildKit should be used or the legacy builder, but hard-coded using the
legacy builder for Windows daemons.
While Windows / WCOW does not yet support BuildKit by default, there is
work in progress to implement it, so we should not hard-code the assumption
that a Windows daemon cannot support BuildKit.
On the daemon-side, [moby@7b153b9] (Docker v23.0) changed the default as
advertised by the daemon to be BuildKit for Linux daemons. That change
still hardcoded BuildKit to be unsupported for Windows daemons (and does
not yet allow overriding the config), but this may change for future
versions of the daemon, or test-builds.
This patch:
- Re-introduces checks for the BuildkitVersion field in the "Ping" response.
- If the Ping response from the daemon advertises that it supports BuildKit,
the CLI will now use BuildKit as builder.
- If we didn't get a Ping response, or the Ping response did NOT advertise
that the daemon supported BuildKit, we continue to use the current
defaults (BuildKit for Linux daemons, and the legacy builder for Windows)
- Handling of the DOCKER_BUILDKIT environment variable is unchanged; for
CLI.BuildKitEnabled, DOCKER_BUILDKIT always takes precedence, and for
processBuilder the value is taken into account, but will print a warning
when BuildKit is disabled and a Linux daemon is used. For Windows daemons,
no warning is printed.
[moby@7b153b9]: 7b153b9e28
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since 509123f935, we've been leaking sockets
in the filesystem on platforms where abstract sockets aren't supported.
That change relied on Go to cleanup our sockets for us, which Go will happily
do as long as we make sure to close the listener, which we weren't previously
doing unless to signal the plugin to terminate.
This change adds a deferred call to `PluginServer.Close()`, which makes sure we
close the plugin server at the end of the plugin execution, so that we never exit
without cleaning up.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Explicitly create the context and set it on the CLI, instead of depending on
NewDockerCli() to instance a default context.
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Before, for plugin commands, only the plugin name (such as `buildx`)
would be both included as `RootCmd` when passed to the hook plugin,
which isn't enough information for a plugin to decide whether to execute
a hook or not since plugins implement multiple varied commands (`buildx
build`, `buildx prune`, etc.).
This commit changes the hook logic to account for this situation, so
that the the entire configured hook is passed, i.e., if a user has a
hook configured for `buildx imagetools inspect` and the command
`docker buildx imagetools inspect alpine` is called, then the plugin
hooks will be passed `buildx imagetools inspect`.
This logic works for aliased commands too, so whether `docker build ...`
or `docker buildx build` is executed (unless Buildx is disabled) the
hook will be invoked with `buildx build`.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
hooks: include full match when invoking plugins
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Particularly for cases such as `docker exec -it`, it's relevant that the CLI
still executes hooks even if the exec exited with a non-zero exit code,
since this is can be part of a normal `docker exec` invocation depending on
how the user exits.
In the future, this might also be interesting to allow plugins to run
hooks after an error so they can offer error-state recovery suggestions,
although this would require additional work to give the plugin more
information about the failed execution.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This commit adds a "terminal" attribute to `BaseMetricAttributes`
that allows us to discern whether an invocation was from an interactive
terminal or not.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This adds a default otel error handler for the cli in the debug package.
It uses logrus to log the error on the debug level and should work out
of the box with the `--debug` flag and `DEBUG` environment variable.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This changes things to rely on a plugin server that manages all
connections made to the server.
An optional handler can be passed into the server when the caller wants
to do extra things with the connection.
It is the caller's responsibility to close the server.
When the server is closed, first all existing connections are closed
(and new connections are prevented).
Now the signal loop only needs to close the server and not deal with
`net.Conn`'s directly (or double-indirects as the case was before this
change).
The socket, when present in the filesystem, is no longer unlinked
eagerly, as reconnections require it to be present for the lifecycle of
the plugin server.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
The command path sent for `docker build` should be `docker` rather than
`docker build` to be consistent with the other command paths.
* `docker buildx build` has a command path of `docker buildx`
* `docker builder build` has a command path of `docker builder`
* `docker image build` has a command path of `docker image`
The reason this gets set to `docker buildx` rather than `docker buildx
build` is because the `build` portion of the command path is processed
by the plugin. So the command path only contains the portions of the
command path that were processed by this tool.
Since the `build` of `docker build` gets forwarded to `buildx`, it is
not included in the command path.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
When a plugin is invoked, the docker cli will now set
`OTEL_RESOURCE_ATTRIBUTES` to pass OTEL resource attribute names to the
plugin as additional resource attributes. At the moment, the only
resource attribute passed is `cobra.command_path`.
All resource attributes passed by the CLI are prepended with the
namespace `docker.cli` to avoid clashing with existing ones the plugin
uses or ones defined by the user.
For aliased commands like the various builder commands, the command path
is overwritten to match with the original name (such as `docker
builder`) instead of the forwarded name (such as `docker buildx build`).
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
In order to solve the "double notification" issue (see:
ef5e5fa03f)
without running the plugin process under a new pgid (see:
https://github.com/moby/moby/issues/47073) we instead check if we're
attached to a TTY, and if so skip signalling the plugin process since it
will already be signalled.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This reverts commit ef5e5fa03f.
Running new plugins under a new pgid isn't a viable solution due to
it causing issues with plugin processes attempting to read from the
TTY (see: https://github.com/moby/moby/issues/47073).
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Changes were made in 1554ac3b5f to provide
a mechanism for the CLI to notify running plugin processes that they
should exit, in order to improve the general CLI/plugin UX. The current
implementation boils down to:
1. The CLI creates a socket
2. The CLI executes the plugin
3. The plugin connects to the socket
4. (When) the CLI receives a termination signal, it uses the socket to
notify the plugin that it should exit
5. The plugin's gets notified via the socket, and cancels it's `cmd.Context`,
which then gets handled appropriately
This change works in most cases and fixes the issue it sets out to solve
(see: https://github.com/docker/compose/pull/11292) however, in the case
where the user has a TTY attached and the plugin is not already handling
received signals, steps 4+ changes:
4. (When) the CLI receives a termination signal, before it can use the
socket to notify the plugin that it should exit, the plugin process
also receives a signal due to sharing the pgid with the CLI
Since we now have a proper "job control" mechanism, we can simplify the
scenarios by executing the plugins with their own process group id,
thereby removing the "double notification" issue and making it so that
plugins can handle the same whether attached to a TTY or not.
In order to make this change "plugin-binary" backwards-compatible, in
the case that a plugin does not connect to the socket, the CLI passes
the signal to the plugin process.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Update this function to accept a smaller interface, as it doesn't need
all of "CLI". Also return errors encountered during its operation (although
the caller currently has no error return on its own).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a follow-up to 0e73168b7e
This repository is not yet a module (i.e., does not have a `go.mod`). This
is not problematic when building the code in GOPATH or "vendor" mode, but
when using the code as a module-dependency (in module-mode), different semantics
are applied since Go1.21, which switches Go _language versions_ on a per-module,
per-package, or even per-file base.
A condensed summary of that logic [is as follows][1]:
- For modules that have a go.mod containing a go version directive; that
version is considered a minimum _required_ version (starting with the
go1.19.13 and go1.20.8 patch releases: before those, it was only a
recommendation).
- For dependencies that don't have a go.mod (not a module), go language
version go1.16 is assumed.
- Likewise, for modules that have a go.mod, but the file does not have a
go version directive, go language version go1.16 is assumed.
- If a go.work file is present, but does not have a go version directive,
language version go1.17 is assumed.
When switching language versions, Go _downgrades_ the language version,
which means that language features (such as generics, and `any`) are not
available, and compilation fails. For example:
# github.com/docker/cli/cli/context/store
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
Note that these fallbacks are per-module, per-package, and can even be
per-file, so _(indirect) dependencies_ can still use modern language
features, as long as their respective go.mod has a version specified.
Unfortunately, these failures do not occur when building locally (using
vendor / GOPATH mode), but will affect consumers of the module.
Obviously, this situation is not ideal, and the ultimate solution is to
move to go modules (add a go.mod), but this comes with a non-insignificant
risk in other areas (due to our complex dependency tree).
We can revert to using go1.16 language features only, but this may be
limiting, and may still be problematic when (e.g.) matching signatures
of dependencies.
There is an escape hatch: adding a `//go:build` directive to files that
make use of go language features. From the [go toolchain docs][2]:
> The go line for each module sets the language version the compiler enforces
> when compiling packages in that module. The language version can be changed
> on a per-file basis by using a build constraint.
>
> For example, a module containing code that uses the Go 1.21 language version
> should have a `go.mod` file with a go line such as `go 1.21` or `go 1.21.3`.
> If a specific source file should be compiled only when using a newer Go
> toolchain, adding `//go:build go1.22` to that source file both ensures that
> only Go 1.22 and newer toolchains will compile the file and also changes
> the language version in that file to Go 1.22.
This patch adds `//go:build` directives to those files using recent additions
to the language. It's currently using go1.19 as version to match the version
in our "vendor.mod", but we can consider being more permissive ("any" requires
go1.18 or up), or more "optimistic" (force go1.21, which is the version we
currently use to build).
For completeness sake, note that any file _without_ a `//go:build` directive
will continue to use go1.16 language version when used as a module.
[1]: 58c28ba286/src/cmd/go/internal/gover/version.go (L9-L56)
[2]; https://go.dev/doc/toolchain#:~:text=The%20go%20line%20for,file%20to%20Go%201.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Laura Brehm
Jonathan A. Sternberg
Sebastiaan van Stijn
Sebastiaan van Stijn
Alano Terblanche
Christopher Petito
Bjorn Neergaard
Brian Goff