go1.22 and up now produce a unique variable in loops, tehrefore no longer
requiring to capture the variable manually;
cli-plugins/manager/cobra.go:55:4: The copy of the 'for' variable "p" can be deleted (Go 1.22+) (copyloopvar)
p := p
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this patch, output for invalid top-level and sub-commands differed.
For top-level commands, the CLI would print an error-message and a suggestion
to use `--help`. For missing *subcommands*, we would hit a different code-path,
and different output, which includes full "usage" / "help" output.
While it is a common convention to show usage output, and may have been
a nice gesture when docker was still young and only had a few commands
and options ("you did something wrong; here's an overview of what you
can use"), that's no longer the case, and many commands have a _very_
long output.
The result of this is that the error message, which is the relevant
information in this case - "You mis-typed something" - is lost in the
output, and hard to find (sometimes even requiring scrolling back).
The output is also confusing, because it _looks_ like something ran
successfully (most of the output is not about the error!).
Even further; the suggested resolution (try `--help` to see the correct
options) is rather redundant, because running teh command with `--help`
produces _exactly_ the same output as was just showh, baring the error
message. As a fun fact, due to the usage output being printed, the
output even contains not one, but _two_ "call to actions";
- `See 'docker volume --help'.` (under the erro message)
- `Run 'docker volume COMMAND --help' for more information on a command.`
(under the usage output)
In short; the output is too verbose, confusing, and doesn't provide
a good UX. Let's reduce the output produced so that the focus is on the
important information.
This patch:
- Changes the usage to the short-usage.
- Changes the error-message to mention the _full_ command instead of only
the command after `docker` (so `docker no-such-command` instead of
`no-such-command`).
- Prefixes the error message with the binary / root-command name
(usually `docker:`); this is something we can still decide on, but
it's a pattern we already use in some places. The motivation for this
is that `docker` commands can often produce output that's a combination
of output from the CLI itself, output from the daemon, and even output
from the container. The `docker:` prefix helps to distinguish where
the message originated from (the `docker` CLI in this case).
- Adds an empty line between the error-message and the "call to action"
(`Run 'docker volume --help'...` in the example below). This helps
separating the error message ("unkown flag") from the call-to-action.
Before this patch:
Unknown top-level command:
docker nosuchcommand foo
docker: 'nosuchcommand' is not a docker command.
See 'docker --help'
Unknown sub-command:
docker volume nosuchcommand foo
Usage: docker volume COMMAND
Manage volumes
Commands:
create Create a volume
inspect Display detailed information on one or more volumes
ls List volumes
prune Remove unused local volumes
rm Remove one or more volumes
update Update a volume (cluster volumes only)
Run 'docker volume COMMAND --help' for more information on a command.
After this patch:
Unknown top-level command:
docker nosuchcommand foo
docker: unknown command: docker nosuchcommand
Run 'docker --help' for more information
Unknown sub-command:
docker volume nosuchcommand foo
docker: unknown command: 'docker volume nosuchcommand'
Usage: docker volume COMMAND
Run 'docker volume --help' for more information
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add some documentation about their purpose, and document order of preference
when resolving plugins.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
internal/test/cli.go:175:14: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("no notary client available unless defined")
^
cli/command/cli.go:318:29: fmt.Errorf can be replaced with errors.New (perfsprint)
return docker.Endpoint{}, fmt.Errorf("no context store initialized")
^
cli/command/container/attach.go:161:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf(result.Error.Message)
^
cli/command/container/opts.go:577:16: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("--health-start-period cannot be negative")
^
cli/command/container/opts.go:580:16: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("--health-start-interval cannot be negative")
^
cli/command/container/stats.go:221:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("filtering is not supported when specifying a list of containers")
^
cli/command/container/attach_test.go:82:17: fmt.Errorf can be replaced with errors.New (perfsprint)
expectedErr = fmt.Errorf("unexpected error")
^
cli/command/container/create_test.go:234:40: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.CreateResponse{}, fmt.Errorf("shouldn't try to pull image")
^
cli/command/container/list_test.go:150:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("error listing containers")
^
cli/command/container/rm_test.go:40:31: fmt.Errorf can be replaced with errors.New (perfsprint)
return errdefs.NotFound(fmt.Errorf("Error: no such container: " + container))
^
cli/command/container/run_test.go:138:40: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.CreateResponse{}, fmt.Errorf("shouldn't try to pull image")
^
cli/command/image/pull_test.go:115:49: fmt.Errorf can be replaced with errors.New (perfsprint)
return io.NopCloser(strings.NewReader("")), fmt.Errorf("shouldn't try to pull image")
^
cli/command/network/connect.go:88:16: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("invalid key/value pair format in driver options")
^
cli/command/plugin/create_test.go:96:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("Error creating plugin")
^
cli/command/plugin/disable_test.go:32:12: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("Error disabling plugin")
^
cli/command/plugin/enable_test.go:32:12: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("failed to enable plugin")
^
cli/command/plugin/inspect_test.go:55:22: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, nil, fmt.Errorf("error inspecting plugin")
^
cli/command/plugin/install_test.go:43:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("Error installing plugin")
^
cli/command/plugin/install_test.go:51:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("(image) when fetching")
^
cli/command/plugin/install_test.go:95:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("should not try to install plugin")
^
cli/command/plugin/list_test.go:35:41: fmt.Errorf can be replaced with errors.New (perfsprint)
return types.PluginsListResponse{}, fmt.Errorf("error listing plugins")
^
cli/command/plugin/remove_test.go:27:12: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("Error removing plugin")
^
cli/command/registry/login_test.go:36:46: fmt.Errorf can be replaced with errors.New (perfsprint)
return registrytypes.AuthenticateOKBody{}, fmt.Errorf("Invalid Username or Password")
^
cli/command/registry/login_test.go:44:46: fmt.Errorf can be replaced with errors.New (perfsprint)
return registrytypes.AuthenticateOKBody{}, fmt.Errorf(errUnknownUser)
^
cli/command/system/info.go:190:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("errors pretty printing info")
^
cli/command/system/prune.go:77:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf(`ERROR: The "until" filter is not supported with "--volumes"`)
^
cli/command/system/version_test.go:19:28: fmt.Errorf can be replaced with errors.New (perfsprint)
return types.Version{}, fmt.Errorf("no server")
^
cli/command/trust/key_load.go:112:22: fmt.Errorf can be replaced with errors.New (perfsprint)
return []byte{}, fmt.Errorf("could not decrypt key")
^
cli/command/trust/revoke.go:44:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("cannot use a digest reference for IMAGE:TAG")
^
cli/command/trust/revoke.go:105:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("no signed tags to remove")
^
cli/command/trust/signer_add.go:56:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("releases is a reserved keyword, please use a different signer name")
^
cli/command/trust/signer_add.go:60:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("path to a public key must be provided using the `--key` flag")
^
opts/config.go:71:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("source is required")
^
opts/mount.go:168:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("type is required")
^
opts/mount.go:172:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("target is required")
^
opts/network.go:90:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("network name/id is not specified")
^
opts/network.go:129:18: fmt.Errorf can be replaced with errors.New (perfsprint)
return "", "", fmt.Errorf("invalid key value pair format in driver options")
^
opts/opts.go:404:13: fmt.Errorf can be replaced with errors.New (perfsprint)
return 0, fmt.Errorf("value is too precise")
^
opts/opts.go:412:18: fmt.Errorf can be replaced with errors.New (perfsprint)
return "", "", fmt.Errorf("empty string specified for links")
^
opts/parse.go:84:37: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.RestartPolicy{}, fmt.Errorf("invalid restart policy format: no policy provided before colon")
^
opts/parse.go:89:38: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.RestartPolicy{}, fmt.Errorf("invalid restart policy format: maximum retry count must be an integer")
^
opts/port.go:105:13: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("hostip is not supported")
^
opts/secret.go:70:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("source is required")
^
opts/env_test.go:57:11: fmt.Errorf can be replaced with errors.New (perfsprint)
err: fmt.Errorf("invalid environment variable: =a"),
^
opts/env_test.go:93:11: fmt.Errorf can be replaced with errors.New (perfsprint)
err: fmt.Errorf("invalid environment variable: ="),
^
cli-plugins/manager/error_test.go:16:11: fmt.Errorf can be replaced with errors.New (perfsprint)
inner := fmt.Errorf("testing")
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Don't show `Next steps:` with no messages at all when plugin returns an
unitialized value of `HookMessage` (zero-initialization sets its type to
NextSteps and empty template).
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Commit 5011759056 implemented a fix that
caused the current environment to be discarded, using `os.Environ()`.
On Windows, `os.Environ()` may produce an incorrect value for `PWD`,
for which a new function was added in go1.19;
- https://tip.golang.org/doc/go1.19#osexecpkgosexec
- https://go-review.googlesource.com/c/go/+/401340
Replace the use of `os.Environ()` with `cmd.Environ()` to address that.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before, for plugin commands, only the plugin name (such as `buildx`)
would be both included as `RootCmd` when passed to the hook plugin,
which isn't enough information for a plugin to decide whether to execute
a hook or not since plugins implement multiple varied commands (`buildx
build`, `buildx prune`, etc.).
This commit changes the hook logic to account for this situation, so
that the the entire configured hook is passed, i.e., if a user has a
hook configured for `buildx imagetools inspect` and the command
`docker buildx imagetools inspect alpine` is called, then the plugin
hooks will be passed `buildx imagetools inspect`.
This logic works for aliased commands too, so whether `docker build ...`
or `docker buildx build` is executed (unless Buildx is disabled) the
hook will be invoked with `buildx build`.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
hooks: include full match when invoking plugins
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
During normal plugin execution (from the CLI), the CLI configures the
plugin command it's about to execute in order to pass all environment
variables on, as well as to set the ReExec env var that informs the
plugin about how it was executed, and which plugins rely on to check
whether they are being run standalone or not.
This commit adds the same behavior to hook invocations, which is
necessary for some plugins to know that they are not running standalone
so that they expose their root command at the correct level.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
When a plugin is invoked, the docker cli will now set
`OTEL_RESOURCE_ATTRIBUTES` to pass OTEL resource attribute names to the
plugin as additional resource attributes. At the moment, the only
resource attribute passed is `cobra.command_path`.
All resource attributes passed by the CLI are prepended with the
namespace `docker.cli` to avoid clashing with existing ones the plugin
uses or ones defined by the user.
For aliased commands like the various builder commands, the command path
is overwritten to match with the original name (such as `docker
builder`) instead of the forwarded name (such as `docker buildx build`).
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This reverts commit ef5e5fa03f.
Running new plugins under a new pgid isn't a viable solution due to
it causing issues with plugin processes attempting to read from the
TTY (see: https://github.com/moby/moby/issues/47073).
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Changes were made in 1554ac3b5f to provide
a mechanism for the CLI to notify running plugin processes that they
should exit, in order to improve the general CLI/plugin UX. The current
implementation boils down to:
1. The CLI creates a socket
2. The CLI executes the plugin
3. The plugin connects to the socket
4. (When) the CLI receives a termination signal, it uses the socket to
notify the plugin that it should exit
5. The plugin's gets notified via the socket, and cancels it's `cmd.Context`,
which then gets handled appropriately
This change works in most cases and fixes the issue it sets out to solve
(see: https://github.com/docker/compose/pull/11292) however, in the case
where the user has a TTY attached and the plugin is not already handling
received signals, steps 4+ changes:
4. (When) the CLI receives a termination signal, before it can use the
socket to notify the plugin that it should exit, the plugin process
also receives a signal due to sharing the pgid with the CLI
Since we now have a proper "job control" mechanism, we can simplify the
scenarios by executing the plugins with their own process group id,
thereby removing the "double notification" issue and making it so that
plugins can handle the same whether attached to a TTY or not.
In order to make this change "plugin-binary" backwards-compatible, in
the case that a plugin does not connect to the socket, the CLI passes
the signal to the plugin process.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
This is a follow-up to 0e73168b7e
This repository is not yet a module (i.e., does not have a `go.mod`). This
is not problematic when building the code in GOPATH or "vendor" mode, but
when using the code as a module-dependency (in module-mode), different semantics
are applied since Go1.21, which switches Go _language versions_ on a per-module,
per-package, or even per-file base.
A condensed summary of that logic [is as follows][1]:
- For modules that have a go.mod containing a go version directive; that
version is considered a minimum _required_ version (starting with the
go1.19.13 and go1.20.8 patch releases: before those, it was only a
recommendation).
- For dependencies that don't have a go.mod (not a module), go language
version go1.16 is assumed.
- Likewise, for modules that have a go.mod, but the file does not have a
go version directive, go language version go1.16 is assumed.
- If a go.work file is present, but does not have a go version directive,
language version go1.17 is assumed.
When switching language versions, Go _downgrades_ the language version,
which means that language features (such as generics, and `any`) are not
available, and compilation fails. For example:
# github.com/docker/cli/cli/context/store
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
Note that these fallbacks are per-module, per-package, and can even be
per-file, so _(indirect) dependencies_ can still use modern language
features, as long as their respective go.mod has a version specified.
Unfortunately, these failures do not occur when building locally (using
vendor / GOPATH mode), but will affect consumers of the module.
Obviously, this situation is not ideal, and the ultimate solution is to
move to go modules (add a go.mod), but this comes with a non-insignificant
risk in other areas (due to our complex dependency tree).
We can revert to using go1.16 language features only, but this may be
limiting, and may still be problematic when (e.g.) matching signatures
of dependencies.
There is an escape hatch: adding a `//go:build` directive to files that
make use of go language features. From the [go toolchain docs][2]:
> The go line for each module sets the language version the compiler enforces
> when compiling packages in that module. The language version can be changed
> on a per-file basis by using a build constraint.
>
> For example, a module containing code that uses the Go 1.21 language version
> should have a `go.mod` file with a go line such as `go 1.21` or `go 1.21.3`.
> If a specific source file should be compiled only when using a newer Go
> toolchain, adding `//go:build go1.22` to that source file both ensures that
> only Go 1.22 and newer toolchains will compile the file and also changes
> the language version in that file to Go 1.22.
This patch adds `//go:build` directives to those files using recent additions
to the language. It's currently using go1.19 as version to match the version
in our "vendor.mod", but we can consider being more permissive ("any" requires
go1.18 or up), or more "optimistic" (force go1.21, which is the version we
currently use to build).
For completeness sake, note that any file _without_ a `//go:build` directive
will continue to use go1.16 language version when used as a module.
[1]: 58c28ba286/src/cmd/go/internal/gover/version.go (L9-L56)
[2]; https://go.dev/doc/toolchain#:~:text=The%20go%20line%20for,file%20to%20Go%201.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was marked deprecated in 977d3ae046,
which is part of Docker 20.10 and up.
This patch removes the field.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
the "golang.org/x/sys/execabs" package was introduced to address a security
issue on Windows, and changing the default behavior of os/exec was considered
a breaking change. go1.19 applied the behavior that was previously implemented
in the execabs package;
from the release notes: https://go.dev/doc/go1.19#os-exec-path
> Command and LookPath no longer allow results from a PATH search to be found
> relative to the current directory. This removes a common source of security
> problems but may also break existing programs that depend on using, say,
> exec.Command("prog") to run a binary named prog (or, on Windows, prog.exe)
> in the current directory. See the os/exec package documentation for information
> about how best to update such programs.
>
> On Windows, Command and LookPath now respect the NoDefaultCurrentDirectoryInExePath
> environment variable, making it possible to disable the default implicit search
> of “.” in PATH lookups on Windows systems.
With those changes, we no longer need to use the execabs package, and we can
switch back to os/exec.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This field was marked deprecated in 977d3ae046,
which is part of v20.10 and up, but the comment was missing a newline before
the deprecation message, which may be picked up by IDEs, but is not matching
the correct format, so may not be picked up by linters.
This patch fixes the format, to make sure linters pick up that the field is
deprecated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This reverts commit 62f2358b99.
Spawning a goroutine for each iteration in the loop when listing
plugins is racy unfortunately. `plugins` slice is protected with
a mutex so not sure why it fails.
I tried using a channel to collect the plugins instead of a slice
to guarantee that they will be appended to the list in the order
they are processed but no dice.
I also tried without errgroup package and simply use sync.WaitGroup
but same. I have also created an extra channel to receive errors
from the goroutines but racy too.
I think the change in this function is not related to the race
condition but newPlugin is. So revert in the meantime :(
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
We are currently loading plugin command stubs for every
invocation which still has a significant performance hit.
With this change we are doing this operation only if cobra
completion arg request is found.
- 20.10.23: `docker --version` takes ~15ms
- 23.0.1: `docker --version` takes ~93ms
With this change `docker --version` takes ~9ms
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
We are currently loading plugin commands stubs for every
command invocation to add support for Cobra v2 completion.
This cause a significant performance hit if there is a
lot of plugins in the user space (7 atm in Docker Desktop):
`docker --version` takes in current 23.0.1 ~93ms
Instead of removing completion for plugins to fix the
regression, we can slightly improve plugins discovery by
spawning a goroutine for each iteration in the loop when
listing plugins:
`docker --version` now takes ~38ms
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
The test used `gopkg.in/yaml.v2` to verify the TextMarshaller implementation,
which was implemented to allow printing the errors in JSON formatted output;
> This exists primarily to implement encoding.TextMarshaller such that
> rendering a plugin as JSON (e.g. for `docker info -f '{{json .CLIPlugins}}'`)
> renders the Err field as a useful string and not just `{}`.
Given that both yaml.Marshal and json.Marshal use this, we may as well use
Go's stdlib.
While updating, also changed some of the assertions to checks, so that we don't
fail the test early.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like the linter uses an explicit -lang, which (for go1.19)
results in some additional formatting for octal values.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Older versions of Go do not format these comments, so we can already
reformat them ahead of time to prevent gofmt linting failing once
we update to Go 1.19 or up.
Result of:
gofmt -s -w $(find . -type f -name '*.go' | grep -v "/vendor/")
With some manual adjusting.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
On Windows, the os/exec.{Command,CommandContext,LookPath} functions
resolve command names that have neither path separators nor file extension
(e.g., "git") by first looking in the current working directory before
looking in the PATH environment variable.
Go maintainers intended to match cmd.exe's historical behavior.
However, this is pretty much never the intended behavior and as an abundance of precaution
this patch prevents that when executing commands.
Example of commands that docker.exe may execute: `git`, `docker-buildx` (or other cli plugin), `docker-credential-wincred`, `docker`.
Note that this was prompted by the [Go 1.15.7 security fixes](https://blog.golang.org/path-security), but unlike in `go.exe`,
the windows path lookups in docker are not in a code path allowing remote code execution, thus there is no security impact on docker.
Signed-off-by: Tibor Vass <tibor@docker.com>
The CLI disabled experimental features by default, requiring users
to set a configuration option to enable them.
Disabling experimental features was a request from Enterprise users
that did not want experimental features to be accessible.
We are changing this policy, and now enable experimental features
by default. Experimental features may still change and/or removed,
and will be highlighted in the documentation and "usage" output.
For example, the `docker manifest inspect --help` output now shows:
EXPERIMENTAL:
docker manifest inspect is an experimental feature.
Experimental features provide early access to product functionality. These features
may change between releases without warning or can be removed entirely from a future
release. Learn more about experimental features: https://docs.docker.com/go/experimental/
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The vanity domain is down, and the project has moved
to a new location.
vendor check started failing because of this:
Collecting initial packages
Download dependencies
unrecognized import path "vbom.ml/util" (https fetch: Get https://vbom.ml/util?go-get=1: dial tcp: lookup vbom.ml on 169.254.169.254:53: no such host)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Sebastiaan van Stijn
Paweł Gronowski
Alano Terblanche
Laura Brehm
Tonis Tiigi
Jonathan A. Sternberg
Sebastiaan van Stijn
Kevin Alvarez
Nicolas De Loof
Tibor Vass
Silvin Lubecki