Particularly for cases such as `docker exec -it`, it's relevant that the CLI
still executes hooks even if the exec exited with a non-zero exit code,
since this is can be part of a normal `docker exec` invocation depending on
how the user exits.
In the future, this might also be interesting to allow plugins to run
hooks after an error so they can offer error-state recovery suggestions,
although this would require additional work to give the plugin more
information about the failed execution.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This commit adds a "terminal" attribute to `BaseMetricAttributes`
that allows us to discern whether an invocation was from an interactive
terminal or not.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This adds a default otel error handler for the cli in the debug package.
It uses logrus to log the error on the debug level and should work out
of the box with the `--debug` flag and `DEBUG` environment variable.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This changes things to rely on a plugin server that manages all
connections made to the server.
An optional handler can be passed into the server when the caller wants
to do extra things with the connection.
It is the caller's responsibility to close the server.
When the server is closed, first all existing connections are closed
(and new connections are prevented).
Now the signal loop only needs to close the server and not deal with
`net.Conn`'s directly (or double-indirects as the case was before this
change).
The socket, when present in the filesystem, is no longer unlinked
eagerly, as reconnections require it to be present for the lifecycle of
the plugin server.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
In order to solve the "double notification" issue (see:
ef5e5fa03f)
without running the plugin process under a new pgid (see:
https://github.com/moby/moby/issues/47073) we instead check if we're
attached to a TTY, and if so skip signalling the plugin process since it
will already be signalled.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This reverts commit ef5e5fa03f.
Running new plugins under a new pgid isn't a viable solution due to
it causing issues with plugin processes attempting to read from the
TTY (see: https://github.com/moby/moby/issues/47073).
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Changes were made in 1554ac3b5f to provide
a mechanism for the CLI to notify running plugin processes that they
should exit, in order to improve the general CLI/plugin UX. The current
implementation boils down to:
1. The CLI creates a socket
2. The CLI executes the plugin
3. The plugin connects to the socket
4. (When) the CLI receives a termination signal, it uses the socket to
notify the plugin that it should exit
5. The plugin's gets notified via the socket, and cancels it's `cmd.Context`,
which then gets handled appropriately
This change works in most cases and fixes the issue it sets out to solve
(see: https://github.com/docker/compose/pull/11292) however, in the case
where the user has a TTY attached and the plugin is not already handling
received signals, steps 4+ changes:
4. (When) the CLI receives a termination signal, before it can use the
socket to notify the plugin that it should exit, the plugin process
also receives a signal due to sharing the pgid with the CLI
Since we now have a proper "job control" mechanism, we can simplify the
scenarios by executing the plugins with their own process group id,
thereby removing the "double notification" issue and making it so that
plugins can handle the same whether attached to a TTY or not.
In order to make this change "plugin-binary" backwards-compatible, in
the case that a plugin does not connect to the socket, the CLI passes
the signal to the plugin process.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Update this function to accept a smaller interface, as it doesn't need
all of "CLI". Also return errors encountered during its operation (although
the caller currently has no error return on its own).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, long lived CLI plugin processes weren't
properly handled
(see: https://github.com/docker/cli/issues/4402)
resulting in plugin processes being left behind
running, after the CLI process exits.
This commit changes the plugin handling code to open
an abstract unix socket before running the plugin and
passing it to the plugin process, and changes the
signal handling on the CLI side to close this socket
which tells the plugin that it should exit.
This implementation makes use of sockets instead of
simply setting PDEATHSIG on the plugin process
so that it will work on both BSDs, assorted UNIXes
and Windows.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This copies the github.com/moby/buildkit/util/appcontext
package as an internal package. The appcontext package from
BuildKit was the only remaining dependency on BuildKit, and
while we may need some of its functionality, the implementation
is not correct for how it's used in docker/cli (so would need
a rewrite).
Moving a copy of the code into the docker/cli (but as internal
package to prevent others from depending on it) is a first step
in that process, and removes the circular dependency between
BuildKit and the CLi.
We are only using these:
tree vendor/github.com/moby/buildkit
vendor/github.com/moby/buildkit
├── AUTHORS
├── LICENSE
└── util
└── appcontext
├── appcontext.go
├── appcontext_unix.go
├── appcontext_windows.go
└── register.go
3 directories, 6 files
Before this:
go mod graph | grep ' github.com/docker/cli'
github.com/moby/buildkit@v0.11.6 github.com/docker/cli@v23.0.0-rc.1+incompatible
After this:
go mod graph | grep ' github.com/docker/cli'
# (nothing)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a similar fix as 006c946389, which
fixed this for detection of commands that were executed. Make sure we don't
call the "/_ping" endpoint if we don't need to.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commands comes in the form of `rootCmd 1stCmd 2ndCmd` and the `tryRunPluginHelp()` function takes the command name to run plugin help in the form `docker-<command name>`, if command name matches a plugin name it will display the wrong help message.
to make sure we're using 1stCmd level we check if the root command is the same as the command parent.
e.g. `docker swarm init --help`
```
docker: is the root command
swarm : 1st level command
init : 2nd level command
--help: is a flag
```
this approach is not taking in consideration if a plugin and 1st level command matches, for example `docker foo` command and `docker-foo` plugin.
Signed-off-by: Hernan Garcia <hernandanielg@gmail.com>
Signed-off-by: Hernan Garcia <hernan.garcia@percona.com>
The flag-set that was returned is a pointer to the command's Flags(), which
is in itself passed by reference (as it is modified / set up).
This patch removes the flags return, to prevent assuming it's different than
the command's flags.
While SetupRootCommand is exported, a search showed that it's only used internally,
so changing the signature should not be a problem.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We are currently loading plugin command stubs for every
invocation which still has a significant performance hit.
With this change we are doing this operation only if cobra
completion arg request is found.
- 20.10.23: `docker --version` takes ~15ms
- 23.0.1: `docker --version` takes ~93ms
With this change `docker --version` takes ~9ms
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Both the DockerCLI and Cobra Commands provide accessors for Input, Output,
and Error streams (usually STDIN, STDOUT, STDERR). While we were already
passing DockerCLI's Output to Cobra, we were not doing so for the other
streams (and were passing none for plugin commands), potentially resulting
in DockerCLI output/input to mean something else than a Cobra Command's
intput/output/error.
This patch sets them to the same streams when constructing the Cobra
command.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 20ba591b7f fixed incorrect feature
detection in the CLI, but introduced a regression; previously the "ping"
would only be executed if needed (see b39739123b),
but by not inlining the call to `ServerInfo()` would now always be called.
This patch inlines the code again to only execute the "ping" conditionally,
which allows it to be executed lazily (and omitted for commands that don't
require a daemon connection).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When server is unreachable and docker checkpoint (or any command that
needs to check the server type) is run, incorrect error was returned.
When checking if the daemon had the right OS, we compared the OSType
from the clients ServerInfo(). In situations where the client cannot
connect to the daemon, a "stub" Info is used for this, in which we
assume the daemon has experimental enabled, and is running the latest
API version.
However, we cannot fill in the correct OSType, so this field is empty
in this situation.
This patch only compares the OSType if the field is non-empty, otherwise
assumes the platform matches.
before this:
docker -H unix:///no/such/socket.sock checkpoint create test test
docker checkpoint create is only supported on a Docker daemon running on linux, but the Docker daemon is running on
with this patch:
docker -H unix:///no/such/socket.sock checkpoint create test test
Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?
Co-authored-by: Adyanth Hosavalike <ahosavalike@ucsd.edu>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This internalizes constructing the Client(), which allows us to provide
fallbacks when trying to determin the current API version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit cbec75e2f3 updated `runDocker()` to load
plugin-stubs before `processAliases()` was executed. As a result, plugin
stubs were considered as "builtin commands", causing the alias verification
to fail;
Without alias installed:
```bash
docker version
Client:
Version: 22.06.0-beta.0-140-g3dad26ca2.m
API version: 1.42
Go version: go1.19.1
Git commit: 3dad26ca2
Built: Wed Sep 28 22:36:09 2022
OS/Arch: darwin/arm64
Context: default
...
```
After running `docker buildx install`;
```bash
./build/docker buildx install
cat ~/.docker/config.json
{
"aliases": {
"builder": "buildx"
}
}
./build/docker version
not allowed to alias with builtin "buildx" as target
```
This patch moves loading the stubs _after_ the call to `processAliases()`, so
that verification passes. As an extra precaution, the `processAliases()` function
is also updated to exclude plugin-stub commands.
Note that cbec75e2f3 also introduced a performance
regression, which may be related to the early loading of plugins (and creating
stubs); it looks like various other code locations may also be loading plugins,
for example `tryPluginRun()` calls `pluginmanager.PluginRunCommand()`, which
also traverses plugin directories.
We should look under what circumstances the plugin stub-commands are actually
needed, and make sure that they're only created in those situations.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The information in this struct was basically fixed (there's
some discrepancy around the "DefaultVersion" which, probably,
should never vary, and always be set to the Default (maximum)
API version supported by the client.
Experimental is now always enabled, so this information did
not require any dynamic info as well.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
While running a plugin and canceling with SIGTERM, main process will
close right away without letting the plugin close itself down and handle
the exit code properly. Add appcontext that is useful for handling
sigterm, as well as supporting sigkill when things go wrong.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The CLI disabled experimental features by default, requiring users
to set a configuration option to enable them.
Disabling experimental features was a request from Enterprise users
that did not want experimental features to be accessible.
We are changing this policy, and now enable experimental features
by default. Experimental features may still change and/or removed,
and will be highlighted in the documentation and "usage" output.
For example, the `docker manifest inspect --help` output now shows:
EXPERIMENTAL:
docker manifest inspect is an experimental feature.
Experimental features provide early access to product functionality. These features
may change between releases without warning or can be removed entirely from a future
release. Learn more about experimental features: https://docs.docker.com/go/experimental/
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These annotations use `nil` as value, which caused the flag-hiding functions
to ignore the annotations, and therefore not hiding the flags.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Perform feature detection when actually needed, instead of during
initializing
- Version negotiation is performed either when making an API request,
or when (e.g.) running `docker help` (to hide unsupported features)
- Use a 2 second timeout when 'pinging' the daemon; this should be
sufficient for most cases, and when feature detection failed, the
daemon will still perform validation (and produce an error if needed)
- context.WithTimeout doesn't currently work with ssh connections (connhelper),
so we're only applying this timeout for tcp:// connections, otherwise
keep the old behavior.
Before this change:
time sh -c 'DOCKER_HOST=tcp://42.42.42.41:4242 docker help &> /dev/null'
real 0m32.919s
user 0m0.370s
sys 0m0.227s
time sh -c 'DOCKER_HOST=tcp://42.42.42.41:4242 docker context ls &> /dev/null'
real 0m32.072s
user 0m0.029s
sys 0m0.023s
After this change:
time sh -c 'DOCKER_HOST=tcp://42.42.42.41:4242 docker help &> /dev/null'
real 0m 2.28s
user 0m 0.03s
sys 0m 0.03s
time sh -c 'DOCKER_HOST=tcp://42.42.42.41:4242 docker context ls &> /dev/null'
real 0m 0.13s
user 0m 0.02s
sys 0m 0.02s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The linter is complaining:
cmd/docker/docker.go:72:23⚠️ dockerCli can be github.com/docker/cli/cli/command.Cli (interfacer)
Unclear precisely which change in the preceeding commits caused it to notice
this possibility.
Signed-off-by: Ian Campbell <ijc@docker.com>
Hernan Garcia
Laura Brehm
Bjorn Neergaard
Jonathan A. Sternberg
Sebastiaan van Stijn
Christopher Petito
Alano Terblanche
Brian Goff
Sebastiaan van Stijn
Hernan Garcia
Kevin Alvarez
Adyanth Hosavalike
Nicolas De Loof
Silvin Lubecki
Tonis Tiigi
Ian Campbell