Since 509123f935, we've been leaking sockets
in the filesystem on platforms where abstract sockets aren't supported.
That change relied on Go to cleanup our sockets for us, which Go will happily
do as long as we make sure to close the listener, which we weren't previously
doing unless to signal the plugin to terminate.
This change adds a deferred call to `PluginServer.Close()`, which makes sure we
close the plugin server at the end of the plugin execution, so that we never exit
without cleaning up.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Explicitly create the context and set it on the CLI, instead of depending on
NewDockerCli() to instance a default context.
Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Co-authored-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Signed-off-by: Alano Terblanche <18033717+Benehiko@users.noreply.github.com>
Before, for plugin commands, only the plugin name (such as `buildx`)
would be both included as `RootCmd` when passed to the hook plugin,
which isn't enough information for a plugin to decide whether to execute
a hook or not since plugins implement multiple varied commands (`buildx
build`, `buildx prune`, etc.).
This commit changes the hook logic to account for this situation, so
that the the entire configured hook is passed, i.e., if a user has a
hook configured for `buildx imagetools inspect` and the command
`docker buildx imagetools inspect alpine` is called, then the plugin
hooks will be passed `buildx imagetools inspect`.
This logic works for aliased commands too, so whether `docker build ...`
or `docker buildx build` is executed (unless Buildx is disabled) the
hook will be invoked with `buildx build`.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
hooks: include full match when invoking plugins
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Particularly for cases such as `docker exec -it`, it's relevant that the CLI
still executes hooks even if the exec exited with a non-zero exit code,
since this is can be part of a normal `docker exec` invocation depending on
how the user exits.
In the future, this might also be interesting to allow plugins to run
hooks after an error so they can offer error-state recovery suggestions,
although this would require additional work to give the plugin more
information about the failed execution.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This commit adds a "terminal" attribute to `BaseMetricAttributes`
that allows us to discern whether an invocation was from an interactive
terminal or not.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This adds a default otel error handler for the cli in the debug package.
It uses logrus to log the error on the debug level and should work out
of the box with the `--debug` flag and `DEBUG` environment variable.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This changes things to rely on a plugin server that manages all
connections made to the server.
An optional handler can be passed into the server when the caller wants
to do extra things with the connection.
It is the caller's responsibility to close the server.
When the server is closed, first all existing connections are closed
(and new connections are prevented).
Now the signal loop only needs to close the server and not deal with
`net.Conn`'s directly (or double-indirects as the case was before this
change).
The socket, when present in the filesystem, is no longer unlinked
eagerly, as reconnections require it to be present for the lifecycle of
the plugin server.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
In order to solve the "double notification" issue (see:
ef5e5fa03f)
without running the plugin process under a new pgid (see:
https://github.com/moby/moby/issues/47073) we instead check if we're
attached to a TTY, and if so skip signalling the plugin process since it
will already be signalled.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This reverts commit ef5e5fa03f.
Running new plugins under a new pgid isn't a viable solution due to
it causing issues with plugin processes attempting to read from the
TTY (see: https://github.com/moby/moby/issues/47073).
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Changes were made in 1554ac3b5f to provide
a mechanism for the CLI to notify running plugin processes that they
should exit, in order to improve the general CLI/plugin UX. The current
implementation boils down to:
1. The CLI creates a socket
2. The CLI executes the plugin
3. The plugin connects to the socket
4. (When) the CLI receives a termination signal, it uses the socket to
notify the plugin that it should exit
5. The plugin's gets notified via the socket, and cancels it's `cmd.Context`,
which then gets handled appropriately
This change works in most cases and fixes the issue it sets out to solve
(see: https://github.com/docker/compose/pull/11292) however, in the case
where the user has a TTY attached and the plugin is not already handling
received signals, steps 4+ changes:
4. (When) the CLI receives a termination signal, before it can use the
socket to notify the plugin that it should exit, the plugin process
also receives a signal due to sharing the pgid with the CLI
Since we now have a proper "job control" mechanism, we can simplify the
scenarios by executing the plugins with their own process group id,
thereby removing the "double notification" issue and making it so that
plugins can handle the same whether attached to a TTY or not.
In order to make this change "plugin-binary" backwards-compatible, in
the case that a plugin does not connect to the socket, the CLI passes
the signal to the plugin process.
Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Update this function to accept a smaller interface, as it doesn't need
all of "CLI". Also return errors encountered during its operation (although
the caller currently has no error return on its own).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Previously, long lived CLI plugin processes weren't
properly handled
(see: https://github.com/docker/cli/issues/4402)
resulting in plugin processes being left behind
running, after the CLI process exits.
This commit changes the plugin handling code to open
an abstract unix socket before running the plugin and
passing it to the plugin process, and changes the
signal handling on the CLI side to close this socket
which tells the plugin that it should exit.
This implementation makes use of sockets instead of
simply setting PDEATHSIG on the plugin process
so that it will work on both BSDs, assorted UNIXes
and Windows.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
This copies the github.com/moby/buildkit/util/appcontext
package as an internal package. The appcontext package from
BuildKit was the only remaining dependency on BuildKit, and
while we may need some of its functionality, the implementation
is not correct for how it's used in docker/cli (so would need
a rewrite).
Moving a copy of the code into the docker/cli (but as internal
package to prevent others from depending on it) is a first step
in that process, and removes the circular dependency between
BuildKit and the CLi.
We are only using these:
tree vendor/github.com/moby/buildkit
vendor/github.com/moby/buildkit
├── AUTHORS
├── LICENSE
└── util
└── appcontext
├── appcontext.go
├── appcontext_unix.go
├── appcontext_windows.go
└── register.go
3 directories, 6 files
Before this:
go mod graph | grep ' github.com/docker/cli'
github.com/moby/buildkit@v0.11.6 github.com/docker/cli@v23.0.0-rc.1+incompatible
After this:
go mod graph | grep ' github.com/docker/cli'
# (nothing)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is a similar fix as 006c946389, which
fixed this for detection of commands that were executed. Make sure we don't
call the "/_ping" endpoint if we don't need to.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The flag-set that was returned is a pointer to the command's Flags(), which
is in itself passed by reference (as it is modified / set up).
This patch removes the flags return, to prevent assuming it's different than
the command's flags.
While SetupRootCommand is exported, a search showed that it's only used internally,
so changing the signature should not be a problem.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We are currently loading plugin command stubs for every
invocation which still has a significant performance hit.
With this change we are doing this operation only if cobra
completion arg request is found.
- 20.10.23: `docker --version` takes ~15ms
- 23.0.1: `docker --version` takes ~93ms
With this change `docker --version` takes ~9ms
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Both the DockerCLI and Cobra Commands provide accessors for Input, Output,
and Error streams (usually STDIN, STDOUT, STDERR). While we were already
passing DockerCLI's Output to Cobra, we were not doing so for the other
streams (and were passing none for plugin commands), potentially resulting
in DockerCLI output/input to mean something else than a Cobra Command's
intput/output/error.
This patch sets them to the same streams when constructing the Cobra
command.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 20ba591b7f fixed incorrect feature
detection in the CLI, but introduced a regression; previously the "ping"
would only be executed if needed (see b39739123b),
but by not inlining the call to `ServerInfo()` would now always be called.
This patch inlines the code again to only execute the "ping" conditionally,
which allows it to be executed lazily (and omitted for commands that don't
require a daemon connection).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When server is unreachable and docker checkpoint (or any command that
needs to check the server type) is run, incorrect error was returned.
When checking if the daemon had the right OS, we compared the OSType
from the clients ServerInfo(). In situations where the client cannot
connect to the daemon, a "stub" Info is used for this, in which we
assume the daemon has experimental enabled, and is running the latest
API version.
However, we cannot fill in the correct OSType, so this field is empty
in this situation.
This patch only compares the OSType if the field is non-empty, otherwise
assumes the platform matches.
before this:
docker -H unix:///no/such/socket.sock checkpoint create test test
docker checkpoint create is only supported on a Docker daemon running on linux, but the Docker daemon is running on
with this patch:
docker -H unix:///no/such/socket.sock checkpoint create test test
Cannot connect to the Docker daemon at unix:///no/such/socket.sock. Is the docker daemon running?
Co-authored-by: Adyanth Hosavalike <ahosavalike@ucsd.edu>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This internalizes constructing the Client(), which allows us to provide
fallbacks when trying to determin the current API version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit cbec75e2f3 updated `runDocker()` to load
plugin-stubs before `processAliases()` was executed. As a result, plugin
stubs were considered as "builtin commands", causing the alias verification
to fail;
Without alias installed:
```bash
docker version
Client:
Version: 22.06.0-beta.0-140-g3dad26ca2.m
API version: 1.42
Go version: go1.19.1
Git commit: 3dad26ca2
Built: Wed Sep 28 22:36:09 2022
OS/Arch: darwin/arm64
Context: default
...
```
After running `docker buildx install`;
```bash
./build/docker buildx install
cat ~/.docker/config.json
{
"aliases": {
"builder": "buildx"
}
}
./build/docker version
not allowed to alias with builtin "buildx" as target
```
This patch moves loading the stubs _after_ the call to `processAliases()`, so
that verification passes. As an extra precaution, the `processAliases()` function
is also updated to exclude plugin-stub commands.
Note that cbec75e2f3 also introduced a performance
regression, which may be related to the early loading of plugins (and creating
stubs); it looks like various other code locations may also be loading plugins,
for example `tryPluginRun()` calls `pluginmanager.PluginRunCommand()`, which
also traverses plugin directories.
We should look under what circumstances the plugin stub-commands are actually
needed, and make sure that they're only created in those situations.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Sebastiaan van Stijn
Alano Terblanche
Laura Brehm
Christopher Petito
Bjorn Neergaard
Jonathan A. Sternberg
Sebastiaan van Stijn
Brian Goff
Kevin Alvarez
Adyanth Hosavalike
Nicolas De Loof