2017-09-26 18:15:04 -04:00
|
|
|
package manifest
|
|
|
|
|
|
|
|
import (
|
2018-05-03 21:02:44 -04:00
|
|
|
"context"
|
2022-02-25 07:11:27 -05:00
|
|
|
"io"
|
2017-09-26 18:15:04 -04:00
|
|
|
"testing"
|
|
|
|
|
2023-08-30 18:36:58 -04:00
|
|
|
"github.com/distribution/reference"
|
2022-02-25 07:11:27 -05:00
|
|
|
"github.com/docker/cli/cli/manifest/store"
|
2017-09-26 18:15:04 -04:00
|
|
|
manifesttypes "github.com/docker/cli/cli/manifest/types"
|
|
|
|
"github.com/docker/cli/internal/test"
|
|
|
|
"github.com/pkg/errors"
|
2020-02-22 12:12:14 -05:00
|
|
|
"gotest.tools/v3/assert"
|
|
|
|
is "gotest.tools/v3/assert/cmp"
|
|
|
|
"gotest.tools/v3/golden"
|
2017-09-26 18:15:04 -04:00
|
|
|
)
|
|
|
|
|
|
|
|
func TestManifestCreateErrors(t *testing.T) {
|
|
|
|
testCases := []struct {
|
|
|
|
args []string
|
|
|
|
expectedError string
|
|
|
|
}{
|
|
|
|
{
|
|
|
|
args: []string{"too-few-arguments"},
|
|
|
|
expectedError: "requires at least 2 arguments",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
args: []string{"th!si'sa/fa!ke/li$t/name", "example.com/alpine:3.0"},
|
|
|
|
expectedError: "error parsing name for manifest list",
|
|
|
|
},
|
|
|
|
}
|
|
|
|
|
|
|
|
for _, tc := range testCases {
|
test spring-cleaning
This makes a quick pass through our tests;
Discard output/err
----------------------------------------------
Many tests were testing for error-conditions, but didn't discard output.
This produced a lot of noise when running the tests, and made it hard
to discover if there were actual failures, or if the output was expected.
For example:
=== RUN TestConfigCreateErrors
Error: "create" requires exactly 2 arguments.
See 'create --help'.
Usage: create [OPTIONS] CONFIG file|- [flags]
Create a config from a file or STDIN
Error: "create" requires exactly 2 arguments.
See 'create --help'.
Usage: create [OPTIONS] CONFIG file|- [flags]
Create a config from a file or STDIN
Error: error creating config
--- PASS: TestConfigCreateErrors (0.00s)
And after discarding output:
=== RUN TestConfigCreateErrors
--- PASS: TestConfigCreateErrors (0.00s)
Use sub-tests where possible
----------------------------------------------
Some tests were already set-up to use test-tables, and even had a usable
name (or in some cases "error" to check for). Change them to actual sub-
tests. Same test as above, but now with sub-tests and output discarded:
=== RUN TestConfigCreateErrors
=== RUN TestConfigCreateErrors/requires_exactly_2_arguments
=== RUN TestConfigCreateErrors/requires_exactly_2_arguments#01
=== RUN TestConfigCreateErrors/error_creating_config
--- PASS: TestConfigCreateErrors (0.00s)
--- PASS: TestConfigCreateErrors/requires_exactly_2_arguments (0.00s)
--- PASS: TestConfigCreateErrors/requires_exactly_2_arguments#01 (0.00s)
--- PASS: TestConfigCreateErrors/error_creating_config (0.00s)
PASS
It's not perfect in all cases (in the above, there's duplicate "expected"
errors, but Go conveniently adds "#01" for the duplicate). There's probably
also various tests I missed that could still use the same changes applied;
we can improve these in follow-ups.
Set cmd.Args to prevent test-failures
----------------------------------------------
When running tests from my IDE, it compiles the tests before running,
then executes the compiled binary to run the tests. Cobra doesn't like
that, because in that situation `os.Args` is taken as argument for the
command that's executed. The command that's tested now sees the test-
flags as arguments (`-test.v -test.run ..`), which causes various tests
to fail ("Command XYZ does not accept arguments").
# compile the tests:
go test -c -o foo.test
# execute the test:
./foo.test -test.v -test.run TestFoo
=== RUN TestFoo
Error: "foo" accepts no arguments.
The Cobra maintainers ran into the same situation, and for their own
use have added a special case to ignore `os.Args` in these cases;
https://github.com/spf13/cobra/blob/v1.8.1/command.go#L1078-L1083
args := c.args
// Workaround FAIL with "go test -v" or "cobra.test -test.v", see #155
if c.args == nil && filepath.Base(os.Args[0]) != "cobra.test" {
args = os.Args[1:]
}
Unfortunately, that exception is too specific (only checks for `cobra.test`),
so doesn't automatically fix the issue for other test-binaries. They did
provide a `cmd.SetArgs()` utility for this purpose
https://github.com/spf13/cobra/blob/v1.8.1/command.go#L276-L280
// SetArgs sets arguments for the command. It is set to os.Args[1:] by default, if desired, can be overridden
// particularly useful when testing.
func (c *Command) SetArgs(a []string) {
c.args = a
}
And the fix is to explicitly set the command's args to an empty slice to
prevent Cobra from falling back to using `os.Args[1:]` as arguments.
cmd := newSomeThingCommand()
cmd.SetArgs([]string{})
Some tests already take this issue into account, and I updated some tests
for this, but there's likely many other ones that can use the same treatment.
Perhaps the Cobra maintainers would accept a contribution to make their
condition less specific and to look for binaries ending with a `.test`
suffix (which is what compiled binaries usually are named as).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-03 19:29:04 -04:00
|
|
|
tc := tc
|
|
|
|
t.Run(tc.expectedError, func(t *testing.T) {
|
|
|
|
cli := test.NewFakeCli(nil)
|
|
|
|
cmd := newCreateListCommand(cli)
|
|
|
|
cmd.SetArgs(tc.args)
|
|
|
|
cmd.SetOut(io.Discard)
|
|
|
|
cmd.SetErr(io.Discard)
|
|
|
|
assert.ErrorContains(t, cmd.Execute(), tc.expectedError)
|
|
|
|
})
|
2017-09-26 18:15:04 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// create a manifest list, then overwrite it, and inspect to see if the old one is still there
|
|
|
|
func TestManifestCreateAmend(t *testing.T) {
|
2023-11-20 11:38:50 -05:00
|
|
|
manifestStore := store.NewStore(t.TempDir())
|
2017-09-26 18:15:04 -04:00
|
|
|
|
|
|
|
cli := test.NewFakeCli(nil)
|
2023-11-20 11:38:50 -05:00
|
|
|
cli.SetManifestStore(manifestStore)
|
2017-09-26 18:15:04 -04:00
|
|
|
|
|
|
|
namedRef := ref(t, "alpine:3.0")
|
|
|
|
imageManifest := fullImageManifest(t, namedRef)
|
2023-11-20 11:38:50 -05:00
|
|
|
err := manifestStore.Save(ref(t, "list:v1"), namedRef, imageManifest)
|
2018-03-05 18:53:52 -05:00
|
|
|
assert.NilError(t, err)
|
2017-09-26 18:15:04 -04:00
|
|
|
namedRef = ref(t, "alpine:3.1")
|
|
|
|
imageManifest = fullImageManifest(t, namedRef)
|
2023-11-20 11:38:50 -05:00
|
|
|
err = manifestStore.Save(ref(t, "list:v1"), namedRef, imageManifest)
|
2018-03-05 18:53:52 -05:00
|
|
|
assert.NilError(t, err)
|
2017-09-26 18:15:04 -04:00
|
|
|
|
|
|
|
cmd := newCreateListCommand(cli)
|
|
|
|
cmd.SetArgs([]string{"example.com/list:v1", "example.com/alpine:3.1"})
|
|
|
|
cmd.Flags().Set("amend", "true")
|
2022-02-25 07:11:27 -05:00
|
|
|
cmd.SetOut(io.Discard)
|
2017-09-26 18:15:04 -04:00
|
|
|
err = cmd.Execute()
|
2018-03-05 18:53:52 -05:00
|
|
|
assert.NilError(t, err)
|
2017-09-26 18:15:04 -04:00
|
|
|
|
|
|
|
// make a new cli to clear the buffers
|
|
|
|
cli = test.NewFakeCli(nil)
|
2023-11-20 11:38:50 -05:00
|
|
|
cli.SetManifestStore(manifestStore)
|
2017-09-26 18:15:04 -04:00
|
|
|
inspectCmd := newInspectCommand(cli)
|
|
|
|
inspectCmd.SetArgs([]string{"example.com/list:v1"})
|
2018-03-05 18:53:52 -05:00
|
|
|
assert.NilError(t, inspectCmd.Execute())
|
2017-09-26 18:15:04 -04:00
|
|
|
actual := cli.OutBuffer()
|
|
|
|
expected := golden.Get(t, "inspect-manifest-list.golden")
|
2018-03-05 18:53:52 -05:00
|
|
|
assert.Check(t, is.Equal(string(expected), actual.String()))
|
2017-09-26 18:15:04 -04:00
|
|
|
}
|
|
|
|
|
|
|
|
// attempt to overwrite a saved manifest and get refused
|
|
|
|
func TestManifestCreateRefuseAmend(t *testing.T) {
|
2023-11-20 11:38:50 -05:00
|
|
|
manifestStore := store.NewStore(t.TempDir())
|
2017-09-26 18:15:04 -04:00
|
|
|
|
|
|
|
cli := test.NewFakeCli(nil)
|
2023-11-20 11:38:50 -05:00
|
|
|
cli.SetManifestStore(manifestStore)
|
2017-09-26 18:15:04 -04:00
|
|
|
namedRef := ref(t, "alpine:3.0")
|
|
|
|
imageManifest := fullImageManifest(t, namedRef)
|
2023-11-20 11:38:50 -05:00
|
|
|
err := manifestStore.Save(ref(t, "list:v1"), namedRef, imageManifest)
|
2018-03-05 18:53:52 -05:00
|
|
|
assert.NilError(t, err)
|
2017-09-26 18:15:04 -04:00
|
|
|
|
|
|
|
cmd := newCreateListCommand(cli)
|
|
|
|
cmd.SetArgs([]string{"example.com/list:v1", "example.com/alpine:3.0"})
|
2022-02-25 07:11:27 -05:00
|
|
|
cmd.SetOut(io.Discard)
|
test spring-cleaning
This makes a quick pass through our tests;
Discard output/err
----------------------------------------------
Many tests were testing for error-conditions, but didn't discard output.
This produced a lot of noise when running the tests, and made it hard
to discover if there were actual failures, or if the output was expected.
For example:
=== RUN TestConfigCreateErrors
Error: "create" requires exactly 2 arguments.
See 'create --help'.
Usage: create [OPTIONS] CONFIG file|- [flags]
Create a config from a file or STDIN
Error: "create" requires exactly 2 arguments.
See 'create --help'.
Usage: create [OPTIONS] CONFIG file|- [flags]
Create a config from a file or STDIN
Error: error creating config
--- PASS: TestConfigCreateErrors (0.00s)
And after discarding output:
=== RUN TestConfigCreateErrors
--- PASS: TestConfigCreateErrors (0.00s)
Use sub-tests where possible
----------------------------------------------
Some tests were already set-up to use test-tables, and even had a usable
name (or in some cases "error" to check for). Change them to actual sub-
tests. Same test as above, but now with sub-tests and output discarded:
=== RUN TestConfigCreateErrors
=== RUN TestConfigCreateErrors/requires_exactly_2_arguments
=== RUN TestConfigCreateErrors/requires_exactly_2_arguments#01
=== RUN TestConfigCreateErrors/error_creating_config
--- PASS: TestConfigCreateErrors (0.00s)
--- PASS: TestConfigCreateErrors/requires_exactly_2_arguments (0.00s)
--- PASS: TestConfigCreateErrors/requires_exactly_2_arguments#01 (0.00s)
--- PASS: TestConfigCreateErrors/error_creating_config (0.00s)
PASS
It's not perfect in all cases (in the above, there's duplicate "expected"
errors, but Go conveniently adds "#01" for the duplicate). There's probably
also various tests I missed that could still use the same changes applied;
we can improve these in follow-ups.
Set cmd.Args to prevent test-failures
----------------------------------------------
When running tests from my IDE, it compiles the tests before running,
then executes the compiled binary to run the tests. Cobra doesn't like
that, because in that situation `os.Args` is taken as argument for the
command that's executed. The command that's tested now sees the test-
flags as arguments (`-test.v -test.run ..`), which causes various tests
to fail ("Command XYZ does not accept arguments").
# compile the tests:
go test -c -o foo.test
# execute the test:
./foo.test -test.v -test.run TestFoo
=== RUN TestFoo
Error: "foo" accepts no arguments.
The Cobra maintainers ran into the same situation, and for their own
use have added a special case to ignore `os.Args` in these cases;
https://github.com/spf13/cobra/blob/v1.8.1/command.go#L1078-L1083
args := c.args
// Workaround FAIL with "go test -v" or "cobra.test -test.v", see #155
if c.args == nil && filepath.Base(os.Args[0]) != "cobra.test" {
args = os.Args[1:]
}
Unfortunately, that exception is too specific (only checks for `cobra.test`),
so doesn't automatically fix the issue for other test-binaries. They did
provide a `cmd.SetArgs()` utility for this purpose
https://github.com/spf13/cobra/blob/v1.8.1/command.go#L276-L280
// SetArgs sets arguments for the command. It is set to os.Args[1:] by default, if desired, can be overridden
// particularly useful when testing.
func (c *Command) SetArgs(a []string) {
c.args = a
}
And the fix is to explicitly set the command's args to an empty slice to
prevent Cobra from falling back to using `os.Args[1:]` as arguments.
cmd := newSomeThingCommand()
cmd.SetArgs([]string{})
Some tests already take this issue into account, and I updated some tests
for this, but there's likely many other ones that can use the same treatment.
Perhaps the Cobra maintainers would accept a contribution to make their
condition less specific and to look for binaries ending with a `.test`
suffix (which is what compiled binaries usually are named as).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-03 19:29:04 -04:00
|
|
|
cmd.SetErr(io.Discard)
|
2017-09-26 18:15:04 -04:00
|
|
|
err = cmd.Execute()
|
2018-03-06 15:54:24 -05:00
|
|
|
assert.Error(t, err, "refusing to amend an existing manifest list with no --amend flag")
|
2017-09-26 18:15:04 -04:00
|
|
|
}
|
|
|
|
|
|
|
|
// attempt to make a manifest list without valid images
|
|
|
|
func TestManifestCreateNoManifest(t *testing.T) {
|
2023-11-20 11:38:50 -05:00
|
|
|
manifestStore := store.NewStore(t.TempDir())
|
2017-09-26 18:15:04 -04:00
|
|
|
|
|
|
|
cli := test.NewFakeCli(nil)
|
2023-11-20 11:38:50 -05:00
|
|
|
cli.SetManifestStore(manifestStore)
|
2017-09-26 18:15:04 -04:00
|
|
|
cli.SetRegistryClient(&fakeRegistryClient{
|
|
|
|
getManifestFunc: func(_ context.Context, ref reference.Named) (manifesttypes.ImageManifest, error) {
|
|
|
|
return manifesttypes.ImageManifest{}, errors.Errorf("No such image: %v", ref)
|
|
|
|
},
|
|
|
|
getManifestListFunc: func(ctx context.Context, ref reference.Named) ([]manifesttypes.ImageManifest, error) {
|
|
|
|
return nil, errors.Errorf("No such manifest: %s", ref)
|
|
|
|
},
|
|
|
|
})
|
|
|
|
|
|
|
|
cmd := newCreateListCommand(cli)
|
|
|
|
cmd.SetArgs([]string{"example.com/list:v1", "example.com/alpine:3.0"})
|
2022-02-25 07:11:27 -05:00
|
|
|
cmd.SetOut(io.Discard)
|
test spring-cleaning
This makes a quick pass through our tests;
Discard output/err
----------------------------------------------
Many tests were testing for error-conditions, but didn't discard output.
This produced a lot of noise when running the tests, and made it hard
to discover if there were actual failures, or if the output was expected.
For example:
=== RUN TestConfigCreateErrors
Error: "create" requires exactly 2 arguments.
See 'create --help'.
Usage: create [OPTIONS] CONFIG file|- [flags]
Create a config from a file or STDIN
Error: "create" requires exactly 2 arguments.
See 'create --help'.
Usage: create [OPTIONS] CONFIG file|- [flags]
Create a config from a file or STDIN
Error: error creating config
--- PASS: TestConfigCreateErrors (0.00s)
And after discarding output:
=== RUN TestConfigCreateErrors
--- PASS: TestConfigCreateErrors (0.00s)
Use sub-tests where possible
----------------------------------------------
Some tests were already set-up to use test-tables, and even had a usable
name (or in some cases "error" to check for). Change them to actual sub-
tests. Same test as above, but now with sub-tests and output discarded:
=== RUN TestConfigCreateErrors
=== RUN TestConfigCreateErrors/requires_exactly_2_arguments
=== RUN TestConfigCreateErrors/requires_exactly_2_arguments#01
=== RUN TestConfigCreateErrors/error_creating_config
--- PASS: TestConfigCreateErrors (0.00s)
--- PASS: TestConfigCreateErrors/requires_exactly_2_arguments (0.00s)
--- PASS: TestConfigCreateErrors/requires_exactly_2_arguments#01 (0.00s)
--- PASS: TestConfigCreateErrors/error_creating_config (0.00s)
PASS
It's not perfect in all cases (in the above, there's duplicate "expected"
errors, but Go conveniently adds "#01" for the duplicate). There's probably
also various tests I missed that could still use the same changes applied;
we can improve these in follow-ups.
Set cmd.Args to prevent test-failures
----------------------------------------------
When running tests from my IDE, it compiles the tests before running,
then executes the compiled binary to run the tests. Cobra doesn't like
that, because in that situation `os.Args` is taken as argument for the
command that's executed. The command that's tested now sees the test-
flags as arguments (`-test.v -test.run ..`), which causes various tests
to fail ("Command XYZ does not accept arguments").
# compile the tests:
go test -c -o foo.test
# execute the test:
./foo.test -test.v -test.run TestFoo
=== RUN TestFoo
Error: "foo" accepts no arguments.
The Cobra maintainers ran into the same situation, and for their own
use have added a special case to ignore `os.Args` in these cases;
https://github.com/spf13/cobra/blob/v1.8.1/command.go#L1078-L1083
args := c.args
// Workaround FAIL with "go test -v" or "cobra.test -test.v", see #155
if c.args == nil && filepath.Base(os.Args[0]) != "cobra.test" {
args = os.Args[1:]
}
Unfortunately, that exception is too specific (only checks for `cobra.test`),
so doesn't automatically fix the issue for other test-binaries. They did
provide a `cmd.SetArgs()` utility for this purpose
https://github.com/spf13/cobra/blob/v1.8.1/command.go#L276-L280
// SetArgs sets arguments for the command. It is set to os.Args[1:] by default, if desired, can be overridden
// particularly useful when testing.
func (c *Command) SetArgs(a []string) {
c.args = a
}
And the fix is to explicitly set the command's args to an empty slice to
prevent Cobra from falling back to using `os.Args[1:]` as arguments.
cmd := newSomeThingCommand()
cmd.SetArgs([]string{})
Some tests already take this issue into account, and I updated some tests
for this, but there's likely many other ones that can use the same treatment.
Perhaps the Cobra maintainers would accept a contribution to make their
condition less specific and to look for binaries ending with a `.test`
suffix (which is what compiled binaries usually are named as).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-07-03 19:29:04 -04:00
|
|
|
cmd.SetErr(io.Discard)
|
2017-09-26 18:15:04 -04:00
|
|
|
err := cmd.Execute()
|
2018-03-06 15:54:24 -05:00
|
|
|
assert.Error(t, err, "No such image: example.com/alpine:3.0")
|
2017-09-26 18:15:04 -04:00
|
|
|
}
|