From 24186d8008ecbd5e00b09185cd42ac88aac6f701 Mon Sep 17 00:00:00 2001 From: Sebastiaan van Stijn Date: Sun, 17 Mar 2024 14:14:52 +0100 Subject: [PATCH 1/2] ci: fix CodeQL 2.16.4 autobuild CodeQL 2.16.4's auto-build added support for multi-module repositories, and is trying to be smart by searching for modules in every directory, including vendor directories. If no module is found, it's creating one which is ... not what we want, so let's give it a "go.mod". Here's from a run in CI; /opt/hostedtoolcache/CodeQL/2.16.4/x64/codeql/codeql version --format=json { "productName" : "CodeQL", "vendor" : "GitHub", "version" : "2.16.4", "sha" : "9727ba3cd3d5a26f8b9347bf3c3eb4f565ac077b", "branches" : [ "codeql-cli-2.16.4" ], "copyright" : "Copyright (C) 2019-2024 GitHub, Inc.", "unpackedLocation" : "/opt/hostedtoolcache/CodeQL/2.16.4/x64/codeql", "configFileLocation" : "/home/runner/.config/codeql/config", "configFileFound" : false, "features" : { "analysisSummaryV2Option" : true, "buildModeOption" : true, "bundleSupportsIncludeDiagnostics" : true, "featuresInVersionResult" : true, "indirectTracingSupportsStaticBinaries" : false, "informsAboutUnsupportedPathFilters" : true, "supportsPython312" : true, "mrvaPackCreate" : true, "threatModelOption" : true, "traceCommandUseBuildMode" : true, "v2ramSizing" : true, "mrvaPackCreateMultipleQueries" : true, "setsCodeqlRunnerEnvVar" : true } } With 2.16.4, first it is unable to correlate files with the project, considering them "stray" files; Attempting to automatically build go code /opt/hostedtoolcache/CodeQL/2.16.4/x64/codeql/go/tools/autobuild.sh 2024/03/16 15:54:34 Autobuilder was built with go1.22.0, environment has go1.21.8 2024/03/16 15:54:34 LGTM_SRC is /home/runner/work/cli/cli 2024/03/16 15:54:34 Found no go.work files in the workspace; looking for go.mod files... 2024/03/16 15:54:34 Found stray Go source file in cli/cobra.go. 2024/03/16 15:54:34 Found stray Go source file in cli/cobra_test.go. 2024/03/16 15:54:34 Found stray Go source file in cli/command/builder/client_test.go. 2024/03/16 15:54:34 Found stray Go source file in cli/command/builder/cmd.go. ... It then tries to build the binary, but in go modules mode, which fails (it also seems to be doing this for each and every directory); Use "make dev" to start an interactive development container, use "make -f docker.Makefile " to execute this target in a container, or set DISABLE_WARN_OUTSIDE_CONTAINER=1 to disable this warning. Press Ctrl+C now to abort, or wait for the script to continue.. ./scripts/build/binary Building static docker-linux-amd64 + go build -o build/docker-linux-amd64 -tags osusergo pkcs11 -ldflags -X "github.com/docker/cli/cli/version.GitCommit=38c3ff6" -X "github.com/docker/cli/cli/version.BuildTime=2024-03-16T17:20:38Z" -X "github.com/docker/cli/cli/version.Version=38c3ff6.m" -extldflags -static -buildmode=pie github.com/docker/cli/cmd/docker cannot find package "github.com/docker/cli/cmd/docker" in any of: /opt/hostedtoolcache/go/1.21.8/x64/src/github.com/docker/cli/cmd/docker (from $GOROOT) /home/runner/go/src/github.com/docker/cli/cmd/docker (from $GOPATH) make: *** [Makefile:62: binary] Error 1 2024/03/16 17:20:38 Running /usr/bin/make [make] failed, continuing anyway: exit status 2 2024/03/16 17:20:38 Build failed, continuing to install dependencies. 2024/03/16 17:20:38 The code in vendor/gotest.tools/v3/skip seems to be missing a go.mod file. Attempting to initialize one... 2024/03/16 17:20:38 Import path is 'github.com/docker/cli' If also seems to be doing this for ... every package? cat 0_codeql.log | grep 'you are not in a container' | wc -l 497 After which it starts to create modules out of every directory; The code in internal/test/network seems to be missing a go.mod file. Attempting to initialize one... The code in internal/test/notary seems to be missing a go.mod file. Attempting to initialize one... The code in internal/test/output seems to be missing a go.mod file. Attempting to initialize one... The code in opts seems to be missing a go.mod file. Attempting to initialize one... The code in service seems to be missing a go.mod file. Attempting to initialize one... The code in service/logs seems to be missing a go.mod file. Attempting to initialize one... The code in templates seems to be missing a go.mod file. Attempting to initialize one... The code in vendor seems to be missing a go.mod file. Attempting to initialize one... The code in vendor/dario.cat seems to be missing a go.mod file. Attempting to initialize one... The code in vendor/dario.cat/mergo seems to be missing a go.mod file. Attempting to initialize one... ... Skipping dependency package regexp. Skipping dependency package github.com/opencontainers/go-digest. Skipping dependency package github.com/distribution/reference. Extracting /home/runner/work/cli/cli/cli/command/go.mod Done extracting /home/runner/work/cli/cli/cli/command/go.mod (1ms) Extracting /home/runner/work/cli/cli/cli/command/go.mod Done extracting /home/runner/work/cli/cli/cli/command/go.mod (0ms) Extracting /home/runner/work/cli/cli/cli/command/go.mod Done extracting /home/runner/work/cli/cli/cli/command/go.mod (0ms) Signed-off-by: Sebastiaan van Stijn --- .github/workflows/codeql.yml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index cb3413317b..60a918ad2b 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -52,6 +52,16 @@ jobs: uses: github/codeql-action/init@v3 with: languages: go + # CodeQL 2.16.4's auto-build added support for multi-module repositories, + # and is trying to be smart by searching for modules in every directory, + # including vendor directories. If no module is found, it's creating one + # which is ... not what we want, so let's give it a "go.mod". + # see: https://github.com/docker/cli/pull/4944#issuecomment-2002034698 + - + name: Create go.mod + run: | + ln -s vendor.mod go.mod + ln -s vendor.sum go.sum - name: Autobuild uses: github/codeql-action/autobuild@v3 From b120b96ac705f585652ae8a63bff748b4c500252 Mon Sep 17 00:00:00 2001 From: Sebastiaan van Stijn Date: Sun, 17 Mar 2024 14:26:32 +0100 Subject: [PATCH 2/2] ci: set DISABLE_WARN_OUTSIDE_CONTAINER=1 for CodeQL action CodeQL autobuild uses the makefile, but outside of a container, so let's set this variable to prevent it having to wait 10 seconds; Use "make dev" to start an interactive development container, use "make -f docker.Makefile " to execute this target in a container, or set DISABLE_WARN_OUTSIDE_CONTAINER=1 to disable this warning. Press Ctrl+C now to abort, or wait for the script to continue.. Signed-off-by: Sebastiaan van Stijn --- .github/workflows/codeql.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 60a918ad2b..3b8884ca2d 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -26,6 +26,8 @@ jobs: codeql: runs-on: 'ubuntu-latest' timeout-minutes: 360 + env: + DISABLE_WARN_OUTSIDE_CONTAINER: '1' permissions: actions: read contents: read