DockerCLI/circle.yml

124 lines
3.6 KiB
YAML
Raw Normal View History

version: 2
jobs:
lint:
working_directory: /work
docker: [{image: 'docker:18.03-git'}]
steps:
- checkout
- setup_remote_docker:
version: 18.03.1-ce
reusable: true
exclusive: false
- run:
command: docker version
- run:
name: "Lint"
command: |
Do not patch Dockerfiles in CI When building the Dockerfiles for development, those images are mainly used to create a reproducible build-environment. The source code is bind-mounted into the image at runtime; there is no need to create an image with the actual source code, and copying the source code into the image would lead to a new image being created for each code-change (possibly leading up to many "dangling" images for previous code-changes). However, when building (and using) the development images in CI, bind-mounting is not an option, because the daemon is running remotely. To make this work, the circle-ci script patched the Dockerfiles when CI is run; adding a `COPY` to the respective Dockerfiles. Patching Dockerfiles is not really a "best practice" and, even though the source code does not and up in the image, the source would still be _sent_ to the daemon for each build (unless BuildKit is used). This patch updates the makefiles, circle-ci script, and Dockerfiles; - When building the Dockerfiles locally, pipe the Dockerfile through stdin. Doing so, prevents the build-context from being sent to the daemon. This speeds up the build, and doesn't fill up the Docker "temp" directory with content that's not used - Now that no content is sent, add the COPY instructions to the Dockerfiles, and remove the code in the circle-ci script to "live patch" the Dockerfiles. Before this patch is applied (with cache): ``` $ time make -f docker.Makefile build_shell_validate_image docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck . Sending build context to Docker daemon 41MB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 2.75 real 0.45 user 0.56 sys ``` After this patch is applied (with cache):: ``` $ time make -f docker.Makefile build_shell_validate_image cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate - Sending build context to Docker daemon 2.048kB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 0.33 real 0.07 user 0.08 sys ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-28 19:06:10 -05:00
docker build -f dockerfiles/Dockerfile.lint --tag cli-linter:$CIRCLE_BUILD_NUM .
docker run --rm cli-linter:$CIRCLE_BUILD_NUM
cross:
working_directory: /work
docker: [{image: 'docker:18.03-git'}]
parallelism: 3
steps:
- checkout
- setup_remote_docker:
version: 18.03.1-ce
reusable: true
exclusive: false
- run:
name: "Cross"
command: |
Do not patch Dockerfiles in CI When building the Dockerfiles for development, those images are mainly used to create a reproducible build-environment. The source code is bind-mounted into the image at runtime; there is no need to create an image with the actual source code, and copying the source code into the image would lead to a new image being created for each code-change (possibly leading up to many "dangling" images for previous code-changes). However, when building (and using) the development images in CI, bind-mounting is not an option, because the daemon is running remotely. To make this work, the circle-ci script patched the Dockerfiles when CI is run; adding a `COPY` to the respective Dockerfiles. Patching Dockerfiles is not really a "best practice" and, even though the source code does not and up in the image, the source would still be _sent_ to the daemon for each build (unless BuildKit is used). This patch updates the makefiles, circle-ci script, and Dockerfiles; - When building the Dockerfiles locally, pipe the Dockerfile through stdin. Doing so, prevents the build-context from being sent to the daemon. This speeds up the build, and doesn't fill up the Docker "temp" directory with content that's not used - Now that no content is sent, add the COPY instructions to the Dockerfiles, and remove the code in the circle-ci script to "live patch" the Dockerfiles. Before this patch is applied (with cache): ``` $ time make -f docker.Makefile build_shell_validate_image docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck . Sending build context to Docker daemon 41MB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 2.75 real 0.45 user 0.56 sys ``` After this patch is applied (with cache):: ``` $ time make -f docker.Makefile build_shell_validate_image cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate - Sending build context to Docker daemon 2.048kB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 0.33 real 0.07 user 0.08 sys ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-28 19:06:10 -05:00
docker build -f dockerfiles/Dockerfile.cross --tag cli-builder:$CIRCLE_BUILD_NUM .
name=cross-$CIRCLE_BUILD_NUM-$CIRCLE_NODE_INDEX
docker run \
-e CROSS_GROUP=$CIRCLE_NODE_INDEX \
--name $name cli-builder:$CIRCLE_BUILD_NUM \
make cross
docker cp \
$name:/go/src/github.com/docker/cli/build \
/work/build
- store_artifacts:
path: /work/build
test:
working_directory: /work
docker: [{image: 'docker:18.03-git'}]
steps:
- checkout
- setup_remote_docker:
version: 18.03.1-ce
reusable: true
exclusive: false
- run:
name: "Unit Test with Coverage"
command: |
mkdir -p test-results/unit-tests
Do not patch Dockerfiles in CI When building the Dockerfiles for development, those images are mainly used to create a reproducible build-environment. The source code is bind-mounted into the image at runtime; there is no need to create an image with the actual source code, and copying the source code into the image would lead to a new image being created for each code-change (possibly leading up to many "dangling" images for previous code-changes). However, when building (and using) the development images in CI, bind-mounting is not an option, because the daemon is running remotely. To make this work, the circle-ci script patched the Dockerfiles when CI is run; adding a `COPY` to the respective Dockerfiles. Patching Dockerfiles is not really a "best practice" and, even though the source code does not and up in the image, the source would still be _sent_ to the daemon for each build (unless BuildKit is used). This patch updates the makefiles, circle-ci script, and Dockerfiles; - When building the Dockerfiles locally, pipe the Dockerfile through stdin. Doing so, prevents the build-context from being sent to the daemon. This speeds up the build, and doesn't fill up the Docker "temp" directory with content that's not used - Now that no content is sent, add the COPY instructions to the Dockerfiles, and remove the code in the circle-ci script to "live patch" the Dockerfiles. Before this patch is applied (with cache): ``` $ time make -f docker.Makefile build_shell_validate_image docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck . Sending build context to Docker daemon 41MB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 2.75 real 0.45 user 0.56 sys ``` After this patch is applied (with cache):: ``` $ time make -f docker.Makefile build_shell_validate_image cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate - Sending build context to Docker daemon 2.048kB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 0.33 real 0.07 user 0.08 sys ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-28 19:06:10 -05:00
docker build -f dockerfiles/Dockerfile.dev --tag cli-builder:$CIRCLE_BUILD_NUM .
docker run \
-e GOTESTSUM_JUNITFILE=/tmp/junit.xml \
--name \
test-$CIRCLE_BUILD_NUM cli-builder:$CIRCLE_BUILD_NUM \
make test-coverage
docker cp \
test-$CIRCLE_BUILD_NUM:/tmp/junit.xml \
./test-results/unit-tests/junit.xml
- run:
name: "Upload to Codecov"
command: |
docker cp \
test-$CIRCLE_BUILD_NUM:/go/src/github.com/docker/cli/coverage.txt \
coverage.txt
apk add -U bash curl
curl -s https://codecov.io/bash | bash || \
echo 'Codecov failed to upload'
- store_test_results:
path: test-results
- store_artifacts:
path: test-results
validate:
working_directory: /work
docker: [{image: 'docker:18.03-git'}]
steps:
- checkout
- setup_remote_docker:
version: 18.03.1-ce
reusable: true
exclusive: false
- run:
name: "Validate Vendor, Docs, and Code Generation"
command: |
rm -f .dockerignore # include .git
Do not patch Dockerfiles in CI When building the Dockerfiles for development, those images are mainly used to create a reproducible build-environment. The source code is bind-mounted into the image at runtime; there is no need to create an image with the actual source code, and copying the source code into the image would lead to a new image being created for each code-change (possibly leading up to many "dangling" images for previous code-changes). However, when building (and using) the development images in CI, bind-mounting is not an option, because the daemon is running remotely. To make this work, the circle-ci script patched the Dockerfiles when CI is run; adding a `COPY` to the respective Dockerfiles. Patching Dockerfiles is not really a "best practice" and, even though the source code does not and up in the image, the source would still be _sent_ to the daemon for each build (unless BuildKit is used). This patch updates the makefiles, circle-ci script, and Dockerfiles; - When building the Dockerfiles locally, pipe the Dockerfile through stdin. Doing so, prevents the build-context from being sent to the daemon. This speeds up the build, and doesn't fill up the Docker "temp" directory with content that's not used - Now that no content is sent, add the COPY instructions to the Dockerfiles, and remove the code in the circle-ci script to "live patch" the Dockerfiles. Before this patch is applied (with cache): ``` $ time make -f docker.Makefile build_shell_validate_image docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck . Sending build context to Docker daemon 41MB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 2.75 real 0.45 user 0.56 sys ``` After this patch is applied (with cache):: ``` $ time make -f docker.Makefile build_shell_validate_image cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate - Sending build context to Docker daemon 2.048kB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 0.33 real 0.07 user 0.08 sys ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-28 19:06:10 -05:00
docker build -f dockerfiles/Dockerfile.dev --tag cli-builder-with-git:$CIRCLE_BUILD_NUM .
docker run --rm cli-builder-with-git:$CIRCLE_BUILD_NUM \
make ci-validate
no_output_timeout: 15m
shellcheck:
working_directory: /work
docker: [{image: 'docker:18.03-git'}]
steps:
- checkout
- setup_remote_docker:
version: 18.03.1-ce
reusable: true
exclusive: false
- run:
name: "Run shellcheck"
command: |
Do not patch Dockerfiles in CI When building the Dockerfiles for development, those images are mainly used to create a reproducible build-environment. The source code is bind-mounted into the image at runtime; there is no need to create an image with the actual source code, and copying the source code into the image would lead to a new image being created for each code-change (possibly leading up to many "dangling" images for previous code-changes). However, when building (and using) the development images in CI, bind-mounting is not an option, because the daemon is running remotely. To make this work, the circle-ci script patched the Dockerfiles when CI is run; adding a `COPY` to the respective Dockerfiles. Patching Dockerfiles is not really a "best practice" and, even though the source code does not and up in the image, the source would still be _sent_ to the daemon for each build (unless BuildKit is used). This patch updates the makefiles, circle-ci script, and Dockerfiles; - When building the Dockerfiles locally, pipe the Dockerfile through stdin. Doing so, prevents the build-context from being sent to the daemon. This speeds up the build, and doesn't fill up the Docker "temp" directory with content that's not used - Now that no content is sent, add the COPY instructions to the Dockerfiles, and remove the code in the circle-ci script to "live patch" the Dockerfiles. Before this patch is applied (with cache): ``` $ time make -f docker.Makefile build_shell_validate_image docker build -t docker-cli-shell-validate -f ./dockerfiles/Dockerfile.shellcheck . Sending build context to Docker daemon 41MB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 2.75 real 0.45 user 0.56 sys ``` After this patch is applied (with cache):: ``` $ time make -f docker.Makefile build_shell_validate_image cat ./dockerfiles/Dockerfile.shellcheck | docker build -t docker-cli-shell-validate - Sending build context to Docker daemon 2.048kB Step 1/2 : FROM debian:stretch-slim ... Successfully built 81e14e8ad856 Successfully tagged docker-cli-shell-validate:latest 0.33 real 0.07 user 0.08 sys ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-28 19:06:10 -05:00
docker build -f dockerfiles/Dockerfile.shellcheck --tag cli-validator:$CIRCLE_BUILD_NUM .
docker run --rm cli-validator:$CIRCLE_BUILD_NUM \
make shellcheck
workflows:
version: 2
ci:
jobs:
- lint
- cross
- test
- validate
- shellcheck