Commit Graph

7085 Commits

Author SHA1 Message Date
Silvin Lubecki dbb2a52a74
Merge pull request #2716 from thaJeztah/update_git_remote_description
docs/build: add note about git subdirectories with BuildKit
2020-09-14 12:24:57 +02:00
Sebastiaan van Stijn b4db7e38bc
docs/build: add note about git subdirectories with BuildKit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-14 11:36:55 +02:00
Silvin Lubecki 2955ece024
Merge pull request #2713 from thaJeztah/update_circleci
CircleCI: update to docker 19.03.12
2020-09-11 17:06:41 +02:00
Sebastiaan van Stijn 57326f5e6d
CircleCI: update to docker 19.03.12
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-11 16:44:02 +02:00
Tibor Vass cec8723d8c
Merge pull request #2656 from thaJeztah/bump_buildkit
vendor: buildkit 4d1f260e8490ec438ab66e08bb105577aca0ce06
2020-09-10 17:43:42 -04:00
Tibor Vass 7836597b3d
Merge pull request #2712 from thaJeztah/carry_2660_ulimits
Add ulimits support to docker service and docker stack deploy (carry 2660)
2020-09-10 15:40:25 -04:00
Sebastiaan van Stijn 1778ffbb18
Merge pull request #2709 from thaJeztah/service_caps_reset
Service cap-add/cap-drop: add special "RESET" value
2020-09-10 15:22:41 +02:00
Albin Kerouanton 940907951b
Support ulimits in docker stack deploy
This is related to moby/moby 40639.

Signed-off-by: Albin Kerouanton <albin@akerouanton.name>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-10 14:59:02 +02:00
Albin Kerouanton a9158bdc50
Add ulimits option to docker service create/update/inspect
This is related to moby/moby 40639.

Signed-off-by: Albin Kerouanton <albin@akerouanton.name>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-10 14:41:33 +02:00
Sebastiaan van Stijn 866e4b10a1
opts/UlimitOpt: sort lists by name
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-10 11:56:27 +02:00
Sebastiaan van Stijn 23660be600
Service cap-add/cap-drop: add special "RESET" value
This implements a special "RESET" value that can be used to reset the
list of capabilities to add/drop when updating a service.

Given the following service;

| CapDrop        | CapAdd        |
| -------------- | ------------- |
| CAP_SOME_CAP   |               |

When updating the service, and applying `--cap-drop RESET`, the "drop" list
is reset to its default:

| CapDrop        | CapAdd        |
| -------------- | ------------- |
|                |               |

When updating the service, and applying `--cap-drop RESET`, combined with
`--cap-add CAP_SOME_CAP` and `--cap-drop CAP_SOME_OTHER_CAP`:

| CapDrop        | CapAdd        |
| -------------- | ------------- |
| CAP_FOO_CAP    | CAP_SOME_CAP  |

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-10 11:14:38 +02:00
Sebastiaan van Stijn 7edc00d808
vendor: buildkit 4d1f260e8490ec438ab66e08bb105577aca0ce06
full diff: df35e9818d...4d1f260e84

- moby/buildkit#1551 session: track sessions with a group construct
- moby/buildkit#1534 secrets: allow providing secrets with env
- moby/buildkit#1533 git: support for token authentication
- moby/buildkit#1549 progressui: fix logs time formatting

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 15:19:02 +02:00
Silvin Lubecki 60abe967b5
Merge pull request #2711 from thaJeztah/bump_engine
vendor: bump docker/docker, docker/swarmkit, and dependencies
2020-09-09 15:08:53 +02:00
Sebastiaan van Stijn ed339fa5c1
vendor: update prometheus deps to match docker/docker
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:57:00 +02:00
Sebastiaan van Stijn 338e83837b
vendor: github.com/golang/protobuf v1.3.5
full diff: https://github.com/golang/protobuf/compare/v1.3.3...v1.3.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:09:42 +02:00
Sebastiaan van Stijn 038cc46f88
vendor: github.com/containerd/cgroups 318312a373405e5e91134d8063d04d59768a1bff
full diff: 44306b6a1d...318312a373

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:08:01 +02:00
Sebastiaan van Stijn 9968ccafe8
vendor: github.com/containerd/containerd v1.4.0-rc.1
full diff: c80284d4b5...v1.4.0-rc.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:58 +02:00
Sebastiaan van Stijn 5468092784
vendor: opencontainers/runc v1.0.0-rc92
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc10...v1.0.0-rc92

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:56 +02:00
Sebastiaan van Stijn dbe2f594ed
vendor: golang.org/x/sys ed371f2e16b4b305ee99df548828de367527b76b
full diff: 85ca7c5b95...ed371f2e16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:54 +02:00
Sebastiaan van Stijn e70e756053
vendor golang.org/x/net v0.0.0-20200707034311-ab3426394381
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:52 +02:00
Sebastiaan van Stijn 1c3a97b0ff
vendor golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:50 +02:00
Sebastiaan van Stijn 9f0658fb02
vendor: github.com/gorilla/mux v1.8.0
full diff: https://github.com/gorilla/mux/compare/v1.7.4...v1.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:48 +02:00
Sebastiaan van Stijn 47980a7e49
vendor: github.com/docker/docker f50a40e889fdaeebf14fce1d494f95e60092d21d
full diff: 0f41a77c69...f50a40e889

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:46 +02:00
Sebastiaan van Stijn 0bb70223bd
vendor: github.com/docker/swarmkit d6592ddefd8a5319aadff74c558b816b1a0b2590
full diff: 035d564a36...d6592ddefd

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-09 14:07:44 +02:00
Silvin Lubecki c84ed28d44
Merge pull request #2703 from despreston/des/build-man-fixes
man/docker-build.1:fix --memory-swamp & --network, removing = b/w opts & values
2020-09-09 13:27:16 +02:00
Brian Goff 164802973e
Merge pull request #2687 from thaJeztah/carry_service_caps
[carry 2663] Add capabilities support to stack/service commands
2020-09-08 12:06:27 -07:00
Sebastiaan van Stijn 551ac136e0
Merge pull request #2706 from albers/completion-capabilities-variants
Complete capabilites with and without "CAP_" prefix.
2020-09-08 15:00:50 +02:00
Sebastiaan van Stijn 95037299cb
Service cap-add/cap-drop: handle updates as "tri-state"
Adding/removing capabilities when updating a service is considered a tri-state;

- if the capability was previously "dropped", then remove it from "CapabilityDrop",
  but do NOT add it to "CapabilityAdd". However, if the capability was not yet in
  the service's "CapabilityDrop", then simply add it to the service's "CapabilityAdd"
- likewise, if the capability was previously "added", then remove it from
  "CapabilityAdd", but do NOT add it to "CapabilityDrop". If the capability was
  not yet in the service's "CapabilityAdd", then simply add it to the service's
  "CapabilityDrop".

In other words, given a service with the following:

| CapDrop        | CapAdd        |
| -------------- | ------------- |
| CAP_SOME_CAP   |               |

When updating the service, and applying `--cap-add CAP_SOME_CAP`, the previously
dropped capability is removed:

| CapDrop        | CapAdd        |
| -------------- | ------------- |
|                |               |

When updating the service a second time, applying `--cap-add CAP_SOME_CAP`,
capability is now added:

| CapDrop        | CapAdd        |
| -------------- | ------------- |
|                | CAP_SOME_CAP  |

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-08 14:42:39 +02:00
Sebastiaan van Stijn 190c64b415
Service cap-add/cap-drop: improve handling of combinations and special "ALL" value
When creating and updating services, we need to avoid unneeded service churn.

The interaction of separate lists to "add" and "drop" capabilities, a special
("ALL") capability, as well as a "relaxed" format for accepted capabilities
(case-insensitive, `CAP_` prefix optional) make this rather involved.

This patch updates how we handle `--cap-add` / `--cap-drop` when  _creating_ as
well as _updating_, with the following rules/assumptions applied:

- both existing (service spec) and new (values passed through flags or in
  the compose-file) are normalized and de-duplicated before use.
- the special "ALL" capability is equivalent to "all capabilities" and taken
  into account when normalizing capabilities. Combining "ALL" capabilities
  and other capabilities is therefore equivalent to just specifying "ALL".
- adding capabilities takes precedence over dropping, which means that if
  a capability is both set to be "dropped" and to be "added", it is removed
  from the list to "drop".
- the final lists should be sorted and normalized to reduce service churn
- no validation of capabilities is handled by the client. Validation is
  delegated to the daemon/server.

When deploying a service using a docker-compose file, the docker-compose file
is *mostly* handled as being "declarative". However, many of the issues outlined
above also apply to compose-files, so similar handling is applied to compose
files as well to prevent service churn.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-08 14:38:35 +02:00
Albin Kerouanton c6ec4e081e
service: Add --cap-add & --cap-drop to service cmds
Signed-off-by: Albin Kerouanton <albin@akerouanton.name>
2020-09-08 14:24:55 +02:00
Olli Janatuinen 0db61ff6da
stack: Support cap_add and cap_drop on services
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
Signed-off-by: Albin Kerouanton <albin@akerouanton.name>
2020-09-08 14:24:52 +02:00
Silvin Lubecki ad16982544
Merge pull request #2696 from thaJeztah/multistage_dockerfile
Refactor / cleanup Dockerfile.dev
2020-09-08 11:07:25 +02:00
Sebastiaan van Stijn 01cd748eb6
Dockerfile: use experimental syntax and buildkit cache-mounts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-08 10:30:18 +02:00
Sebastiaan van Stijn 3aee35037a
Makefile: use --mount instead of -v for docker socket
The shorthand `-v` will auto-create the host-path as a directory if
the socket is not yet up, instead of failing the container. To prevent
accidental creation of `/var/run/docker.sock` as a directory, use
the `--mount` flag instead, which does not auto-create host-paths.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-08 10:27:14 +02:00
Sebastiaan van Stijn ab72cc923c
Dockerfile: use bash as default command, remove DOCKER_CLI_SHELL
Bash is installed already, so we might as well use it as a default.

This also removes the DOCKER_CLI_SHELL variable, which  was added
in  b039db985a to allow using bash
instead of (a)sh. Now that Bash is the default, there should be
no need to override, and we can keep things simple.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-08 10:27:12 +02:00
Sebastiaan van Stijn 7c66213f82
Dockerfile: build gotestsum from source
Building form source is slower, but will support other architectures, such as arm64

Before:

     => [gotestsum 1/2] ADD https://github.com/gotestyourself/gotestsum/releases/download/v0.4.0/gotestsum_0.4.0_linux_amd64.tar.gz gotestsum.tar.gz  2.3s
     => [gotestsum 2/2] RUN tar -xf gotestsum.tar.gz gotestsum -C /go/bin                                                                             1.8s

After:

    => [gotestsum 1/1] RUN go get gotest.tools/gotestsum@v0.4.0                                                                                       13.0s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-08 10:27:06 +02:00
Sebastiaan van Stijn 01d6d2e7a7
Dockerfile: refactor to use multi-stage build
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-08 10:25:22 +02:00
Harald Albers 015cb98846 Complete capabilites with and without "CAP_" prefix.
Signed-off-by: Harald Albers <github@albersweb.de>
2020-09-04 11:09:16 +02:00
Des Preston 8df7a9364f man/docker-build.1:fix --memory-swamp and --network, removing = between long-form opts and values
Signed-off-by: Des Preston <despreston@gmail.com>
2020-09-01 12:11:21 -04:00
Silvin Lubecki ed8ce81d58
Merge pull request #2699 from thaJeztah/reduce_table_width
formatter: reduce minimum width for columns in table-view
2020-09-01 10:11:18 +02:00
Tibor Vass 0db7f23957
Merge pull request #2697 from thaJeztah/bump_natsort
vendor: bump github.com/fvbommel/sortorder v1.0.1
2020-08-31 17:35:21 +02:00
Tibor Vass 359ae2407d
Merge pull request #2679 from thaJeztah/simplify_env_handling
opts: simplify ValidateEnv to use os.LookupEnv
2020-08-31 17:32:16 +02:00
Sebastiaan van Stijn dace8fdc75
formatter: reduce minimum width for columns in table-view
The tabwriter was configured to have a min-width for columns of 20 positions.
This seemed quite wide, and caused smaller columns to be printed with a large
gap between.

Before:

    docker container stats

    CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
    29184b3ae391        amazing_shirley     0.00%               800KiB / 1.944GiB     0.04%               1.44kB / 0B         0B / 0B             1
    403c101bad56        agitated_swartz     0.15%               34.31MiB / 1.944GiB   1.72%               10.2MB / 206kB      0B / 0B             51
    0dc4b7f6c6be        container2          0.00%               1.012MiB / 1.944GiB   0.05%               12.9kB / 0B         0B / 0B             5
    2d99abcc6f62        container99         0.00%               972KiB / 1.944GiB     0.05%               13kB / 0B           0B / 0B             5
    9f9aa90173ac        foo                 0.00%               820KiB / 1.944GiB     0.04%               13kB / 0B           0B / 0B             5

    docker container ls

    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
    29184b3ae391        docker-cli-dev      "ash"                    4 hours ago         Up 4 hours                              amazing_shirley
    403c101bad56        docker-dev:master   "hack/dind bash"         3 days ago          Up 3 days                               agitated_swartz
    0dc4b7f6c6be        nginx:alpine        "/docker-entrypoint.…"   4 days ago          Up 4 days           80/tcp              container2
    2d99abcc6f62        nginx:alpine        "/docker-entrypoint.…"   4 days ago          Up 4 days           80/tcp              container99
    9f9aa90173ac        nginx:alpine        "/docker-entrypoint.…"   4 days ago          Up 4 days           80/tcp              foo

    docker image ls

    REPOSITORY          TAG                    IMAGE ID            CREATED             SIZE
    docker-cli-dev      latest                 5f603caa04aa        4 hours ago         610MB
    docker-cli-native   latest                 9dd29f8d387b        4 hours ago         519MB
    docker-dev          master                 8132bf7a199e        3 days ago          2.02GB
    docker-dev          improve-build-errors   69e208994b3f        11 days ago         2.01GB
    docker-dev          refactor-idtools       69e208994b3f        11 days ago         2.01GB

After:

    docker container stats

    CONTAINER ID   NAME              CPU %     MEM USAGE / LIMIT     MEM %     NET I/O          BLOCK I/O   PIDS
    29184b3ae391   amazing_shirley   0.14%     5.703MiB / 1.944GiB   0.29%     1.44kB / 0B      0B / 0B     10
    403c101bad56   agitated_swartz   0.15%     56.97MiB / 1.944GiB   2.86%     10.2MB / 206kB   0B / 0B     51
    0dc4b7f6c6be   container2        0.00%     1016KiB / 1.944GiB    0.05%     12.9kB / 0B      0B / 0B     5
    2d99abcc6f62   container99       0.00%     956KiB / 1.944GiB     0.05%     13kB / 0B        0B / 0B     5
    9f9aa90173ac   foo               0.00%     980KiB / 1.944GiB     0.05%     13kB / 0B        0B / 0B     5

    docker container ls

    CONTAINER ID   IMAGE               COMMAND                  CREATED          STATUS          PORTS     NAMES
    29184b3ae391   docker-cli-dev      "ash"                    12 minutes ago   Up 12 minutes             amazing_shirley
    403c101bad56   docker-dev:master   "hack/dind bash"         3 days ago       Up 3 days                 agitated_swartz
    0dc4b7f6c6be   nginx:alpine        "/docker-entrypoint.…"   4 days ago       Up 4 days       80/tcp    container2
    2d99abcc6f62   nginx:alpine        "/docker-entrypoint.…"   4 days ago       Up 4 days       80/tcp    container99
    9f9aa90173ac   nginx:alpine        "/docker-entrypoint.…"   4 days ago       Up 4 days       80/tcp    foo

    docker image ls

    REPOSITORY          TAG                    IMAGE ID       CREATED         SIZE
    docker-cli-dev      latest                 5f603caa04aa   4 hours ago     610MB
    docker-cli-native   latest                 9dd29f8d387b   4 hours ago     519MB
    docker-dev          master                 8132bf7a199e   3 days ago      2.02GB
    docker-dev          improve-build-errors   69e208994b3f   11 days ago     2.01GB
    docker-dev          refactor-idtools       69e208994b3f   11 days ago     2.01GB

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-31 16:31:15 +02:00
Silvin Lubecki f784262d07
Merge pull request #2700 from thaJeztah/unconvert
formatter: minor refactor to reduce needless conversions
2020-08-31 09:46:11 +02:00
Sebastiaan van Stijn b951b5b8c9
formatter: minor refactor to reduce needless conversions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-28 23:09:27 +02:00
Sebastiaan van Stijn 83eda5313b
opts: simplify ValidateEnv to use os.LookupEnv
os.LookupEnv() was not available yet at the time this was
implemented, but now provides the functionality we need,
so replacing our custom handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-28 18:31:41 +02:00
Sebastiaan van Stijn c5bf79c37e
Merge pull request #2693 from thaJeztah/fix_dynbinary_in_devcontainer
Dockerfile: fix dynbinary inside dev container
2020-08-28 18:13:00 +02:00
Sebastiaan van Stijn 33510e0a68
vendor: bump github.com/fvbommel/sortorder v1.0.1
drops the dependency on github.com/xlab/handysort from go.mod

diff: https://github.com/fvbommel/sortorder/compare/v1.0.0...v1.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-28 18:10:11 +02:00
Sebastiaan van Stijn 0b8f46b061
Dockerfile: fix dynbinary inside dev container
Running `make dynbinary` inside the dev-container failed because
gcc was not installed. Adding the `build-base` metapackage, which
has gcc, make, g++ and other packages that are needed.

Before this change:

    $ make -f docker.Makefile shell
    $$ make dynbinary
    ./scripts/build/dynbinary
    Building dynamically linked build/docker-linux-amd64
    # github.com/docker/cli/cmd/docker
    /usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exec: "gcc": executable file not found in $PATH
    make: *** [Makefile:63: dynbinary] Error 2

With this patch applied

    $ make -f docker.Makefile shell
    $$ make dynbinary
    ./scripts/build/dynbinary
    Building dynamically linked build/docker-linux-amd64
    $$ echo $?
    0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-28 16:25:15 +02:00
Tibor Vass 8fc038afe7
Merge pull request #2694 from thaJeztah/fix_vbomml_util
Replace vbom.ml/util with fvbommel/sortorder
2020-08-28 16:00:24 +02:00