Description
Description
About two months ago I discovered a bug in Docker Compose when I wanted to use required environment variables. Everything works as expected when I have only one variable, but when I have multiple variables in a string, only the last one is validated. It is more complex, because it is actually validated and the error message is just overridden by the next non-empty variable.
I also fixed it in "compose-spec/compose-go": compose-spec/compose-go#242
However it was not merged and now the latest development version of "compose-spec/compose-go" is not compatible with Docker Compose. I don't exactly know the relation between the compose and compose-go developer teams, so I don't know what the next step could be, but I would like Docker Compose v2 to work properly so I offer any help I can give you. In the end, people will see and remember bugs in Docker Compose and not in its modules.
I create this issue because even if my pull request were merged, considering the current branches and the process of the development, you could not apply that in Docker Compose until you make Docker Compose compatible with the new changes in compose-go in which ExtraHosts is not []string anymore but type HostsList map[string]string . You might have a better idea how it could be fixed in Compose v2 as well without waiting for an other minor version or you can assure us that a new minor version with that fix could be released soon.
If I can see that bugfixes like these can happen, I am happy to continue to work on other bugs as well if it helps.
Steps to reproduce the issue:
Use a compose file like this:
services:
bash:
image: bash:5.1
command:
- bash
- -c
- "echo VARIABLES: ${COMPOSE_TEST_A:?A is empty} ${COMPOSE_TEST_B:?B is empty} ${COMPOSE_TEST_C:?C is empty}"with this .env file:
COMPOSE_TEST_A=A
COMPOSE_TEST_B=B
COMPOSE_TEST_C=C
Run
docker compose up
COMPOSE_TEST_B="" docker compose up
COMPOSE_TEST_C="" docker compose up
Describe the results you received:
The output of the above example is:
[+] Running 1/0
⠿ Container compose-test-bash-1 Recreated 0.0s
Attaching to compose-test-bash-1
compose-test-bash-1 | VARIABLES: A B C
compose-test-bash-1 exited with code 0
[+] Running 1/0
⠿ Container compose-test-bash-1 Created 0.0s
Attaching to compose-test-bash-1
compose-test-bash-1 | VARIABLES: A C
compose-test-bash-1 exited with code 0
invalid interpolation format for services.bash.command.[]: "required variable COMPOSE_TEST_C is missing a value: C is empty". You may need to escape any $ with another $
Describe the results you expected:
Any empty but required variable should result an error message started with the first error:
[+] Running 1/0
⠿ Container compose-test-bash-1 Recreated 0.0s
Attaching to compose-test-bash-1
compose-test-bash-1 | VARIABLES: A B C
compose-test-bash-1 exited with code 0
invalid interpolation format for services.bash.command.[]: "required variable COMPOSE_TEST_B is missing a value: B is empty". You may need to escape any $ with another $
invalid interpolation format for services.bash.command.[]: "required variable COMPOSE_TEST_C is missing a value: C is empty". You may need to escape any $ with another $
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.8.2)
compose: Docker Compose (Docker Inc., v2.5.0)
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
scan: Docker Scan (Docker Inc., v0.17.0)
Server:
Containers: 3
Running: 1
Paused: 0
Stopped: 2
Images: 65
Server Version: 20.10.14
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: z6hojlb8sew26nf77ip1xwzzh
Is Manager: true
ClusterID: ibg3se3bghfvjz73apiicsu6l
Managers: 1
Nodes: 1
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 3
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 192.168.65.3
Manager Addresses:
192.168.65.3:2377
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc version: v1.0.3-0-gf46b6ba
init version: de40ad0
Security Options:
seccomp
Profile: default
cgroupns
Kernel Version: 5.10.104-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 4
Total Memory: 7.75GiB
Name: docker-desktop
ID: PH52:XTVE:4E2N:YEUX:LQ6J:6TQ2:Q7LQ:2IMB:TS4F:2OSW:YGK6:QKNT
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal,192.168.65.4,192.168.65.4
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5000
127.0.0.0/8
Live Restore Enabled: false
Additional environment details:
The environment is not relevant here since I already found the cause which was error messages overriding each other in a callback function, however I tried on MacOs and Linux mainly.