BuildKit: caching, parallelism and reproducibility

Key points

• BuildKit builds a graph, not a script. Independent stages run in parallel, and anything not needed for the requested target is skipped entirely.
• Cache mounts are the big win for JVM builds. A cache mount on the Gradle or Maven home persists between builds without becoming an image layer, so dependencies are not re-downloaded and not shipped.
• That is different from a normal layer cache: the layer cache is invalidated by any earlier change, while a cache mount survives because it is storage, not content.
• Secret mounts expose a file only for the duration of one RUN. It never becomes a layer, which is the correct way to use a private repository credential during a build.
• In CI the local layer cache is usually empty on every run. --cache-to and --cache-from against a registry give a shared cache across runners and turn a cold CI build into a warm one.
• For reproducibility, pin the base by digest and set SOURCE_DATE_EPOCH. Without it, timestamps differ on every build and two builds of the same commit produce different digests.

Example

# syntax=docker/dockerfile:1

FROM eclipse-temurin:21-jdk AS deps
WORKDIR /src
COPY gradlew settings.gradle build.gradle ./
COPY gradle/ gradle/
# Cache mount: persists across builds, and is NOT part of any layer.
RUN --mount=type=cache,target=/root/.gradle,sharing=locked \
    ./gradlew dependencies --no-daemon

FROM deps AS build
COPY src/ src/
RUN --mount=type=cache,target=/root/.gradle,sharing=locked \
    ./gradlew bootJar --no-daemon -x test

# Runs in PARALLEL with build - BuildKit sees they are independent.
FROM deps AS test
COPY src/ src/
RUN --mount=type=cache,target=/root/.gradle,sharing=locked \
    ./gradlew test --no-daemon

FROM gcr.io/distroless/java21-debian12:nonroot
COPY --from=build /src/build/libs/*.jar /app/app.jar
ENTRYPOINT ["java", "-jar", "/app/app.jar"]

---
# A private repository credential, never written to a layer:
# RUN --mount=type=secret,id=gradle_props,target=/root/.gradle/gradle.properties \
#     ./gradlew publish
#
#   docker build --secret id=gradle_props,src=$HOME/.gradle/gradle.properties .

---
# CI: the local cache is cold every run. Share it through the registry.
$ docker buildx build \
    --cache-from type=registry,ref=registry.example.com/app:buildcache \
    --cache-to   type=registry,ref=registry.example.com/app:buildcache,mode=max \
    --platform linux/amd64,linux/arm64 \
    --push -t registry.example.com/app:1.4.2 .

# Reproducible builds - same commit, same digest:
$ SOURCE_DATE_EPOCH=$(git log -1 --format=%ct) \
  docker buildx build --output type=registry,rewrite-timestamp=true .

Cache mounts survive changes that invalidate the layer cache - and in CI a registry cache is what turns every cold build warm.