Now that Docker image building is called from Python, we can start to
do advanced stuff with it.
With this commit, we switch from building Docker images directly from
the source directory ("the Docker way") to using our custom Docker image
build contexts.
The main advantage of this is that locally-built Docker images can now
use our custom Dockerfile syntax to include extra files in the build
context!
The code for building a Docker image from a context has been extracted
to its own standalone function. I have nefarious plans for this in the
future, such as the ability to override the FROM syntax to specify
URLs of images. This would allow us to host base images on our own
server, which removes a dependency on Docker Hub and improves
determinism, since images on Docker Hub change all the time.
MozReview-Commit-ID: 5lTdV8yEHkc
We already had code for resolving the image registry and tag. We
refactored it slightly to be more useful then changed build.sh to
accept the tag as an argument.
At this point, build.sh is basically a wrapper around `docker`. But
there's a special case for executing custom "build.sh" files we
need to eliminate first...
MozReview-Commit-ID: A9HVvxgCdG2
A limitation of traditional docker build context generation is it
only includes files from the same directory as the Dockerfile. When
repositories have multiple, related Dockerfiles, this limitation
results file duplication or putting all Dockerfiles in the same
directory (which isn't feasible for mozilla-central since they would
need to be in the root directory).
This commit enhances Dockerfiles to allow *any* file from the
repository checkout to be ADDed to the docker build context.
Using the syntax "# %include <path>" you are able to include paths
or directories (relative from the top source directory root) in the
generated context archive. Files add this way are available under the
"topsrcdir/" path and can be ADDed to Docker images.
Since context archive generation is deterministic and the hash of
the resulting archive is used to determine when images need to be
rebuilt, any extra included file that changes will change the hash
of the context archive and force image regeneration.
Basic tests for the new feature have been added.
MozReview-Commit-ID: 4hPZesJuGQV
This restores order to only having a single hash for a context
directory.
Using a tempfile here is a bit unfortunate. It can be optimized later,
if needed.
MozReview-Commit-ID: LMNsvt3fDYx
Relying on global variables like GECKO is a bit dangerous. To facilitate
testing of archive generation in subsequent commits, let's pass an
path into this function.
The argument is currently unused.
MozReview-Commit-ID: Et1UYraflDP
We recently implemented code in mozpack for performing deterministic
tar file creation. It normalizes things like uids, gids, and mtimes
that creep into archives.
MozReview-Commit-ID: 1tn5eXkqACQ
Upcoming commits will refactor how context tarballs are created. In
preparation for this, we establish a standalone function for creating
context tarballs and refactor docker_image.py to use it.
MozReview-Commit-ID: KEW6ppO1vCl
