So we can change a string in this file to force the use of new caches,
thus ensuring a clean break from one changeset to the next.
MozReview-Commit-ID: EZsR23a1PZE
The image builder image we use to build docker images is updated
manually, and not necessarily when changes occur in tree that should be
reflected by a new image builder image. For instance, its run-task is
currently outdated. Not enough that it's actually a problem, but it
could rapidly become a problem.
There is also a lot of friction when trying to make changes in how
docker images are built, and while last time I tried, I ended up not
being able to do the changes I wanted to make because the docker version
on the host is too old, but this is already the second time I've been
trying to make things better and hit a wall because the the image
builder is essentially fixed in stone on the docker hub.
So with this change, we make all the docker images use the in-tree image
builder image, except itself, obviously. That one uses the last version
that was uploaded. We may want to update it at some point, but not doing
so will only impact building the image builder image itself, not the
other ones.
When trying to remove an ubiquitous group like tc(), it's hard to tell where the
error was located without grepping my filesystem. This makes it a bit easier to
find and fix these errors.
MozReview-Commit-ID: 8NjvB5zOoqb
Relying on the various transforms setting it manually is error prone,
and, in fact, is why bug 1430037 busted beta. This change makes this
setting happen at a single place. This yields the same full task graph
as before, except for *more* chain-of-trust inputs being set now: they
were missing for toolchain tasks (which makes us closer to bug 1384430).
The image builder image we use to build docker images is updated
manually, and not necessarily when changes occur in tree that should be
reflected by a new image builder image. For instance, its run-task is
currently outdated. Not enough that it's actually a problem, but it
could rapidly become a problem.
There is also a lot of friction when trying to make changes in how
docker images are built, and while last time I tried, I ended up not
being able to do the changes I wanted to make because the docker version
on the host is too old, but this is already the second time I've been
trying to make things better and hit a wall because the the image
builder is essentially fixed in stone on the docker hub.
So with this change, we make all the docker images use the in-tree image
builder image, except itself, obviously. That one uses the last version
that was uploaded. We may want to update it at some point, but not doing
so will only impact building the image builder image itself, not the
other ones.
There are e.g. some build infrastructure changes that we want to have a
controlled impact on the Firefox builds we produce. We have, in multiple
occasions, gone through manual work to compare Firefox builds, most of
the time using the diffoscope tool (https://diffoscope.org/).
This change introduces a new task kind that takes two Firefox builds as
input, either by name (reference to a build from the current task graph)
or by index (reference to a build from a previous push), and compares
them.
In order to get a Firefox build by index, we rely on dummy tasks with
an optimization we expect to always hit, so we add the necessary bits
to ensure those dummy tasks can go through up to the optimization phase
and be optimized out there.
These two new attributes will help us determine which tasks belong in which release promotion graphs. In the future, we'll specify these for all shippable builds as well, and we can reduce the usage of the `product` keys. However, sometimes we need `product` to differ from `shipping-product` (e.g., `mobile` vs `fennec`; in this case we mean `stage_product` vs `shipping_product`), so I haven't yet touched those key/value pairs.
MozReview-Commit-ID: LEuf4CS277Q
In bug 1412690, Dustin noted that the scopes and routes don't belong at the worker level.
To deal with the release indexes, we now have a new `index_builder`. We also add the default
release bbb scope in `build_buildbot_bridge_payload`.
We can and should still move the product info to attributes. I left that for another patch.
MozReview-Commit-ID: 4ZqvnY577S7
Tasks that have the 'always_target' attribute set will be always be included
in the target_task_graph, regardless of target task filtering.
Furthermore, if they were only added because of this attribute (i.e, the
filters would have excluded the task), then the task will be a candidate for
optimization even if the 'optimize_target_tasks' parameter is False.
MozReview-Commit-ID: 9eoVJ5qpAMO
This patch adds the `release-notify-promote` and `release-notify-publish` kinds. It also genericizes all the notifications, and updates the kinds that use those notifications.
MozReview-Commit-ID: 9ymXKzthVF4
This patch adds the beetmover-cdns kind, and adds it to
`publish_fennec`.
This was the first non-buildbot-bridge, non-dummy relpro task, so this
needed a new transform.
This patch also updates the `previous_graph_kinds` and updates the
beetmover scopes in scriptworker.py.
MozReview-Commit-ID: 3rpkjuLjjXz
This patch adds per-task pulse notifications, as well as general support
to handle them.
Longer term we may move away from pulse-notifications, but this allows
us to proceed with pulse-notifications until that time.
MozReview-Commit-ID: 1uB4X682yLT
