When trying to remove an ubiquitous group like tc(), it's hard to tell where the
error was located without grepping my filesystem. This makes it a bit easier to
find and fix these errors.
MozReview-Commit-ID: 8NjvB5zOoqb
Relying on the various transforms setting it manually is error prone,
and, in fact, is why bug 1430037 busted beta. This change makes this
setting happen at a single place. This yields the same full task graph
as before, except for *more* chain-of-trust inputs being set now: they
were missing for toolchain tasks (which makes us closer to bug 1384430).
The image builder image we use to build docker images is updated
manually, and not necessarily when changes occur in tree that should be
reflected by a new image builder image. For instance, its run-task is
currently outdated. Not enough that it's actually a problem, but it
could rapidly become a problem.
There is also a lot of friction when trying to make changes in how
docker images are built, and while last time I tried, I ended up not
being able to do the changes I wanted to make because the docker version
on the host is too old, but this is already the second time I've been
trying to make things better and hit a wall because the the image
builder is essentially fixed in stone on the docker hub.
So with this change, we make all the docker images use the in-tree image
builder image, except itself, obviously. That one uses the last version
that was uploaded. We may want to update it at some point, but not doing
so will only impact building the image builder image itself, not the
other ones.
There are e.g. some build infrastructure changes that we want to have a
controlled impact on the Firefox builds we produce. We have, in multiple
occasions, gone through manual work to compare Firefox builds, most of
the time using the diffoscope tool (https://diffoscope.org/).
This change introduces a new task kind that takes two Firefox builds as
input, either by name (reference to a build from the current task graph)
or by index (reference to a build from a previous push), and compares
them.
In order to get a Firefox build by index, we rely on dummy tasks with
an optimization we expect to always hit, so we add the necessary bits
to ensure those dummy tasks can go through up to the optimization phase
and be optimized out there.
These two new attributes will help us determine which tasks belong in which release promotion graphs. In the future, we'll specify these for all shippable builds as well, and we can reduce the usage of the `product` keys. However, sometimes we need `product` to differ from `shipping-product` (e.g., `mobile` vs `fennec`; in this case we mean `stage_product` vs `shipping_product`), so I haven't yet touched those key/value pairs.
MozReview-Commit-ID: LEuf4CS277Q
In bug 1412690, Dustin noted that the scopes and routes don't belong at the worker level.
To deal with the release indexes, we now have a new `index_builder`. We also add the default
release bbb scope in `build_buildbot_bridge_payload`.
We can and should still move the product info to attributes. I left that for another patch.
MozReview-Commit-ID: 4ZqvnY577S7
Tasks that have the 'always_target' attribute set will be always be included
in the target_task_graph, regardless of target task filtering.
Furthermore, if they were only added because of this attribute (i.e, the
filters would have excluded the task), then the task will be a candidate for
optimization even if the 'optimize_target_tasks' parameter is False.
MozReview-Commit-ID: 9eoVJ5qpAMO
This patch adds the `release-notify-promote` and `release-notify-publish` kinds. It also genericizes all the notifications, and updates the kinds that use those notifications.
MozReview-Commit-ID: 9ymXKzthVF4
This patch adds the beetmover-cdns kind, and adds it to
`publish_fennec`.
This was the first non-buildbot-bridge, non-dummy relpro task, so this
needed a new transform.
This patch also updates the `previous_graph_kinds` and updates the
beetmover scopes in scriptworker.py.
MozReview-Commit-ID: 3rpkjuLjjXz
This patch adds per-task pulse notifications, as well as general support
to handle them.
Longer term we may move away from pulse-notifications, but this allows
us to proceed with pulse-notifications until that time.
MozReview-Commit-ID: 1uB4X682yLT
This patch adds the initial `release-bouncer-aliases` kind, and adds it
to the `publish_fennec` `target_tasks_method`.
It also adds the ability to specify the `tuxedo_server_url`
`by-project`.
MozReview-Commit-ID: 9I4IaUlbCCD
This patch adds the first releasetask as a new kind. To support this, we
added a new `release-promotion` flag in the buildbot job. If this is
set, we use the new `bb_release_worker` function; otherwise we fall back
to the `bb_ci_worker` function (this is the old behavior, factored out
into a separate function).
We also added `build_number` and `release_promotion` attributes in the
task definition. Finally, `build_number` now defaults to 1, allowing us
to create the task graph locally without forcing us to set
`BUILD_NUMBER` in the environment.
MozReview-Commit-ID: 8vNMHJemqAG
We now have a "source" task that produces JSON files with per-file
Bugzilla components and a list of files missing a declared Bugzilla
component. gzip variations are also produced.
The files are published in the index so clients can query e.g.
gecko.v2.mozilla-central.latest.source.source-bugzilla-info/public/components.json
and get the latest metadata. This should help alleviate the need for
querying the moz.build evaluation API on hg.mozilla.org - or at least
facilitate bulk queries of the data from a static source.
MozReview-Commit-ID: 9fAoPSt4bxq
This adds a new `tc-A` Tree Herder group, similar to the `A` Autophone
(or Android) group. (I don't want to include a `g` for Gradle because
eventually there will be nothing that is _not_ Gradle.)
Per https://bugzilla.mozilla.org/show_bug.cgi?id=1371445#c31, the
sheriffs are satisfied with the test output.
As to the rest of
https://bugzilla.mozilla.org/show_bug.cgi?id=1372075#c0, the
documentation is in place, and |mach try fuzzy| has eclipsed
trychooser, so I think we should not update trychooser.
MozReview-Commit-ID: 2OWDEmGCd11
