Before, we were returning None, which gets converted to 0.
Derp.
Also fix a flake8 failure introduced by 9f5fbb3066c9.
We'll also need to generate a new decision image. But that will
require someone with TC privileges to be around. That can be done
in a separate commit to unblock this from landing and fixing
consumers of run-task that aren't the decision image.
MozReview-Commit-ID: 6XuoIxjDozF
Before, we were returning None, which gets converted to 0.
Derp.
Also fix a flake8 failure introduced by 9f5fbb3066c9.
We'll also need to generate a new decision image. But that will
require someone with TC privileges to be around. That can be done
in a separate commit to unblock this from landing and fixing
consumers of run-task that aren't the decision image.
MozReview-Commit-ID: 6XuoIxjDozF
Until now Gecko decision task artifacts would expire after a week. Unfortunately, some of these artifacts are
needed in order for Treeherder to determine which tasks it can schedule on a given push.
In this change, we're changing the expiration of all artifacts except those that are related to docker image
optimizations, which, are very large files and we only want to keep for a week.
MozReview-Commit-ID: 96ahyWmoI7W
The function was only used once and was providing little to no value.
A test of this function has been removed. Tests for the lower-level
context creation function are sufficient.
MozReview-Commit-ID: D9EhmZQlqG5
This restores order to only having a single hash for a context
directory.
Using a tempfile here is a bit unfortunate. It can be optimized later,
if needed.
MozReview-Commit-ID: LMNsvt3fDYx
Relying on global variables like GECKO is a bit dangerous. To facilitate
testing of archive generation in subsequent commits, let's pass an
path into this function.
The argument is currently unused.
MozReview-Commit-ID: Et1UYraflDP
Now that tar file generation is deterministic, we can use the hash
of the created archive rather than the hash of the files that are
(presumably) in the archive.
This temporarily breaks consistent hashing by using independent
hashing mechanisms. This will be cleaned up in a subsequent commit.
MozReview-Commit-ID: CWooVGfDKZO
Upcoming commits will refactor how context tarballs are created. In
preparation for this, we establish a standalone function for creating
context tarballs and refactor docker_image.py to use it.
MozReview-Commit-ID: KEW6ppO1vCl
This introduces a completely new way of specifying test task in-tree,
completely replacing the old spider-web of YAML files.
The high-level view is this:
- some configuration files are used to determine which test suites to run
for each test platform, and against which build platforms
- each test suite is then represented by a dictionary, and modified by a
sequence of transforms, duplicating as necessary (e.g., chunks), until
it becomes a task definition
The transforms allow sufficient generality to support just about any desired
configuration, with the advantage that common configurations are "easy" while
unusual configurations are supported but notable for their oddness (they
require a custom transform).
As of this commit, this system produces the same set of test graphs as the
existing YAML, modulo:
- extra.treeherder.groupName -- this was not consistent in the YAML
- extra.treeherder.build -- this is ignored by taskcluster-treeherder anyway
- mozharness command argument order
- boolean True values for environment variables are now the string "true"
- metadata -- this is now much more consistent, with task name being the label
Testing of this commit demonstrates that it produces the same set of test tasks for
the following projects (those which had special cases defined in the YAML):
- autoland
- ash (*)
- willow
- mozilla-inbound
- mozilla-central
- try:
-b do -p all -t all -u all
-b d -p linux64,linux64-asan -u reftest -t none
-b d -p linux64,linux64-asan -u reftest[x64] -t none[x64]
(*) this patch omits the linux64/debug tc-M-e10s(dt) test, which is enabled on
ash; ash will require a small changeset to re-enable this test.
IGNORE BAD COMMIT MESSAGES (because the hook flags try syntax!)
MozReview-Commit-ID: G34dg9f17Hq
