The command helpfully downloads a docker image created by a taskcluster task,
which can be helpful when trying to replicate subtle bugs that only occur
in-tree.
This also fixes a bug in hashing Dockerfile directories, where the full
pathname was taken into account. While this pathname is consistent from
decision task to decision task, it is not consistent with developers' home
directories. This change omits the directory prefix, which will cause a
one-time shift in all directory hashes.
MozReview-Commit-ID: EamQzUGG5qY
Firefox's automation currently tends to run all the jobs all the time.
It is wasteful to do this. For example, running ESLint when the commit
only changes a .cpp file adds no value.
This commit adds support for only running tasks when certain files
change. The new-style tasks introduced by the previous commit have been
taught a "when" dictionary property that defines conditions that should
hold for the task to be executed. We define a "file_patterns" list that
defines lists of mozpack path matching expressions that will be matched
against the set of files changed by the changesets relevant to the
changeset being built. The eslint task has been updated to only run if
files related to it change.
Because conditions may not be accurate, we add a CLI argument to ignore
conditions and force all would-be-filtered tasks to run.
MozReview-Commit-ID: 3OeBSKAQAeg
We're about to introduce a mechanism to influence which tasks run based
on what files change. To help debug what's happening, print out the list
of commits that influence the task selection.
MozReview-Commit-ID: Kfj2pf1PSIS
Over in bug 1247802 we deployed a new JSON web API on hg.mozilla.org
that returns JSON metadata for changesets that are relevant for build
automation. It returns a superset of what is returned by the pushlog
JSON API. So we switch to it.
MozReview-Commit-ID: 6X3NANo1mgq
Before, we attempted to build and query a URL that potentially had
"None" in it. This printed some wonky messages in the log and may have
contributed to added latency due to the HTTP request that was doomed to
fail.
MozReview-Commit-ID: JrR5PK33vCn
requests should *always* be used for performing HTTP requests because it
has a better API *and* has sane security defaults compared to the HTTP
request APIs in the Python standard library. Although, Python 2.7.9+
does have slightly saner defaults in the standard library. I still trust
requests more.
MozReview-Commit-ID: GqohpfYYGBw
This adds a `--level` option to taskcluster-graph, and passes the level
supplied from mozilla-taskcluster. It then substitutes that into cache names
for just about every cache (tooltool being the exception, as it verifies hashes
and is thus immune to poisoning). The scopes for these new cache names are
already included in the relevant `moz-tree:level:*` roles.
This also strips `-c6` from cache names; I added this when we were
transitioning from the Ubuntu-based build images, to ensure I got clean caches.
It's no longer necessary.
This adds a `--level` option to taskcluster-graph, and passes the level
supplied from mozilla-taskcluster. It then substitutes that into cache names
for just about every cache (tooltool being the exception, as it verifies hashes
and is thus immune to poisoning). The scopes for these new cache names are
already included in the relevant `moz-tree:level:*` roles.
This also strips `-c6` from cache names; I added this when we were
transitioning from the Ubuntu-based build images, to ensure I got clean caches.
It's no longer necessary.
We're already casting the list of scopes to a set later. So we might as
well use a set from the beginning.
While we're here, sort the final value so output is more deterministic
and easier to read. This sort shouldn't matter since the existing set()
would have lost ordering.
This adds test configs for desktop linux64 unittests, including: mochitest-plain,
mochitest-browser-chrome, mochitest-devtools-chrome, reftest and xpcshell. It
also does a minor refactor of the b2g configs to remove some b2g-specific logic
from the base 'test.yml' config.
This does *not* schedule these tests anywhere just yet.
This adds test configs for desktop linux64 unittests, including: mochitest-plain,
mochitest-browser-chrome, mochitest-devtools-chrome, reftest and xpcshell. It
also does a minor refactor of the b2g configs to remove some b2g-specific logic
from the base 'test.yml' config.
This does *not* schedule these tests anywhere just yet.
This included adding the slugid 1.0.6 python source code in /python since slugid
is now a dependency of the ./mach taskcluster-graph command, as well us updating
references that used it. Previously the implementation was in-tree.
taskcluster-graph and taskcluster-build mach command now has the
--interactive option and try supports the "-i" flag.
This makes all tasks interactive.
This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moving them into
mozharness itself.
This required moving the custom mozharness configs out of the builder image
(which puts them in ~, where mozharness no longer looks) and moves them into
mozharness itself.
Now that mozharness is in tree, we don't need references to mozharness
repositories and revisions.
We don't cleanup all mozharness stuff because android and desktop builds
still references external mozharness.
