The `taskgraph` package generates TaskCluster task graphs based on collections
of task "kinds". Initially, there is only one kind, the "legacy" kind, which
reads the YAML files from `testing/taskcluster/tasks` to generate the task
graph.
Try syntax is implemented by filtering the tasks in the taskgraph after it has
been created, then extending the result to include any prerequisite tasks.
A collection of `mach taskgraph` subcommands are provided for developers to
extend or debug the task-graph generation process.
MozReview-Commit-ID: 1TJCns4XxZ8
We can now define a list of "tags" for a task. Specifying "-j <tag>"
in Try syntax will run all tasks having that tag.
MozReview-Commit-ID: Ih9Z0tRZ5VA
Firefox's automation currently tends to run all the jobs all the time.
It is wasteful to do this. For example, running ESLint when the commit
only changes a .cpp file adds no value.
This commit adds support for only running tasks when certain files
change. The new-style tasks introduced by the previous commit have been
taught a "when" dictionary property that defines conditions that should
hold for the task to be executed. We define a "file_patterns" list that
defines lists of mozpack path matching expressions that will be matched
against the set of files changed by the changesets relevant to the
changeset being built. The eslint task has been updated to only run if
files related to it change.
Because conditions may not be accurate, we add a CLI argument to ignore
conditions and force all would-be-filtered tasks to run.
MozReview-Commit-ID: 3OeBSKAQAeg
Currently, tasks are either "build" or "test" tasks. And "test" tasks
are dependent on "build" tasks, so they are effectively an extension of
"build" tasks.
Not everything is a "build" task. Not everything is associated with a
specific platform.
This commit introduces support for defining non-build "tasks" under the
"tasks" top-level element of a jobs YAML file. Interally, they are
treated as "build" tasks but are declared differently.
By default, all these tasks run.
The -j/--job argument has been added to the try syntax parser. It
specifies an opt-in list of these non-build tasks to run. By default, it
runs all of them.
The eslint-gecko "build" task has been moved to this new mechanism.
Documentation for the new task type have been added.
There is definitely some wonkiness in this implementation. For example,
there are references to "build_name," "build_type," and "build_product,"
which arguably are no longer relevant to generic tasks. However, they
appear to be so integrated into task processing (including route names)
that I'm a bit scared to change them.
MozReview-Commit-ID: BY219tLFb6Z
This adds a lot of functionality to the `flags.aliases` field in the try
comment parser, and implements all of the aliases currently supported by
Buildbot's try_parser.py.
The situation changes slightly because of the way chunks are handled; it's no
longer possible to specify chunks using an alias with no leading `-`. This
change should not cause undue hardship.
taskcluster-graph and taskcluster-build mach command now has the
--interactive option and try supports the "-i" flag.
This makes all tasks interactive.
Emulator tasks have some additional parameters not found in other
tasks. Therefore, builds can have an 'additional-parameters' field in
which specific task template replacements can be passed to trygraph.
This field is optional.
Moreover, the syntax for job_flags.yml builds was changed a bit.
Now, instead of:
builds:
<platform>:
types:
opt: <task>
debug: <task>
The syntax is:
builds:
<platform>:
types:
opt:
task: <task>
additional-parameters:
<parameter 1>: <value>
<parameter 2>: <value>
...
debug:
task: <task>
additional-parameters:
<parameter 1>: <value>
<parameter 2>: <value>
...
