With this in place, all `-j`obs will not run by default on try. This will omit
such jobs in most try pushes even if files-changed matches. This is
unfortunate, but better than running them unconditionally. Fuzzy selections,
and later `just try it` pushes, are the ultimate solution here.
With this change, a push with no try syntax or try_task_config.json will schedule
no tasks at all.
MozReview-Commit-ID: FGjqlDW1FT6
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
In preparation for much more thorough optimization of task-graphs, this
makes a few changes:
* optimization is split into thre phases, with task removal in one phase
(following dependency links) and task replacement in the next (in the
reverse order).
* optimization uses class instances instead of functions for optimizations;
this allows different functions for different phases, and also leaves open
the possibility of composing optimizations.
* the replacement phase can also support removal; this is when utility tasks
like symbol uploads can be optimized away iff their parent task is
optimized.
MozReview-Commit-ID: C5QznNpwqXn
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
In preparation for much more thorough optimization of task-graphs, this
makes a few changes:
* optimization is split into thre phases, with task removal in one phase
(following dependency links) and task replacement in the next (in the
reverse order).
* optimization uses class instances instead of functions for optimizations;
this allows different functions for different phases, and also leaves open
the possibility of composing optimizations.
* the replacement phase can also support removal; this is when utility tasks
like symbol uploads can be optimized away iff their parent task is
optimized.
MozReview-Commit-ID: C5QznNpwqXn
Per IRC dicussion, there is no technical or policy restriction on dependencies
within a task kind. Update the documentation to remove the out-of-date mention
of this limitation. In particular, toolchain build tasks tend to depend on
each other.
MozReview-Commit-ID: K6p0mxyjcvY
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
In preparation for much more thorough optimization of task-graphs, this
makes a few changes:
* optimization is split into thre phases, with task removal in one phase
(following dependency links) and task replacement in the next (in the
reverse order).
* optimization uses class instances instead of functions for optimizations;
this allows different functions for different phases, and also leaves open
the possibility of composing optimizations.
* the replacement phase can also support removal; this is when utility tasks
like symbol uploads can be optimized away iff their parent task is
optimized.
MozReview-Commit-ID: C5QznNpwqXn
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
In preparation for much more thorough optimization of task-graphs, this
makes a few changes:
* optimization is split into thre phases, with task removal in one phase
(following dependency links) and task replacement in the next (in the
reverse order).
* optimization uses class instances instead of functions for optimizations;
this allows different functions for different phases, and also leaves open
the possibility of composing optimizations.
* the replacement phase can also support removal; this is when utility tasks
like symbol uploads can be optimized away iff their parent task is
optimized.
MozReview-Commit-ID: C5QznNpwqXn
`run-task` is taught a --sparse-profile argument to be passed down
to `hg robustcheckout` for the main source checkout. It does what
you expect: performs a sparse checkout using the named profile.
The Taskgraph YAML for run-task is taught a "sparse-profile"
property to define the sparse profile. When defined, --sparse-profile
will be passed down to `run-task` and the cache name will be updated
to reflect the use of sparse checkout.
Our cache checking transform is updated to audit for the use of
--sparse-profile without the corresponding "-sparse" cache name
variation.
The reason we need a distinct cache name for sparse is because
clients that aren't sparse aware will be unable to read checkouts
that are sparse. By forcing sparse and non-sparse into different
cache pools, we avoid compatibility issues.
In the ideal world, we probably support sparse profiles on all the
VCS checkouts that `run-task` supports (e.g. --tools-checkout).
Perfect is the enemy of done. All of this is defined in-tree and
it is easy enough to change atomically.
MozReview-Commit-ID: 79k7Vul0hHO
* eliminate heading for test kinds, of which there is now only one
* make the caches document have a single heading in the TOC
* break out mach commands into a separate document, add ./mach taskgraph morphed
* remove docs for YAML templates support (the .yml file wasn't actually
used -- I expect it was a merge leftover); these are still used for actions.yml,
but once that is gone the code should be removed, too.
* break try out into its own document, edit to distinguish "how to run try"
from "how to generate config"
MozReview-Commit-ID: 76ZopWA9TPL
Today, cache names are mostly static and are brittle as a result.
In theory, when a backwards incompatible change is performed on
something that touches a cache, the cache name needs to be changed
to ensure tasks running the old code don't see cached data from the
new task. (Alternatively, all code is forward compatible, but that is
hard to implement in practice.)
For many things, the process works as planned. However, not everyone
knows that cache names need changed. And, it isn't always obvious
that some things require fresh caches. When mistakes are made, tasks
break intermittently due to cache wonkiness.
One area where we get into trouble is with UID and GID mismatch.
Task A will use a Docker image where our standard "worker" user/group
is UID/GID 1000:1000. Then Task B will use UID/GID 500:500. (This is
common when mixing Debian and RedHel based distros.) If they use the
same cache, then Task B needs to chown/chmod all files in the cache
or there could be a permissions problem. This is exactly why
run-task recursively chowns certain paths before dropping root
privileges.
Permissions setting in run-task solves permissions problems. But
it doesn't solve content incompatibility problems. For that, you
need to change cache names, not use caches, or blow away content
when incompatibilities are detected.
This commit starts the process of adding a little bit more coherence
to our caching story.
There are two main features in this commit:
1) Cache names tied to run-task content
2) Cache validation in run-task
Taskgraph now detects when a task is using caches with run-task. When
caches and run-task are both being used, the cache name is adjusted to
contain a hash of run-task's content. When run-task changes, the cache
name changes. So, changing run-task ensures that all caches from that point
forward are "clean." This frees run-task and any functionality related
to run-task (such as maintaining version control checkouts) from
having to maintain backwards or forwards compatibility with any other
version of run-task. This does mean that any changes to run-task
effectively wipe out caches. But changes to run-task tend to be
seldom, so this should be acceptable.
The second part of this change is code in run-task to record per-cache
properties and validate whether a populated cache is appropriate for
use. To enable this, taskgraph passes a list of cache paths via an
environment variable. For each cache path, run-task looks for a
well-defined file containing a list of "requirements." Right now,
that list is simply a version string. But other features will be
worked into it. If the cache is empty, we simply write out a new
requirements file and are done. If the file exists, we compare
requirements and fail fast if there is a mismatch. If the cache
has content but not this special file, then we abort (because this
should never happen).
The "requirements" validation isn't very useful now because the only
entry comes from run-task's source code and modifying run-task will
change the hash and cause a new cache to be used. The implementation
at this point is more demonstrating the concept than doing anything
terribly useful with it.
MozReview-Commit-ID: HtpXIc7OD1k
Caches shared across levels scare me, even if readers are purported to
perform content verification. We shouldn't take any risks with released
Firefox builds being contaminated by e.g. Try tasks.
Also, the old cache name interferes with my desire to make cache
names dynamic. This requires dynamic scopes. We already have
have level-{{level}}-* scopes for caches. So having all caches
prefixed with this makes things flexible.
MozReview-Commit-ID: LsrcxIYoEh1
This change adds an upload-generated-sources task kind that runs after nightly
builds, fetches their `target.generated-files.tar.gz` artifact, and uploads
all the contained files to an S3 bucket. For actual nightly and release builds
on SCM level 3 trees, the S3 bucket is configured to be publicly accessible,
so that tools like Socorro will be able to fetch generated source files that
appear in crash reports, and debuggers will be able to fetch generated sources
when they show up while debugging Nightly or Release builds.
There are also level-2 and level-1 S3 buckets configured for builds happening
on trees of other levels such as try. They are not configured as publicly
accessible, but they exist so that these tasks can be tested in try.
MozReview-Commit-ID: Js1HRftbtep
This provides a mechanism to modify the behaviour of tasks from a try push. The try_task_config.json
looks something like:
{
"tasks": ["build-linux64/opt", "test-linux64/opt-mochitest-e10s-1"],
"templates": {
"artifact": {"enabled": 1}
}
}
This tells taskgraph to apply the 'artifact' template to all tasks. Templates are JSONe based
.yml files that live under taskcluster/taskgraph/templates. Taskgraph will render every template
against every task definition. The templates themselves can then use JSONe condition statements to
filter out which tasks they should or shouldn't apply to.
MozReview-Commit-ID: J8HVZzOt4mX
It is desirable for e.g. smooth toolchain transitions, to be able to
refer them with generic name from toolchain jobs, while they have more
specific names, including version numbers. For example, in a near
future, there could be a linux64-gcc-4.9 toolchain and a linux64-gcc-6.
The default would be former, but at some point we'd want to switch to
the latter, without having to change all the toolchain definitions.
Moreover, when the switch happens, it would be desirable to have some
jobs stick with the old version, which is hard to keep track of when
all the toolchain definitions for build jobs use the same versioned
toolchain. With an alias, jobs that want the default use the alias, and
jobs that want to use a specific version use the versioned toolchain
name.
This introduces a 'try_task_config' method of scheduling. En lieu of (or in addition to) try
syntax, you can now check in a file called 'try_task_config.json' to the root of the source
tree. The format is either a list of task labels, or dict where task labels are the keys.
Taskcluster will simply schedule any tasks that are listed there.
This file is primarily meant to be generated by tools (which don't exist yet), as the json
format is much easier for tools to generate or consume. These tools should use an in-memory
commit to add the file so it is automatically removed again after the push.
A server-side hook will be added in bug 1380357 to prevent this file from accidentally
landing on non-try trees.
MozReview-Commit-ID: 2zKfZXuuDhH
