With this in place, all `-j`obs will not run by default on try. This will omit
such jobs in most try pushes even if files-changed matches. This is
unfortunate, but better than running them unconditionally. Fuzzy selections,
and later `just try it` pushes, are the ultimate solution here.
With this change, a push with no try syntax or try_task_config.json will schedule
no tasks at all.
MozReview-Commit-ID: FGjqlDW1FT6
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
This sets the try_mode property, and parses the try message (if given), early
in the decision task and puts the results into the parameters.
The proximate need is to set optimze_target_tasks for some try modes and not
others. This also replaces the existing logic for parsing messages for certain
kinds, and makes the distinction between the different try modes a little
clearer.
MozReview-Commit-ID: AXJEGLh6pEV
This fixes the issue where "-u web-platform-tests-wdspec" scheduled
nothing. Now it will schedule a e10s-enabled run of wdspec tests.
MozReview-Commit-ID: AdHZqtk1hLy
The upload now uses MOZ_SCM_LEVEL to determine which secret and bucket to
upload to, so it can potentially run at any level.
This also modifies task descriptions to allow {level} in scopes, and updates
try syntax to allow `-j doc-upload` even though run-on-tasks says it doesn't
run on try by default.
MozReview-Commit-ID: Dm27TGPa7IM
The first version of the patch didn't account for the fact that if we did -p all we still couldn't do -u foo[Windows 8]
specifically: -p all means self.platforms is None
https://hg.mozilla.org/mozilla-central/file/16ffc1d05422/taskcluster/taskgraph/try_option_syntax.py#l563
Then we check if the *test* has a run_on_projects that has either `all` or `try`, it doesn't so we abort.
In new code, we merely set a flag to say don't-run-me-by-default and use that in a few places.
MozReview-Commit-ID: 6rjNVZvpYGA
It's a bit hacky to single out 'build' dependencies, but most tasks have a dependency on a docker
image, so we can't blanket skip all 'job' tasks with any dependencies at all. This is far from ideal
but is an improvement on the current behaviour of running build dependencies all the time, even if
the 'job' task gets optimized away.
There is likely a cleverer solution, but that can be follow-up work.
MozReview-Commit-ID: 6T68LT5VSrg
This copies the behavior of mozilla-taskcluster when it
submits try pushes. This will allow us to eventually stop
using mozilla-taskcluster.
MozReview-Commit-ID: J9zC92AE7HZ
The logic here is a bit tricky to grok, but essentially there are two kinds of "job" tasks, those that
should always be considered (and possibly be optimized away later due to "skip-unless-changed"), and
those that should only be considered if their associated build-platform is also going to be scheduled.
If -j is specified, that should supercede both cases.
This patch uses the prescence of the 'build_platform' attribute to draw the distinction.
MozReview-Commit-ID: H9SjeYuZ8F0
We have a `unittest_try_name` for all unit test tasks, and similarly a
`talos_try_name`. The `-j` flag controls tasks known as "jobs" (although the
word has dozens of other meanings, too). Some of those set `job_try_name`, but
others do not and have special-case support in `try_option_syntax.py`. With
this change, all "jobs" set `job_try_name` and the special-case is removed.
MozReview-Commit-ID: 9hvW7wBIl2B
In order to migrate from buildbot to taskcluster gradually, we read an
external json file to tells the percentage of jobs that must run in
Taskcluster.
MozReview-Commit-ID: JQhvsXKpcLz
In order to migrate from buildbot to taskcluster gradually, we read an
external json file to tells the percentage of jobs that must run in
Taskcluster.
MozReview-Commit-ID: K9KZMhe36Cu
This uses the email provided by mozilla-taskcluster to find who to
email about failed decision tasks. It also adds some validation of
the try syntax that we've previously ignored.
Any platforms or build types specified in try sytax that don't exist
in the full task graph will throw an error.
MozReview-Commit-ID: JOKkLle7hEe
this patch:
- adds linux{32,64}-nightly/opt test platforms that mirror the non-nightly test platforms.
- adds an `include_nightly` per-project parameter; this is refered to in the default `target_task_method`. It's still possible to launch custom `target_task_method`s to trigger nightlies against, say, try.
- adds a `filter_for_project` method in `target_tasks.py` that allows for `include_nightly` and `run_on_projects` filtering in the various `target_task_method`s.
- adds nightly filtering into the `TryOptionSyntax` object. By default, this will be off. To trigger nightly tests on try, either submit a new decision task with a different `target_task_method` (e.g. `nightly_fennec`) or flip the `include_nightly` flag to True.
- adds the `nightly` attribute to tests if their builds have that attribute.
MozReview-Commit-ID: DttIZH0BHS2
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
The name 'source-check' is a bit of a misnomer, because it already includes a bunch
of tasks that are proper unittests, not lints. Some of these unittests will soon
depend on a build task, which makes 'source-check' feel even more wrong.
They still have a lot in common with the lint tasks though, so it's probably not
worth splitting this into two separate kinds. Instead, let's just rename this kind
to 'source-test', which means, any task that tests stuff and is run from the source
directory (instead of a tests.zip). I think both lints and python-tests fall squarely
under this definition.
MozReview-Commit-ID: K0gZ5rVLyeD
-w, --taskcluster-worker: schedule taskcluster-worker jobs.
This is necessary to relieve the load of the small pool of data center
machines.
MozReview-Commit-ID: IiOLHUs2ALi
We add the following command line options to Taskcluster try syntax:
--spsProfile - enable profile mode.
--rebuild-talos <N> - retrigger talos tests N times.
--setenv <VAR>=<val> - add extra environments variables.
--tag <TAG> - run tests only the tag TAG.
--no-retry - doesn't retry failed jobs.
We have a chicken-egg problem, as we first generate the full task graph
and then parse the try message. But the graph generation step needs to
know the try message to process the aforementioned options. The
solution is to parse the message before graph generation and then
pass the command line options to the transforms. Then, each transform
can look at the option that interests it and process it accordingly.
The message parse function is configured in kind.yml, which gives some
flexibility for future implementations of alternative syntaxes.
MozReview-Commit-ID: GPFdi0FD6Vn
