Currently, if a task defines its own expiry with a very large value,
that will be respected even on try, where we actually don't want that to
happen.
This also helps simplify the setup for docker images.
We also take on the occasion to remove the discrepancy between the
default expiry for tasks in general and tests in particular. Bug 1258497
set the original expiry to 14 days, bug 1281004 added another place
where the expiry was set to 14 days for tests specifically, and then bug
1304180 changed the expiry to 28 days, but it just seems the location
for tests was overlooked rather than deliberately left to 14 days.
Differential Revision: https://phabricator.services.mozilla.com/D96962
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
In order for Thunderbird to effectively cut down on excess builds, there needs
to be a way to define taskgraph optimization strategies beyond what is made
available in the mozilla repository.
taskgraph.optimize.register_strategy gets half of the job done. The other piece
is task schema validation which checks optimizations against OptimizationSchema.
OptimizationSchema gets moved to taskgraph.optimize.schema so that it is not set
too early by taskgraph.util.schema. Projects then call set_optimization_schema()
to replace the default schema object. set_optimization_schema() needs to be called
in the project's taskgraph:register function before any transform code is loaded.
Differential Revision: https://phabricator.services.mozilla.com/D88429
Certain tasks are meant to perform follow-up work to another task. For example,
a 'signing' task signs an artifact from a dependency task. An 'upload-symbols'
tasks uploads an artifact from a dependency task.
In general, we only want to run these types of tasks when the dependency task
is running. But in practice, these tasks often cause the dependency to get
pulled in.
We've tried to fix this in the past by making the 'run-on-projects' and
'optimization' keys match their primary dependency. But it's very easy to
mess this up, and some optimizations (like bugbug) take the content of the
task definition into account. So even if both tasks use the same optimization,
they could have different results.
This revision adds a new 'if-dependencies' key that denotes a task that should
only run if one of the dependencies specified in this list is also run. It
will allow us to stop playing whack-a-mole when trying to make sure tasks
aren't being pulled in solely due to these types of dependencies.
This feature is implemented independently of the 'optimization', so tasks that
specify this can still be optimized even if their primary dependency does run.
Differential Revision: https://phabricator.services.mozilla.com/D80508
When links to specific tasks in treeherder was added to the description in
Bug 1630710, This caused the descriptions for several derived tasks to include
the `{'task-reference': '...'}` wrapper in the derived description. This is
in addition to the already duplicate links to treeherder those tasks had.
To ameliorate this issue, keep track of the original description without the
treeherder link and use that in derived tasks.
Differential Revision: https://phabricator.services.mozilla.com/D84047
I'm going to change this from being autodected, to specified explicitly, and so
`use-sccache` better captures the meaning.
Differential Revision: https://phabricator.services.mozilla.com/D82483
CLOSED TREE
Backed out changeset 93b955e5c048 (bug 1637544)
Backed out changeset be0717d76643 (bug 1637544)
Backed out changeset 447fea64b68d (bug 1637544)
I'm going to change this from being autodected, to specified explicitly, and so
`use-sccache` better captures the meaning.
Differential Revision: https://phabricator.services.mozilla.com/D82483
This adds an optional routes key to the task_task_config schema,
which is a list of strings. Anything in this key is added to the list
of routes for tasks scheduled by the decision task.
Differential Revision: https://phabricator.services.mozilla.com/D67827
Remove the remainder of automation.py.in and the build support for generating
automation.py.
Some of this functionality was in use, especially for android tests.
Some code was moved or re-implemented in remoteautomation.py or in the affected
harness(es). Some features were removed: There are some minor changes in behavior.
For instance, instead of using a different server startup timeout for debug builds,
one value is used for all builds (due to performance improvements over time,
the longer timeout is no longer needed).
Differential Revision: https://phabricator.services.mozilla.com/D66839
Adjust runme to be executable
Rm sdk/pltf install.Manually add meta
Enforce firefox run instead of notify
Add policy to disable updates
Temp hack to default to firefox instead of notify-send
Fix mach linters
Remove firefox command hack. Proper fix
Remove duplicate cmd in runme
Fix indentantion in kind
Fix more linters
Differential Revision: https://phabricator.services.mozilla.com/D59561
Bug 1602446 disabled coalescing for tasks which was designed to reduce load on
the pools which ran those tasks by skipping some of them. After coalescing was
affecting almost no tasks, it started to coalesce more tasks in November 2019
(bug 1602446) up to a level for which too few tasks ran and test coverage was
impacted. Coalescing got disabled and the conclusion is that coalescing is
currently not needed.
Differential Revision: https://phabricator.services.mozilla.com/D65733
Since the workspace is no longer cached, but needs to be a volume for
performance reasons, add a check to ensure that is the case.
Differential Revision: https://phabricator.services.mozilla.com/D62484
Calling the merge day automation requires an action so that we can pass in parameters such as source and destination repository and branch.
Differential Revision: https://phabricator.services.mozilla.com/D62763
