Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Certain tasks are meant to perform follow-up work to another task. For example,
a 'signing' task signs an artifact from a dependency task. An 'upload-symbols'
tasks uploads an artifact from a dependency task.
In general, we only want to run these types of tasks when the dependency task
is running. But in practice, these tasks often cause the dependency to get
pulled in.
We've tried to fix this in the past by making the 'run-on-projects' and
'optimization' keys match their primary dependency. But it's very easy to
mess this up, and some optimizations (like bugbug) take the content of the
task definition into account. So even if both tasks use the same optimization,
they could have different results.
This revision adds a new 'if-dependencies' key that denotes a task that should
only run if one of the dependencies specified in this list is also run. It
will allow us to stop playing whack-a-mole when trying to make sure tasks
aren't being pulled in solely due to these types of dependencies.
This feature is implemented independently of the 'optimization', so tasks that
specify this can still be optimized even if their primary dependency does run.
Differential Revision: https://phabricator.services.mozilla.com/D80508
When links to specific tasks in treeherder was added to the description in
Bug 1630710, This caused the descriptions for several derived tasks to include
the `{'task-reference': '...'}` wrapper in the derived description. This is
in addition to the already duplicate links to treeherder those tasks had.
To ameliorate this issue, keep track of the original description without the
treeherder link and use that in derived tasks.
Differential Revision: https://phabricator.services.mozilla.com/D84047
If we are generating only a part of the graph, to given kind, don't fail if a
build is packaging tests and there is no corresponding test task, as the tests
may not have been generated.
Differential Revision: https://phabricator.services.mozilla.com/D82097
To support using kaniko[1] for building images, we need to generate the docker
contexts in a seprate task from the docker-image task. Since we use the hash of
the context as the cached-task digest, we generate the context in the decision
task already, so this adds a way to write that out to be used by downstream
tasks.
[1] https://github.com/GoogleContainerTools/kaniko
Differential Revision: https://phabricator.services.mozilla.com/D77839
This allows us to change the default optimization strategy used in try pushes.
While probably not super useful to developers, it can help us easily test
changes to new and experimental optimizations on try.
This also changes the default to the 'bugbug_push_schedules' strategy, since
SETA is more or less random and shouldn't be used by 'mach try auto'. In the
future, we'll switch this back to simply using the default optimization as the
default will ideally be the best one that we have.
Differential Revision: https://phabricator.services.mozilla.com/D65746
For mozilla-central, all the code related to taskgraph lives in
taskcluster/taskgraph. Thunderbird's build requirements are evolving, and
we want to be able to have repository-specific code. The natural place for it
to live is an a package beside taskcluster/ci. Add that to python path,
and provide some hooks for adding to the various registries in taskgraph.
Differential Revision: https://phabricator.services.mozilla.com/D60540
This fixes an issue when running "mach taskgraph" with the --root parameter
from the commandline. We use this regularly when working on Thunderbird task
configuration.
The initial list of kind names is read with os.listdir(), which returns
non-unicode strings when given a non-unicode string. As a result, some tasks
fail to validate because there's a non-unicode string where a unicode string
is expected.
Differential Revision: https://phabricator.services.mozilla.com/D60775
This allows users to set TASKGRAPH_OPTIMIZE_STRATEGIES to a
python_path.find_object string. E.g:
TASKGRAPH_OPTIMIZE_STRATEGIES="module:strategies" ./mach taskgraph optimized
This opens the door to swap in external strategies at runtime and will be
used for back testing experimental strategies.
Differential Revision: https://phabricator.services.mozilla.com/D33203
`mach artifact toolchain` gets task definitions from taskgraph, to get the
index path to find the artifacts at. Now that these index paths depend on the
digests of fetch tasks, those kinds need to be loaded as well. This adds a
supported API to get task definitions for a given kind, which loads all the
kind dependencies.
Differential Revision: https://phabricator.services.mozilla.com/D12161
