Commit Graph

156 Commits

Author SHA1 Message Date
Johan Lorenzo
9f70b474a2 Bug 1789668 - Remove comm- specific argument processing from "mach taskgraph decision". r=releng-reviewers,gbrown
Differential Revision: https://phabricator.services.mozilla.com/D157389
2022-09-20 09:29:03 +00:00
Andrew Halberstadt
108950729a Bug 1746414 - Use standalone taskgraph's parameters module in Gecko, r=taskgraph-reviewers,jmaher
Rather than defining a distinct `Parameter` class for Gecko, this now uses the
`extend_parameters_schema` utility function (which mobile repos are already
using).

As a consequence, shared parameters are now defined in standalone taskgraph.
And only Gecko-specific parameters are listed in
`gecko_taskgraph/parameters.py`

The only exception is `project` which gets redefined so we can override the
standalone taskgraph default (since it derives `project` from the repo name,
which doesn't work for Gecko).

Differential Revision: https://phabricator.services.mozilla.com/D134515
2022-01-31 17:03:19 +00:00
Andreea Pavel
83e7ecfe5f Backed out changeset 976680f20082 (bug 1746414) for causing local bootstrap bustage a=backout 2022-01-28 02:03:53 +02:00
Andrew Halberstadt
b8facd6d39 Bug 1746414 - Use standalone taskgraph's parameters module in Gecko, r=taskgraph-reviewers,jmaher
Rather than defining a distinct `Parameter` class for Gecko, this now uses the
`extend_parameters_schema` utility function (which mobile repos are already
using).

As a consequence, shared parameters are now defined in standalone taskgraph.
And only Gecko-specific parameters are listed in
`gecko_taskgraph/parameters.py`

The only exception is `project` which gets redefined so we can override the
standalone taskgraph default (since it derives `project` from the repo name,
which doesn't work for Gecko).

Differential Revision: https://phabricator.services.mozilla.com/D134515
2022-01-27 14:45:19 +00:00
Alexandru Michis
abce22a3df Backed out changeset 34f8cfa0813b (bug 1746414) for breaking local builds.
CLOSED TREE
2022-01-06 00:01:54 +02:00
Andrew Halberstadt
103f95cd25 Bug 1746414 - Use standalone taskgraph's parameters module in Gecko, r=taskgraph-reviewers,jmaher
Rather than defining a distinct `Parameter` class for Gecko, this now uses the
`extend_parameters_schema` utility function (which mobile repos are already
using).

As a consequence, shared parameters are now defined in standalone taskgraph.
And only Gecko-specific parameters are listed in
`gecko_taskgraph/parameters.py`

The only exception is `project` which gets redefined so we can override the
standalone taskgraph default (since it derives `project` from the repo name,
which doesn't work for Gecko).

Differential Revision: https://phabricator.services.mozilla.com/D134515
2022-01-05 16:37:43 +00:00
Andrew Halberstadt
e4a75ac2ce Bug 1732723 - Rename "taskgraph" Python module to "gecko_taskgraph". r=jmaher
For a long time two copies of the 'taskgraph' module have existed in parallel.
We've attempted to keep them in sync, but over time they have diverged and the
maintenance burden has increased.

In order to reduce this burden, we'd like to re-join the two code bases. The
canonical repo will be the one that lives outside of mozilla-central, and this
module will depend on it. Since they both have the same module name (taskgraph)
we need to rename the version in mozilla-central to avoid collisions.

Other consumers of 'taskgraph' (like mobile repos) have standardized on
'<project>_taskgraph' as their module names. So replicating that here as well.

Differential Revision: https://phabricator.services.mozilla.com/D127118
2021-09-30 09:50:08 -04:00
Alex Lopez
5db2d91e9f Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-27 18:12:51 +00:00
Butkovits Atila
776d615efa Backed out changeset 53b1fa0faa6d (bug 1696251) for breaking the static-analysis integration. a=backout 2021-09-23 13:06:40 +03:00
Alex Lopez
05519dde6e Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-21 20:38:16 +00:00
Butkovits Atila
c1ea191790 Backed out changeset 510dd46a9de7 (bug 1696251) for causing Android build bustages. 2021-09-21 05:16:50 +03:00
Alex Lopez
f3dec7c4dd Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-20 20:21:07 +00:00
Noemi Erli
d04aa2bc01 Backed out changeset 5f5b612878f3 (bug 1696251) for causing multiple bustages 2021-09-18 02:22:39 +03:00
Alex Lopez
9d8b080fa8 Bug 1696251: Allow mach commands as stand-alone functions and adapt existing commands. r=mhentges,webdriver-reviewers,perftest-reviewers,sparky,whimboo
This removes the `@CommandProvider` decorator and the need to implement
mach commands inside subclasses of `MachCommandBase`, and moves all
existing commands out from classes to module level functions.

Differential Revision: https://phabricator.services.mozilla.com/D121512
2021-09-17 19:00:39 +00:00
Andrew Halberstadt
194f57321d Bug 1729060 - [taskgraph] Use --diff implementation from 'main.py', r=taskgraph-reviewers,bhearsum
This replaces Gecko's old '--diff' implementation with the one included with
standalone taskgraph.

Note this revision is a straight sync from standalone taskgraph and should not
contain any additional changes.

Depends on D124835

Differential Revision: https://phabricator.services.mozilla.com/D124836
2021-09-08 20:31:46 +00:00
Kershaw Chang
62ba7857e4 Bug 1716566 - Enable mochitest for socket process, r=jmaher
Differential Revision: https://phabricator.services.mozilla.com/D123109
2021-09-07 14:42:44 +00:00
Andrew Halberstadt
6638804fad Bug 1725404 - Only print warning about -J if '--diff' was specified, r=bhearsum
A drive-by fix I noticed.

Depends on D122778

Differential Revision: https://phabricator.services.mozilla.com/D122779
2021-08-17 13:35:13 +00:00
Andrew Halberstadt
d91cfd7d38 Bug 1725404 - Refactor '--target-kind' / '--output-file' logic to use standalone taskgraph's implementation, r=bhearsum
This syncs in the latest 'main.py' from standalone taskgraph and refactors
mach_commands.py to use it.

Differential Revision: https://phabricator.services.mozilla.com/D122778
2021-08-17 13:35:12 +00:00
Andrew Halberstadt
c6d39c2686 Bug 1725404 - [taskgraph] Implement 'mach taskcluster-image-digest' command, r=bhearsum
I'm not sure if this is needed in Gecko, but it matches the similar command in
standalone taskgraph so makes the two more in sync. I don't think having it
will hurt.

Differential Revision: https://phabricator.services.mozilla.com/D122527
2021-08-16 14:56:27 +00:00
Andrew Halberstadt
b8cd58c34f Bug 1725404 - [taskgraph] Re-use 'action-callback' subcommand args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122526
2021-08-16 14:56:26 +00:00
Andrew Halberstadt
cdfce8a9b7 Bug 1725404 - [taskgraph] Re-use 'decision' args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122524
2021-08-16 14:56:26 +00:00
Andrew Halberstadt
c7757aecb4 Bug 1725404 - [taskgraph] Use copy of 'get_filtered_taskgraph' from main.py, r=bhearsum
These two functions were identical.

Differential Revision: https://phabricator.services.mozilla.com/D122523
2021-08-16 14:56:26 +00:00
Andrew Halberstadt
becc3978d1 Bug 1725404 - [taskgraph] Re-use 'docker' subcommand args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122522
2021-08-16 14:56:25 +00:00
Andrew Halberstadt
48e9c59a24 Bug 1725404 - [taskgraph] Re-use 'show' subcommand args from main.py, r=bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D122521
2021-08-16 14:56:25 +00:00
Alex Lopez
25a0a17cae Bug 1696251 - Replace self with command_context where possible in existing mach commands. r=mhentges,webdriver-reviewers,perftest-reviewers,whimboo
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.

Differential Revision: https://phabricator.services.mozilla.com/D118058
2021-07-19 16:04:25 +00:00
Butkovits Atila
44ca1b9737 Backed out changeset e1921c5112d8 (bug 1696251) for causing bustages complaining about 'CommandContext'. CLOSED TREE 2021-07-16 20:35:55 +03:00
Alex Lopez
eaf2553782 Bug 1696251 - Replace self with command_context where possible in existing mach commands. r=mhentges,webdriver-reviewers,perftest-reviewers,whimboo
This step removes all the dependencies of mach commands to
having a MachCommandBase as the `self` by using the `command_context`
argument instead. This also removes any remaining statefulness from those
classes that implement mach commands, ultimately making it easier to move
existing commands out of classes in a follow-up.

Differential Revision: https://phabricator.services.mozilla.com/D118058
2021-07-16 15:51:29 +00:00
Ben Hearsum
d5d998273e No bug: Add helpful suggestion to ./mach taskgraph --diff for some circumstances r=ahal
This helps hint that in some cases, a user may have forgotten to diff
with `-J` if they were changing task bodies/payloads instead of adding
or removing a task.

Differential Revision: https://phabricator.services.mozilla.com/D116384
2021-06-01 14:10:07 +00:00
Rob Lemley
b63366c1ea Bug 1696251 - Fix "mach taskgraph full" command. r=mhentges
Differential Revision: https://phabricator.services.mozilla.com/D115543
2021-05-19 20:50:05 +00:00
Alex Lopez
bc8dd276ea Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-05-17 16:15:58 +00:00
Andrew Halberstadt
3120ed6b0b No Bug - [taskgraph] Add 'check=True' to subprocess when running with --diff, r=taskgraph-reviewers,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D113860
2021-05-03 20:55:19 +00:00
Andrew Halberstadt
cd9f0c33f6 No Bug - [taskgraph] Generate current taskgraph before base when running with --diff, r=taskgraph-reviewers,aki
This ensures that when your patch has an error, taskgraph generation will fail
faster rather than needing to wait for the base generation to complete.

Depends on D113854

Differential Revision: https://phabricator.services.mozilla.com/D113859
2021-05-03 20:55:18 +00:00
Andrew Halberstadt
7e90fb3b15 Bug 1702819 - [taskgraph] Support Python 3.6 with ./mach taskgraph <sub> --diff, r=taskgraph-reviewers,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D113854
2021-05-03 20:55:18 +00:00
Butkovits Atila
9cdb8e2e2a Backed out 2 changesets (bug 1696251) for causing js-bench-sm failures. CLOSED TREE
Backed out changeset 1c84c9a34575 (bug 1696251)
Backed out changeset e169193b7423 (bug 1696251)
2021-04-23 02:53:36 +03:00
Alex Lopez
cb583c215f Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-04-22 18:56:15 +00:00
Cosmin Sabou
9c545e95b4 Backed out changeset d4a5d8567977 (bug 1696251) for non-unified build bustages. CLOSED TREE 2021-04-19 19:43:31 +03:00
Alex Lopez
4eb9b0babc Bug 1696251 - Pass MachCommandBase object as first argument for Mach Commands. r=mhentges,remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers
As an intermediate step to allow mach commands as standalone functions, the MachCommandBase
subclass instance that currently corresponds to self has to be made available as a separate
argument (named command_context).

Differential Revision: https://phabricator.services.mozilla.com/D109650
2021-04-19 16:15:11 +00:00
Ben Hearsum
02bd2587b7 No bug: ensure ./mach taskgraph --diff returns you to your starting branch when using git. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D111848
2021-04-14 13:49:01 +00:00
Andrew Halberstadt
0f9578d90b Bug 1409733 - [taskcluster] Add a mach setting to override the diff command in |mach taskgraph --diff|, r=taskgraph-reviewers,bhearsum
Differential Revision: https://phabricator.services.mozilla.com/D107542
2021-03-09 16:18:25 +00:00
Andrew Halberstadt
da850de6a8 Bug 1409733 - [taskcluster] Implement a --diff flag on |mach taskgraph|, r=taskgraph-reviewers,bhearsum
This implements a --diff flag on the suite of |mach taskgraph| commands. E.g:

    ./mach taskgraph target -p project=autoland -J --diff

The above will update to the base revision of your stack (e.g the public
revision), generate the taskgraph, update to the current revision, generate the
taskgraph again, then display a diff of the two.

You can also specify a specifier to arg, e.g:

    ./mach taskgraph target -J --diff .~1

This will diff against the parent revision in mercurial (use 'HEAD~1' for git).

Differential Revision: https://phabricator.services.mozilla.com/D107274
2021-03-09 16:18:24 +00:00
Andrew Halberstadt
56a33667c6 Bug 1409733 - [taskcluster] Refactor |mach taskgraph| generation into separate format function, r=taskgraph-reviewers,bhearsum
This will allow us to more easily generate and format the taskgraph multiple
times from the same session (e.g, needed for diffing multiple graphs).

Differential Revision: https://phabricator.services.mozilla.com/D107273
2021-03-09 16:18:24 +00:00
Andrew Halberstadt
96c042070b Bug 1409733 - [taskcluster] Sort imports in |mach_commands.py|, r=taskgraph-reviewers,bhearsum
This was generated using isort and pyright.

Differential Revision: https://phabricator.services.mozilla.com/D107272
2021-03-09 16:18:24 +00:00
Mitchell Hentges
94604ead81 Bug 1661153: Removes _ensure_zstd() and usages r=firefox-build-system-reviewers,glandium
Removes a bunch of zstandard installations that were ran from
within the mach virtualenv, which should already have zstandard
installed.

Differential Revision: https://phabricator.services.mozilla.com/D98387
2020-12-03 22:58:52 +00:00
Cosmin Sabou
73a291a894 Backed out changeset f1d785de7ab3 (bug 1661153) for causing symbols bustages. a=backout. 2020-12-03 00:49:17 +02:00
Mitchell Hentges
bf9c59931b Bug 1661153: Removes _ensure_zstd() and usages r=firefox-build-system-reviewers,glandium
The zstandard package is always installed in the mach virtualenv.
The patch assumes that zstandard is only used from the mach virtualenv,
and never the build virtualenv.

Differential Revision: https://phabricator.services.mozilla.com/D98387
2020-12-02 15:16:14 +00:00
Ricky Stewart
31b8ca9716 Bug 1654103: Standardize on Black for Python code in mozilla-central.
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-26 18:34:53 +00:00
Bogdan Tara
612312a64c Backed out 10 changesets (bug 1654103, bug 1672023, bug 1518999) for PanZoomControllerTest.touchEventForResult gv-junit failures CLOSED TREE
Backed out changeset ff3fb0b4a512 (bug 1672023)
Backed out changeset e7834b600201 (bug 1654103)
Backed out changeset 807893ca8069 (bug 1518999)
Backed out changeset 13e6b92440e9 (bug 1518999)
Backed out changeset 8b2ac5a6c98a (bug 1518999)
Backed out changeset 575748295752 (bug 1518999)
Backed out changeset 65f07ce7b39b (bug 1518999)
Backed out changeset 4bb80556158d (bug 1518999)
Backed out changeset 8ac8461d7bd7 (bug 1518999)
Backed out changeset e8ba13ee17f5 (bug 1518999)
2020-10-24 03:36:18 +03:00
Ricky Stewart
065d2eb893 Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-23 20:40:42 +00:00
Dorel Luca
95b44c982f Backed out changeset 7558c8821a07 (bug 1654103) for multiple failures. CLOSED TREE 2020-10-22 03:51:06 +03:00
Ricky Stewart
43baed3c18 Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-21 21:27:27 +00:00