Commit Graph

60 Commits

Author SHA1 Message Date
Ricky Stewart
31b8ca9716 Bug 1654103: Standardize on Black for Python code in mozilla-central.
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-26 18:34:53 +00:00
Bogdan Tara
612312a64c Backed out 10 changesets (bug 1654103, bug 1672023, bug 1518999) for PanZoomControllerTest.touchEventForResult gv-junit failures CLOSED TREE
Backed out changeset ff3fb0b4a512 (bug 1672023)
Backed out changeset e7834b600201 (bug 1654103)
Backed out changeset 807893ca8069 (bug 1518999)
Backed out changeset 13e6b92440e9 (bug 1518999)
Backed out changeset 8b2ac5a6c98a (bug 1518999)
Backed out changeset 575748295752 (bug 1518999)
Backed out changeset 65f07ce7b39b (bug 1518999)
Backed out changeset 4bb80556158d (bug 1518999)
Backed out changeset 8ac8461d7bd7 (bug 1518999)
Backed out changeset e8ba13ee17f5 (bug 1518999)
2020-10-24 03:36:18 +03:00
Ricky Stewart
065d2eb893 Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-23 20:40:42 +00:00
Dorel Luca
95b44c982f Backed out changeset 7558c8821a07 (bug 1654103) for multiple failures. CLOSED TREE 2020-10-22 03:51:06 +03:00
Ricky Stewart
43baed3c18 Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-21 21:27:27 +00:00
Tom Prince
63f5651320 Bug 1653717: [taskgraph] Handle composite description nicely; r=Callek
When links to specific tasks in treeherder was added to the description in
Bug 1630710, This caused the descriptions for several derived tasks to include
the `{'task-reference': '...'}` wrapper in the derived description. This is
in addition to the already duplicate links to treeherder those tasks had.

To ameliorate this issue, keep track of the original description without the
treeherder link and use that in derived tasks.

Differential Revision: https://phabricator.services.mozilla.com/D84047
2020-07-18 15:59:27 +00:00
Adam Gashlin
0b37bff0ad Bug 1645162 - Use nightly key for oak MARs. r=sfraser
Differential Revision: https://phabricator.services.mozilla.com/D79401
2020-06-12 08:06:51 +00:00
Mihai Tabara
e8247a3daa Bug 1632531 - bugfix key verification in partials.r=sfraser,rail
Differential Revision: https://phabricator.services.mozilla.com/D73756
2020-05-04 22:03:13 +00:00
Dorel Luca
9e9aa2a956 Backed out changeset 902ed66de7da (bug 1632531) for linting failure in gecko/taskcluster/taskgraph/transforms/partials.py. CLOSED TREE 2020-05-05 00:56:51 +03:00
Mihai Tabara
6457b4de86 Bug 1632531 - bugfix key verification in partials.r=sfraser,rail
Differential Revision: https://phabricator.services.mozilla.com/D73756
2020-05-04 21:47:54 +00:00
Simon Fraser
43e8294570 Bug 1632793 Update dep key in partials generation r=mtabara
Differential Revision: https://phabricator.services.mozilla.com/D72357
2020-05-04 11:45:26 +00:00
Simon Fraser
56b6f8be9f Bug 1632531 - Ensure unique working directory for parallel use. r=mtabara
Bug 1632531 Refactor funsize/partials to remove unused code, add safer checks

Differential Revision: https://phabricator.services.mozilla.com/D72162
2020-04-23 21:37:44 +00:00
Tom Prince
3464dac803 Bug 1607857: Don't use treeherder platform for partials; r=sfraser
Differential Revision: https://phabricator.services.mozilla.com/D59400
2020-01-10 18:24:13 +00:00
Tom Prince
f7b6cce06b Bug 1607857: [partials] Don't pass platform to funsize; r=sfraser
The only thing it used it for was for a default filename, but we
always pass an explicit name, so remove the platform option and the
default filename.

Differential Revision: https://phabricator.services.mozilla.com/D59399
2020-01-10 18:21:20 +00:00
Daniel Varga
2732af42b7 Backed out 2 changesets (bug 1607857) for causing lint failure at /builds/worker/checkouts/gecko/taskcluster/taskgraph/transforms/mar_signing.py
CLOSED TREE

Backed out changeset ec6201138386 (bug 1607857)
Backed out changeset a6d10498ee87 (bug 1607857)
2020-01-10 20:16:47 +02:00
Tom Prince
1df845c35c Bug 1607857: Don't use treeherder platform for partials; r=sfraser
Differential Revision: https://phabricator.services.mozilla.com/D59400
2020-01-10 17:22:18 +00:00
Tom Prince
0394ace50e Bug 1607857: [partials] Don't pass platform to funsize; r=sfraser
The only thing it used it for was for a default filename, but we
always pass an explicit name, so remove the platform option and the
default filename.

Differential Revision: https://phabricator.services.mozilla.com/D59399
2020-01-10 09:55:27 +00:00
Simon Fraser
533e11cd8c Bug 1607857 Change platform field used in partials tasks r=mtabara
The new code is a superset of the old, so I've left it much the same to avoid conflicts until we come up with a more long term solution.

Differential Revision: https://phabricator.services.mozilla.com/D59181
2020-01-09 15:37:35 +00:00
Sebastian Hengst
7bede697c4 Bug 1547952 - Increase maximum task run time for partials generation on non-asan platforms to 15 minutes. r=mhentges
Differential Revision: https://phabricator.services.mozilla.com/D39249
2019-07-25 11:20:49 +00:00
Simon Fraser
b16423fe9f Bug 1566070 Remove references to datadog from partials r=mtabara
Our datadog contract is coming to an end. Soon we'll add influxdb data collection

Differential Revision: https://phabricator.services.mozilla.com/D38086
2019-07-22 07:11:45 +00:00
Tom Prince
27760efa67 Bug 1458385: [partials] Set the mar-channel-id of partials explicitly; r=sfraser
Since the taskgraph knows the mar-channel-id to use, pass it explicitly to
the partial tasks, rather than extracting the channel from a MAR.

Differential Revision: https://phabricator.services.mozilla.com/D37482
2019-07-11 16:40:09 +00:00
Tom Prince
08ece47786 Bug 1532783: [taskgraph] Switch to using worker aliases for docker-worker and generic-worker; r=dustin
Differential Revision: https://phabricator.services.mozilla.com/D24238
2019-04-30 22:58:06 +00:00
Simon Fraser
5e002b6c22 Bug 1532236 Convert level into integer in partials transform r=mtabara
Differential Revision: https://phabricator.services.mozilla.com/D22108
2019-03-05 17:04:49 +00:00
Noemi Erli
a86e9076c1 Backed out changeset ce3dfcdb5861 (bug 1532236) for linting opt failure in partials.py CLOSED TREE 2019-03-05 18:54:16 +02:00
Simon Fraser
ef143a3f13 Bug 1532236 Convert level into integer in partials transform r=mtabara
Differential Revision: https://phabricator.services.mozilla.com/D22108
2019-03-05 16:12:08 +00:00
Nick Thomas
db8b065556 Bug 1532236 - longer timeout for asan partial generation, r=tomprince
Differential Revision: https://phabricator.services.mozilla.com/D22038
2019-03-05 00:13:11 +00:00
Simon Fraser
b7ab3a2aed Bug 1532236 Improve logging and timeouts in partials generation r=mtabara
Differential Revision: https://phabricator.services.mozilla.com/D21909
2019-03-04 11:56:47 +00:00
Tom Prince
1242b722ee Bug 1524639: [taksgraph] Simplify dependency creation in single_dep transforms; r=aki
Differential Revision: https://phabricator.services.mozilla.com/D18747
2019-02-05 23:21:39 +00:00
Tom Prince
aeb0adc4fa Bug 1492664: Replace uses of get_taskcluster_artifact_url with artifact-reference; r=dustin 2018-12-18 15:35:18 +00:00
Sebastian Hengst
e6609388b4 Backed out 21 changesets (bug 1492664) for breaking cron task for nightlies. a=backout
Backed out changeset a7d50dbb2c8e (bug 1492664)
Backed out changeset 2d876c4ece8b (bug 1492664)
Backed out changeset c82285d253de (bug 1492664)
Backed out changeset bf6d089640eb (bug 1492664)
Backed out changeset d9a7f2ce49c3 (bug 1492664)
Backed out changeset 06c466ab4323 (bug 1492664)
Backed out changeset c1ea4a10cc8d (bug 1492664)
Backed out changeset 4c63a04fdd47 (bug 1492664)
Backed out changeset 742b038bb1dd (bug 1492664)
Backed out changeset 911b4b0fb683 (bug 1492664)
Backed out changeset 870c8cec99e5 (bug 1492664)
Backed out changeset 77699b51336b (bug 1492664)
Backed out changeset 29f33f22fd8b (bug 1492664)
Backed out changeset e7f305408708 (bug 1492664)
Backed out changeset 335a92b1f424 (bug 1492664)
Backed out changeset c566f1c8dcdf (bug 1492664)
Backed out changeset c77ae59aba41 (bug 1492664)
Backed out changeset 9c35dd209c6b (bug 1492664)
Backed out changeset a972d6b4434e (bug 1492664)
Backed out changeset 5ea6f03f845e (bug 1492664)
Backed out changeset 0699d3873e44 (bug 1492664)
2018-12-20 12:43:22 +02:00
Andreea Pavel
f3978e7d28 NO BUG - fix typo in copy_attributes_from_dependent_job a=merge-fix 2018-12-20 07:25:33 +02:00
Razvan Maries
521c2bbb44 Merge mozilla-inbound to mozilla-central a=merge 2018-12-20 07:04:06 +02:00
Tom Prince
9b0f7c3319 Bug 1514124: [win64-aarch64] Pass the architecture to mar generation, so that it can pass the appropriate arch flag to xz; r=aki,Callek
XZ supports rewritting addresses in executable code, which is architechture
specific. The updater is compiled with support for the target architecture
only, so we can't always compress updates passing `--x86` to XZ. This threads
the architecture through to the repackage steps, so we can pass the appropraite
flags to the update packaging scripts.

Differential Revision: https://phabricator.services.mozilla.com/D14601
2018-12-17 21:45:09 +00:00
Margareta Eliza Balazs
c18ee639c6 Backed out 16 changesets (bug 1492664) for breaking developer artifact builds, requested by standard8 a=backout
Backed out changeset 31e500489665 (bug 1492664)
Backed out changeset f4945658d45f (bug 1492664)
Backed out changeset 6d17291b8b92 (bug 1492664)
Backed out changeset 90f3faa36137 (bug 1492664)
Backed out changeset 0b229b00818a (bug 1492664)
Backed out changeset 5eb2c77d70a9 (bug 1492664)
Backed out changeset e1ebad5d89c5 (bug 1492664)
Backed out changeset 3017e5890739 (bug 1492664)
Backed out changeset c8b7e620eabf (bug 1492664)
Backed out changeset d3dfbd848236 (bug 1492664)
Backed out changeset 5c92bb5ac895 (bug 1492664)
Backed out changeset fb7cfca6ebc3 (bug 1492664)
Backed out changeset 0c4101230d4d (bug 1492664)
Backed out changeset b93a0fcc86f3 (bug 1492664)
Backed out changeset 6dc9522ee0bf (bug 1492664)
Backed out changeset 85d7f8b330eb (bug 1492664)
2018-12-19 11:45:29 +02:00
Tom Prince
65fe94b552 Bug 1492664: Replace uses of get_taskcluster_artifact_url with artifact-reference; r=dustin
Differential Revision: https://phabricator.services.mozilla.com/D14812
2018-12-18 17:23:10 +00:00
Aki Sasaki
8e7d2bd834 Bug 1501878 - point partials at unsigned complete mars. r=tomprince
Partial MAR generation doesn't need a signed complete MAR.
By pointing the partials tasks at the unsigned complete MARs, we are
able to defer signing the complete MARs without deferring partials
creation.

Differential Revision: https://phabricator.services.mozilla.com/D11729
2018-11-20 21:26:04 +00:00
Andreea Pavel
b6d1437ae8 Backed out 7 changesets (bug 1501878) for breaking win partial generation nightlies a=backout
Backed out changeset 00f109437e82 (bug 1501878)
Backed out changeset f1843fdd4c0f (bug 1501878)
Backed out changeset af0ea80f7b08 (bug 1501878)
Backed out changeset 01311f87d287 (bug 1501878)
Backed out changeset 4e2dc6c23463 (bug 1501878)
Backed out changeset 050c4bba5d71 (bug 1501878)
Backed out changeset 9198c5a03b59 (bug 1501878)
2018-11-17 18:37:39 +02:00
Aki Sasaki
b888c47f71 Bug 1501878 - point partials at unsigned complete mars. r=tomprince
Partial MAR generation doesn't need a signed complete MAR.
By pointing the partials tasks at the unsigned complete MARs, we are
able to defer signing the complete MARs without deferring partials
creation.

Differential Revision: https://phabricator.services.mozilla.com/D11729
2018-11-16 21:01:13 +00:00
Tom Prince
0055ca294b Bug 1492664: Replace uses of get_taskcluster_artifact_url with artifact-reference; r=dustin 2018-12-18 15:35:18 +00:00
Tom Prince
c7cde5a4ee Bug 1501776: [taskgraph] Make single_dep use primary-dependency to match multi_dep; r=aki
To make switching between `single_dep` and `multi_dep` easier, change
the `dependent-task` to `primary-dependency`.

Differential Revision: https://phabricator.services.mozilla.com/D9722
2018-10-24 22:41:38 +00:00
Tom Prince
d59427a373 Bug 1473416: [release] Allow staging releases to build partials from http://ftp.stage.mozaws.net/; r=sfraser
The partial generation code checks the URLs of the source versions. To allow
building partials from staging releases, allow the staging CDN when generating
partials.

Differential Revision: https://phabricator.services.mozilla.com/D5710
2018-10-12 17:53:20 +00:00
Justin Wood
ded109876a Bug 1469436 - Make repackage-signing distinct from its l10n, so we can depend on it in l10n for windows. r=tomprince
This is needed to not have a circular kind dependency when we actually spell out all dependencies (in a following patch)

Differential Revision: https://phabricator.services.mozilla.com/D1695
2018-06-18 10:49:41 -04:00
Tom Prince
f9bf205d9e Bug 1461784: [release] Only use the partials cache on level 3 jobs; r=sfraser
When running jobs on try, we shouldn't have access to the partials, so
don't try to give them.

Differential Revision: https://phabricator.services.mozilla.com/D1352
2018-05-04 12:07:05 -06:00
Aki Sasaki
e042df0476 bug 1442793 - partials artifact_prefix. r=bhearsum
MozReview-Commit-ID: 2XL2OEpVyWO
2018-03-21 15:13:57 -07:00
Aki Sasaki
3fc8d78d76 bug 1442793 - generate_taskcluster_artifact_prefix. r=bhearsum
MozReview-Commit-ID: 2zRqm5C9ZZQ
2018-03-21 12:07:01 -07:00
Simon Fraser
76a15b2a0c Bug 1436977 Disable partials diff caching for macosx r=rail
MozReview-Commit-ID: i1oe7foWam
2018-02-13 10:42:05 +00:00
Mike Hommey
6c8a0aac83 Bug 1430958 - Set docker-image as a chain-of-trust input centrally. r=dustin
Relying on the various transforms setting it manually is error prone,
and, in fact, is why bug 1430037 busted beta. This change makes this
setting happen at a single place. This yields the same full task graph
as before, except for *more* chain-of-trust inputs being set now: they
were missing for toolchain tasks (which makes us closer to bug 1384430).
2018-01-17 12:05:56 +09:00
Simon Fraser
3ff9bf4ff2 Bug 1419478 Enable S3 caching for binary diff patch files in partial update tasks r=rail
MozReview-Commit-ID: 23vwFcBkqKd
2017-12-15 14:07:31 -06:00
Aki Sasaki
f837815479 bug 1423081 - add release partials support. r=callek
MozReview-Commit-ID: GCFevQVeIyy
2017-12-05 19:08:06 -08:00
Simon Fraser
ef59cdbae1 Bug 1424959 Add reqd environment for partial mar metric collection r=rail
MozReview-Commit-ID: LJBt3tiY8Bp
2017-12-12 22:15:45 -06:00