Commit Graph

7844 Commits

Author SHA1 Message Date
Gabriele Svelto
a4811de5c6 Bug 1774298 - Install watchman only on Debian 11+ r=andi
Differential Revision: https://phabricator.services.mozilla.com/D149358
2022-06-22 10:08:23 +00:00
Mike Hommey
589aa45394 Bug 1760484 - Add webrtc configs for arm64 openbsd. r=mjf
Differential Revision: https://phabricator.services.mozilla.com/D149473
2022-06-21 20:03:56 +00:00
Mike Hommey
202dd1e9b5 Bug 1738845 - Add webrtc configs for arm and ppc64 linux. r=mjf
Adding arm alone breaks ppc64el.

Differential Revision: https://phabricator.services.mozilla.com/D134738
2022-06-21 20:03:56 +00:00
Mike Hommey
9f3e866b5d Bug 1738845 - Add some missing combinations of conditions to GN processing. r=firefox-build-system-reviewers,andi
Ideally, the code would handle things in a more general way that doesn't
require manually dealing with these lists, but this would require more
testing than there is time left before 102 releases.

While here, remove HOST_CPU_ARCH, which is always the same and thus
never appears in a condition.

This changes none of the generated moz.builds for the current
configuration (but changes the outcome when adding new configurations)

Differential Revision: https://phabricator.services.mozilla.com/D149850
2022-06-21 20:03:55 +00:00
Mike Hommey
5d81f2fbf8 Bug 1775183 - Remove fork_interpose.py. r=firefox-build-system-reviewers,nalexander
Its use was removed in bug 1720591.

Differential Revision: https://phabricator.services.mozilla.com/D149840
2022-06-21 06:44:55 +00:00
Mike Hommey
e370947ba6 Bug 1773223 - Remove now irrelevant gn processor tests.
Differential Revision: https://phabricator.services.mozilla.com/D149814
2022-06-21 02:30:46 +00:00
Mike Hommey
8da4c4a7d7 Bug 1773223 - Filter-out _FORTIFY_SOURCE in GN processing. r=firefox-build-system-reviewers,andi
This is redundant with the build system setting it in
toolchain.configure.

Differential Revision: https://phabricator.services.mozilla.com/D149471
2022-06-21 02:30:46 +00:00
Mike Hommey
06d31a31c8 Bug 1773223 - Make the definition of MOZ_X11 independent of the OS. r=firefox-build-system-reviewers,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D149474
2022-06-21 02:30:45 +00:00
Mike Hommey
a1006cfd71 Bug 1773223 - Make the GN processor an independent script. r=firefox-build-system-reviewers,mjf,ahochheiden
The way the processor works currently is that it relies on two different
build backends, one of which produces json files for specific
configurations, and the other which produces moz.build files from the
aggregate of all those configs.

Each of these json files is huge, and we actually don't have enough to
support all the platforms we're supposed to be supporting. Adding more
files is not enticing.

Now that we've made the first step described above work in a single pass
on a single machine (as opposed to multiple passes on multiple machines
previously), we can actually merge both steps and avoid producing the
intermediate json files altogether. This will allow to add more
configurations without having to worry about the weight of those files.

And because this all doesn't need to depend on having the first step
hooked up in the build system, we make the whole an independent script
rather than a build backend.

Differential Revision: https://phabricator.services.mozilla.com/D149210
2022-06-21 02:30:37 +00:00
Ryan VanderMeulen
90fec6ac70 Bug 1738054 - Remove obsolete "skip-if = python == 2" keys from testing manifests and re-enable some skipped tests which now pass. r=ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D149573
2022-06-21 01:46:44 +00:00
smolnar
b059a529e3 Backed out 6 changesets (bug 1738845, bug 1773223, bug 1760484) for causing multiple failures CLOSED TREE
Backed out changeset e759ac4fb646 (bug 1773223)
Backed out changeset 20c541be9a37 (bug 1760484)
Backed out changeset 653d2024a187 (bug 1738845)
Backed out changeset fd9540114bfd (bug 1773223)
Backed out changeset f24f3dfa49aa (bug 1773223)
Backed out changeset 11f3375a4a70 (bug 1773223)
2022-06-21 03:00:53 +03:00
Mike Hommey
52236ada23 Bug 1773223 - Remove now irrelevant gn processor tests.
Differential Revision: https://phabricator.services.mozilla.com/D149814
2022-06-20 22:04:22 +00:00
Mike Hommey
74f972f438 Bug 1760484 - Add webrtc configs for arm64 openbsd. r=mjf
Differential Revision: https://phabricator.services.mozilla.com/D149473
2022-06-20 22:04:22 +00:00
Mike Hommey
02fb73bb3c Bug 1738845 - Add webrtc configs for arm and ppc64 linux. r=mjf
Adding arm alone breaks ppc64el.

Differential Revision: https://phabricator.services.mozilla.com/D134738
2022-06-20 22:04:21 +00:00
Mike Hommey
4c6f6d8d88 Bug 1773223 - Filter-out _FORTIFY_SOURCE in GN processing. r=firefox-build-system-reviewers,andi
This is redundant with the build system setting it in
toolchain.configure.

Differential Revision: https://phabricator.services.mozilla.com/D149471
2022-06-20 22:04:20 +00:00
Mike Hommey
9b92f2e433 Bug 1773223 - Make the definition of MOZ_X11 independent of the OS. r=firefox-build-system-reviewers,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D149474
2022-06-20 22:04:20 +00:00
Mike Hommey
a048256ed4 Bug 1773223 - Make the GN processor an independent script. r=firefox-build-system-reviewers,mjf,ahochheiden
The way the processor works currently is that it relies on two different
build backends, one of which produces json files for specific
configurations, and the other which produces moz.build files from the
aggregate of all those configs.

Each of these json files is huge, and we actually don't have enough to
support all the platforms we're supposed to be supporting. Adding more
files is not enticing.

Now that we've made the first step described above work in a single pass
on a single machine (as opposed to multiple passes on multiple machines
previously), we can actually merge both steps and avoid producing the
intermediate json files altogether. This will allow to add more
configurations without having to worry about the weight of those files.

And because this all doesn't need to depend on having the first step
hooked up in the build system, we make the whole an independent script
rather than a build backend.

Differential Revision: https://phabricator.services.mozilla.com/D149210
2022-06-20 22:04:12 +00:00
Mike Hommey
a7e5303d1c Bug 1773223 - Move some json fixups into the GN processor. r=firefox-build-system-reviewers,andi
As we're shortly going to stop producing the intermediate json files,
we want the fixups to happen in the GN processor.

Ideally, we'd move them all, but cleaning up -isysroot is more involved,
while we won't need it once we don't use intermediate json files, so we
leave the -isysroot cleanup in fixup_json.py for now.

While here, `gn_out["targets"][target_fullname]` doesn't need to be set
on every iteration of the loop.

Differential Revision: https://phabricator.services.mozilla.com/D149209
2022-06-20 21:50:59 +00:00
Mike Hommey
44251d7cbf Bug 1773223 - Remove unused parameters. r=firefox-build-system-reviewers,andi
Also replace uses of config objects, which are only used for topsrcdir.

Differential Revision: https://phabricator.services.mozilla.com/D149208
2022-06-20 21:50:58 +00:00
Mike Hommey
a266eebbce Bug 1773223 - Use the GN binary path passed into generate_gn_config. r=firefox-build-system-reviewers,andi
The path is given to the function, there is no need to get it from
config.substs.

Differential Revision: https://phabricator.services.mozilla.com/D149207
2022-06-20 21:50:58 +00:00
Mike Hommey
566c57333e Bug 1773223 - Remove variable expansion in the GN processor. r=firefox-build-system-reviewers,andi
This was cargo culted from the gyp processor, but is not used.

Differential Revision: https://phabricator.services.mozilla.com/D149206
2022-06-20 21:50:58 +00:00
Mike Hommey
37e2b52b30 Bug 1773223 - Generate webrtc moz.builds for all platforms at once. r=mjf,firefox-build-system-reviewers,ahochheiden
The current script requires to be run on 4 different host platforms each
of which would handle a subset of a total of 32 mozconfigs. That is not
sustainable, and there are already missing configs that break tier-3
platforms.

This replaces the current setup with one that handles all platforms in
one go, although we still keep the internal sequence of GcConfigGen ->
fixup_json -> GnMozbuildWriter.

The downside is that because this relies on the upstream webrtc build
system supporting cross-compilation, and that it actively rejects some
configurations, we need some local hacks to make it work on Linux and
Mac, but for now, we have to leave out Windows, which requires more
work.

For some reason, that removes some duplicated include directories in the
json files, which moves things a little in one moz.build file.

We also remove the mozconfigs we don't use anymore.

Differential Revision: https://phabricator.services.mozilla.com/D149205
2022-06-20 21:50:57 +00:00
Cosmin Sabou
6a41939805 Backed out 11 changesets (bug 1738845, bug 1760484, bug 1773223) for causing build bustages and python failures. CLOSED TREE
Backed out changeset 2faea2e0af45 (bug 1773223)
Backed out changeset 85f0cccbd2a8 (bug 1760484)
Backed out changeset 67ee3490fe22 (bug 1738845)
Backed out changeset 2d0053357489 (bug 1773223)
Backed out changeset 7eccf21b04f2 (bug 1773223)
Backed out changeset 5e1c0d1d4018 (bug 1773223)
Backed out changeset c73d516d7d7f (bug 1773223)
Backed out changeset 05e201f82758 (bug 1773223)
Backed out changeset fe0095ad6576 (bug 1773223)
Backed out changeset 264adb57bb3c (bug 1773223)
Backed out changeset f349d4ad0088 (bug 1773223)
2022-06-21 00:22:01 +03:00
Mike Hommey
81bb728d96 Bug 1773223 - Remove now irrelevant gn processor tests.
Differential Revision: https://phabricator.services.mozilla.com/D149814
2022-06-20 21:16:52 +00:00
Mike Hommey
10b90bad75 Bug 1760484 - Add webrtc configs for arm64 openbsd. r=mjf
Differential Revision: https://phabricator.services.mozilla.com/D149473
2022-06-20 20:58:39 +00:00
Mike Hommey
a5bd885dd2 Bug 1738845 - Add webrtc configs for arm and ppc64 linux. r=mjf
Adding arm alone breaks ppc64el.

Differential Revision: https://phabricator.services.mozilla.com/D134738
2022-06-20 20:58:39 +00:00
Mike Hommey
96eff086ac Bug 1773223 - Filter-out _FORTIFY_SOURCE in GN processing. r=firefox-build-system-reviewers,andi
This is redundant with the build system setting it in
toolchain.configure.

Differential Revision: https://phabricator.services.mozilla.com/D149471
2022-06-20 20:58:38 +00:00
Mike Hommey
c98284493c Bug 1773223 - Make the definition of MOZ_X11 independent of the OS. r=firefox-build-system-reviewers,ahochheiden
Differential Revision: https://phabricator.services.mozilla.com/D149474
2022-06-20 20:58:38 +00:00
Mike Hommey
56eed1647f Bug 1773223 - Make the GN processor an independent script. r=firefox-build-system-reviewers,mjf,ahochheiden
The way the processor works currently is that it relies on two different
build backends, one of which produces json files for specific
configurations, and the other which produces moz.build files from the
aggregate of all those configs.

Each of these json files is huge, and we actually don't have enough to
support all the platforms we're supposed to be supporting. Adding more
files is not enticing.

Now that we've made the first step described above work in a single pass
on a single machine (as opposed to multiple passes on multiple machines
previously), we can actually merge both steps and avoid producing the
intermediate json files altogether. This will allow to add more
configurations without having to worry about the weight of those files.

And because this all doesn't need to depend on having the first step
hooked up in the build system, we make the whole an independent script
rather than a build backend.

Differential Revision: https://phabricator.services.mozilla.com/D149210
2022-06-20 20:58:30 +00:00
Mike Hommey
b7c4666bf8 Bug 1773223 - Move some json fixups into the GN processor. r=firefox-build-system-reviewers,andi
As we're shortly going to stop producing the intermediate json files,
we want the fixups to happen in the GN processor.

Ideally, we'd move them all, but cleaning up -isysroot is more involved,
while we won't need it once we don't use intermediate json files, so we
leave the -isysroot cleanup in fixup_json.py for now.

While here, `gn_out["targets"][target_fullname]` doesn't need to be set
on every iteration of the loop.

Differential Revision: https://phabricator.services.mozilla.com/D149209
2022-06-20 20:58:29 +00:00
Mike Hommey
5fd4d7df63 Bug 1773223 - Remove unused parameters. r=firefox-build-system-reviewers,andi
Also replace uses of config objects, which are only used for topsrcdir.

Differential Revision: https://phabricator.services.mozilla.com/D149208
2022-06-20 20:58:29 +00:00
Mike Hommey
ee1d08e5e9 Bug 1773223 - Use the GN binary path passed into generate_gn_config. r=firefox-build-system-reviewers,andi
The path is given to the function, there is no need to get it from
config.substs.

Differential Revision: https://phabricator.services.mozilla.com/D149207
2022-06-20 20:58:29 +00:00
Mike Hommey
40845f6a32 Bug 1773223 - Remove variable expansion in the GN processor. r=firefox-build-system-reviewers,andi
This was cargo culted from the gyp processor, but is not used.

Differential Revision: https://phabricator.services.mozilla.com/D149206
2022-06-20 20:58:28 +00:00
Mike Hommey
1eb45904c9 Bug 1773223 - Generate webrtc moz.builds for all platforms at once. r=mjf,firefox-build-system-reviewers,ahochheiden
The current script requires to be run on 4 different host platforms each
of which would handle a subset of a total of 32 mozconfigs. That is not
sustainable, and there are already missing configs that break tier-3
platforms.

This replaces the current setup with one that handles all platforms in
one go, although we still keep the internal sequence of GcConfigGen ->
fixup_json -> GnMozbuildWriter.

The downside is that because this relies on the upstream webrtc build
system supporting cross-compilation, and that it actively rejects some
configurations, we need some local hacks to make it work on Linux and
Mac, but for now, we have to leave out Windows, which requires more
work.

For some reason, that removes some duplicated include directories in the
json files, which moves things a little in one moz.build file.

We also remove the mozconfigs we don't use anymore.

Differential Revision: https://phabricator.services.mozilla.com/D149205
2022-06-20 20:58:27 +00:00
andrej
12ad19aea4 Bug 1773621 - Remove panda.tv from list of sites tested on WebPageTest. r=perftest-reviewers,AlexandruIonescu
A week ago we received a notification that we had a test that the WPT chrome tests were perma failing on [[ https://bugzilla.mozilla.org/show_bug.cgi?id=1773621 | bugzilla ]]
After going through the fail logs I realized it was because of website "panda.tv" directing to a unable to connect page message, after some digging it was not returning proper data because panda.tv has not been a company since March 2019(bankruptcy filing), why this only is causing an issue now I believe is because of some kind of update from WPT as I can see a noticeable UI difference on the test results page from before and after the failures started.

My resolution was to remove Panda.tv from our test list and that seems to have resolved the issue.

I also updated the error message to display which website is causing the issue so that if this happens again I don't need to go through each and every webpagetest result to know which of the 40 websites are having an issue.

Differential Revision: https://phabricator.services.mozilla.com/D149642
2022-06-20 14:16:41 +00:00
Jan-Erik Rediger
e447a2597f Bug 1773255 - Update to Glean v50.1.0 r=chutten,supply-chain-reviewers
Including glean_parser 6.1.1

Two important things in there:
* glean_parser: [data-review] Include extra keys' names and descriptions in data review template
* Glean: Derive `serde::{Deserialize, Serialize}` on `Lifetime` and `CommonMetricData`

Differential Revision: https://phabricator.services.mozilla.com/D149381
2022-06-15 15:39:33 +00:00
ahochheiden
e58f9f3f59 Bug 1276019 - Add Microsoft Defender Antivirus exclusions automatically during source bootstrap r=glob
I also added `%USERPROFILE%/.mozbuild` to the exclusion list and updated the windows_build docs to reflect the changes made.

Differential Revision: https://phabricator.services.mozilla.com/D149199
2022-06-15 01:37:11 +00:00
Tooru Fujisawa
36caafe271 Bug 1769405 - Part 4: Detect mismatch between the default mozconfig and the selected build target. r=firefox-build-system-reviewers,glandium
When `./mach bootstrap` is called with the default mozconfig file,
verify the content matches the selected build target, and if not,
show a warning and ask whether to overwrite or not.

Differential Revision: https://phabricator.services.mozilla.com/D146384
2022-06-14 18:13:54 +00:00
Tooru Fujisawa
e43179c6df Bug 1769405 - Part 3: Return project and artifact-builds options from MozbuildObject.get_base_mozconfig_info. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D147012
2022-06-14 18:13:54 +00:00
Tooru Fujisawa
af6c92306f Bug 1769405 - Part 2: Rename mozconfig_and_target to base_mozconfig_info and return dictionary. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D147272
2022-06-14 18:13:54 +00:00
Kash Shampur
f93ae47099 Bug 1754830 - Fix browsertime installation in mozperftest. r=perftest-reviewers,sparky
The patch aims to improve the `_should_install()` method for installing browsertime in mozperftest.
Here, the same approach used in `testing/raptor/mach_commands.py` is used to address potential issues (e.g. KeyError)
that one may encounter when trying to install from the `package.json` through this approach.

Differential Revision: https://phabricator.services.mozilla.com/D148565
2022-06-14 13:47:10 +00:00
Kash Shampur
c706e71fd7 Bug 1754830 - Extend the login-logic to pageload tests in mozperftest. r=perftest-reviewers,sparky
This next patch in the series utilizes the same login-logic in Mozperftest and makes it available to the `pageload_test` method so that we can now automate the logging into of accounts during perftest recordings.
Additional logic is also added to account for if the site requires login, if we are running on CI or locally (and if on CI, accounting for the SCM level), and removal of the verbose flags so secrets do not leak.

Differential Revision: https://phabricator.services.mozilla.com/D147775
2022-06-14 13:47:09 +00:00
Mike Hommey
d8b680e5f9 Bug 1774059 - Try harder to find an appropriate macos SDK. r=firefox-build-system-reviewers,andi
Differential Revision: https://phabricator.services.mozilla.com/D149186
2022-06-14 04:32:29 +00:00
Mike Hommey
fbff717f83 Bug 1773642 - Adjust openbsd webrtc config after bug 1661450. r=mjf
Differential Revision: https://phabricator.services.mozilla.com/D148891
2022-06-14 04:19:03 +00:00
Marian-Vasile Laza
2d41524f31 Backed out 4 changesets (bug 1769405) for causing regression in Bug 1774100. CLOSED TREE
Backed out changeset 813147df341c (bug 1769405)
Backed out changeset 6de9495c467d (bug 1769405)
Backed out changeset 9099ba749b1f (bug 1769405)
Backed out changeset 5f8e0141a0ed (bug 1769405)
2022-06-14 01:36:48 +03:00
Stephen A Pohl
5d9b2bc146 Bug 1696513: Require macOS 11 SDK at configure time and update docs. r=mstange
Differential Revision: https://phabricator.services.mozilla.com/D148933
2022-06-13 17:44:51 +00:00
criss
dfb20805b3 Backed out changeset e186162443a6 (bug 1696513) for causing bustages. CLOSED TREE 2022-06-11 05:31:53 +03:00
Stephen A Pohl
12fee3db3b Bug 1696513: Require macOS 11 SDK at configure time and update docs. r=mstange
Differential Revision: https://phabricator.services.mozilla.com/D148933
2022-06-11 01:05:31 +00:00
Tooru Fujisawa
0034c943df Bug 1769405 - Part 4: Detect mismatch between the default mozconfig and the selected build target. r=firefox-build-system-reviewers,glandium
When `./mach bootstrap` is called with the default mozconfig file,
verify the content matches the selected build target, and if not,
show a warning and ask whether to overwrite or not.

Differential Revision: https://phabricator.services.mozilla.com/D146384
2022-06-10 19:17:00 +00:00
Tooru Fujisawa
49b9a5d613 Bug 1769405 - Part 3: Return project and artifact-builds options from MozbuildObject.get_base_mozconfig_info. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D147012
2022-06-10 19:16:59 +00:00