Commit Graph

8360 Commits

Author SHA1 Message Date
Greg Mierzwinski
864d2a6919 Bug 1784176 - Add a feature for manual logins for recording sites. r=perftest-reviewers,AlexandruIonescu
This patch adds the ability to run manual logins for our websites since it can be simpler, and quicker in some cases. At the same time, a bug with the options handling is fixed.

Differential Revision: https://phabricator.services.mozilla.com/D164590
2023-03-20 14:42:45 +00:00
Andrew Halberstadt
000b2effaa Bug 1811850 - [lint] Fix ruff errors resulting from new configuration, r=webdriver-reviewers,perftest-reviewers,whimboo,sparky,firefox-build-system-reviewers,ahochheiden
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D172359
2023-03-20 13:06:28 +00:00
Andrew Halberstadt
83bc2ca934 Bug 1811850 - [lint] Replace isort linter with ruff, r=taskgraph-reviewers,linter-reviewers,marco,jcristau
This results in some changes from our current `isort` configuration. I'm
unclear if it's because ruff isn't at 100% parity with isort, they choose
different defaults or if I missed some configuration.

Either way, the changes all look reasonable to me (see child commit), so I'm
inclined to just accept the new import format it imposes.

Differential Revision: https://phabricator.services.mozilla.com/D172348
2023-03-20 13:06:27 +00:00
Andrew Halberstadt
69b399c915 Bug 1811850 - [lint] Replace flake8 linter with ruff, r=linter-reviewers,sylvestre
Ruff is a very fast linter implemented in Rust and it can act as a drop-in
replacement for flake8. When running the same set of rules across all files
in mozilla-central (without mozlint), flake8 takes 900 seconds whereas ruff
takes 0.9 seconds.

Ruff also implements rules from other popular Python linters such as pylint,
isort and pyupgrade. There are even plans to implement feature parity with
black in the future. Ultimately, it can become our one stop shop for all Python
linting and formatting.

This stack will swap out all our Python lint tools for ruff (excluding black
for now).

Differential Revision: https://phabricator.services.mozilla.com/D172313
2023-03-20 13:06:27 +00:00
Julien Cristau
31227f5b8e Bug 1822594 - remove webrender test schedule. r=jmaher,ahal
Since webrender was enabled by default this was causing pushes with
webrender-only changes to wrongly optimize out most tests.

Differential Revision: https://phabricator.services.mozilla.com/D172778
2023-03-20 10:45:48 +00:00
Cristian Tuns
e0f5dfb939 Backed out 5 changesets (bug 1811850) for causing linting bustages(bugzilla) CLOSED TREE
Backed out changeset e8fcfc7f8108 (bug 1811850)
Backed out changeset f8950d716c9e (bug 1811850)
Backed out changeset f650123cc188 (bug 1811850)
Backed out changeset d96f90c2c58b (bug 1811850)
Backed out changeset c3b0f9666183 (bug 1811850)
2023-03-16 22:16:30 -04:00
Andrew Halberstadt
0cab218f58 Bug 1811850 - [lint] Fix ruff errors resulting from new configuration, r=webdriver-reviewers,perftest-reviewers,whimboo,sparky,firefox-build-system-reviewers,ahochheiden
# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D172359
2023-03-17 01:54:00 +00:00
Andrew Halberstadt
f661e9875a Bug 1811850 - [lint] Replace isort linter with ruff, r=taskgraph-reviewers,linter-reviewers,marco,jcristau
This results in some changes from our current `isort` configuration. I'm
unclear if it's because ruff isn't at 100% parity with isort, they choose
different defaults or if I missed some configuration.

Either way, the changes all look reasonable to me (see child commit), so I'm
inclined to just accept the new import format it imposes.

Differential Revision: https://phabricator.services.mozilla.com/D172348
2023-03-17 01:53:59 +00:00
Andrew Halberstadt
a89c9f6b7e Bug 1811850 - [lint] Replace flake8 linter with ruff, r=linter-reviewers,sylvestre
Ruff is a very fast linter implemented in Rust and it can act as a drop-in
replacement for flake8. When running the same set of rules across all files
in mozilla-central (without mozlint), flake8 takes 900 seconds whereas ruff
takes 0.9 seconds.

Ruff also implements rules from other popular Python linters such as pylint,
isort and pyupgrade. There are even plans to implement feature parity with
black in the future. Ultimately, it can become our one stop shop for all Python
linting and formatting.

This stack will swap out all our Python lint tools for ruff (excluding black
for now).

Differential Revision: https://phabricator.services.mozilla.com/D172313
2023-03-17 01:53:58 +00:00
Kershaw Chang
2d142517dc Bug 1809843 - Add DoH server and Http/3 server into mochitest, r=necko-reviewers,ahal,valentin,gbrown,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D166644
2023-03-16 21:28:58 +00:00
Barret Rennie
614e74345c Bug 1821243 - Update to jsonschema 4.17.3 r=ahochheiden
We are using features from modern JSONSchema drafts, like dependantRequired, in
the Firefox Messaging System schemas. jsonschema 3.x does not support these
features. Experimenter is also being updated to 4.17.3 and we want to keep our
validation tests in sync with Experimenter.

Differential Revision: https://phabricator.services.mozilla.com/D172457
2023-03-16 20:27:01 +00:00
Cristian Tuns
837b5ca4e5 Backed out 2 changesets (bug 1821243) for causing xpcshell failures on test_RemoteSettingsExperimentLoader_updateRecipes.js CLOSED TREE
Backed out changeset 39df97ca8253 (bug 1821243)
Backed out changeset bbb8006a0784 (bug 1821243)
2023-03-16 15:16:58 -04:00
Barret Rennie
f5a10f7fa3 Bug 1821243 - Update to jsonschema 4.17.3 r=ahochheiden
We are using features from modern JSONSchema drafts, like dependantRequired, in
the Firefox Messaging System schemas. jsonschema 3.x does not support these
features. Experimenter is also being updated to 4.17.3 and we want to keep our
validation tests in sync with Experimenter.

Differential Revision: https://phabricator.services.mozilla.com/D172457
2023-03-16 17:50:21 +00:00
Maxim Cournoyer
bfeb9a29c2 Bug 1817032 - Honor SOURCE_DATE_EPOCH to build langpacks reproducibly. r=eemeli@mozilla.com
Differential Revision: https://phabricator.services.mozilla.com/D169979
2023-03-16 14:01:58 +00:00
Marian-Vasile Laza
46bfe444e2 Backed out 2 changesets (bug 1821243) for xpcshell failures on test_reach_experiments.js. CLOSED TREE
Backed out changeset 44a860152c39 (bug 1821243)
Backed out changeset e34a680accf0 (bug 1821243)
2023-03-16 03:42:46 +02:00
Barret Rennie
10748a3dbe Bug 1821243 - Update to jsonschema 4.17.3 r=ahochheiden
We are using features from modern JSONSchema drafts, like dependantRequired, in
the Firefox Messaging System schemas. jsonschema 3.x does not support these
features. Experimenter is also being updated to 4.17.3 and we want to keep our
validation tests in sync with Experimenter.

Differential Revision: https://phabricator.services.mozilla.com/D172457
2023-03-16 00:23:55 +00:00
Mike Hommey
3f1cfa302b Bug 1820947 - Use "new style" sysroot on Android. r=firefox-build-system-reviewers,andi
For a few NDK releases now, the situation has been simplified wrt
headers and libraries, and while we're currently still using things here
and there because we never changed our ways, we can simplify things a
lot by using the new simplified things. This involves:
- Using a --target that contains the Android version, making clang set
  __ANDROID_API__ itself, and makes it look in $sysroot/usr/lib/$target/$ver
  when linking.
- Using the sysroot that is under toolchains/llvm/prebuilt/*.
- Removing the hacks around libstdc++/libc++.

This ends up emptying stlport compiler flags, which allows to remove a
bunch of things.

Differential Revision: https://phabricator.services.mozilla.com/D172039
2023-03-15 20:44:02 +00:00
serge-sans-paille
e163cd75b2 Bug 1820572 - Make python/mozbuild/mozbuild/test/frontend/data buildable outside of a unified build environment r=andi,glandium
Differential Revision: https://phabricator.services.mozilla.com/D171731
2023-03-15 07:30:36 +00:00
Ben Hearsum
645ad14a82 Bug 1822165: Bump sphinxcontrib-mermaid to latest version r=releng-reviewers,gbrown
This allows us to use more modern mermaid syntax and features. I know we pinned this to an old version quite awhile ago (https://bugzilla.mozilla.org/show_bug.cgi?id=1693140), but it appears that those issues have since been fixed. At the very least, diagrams appear to have no issues rendering.

Differential Revision: https://phabricator.services.mozilla.com/D172563
2023-03-14 20:15:05 +00:00
ogiorgis
d6b7028e1d Bug 1822030 - mach setup - make sure to take the correct README for the description r=sylvestre DONTBUILD
Differential Revision: https://phabricator.services.mozilla.com/D172409
2023-03-14 13:04:09 +00:00
Gijs Kruitbosch
f9f3c7f39f Bug 1738056 - use fluent to insert channel ref into about dialog instead of hoping nothing gets retranslated, r=mkaply,fluent-reviewers,flod
The extant code would fail if the user switched languages while the dialog was
open, as fluent would throw the manually-inserted content away. This patch
properly uses fluent to insert the channel, which also means we don't need to
wait for it to happen after `load` (because of the previous patch in this
stack), it'll be there immediately.

Differential Revision: https://phabricator.services.mozilla.com/D172299
2023-03-14 00:03:42 +00:00
Norisz Fay
f8852215c0 Backed out 2 changesets (bug 1820947) for causing build bustage CLOSED TREE
Backed out changeset eb6419bb8748 (bug 1820947)
Backed out changeset 9990083e8ea2 (bug 1820947)
2023-03-14 01:20:54 +02:00
Mike Hommey
af673cd45c Bug 1820947 - Use "new style" sysroot on Android. r=firefox-build-system-reviewers,andi
For a few NDK releases now, the situation has been simplified wrt
headers and libraries, and while we're currently still using things here
and there because we never changed our ways, we can simplify things a
lot by using the new simplified things. This involves:
- Using a --target that contains the Android version, making clang set
  __ANDROID_API__ itself, and makes it look in $sysroot/usr/lib/$target/$ver
  when linking.
- Using the sysroot that is under toolchains/llvm/prebuilt/*.
- Removing the hacks around libstdc++/libc++.

This ends up emptying stlport compiler flags, which allows to remove a
bunch of things.

Differential Revision: https://phabricator.services.mozilla.com/D172039
2023-03-13 21:14:47 +00:00
Ben Hearsum
87c11a373b Bug 1814727: allow dlsource OR existing required keys when attributing builds r=releng-reviewers,gbrown
We've decided to use a new attribution field when attributing our vanilla builds. This field is valid as the _only_ field in the attribution data.

Differential Revision: https://phabricator.services.mozilla.com/D171592
2023-03-13 19:04:05 +00:00
Ben Hearsum
d005d90a5d Bug 1814727: enhance attribution script args to make it easier to append additional attributions r=releng-reviewers,gbrown
This allows us to easily append attributions, which is helpful when configuring this script in taskgraph. (You can set up some defaults, and then add others for specific jobs.)

As far as I can tell, all current usage of this script uses the environment variables, so it should be safe to remove the current arguments.

Differential Revision: https://phabricator.services.mozilla.com/D170240
2023-03-13 19:04:05 +00:00
Ben Hearsum
fa27368a21 Bug 1814727: Rename partner_attribution.py to a more generic name r=releng-reviewers,gbrown
There's actually nothing partner-specific in this script, and it's about to be used for other types of attribution as well.

Differential Revision: https://phabricator.services.mozilla.com/D170239
2023-03-13 19:04:04 +00:00
Greg Mierzwinski
7a6a17307a Bug 1821705 - Improve testing for perftestetl.py file. r=perftest-reviewers,afinder DONTBUILD
This patch fixes an issue in our perftestetl.py which was causing our coverage percent to drop "artificially". This is happening because there's a bit of code that's unreachable because of an exception that would always be triggered before it. This patch removes the unreachable code, and replaces it with an exception.

Differential Revision: https://phabricator.services.mozilla.com/D172281
2023-03-13 12:50:34 +00:00
Marian-Vasile Laza
65ec8a9faf Backed out 5 changesets (bug 1809843) for causing Cargo related build bustages. CLOSED TREE
Backed out changeset 5c494680f448 (bug 1809843)
Backed out changeset 02742b38edab (bug 1809843)
Backed out changeset 2b9b32ca8294 (bug 1809843)
Backed out changeset 56631cb02ae6 (bug 1809843)
Backed out changeset 37e35a60a71f (bug 1809843)
2023-03-13 13:53:26 +02:00
Kershaw Chang
511f221141 Bug 1809843 - Run mochitest with Http/3 server on try, r=gbrown,necko-reviewers,ci-and-tooling,valentin,suhaib,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D167515
2023-03-13 11:12:33 +00:00
Kershaw Chang
f3923d09b5 Bug 1809843 - Add DoH server and Http/3 server into mochitest, r=necko-reviewers,ahal,valentin,gbrown,jmaher
Differential Revision: https://phabricator.services.mozilla.com/D166644
2023-03-13 11:12:32 +00:00
Andreas Pehrson
d946ca03f9 Bug 1818170 - Load python files directly with our python. r=glandium
This ignores the shebang in python scripts that would be used when executed
directly.

This is unfortunate but needed as libwebrtc/chromium scripts rely on vpython3.

Differential Revision: https://phabricator.services.mozilla.com/D171971
2023-03-08 23:16:50 +00:00
Andreas Pehrson
c10101e5c0 Bug 1818170 - In gn_processor.py pretty print lists when serializing. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D171452
2023-03-08 23:16:49 +00:00
Andreas Pehrson
6f350a9b03 Bug 1818170 - Allow files generated during libwebrtc build in LOCAL_INCLUDES. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D170858
2023-03-08 23:16:49 +00:00
Andreas Pehrson
e1bfcebee5 Bug 1818170 - Make gn_processor.py handle actions that generate files. r=glandium
Differential Revision: https://phabricator.services.mozilla.com/D170857
2023-03-08 23:16:48 +00:00
Sandor Molnar
13787c7779 Backed out 5 changesets (bug 1774569) because changes are incompatible with flake8. CLOSED TREE
Backed out changeset ac6c7441099b (bug 1774569)
Backed out changeset 1081432f3f97 (bug 1774569)
Backed out changeset 111d7f2021c3 (bug 1774569)
Backed out changeset 2cc0b2a54106 (bug 1774569)
Backed out changeset cb45ff55284c (bug 1774569)
2023-03-07 23:21:19 +02:00
Mike Hommey
7d02d9dc56 Bug 1819783 - Avoid logging errors when dealing with records with no params. r=ahal
Under some conditions, logging fails with messages like:

```
--- Logging error ---
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/logging/__init__.py", line 1083, in emit
    msg = self.format(record)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/logging/__init__.py", line 927, in format
    return fmt.format(record)
  File "/Users/glandium/mozilla-unified/python/mach/mach/logging.py", line 156, in format
    formatted_msg = record.msg.format(**record.params)
AttributeError: 'LogRecord' object has no attribute 'params'
```

In those cases, the record does have a message, as expected, but it's
preformatted and there are no params in the record at all, rather than
an empty dict.

Differential Revision: https://phabricator.services.mozilla.com/D171408
2023-03-07 21:04:39 +00:00
Johan Lorenzo
b7b2a3c8aa Bug 1799042 - part 8: Create mach repackage deb-l10n command r=gabriel
Depends on D171728

Differential Revision: https://phabricator.services.mozilla.com/D171124
2023-03-07 19:28:25 +00:00
Johan Lorenzo
fd9d709ab3 Bug 1799042 - part 7: Allow _render_deb_templates() to ignore files r=gabriel
Depends on D171123

Differential Revision: https://phabricator.services.mozilla.com/D171728
2023-03-07 19:28:24 +00:00
Johan Lorenzo
c4aa21d8e9 Bug 1799042 - part 6: Expand _get_build_variables() to support package name and description suffixes r=gabriel
Depends on D171122

Differential Revision: https://phabricator.services.mozilla.com/D171123
2023-03-07 19:28:24 +00:00
Johan Lorenzo
1484e03dd6 Bug 1799042 - part 5: Support chroot path when using debian arch "all" r=gabriel
Depends on D171121

Differential Revision: https://phabricator.services.mozilla.com/D171122
2023-03-07 19:28:23 +00:00
Johan Lorenzo
5978fab8d1 Bug 1799042 - part 4: Create _extract_langpack_metadata() to read manifest.json from XPI files r=gabriel
Depends on D171120

Differential Revision: https://phabricator.services.mozilla.com/D171121
2023-03-07 19:28:23 +00:00
Johan Lorenzo
af3c088dd9 Bug 1799042 - part 3: Create _extract_application_ini_data() which reads from input tar file directly r=gabriel
Depends on D171119

Differential Revision: https://phabricator.services.mozilla.com/D171120
2023-03-07 19:28:23 +00:00
Johan Lorenzo
463c947434 Bug 1799042 - part 2: Rename _extract_application_ini_data() into ..._from_directory() r=gabriel
Depends on D162225

Differential Revision: https://phabricator.services.mozilla.com/D171119
2023-03-07 19:28:21 +00:00
Johan Lorenzo
f04d71b6fe Bug 1799042 - part 1: Make deb dependencies customizable r=gabriel
Differential Revision: https://phabricator.services.mozilla.com/D162225
2023-03-07 19:28:21 +00:00
Alex Hochheiden
1dc46951f9 Bug 1774569 - Removed the patching mechanism that was only needed for poetry version 1.2.0a2 r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D171762
2023-03-07 08:08:47 +00:00
Alex Hochheiden
e1ef1aea92 Bug 1774569 - Update poetry to 1.4.0 r=ahal
https://github.com/python-poetry/poetry/releases/tag/1.4.0https://github.com/python-poetry/poetry/releases/tag/1.4.0
https://github.com/python-poetry/poetry-core/releases/tag/1.5.1

The changes in `third_party/python/requirements.txt` and `mozbuild/test/vendor_requirements.txt` are due to the newer version.

Differential Revision: https://phabricator.services.mozilla.com/D171761
2023-03-07 08:08:46 +00:00
Alex Hochheiden
36fd06274e Bug 1774569 - Vendor jsonschema at 4.10.0 r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D171760
2023-03-07 08:08:45 +00:00
Alex Hochheiden
7c0a32980b Bug 1774569 - Vendor importlib_metadata at 6.0.0 r=firefox-build-system-reviewers,glandium,ahal
This version is compatible with `poetry` `1.4.0` and removes blockers for various other ongoing projects.

https://github.com/python/importlib_metadata/blob/main/CHANGES.rst#v660

Differential Revision: https://phabricator.services.mozilla.com/D150813
2023-03-07 08:08:45 +00:00
Alex Hochheiden
1b18e2053f Bug 1734402 - Regenerate lockfile with Python 3.7 as the minimum version r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D171628
2023-03-07 08:08:44 +00:00
Andi-Bogdan Postelnicu
215b3acc3f Bug 1820494 - mozlint should output file no matter if there are issues for json formatter. r=ahal
Differential Revision: https://phabricator.services.mozilla.com/D171790
2023-03-07 05:53:30 +00:00