This replicates the logic in
testing/web-platform/tests/tools/wptrunner/wptrunner/environment.py to
ensure that all the configured URL base paths are added to the
wptserve router configuration when starting via mach wpt-serve.
Differential Revision: https://phabricator.services.mozilla.com/D96723
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.
5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.
To produce this patch I did all of the following:
1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.
2. Run ./mach lint --linter black --fix
3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.
4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).
# ignore-this-changeset
Differential Revision: https://phabricator.services.mozilla.com/D94045
Lack of support for specialPowers is often cited as a reason for
developers to write mochitests rather than wpt. Although using
specialPowers means that the tests can't be upstreamed, having it
available in gecko-only tests should reduce the need to write
mochitests and potentially means that a patch which previously would
be written with just mochitests would instead be written with just
wpt, a fraction of which would be unsuitable for sharing.
Nevertheless a gecko-only specialPowers-using test should be
considered the option of last resort when cross-browser APIs such as
testdriver aren't sufficient.
This patch installs the specialPowers extension when the path to the
XPI is provided via a --specialpowers-path command line argument. To
ensure this only happens for gecko-only tests, the url_base is added
to the Test object, and the firefox settings() method is updated to
contain a specialpowers setting, which uses the url_base of the test
to determine if this is a gecko-only test.
Because the extension has to be enabled using marionette, we also need
to route this setting through the to executor, along with the provided
XPI path.
Differential Revision: https://phabricator.services.mozilla.com/D92035
Lack of support for specialPowers is often cited as a reason for
developers to write mochitests rather than wpt. Although using
specialPowers means that the tests can't be upstreamed, having it
available in gecko-only tests should reduce the need to write
mochitests and potentially means that a patch which previously would
be written with just mochitests would instead be written with just
wpt, a fraction of which would be unsuitable for sharing.
Nevertheless a gecko-only specialPowers-using test should be
considered the option of last resort when cross-browser APIs such as
testdriver aren't sufficient.
This patch installs the specialPowers extension when the path to the
XPI is provided via a --specialpowers-path command line argument. To
ensure this only happens for gecko-only tests, the url_base is added
to the Test object, and the firefox settings() method is updated to
contain a specialpowers setting, which uses the url_base of the test
to determine if this is a gecko-only test.
Because the extension has to be enabled using marionette, we also need
to route this setting through the to executor, along with the provided
XPI path.
Differential Revision: https://phabricator.services.mozilla.com/D92035
Lack of support for specialPowers is often cited as a reason for
developers to write mochitests rather than wpt. Although using
specialPowers means that the tests can't be upstreamed, having it
available in gecko-only tests should reduce the need to write
mochitests and potentially means that a patch which previously would
be written with just mochitests would instead be written with just
wpt, a fraction of which would be unsuitable for sharing.
Nevertheless a gecko-only specialPowers-using test should be
considered the option of last resort when cross-browser APIs such as
testdriver aren't sufficient.
This patch installs the specialPowers extension when the path to the
XPI is provided via a --specialpowers-path command line argument. To
ensure this only happens for gecko-only tests, the url_base is added
to the Test object, and the firefox settings() method is updated to
contain a specialpowers setting, which uses the url_base of the test
to determine if this is a gecko-only test.
Because the extension has to be enabled using marionette, we also need
to route this setting through the to executor, along with the provided
XPI path.
Differential Revision: https://phabricator.services.mozilla.com/D92035
This also enables py3 linting for testing/web-platform. The external testing/web-platform/tests is excluded from linting.
Differential Revision: https://phabricator.services.mozilla.com/D90744
This utilises the fact that we can now specify the manifest
independently of the metadata path, which wasn't possible before. We
need to edit the default (i.e. Firefox) configuration to update the
metadata path (only) to point at something product-specific so we
don't end up using fx metadata for everything.
Differential Revision: https://phabricator.services.mozilla.com/D89461
wpt previously required Ahem as a system font. That requirement is now
relaxed and installing fonts is only required in order to see bugs
that result from differences between the system font loader and the
webfont loader. Since installing fonts can cause problems on some
systems, make this possible to disable.
Differential Revision: https://phabricator.services.mozilla.com/D61801
This command takes a list of test ids and returns the list of test
files that correspond to those test ids. The mapping is one-to-many
between files and test ids i.e. a single file can generate many tests.
Differential Revision: https://phabricator.services.mozilla.com/D56192
Gecko was trying to pass formatter options in by default, which
doesn't work if the user overrides the formatter. Instead pass in the
default options explicitly using the designed mechanism, which
previously wasn't exposed in wpt.
Differential Revision: https://phabricator.services.mozilla.com/D47117
This command is inteded to be usable as a git mergetool for the
specific case of merging ini files in a way that should always
succeed, produces reasonably likely output, but isn't reliably
correct.
The main use case is for the sync bot where we update metadata on
branches and experience conflicts when we also have changes on
master. We don't necessarily need to resolve these perfectly but we do
need to provide a resolution automatically since otherwise it blocks
syncs. Since the ini parser is in-tree we want to make this an in-tree
command that the sync can use to provide the resolution.
The general strategy is that we want to prefer the "new" metadata
where possible. This won't always be correct e.g. if a test got fixed
on master and simultaneously got edited to go from TIMEOUT to FAIL in
a browser without the fix. But it's not a bad approximation (and
generally a human will struggle to do the merge better by hand, so we
have to assume later try jobs will fix things).
Differential Revision: https://phabricator.services.mozilla.com/D40832
Various cleanup:
- remove extraneous calls to grant_runtime_permissions
- remove unused legacy jimdb support code
- remove "This may take a while" logging
- emphasize x86/x86_64 capabilities of emulator
Differential Revision: https://phabricator.services.mozilla.com/D40302
Use the geckoview TestRunnerActivity, org.mozilla.geckoview.test, by default
for all types of mochitests, reftests, and web-platform tests. TRA is already
the default for gtest and geckoview-junit. Fennec, based on ANDROID_PACKAGE_NAME,
remains the default for robocop and marionette-test and I have no plans to
change those. There is a related issue for xpcshell-test -- not the package
name, but the default apk -- but I am reluctant to handle that until bug 1553225
is resolved.
Differential Revision: https://phabricator.services.mozilla.com/D35479
./mach wpt-unittest command will run tox in testing/web-platform/tests/tools
and testing/web-platform/tests/tools/wptrunner. It does some hacking to update
the requirements.txt files so that we use the in-tree versions of packages
rather than updating from pypi; this involves temporarily rewriting the files
since I couldn't figure out a good way to get tox to install something different.
Differential Revision: https://phabricator.services.mozilla.com/D33811
This avoids a mismatch in the available arguments between upstream pip and
the version vendored into m-c at the cost of the code being more liable
to break when upstream changes.
Differential Revision: https://phabricator.services.mozilla.com/D32991
This parses the metadata files without evaluating the conditional
expressions, producing a JSON summary file in which each possible
condition is represented as a string.
Not all data from the metadata files appears in the summary, only
exceptional cases likely to be of interest to engineers and unlikely
to be on wpt.fyi e.g. tests that have lsan leaks or tests that crash,
not just tests that fail. This is intended to keep the filesize down.
Depends on D24177
Depends on D24177
Differential Revision: https://phabricator.services.mozilla.com/D24178
Add addtest support for per-suite arguments and multiple files. Also
support opening the created test in an editor. This allowed supporting
the wpt suite and replaces `mach wpt-create`.
# Create a wpt test
./mach addtest testing/web-platform/tests/accelerometer/test.html
# Create a wpt reftest
./mach addtest --suite wpt-reftesttesting/web-platform/tests/css/example.html --ref example-ref.html
The files created will be opened in the default editor if --editor is
supplied or a specified editor if the argument is given a value.
Differential Revision: https://phabricator.services.mozilla.com/D26339
This parses the metadata files without evaluating the conditional
expressions, producing a JSON summary file in which each possible
condition is represented as a string.
Not all data from the metadata files appears in the summary, only
exceptional cases likely to be of interest to engineers and unlikely
to be on wpt.fyi e.g. tests that have lsan leaks or tests that crash,
not just tests that fail. This is intended to keep the filesize down.
Depends on D24177
Differential Revision: https://phabricator.services.mozilla.com/D24178
On CI we only want to log screenshots when something unexpected happens since anything
else is rather wasteful of resources. But locally getting screenshots for expected
failures seems helpful for debugging, so worth making the default. Hopefully this isn't
too confusing for people just checking if their patch regresses anything rather than
actively working on fixing failures.
Depends on D16973
Differential Revision: https://phabricator.services.mozilla.com/D16974
When the logging setup moved to earlier in the setup we ended up setting the
option to enable tbpl-style screenshots from mach after the loggers were already
initalised. Move this to earlier in the command so this option starts working again.
Differential Revision: https://phabricator.services.mozilla.com/D16973
Previously we assumed that the path for the lsan_suppressions.txt file was the same as the prefs path,
which is correct in CI but wrong for local builds. This adds a --lsan-dir command line argument and sets
it in mach to the correct value for local builds and uses the prefs directory as a default for backwards
compatibility
Differential Revision: https://phabricator.services.mozilla.com/D16024
When running non-Firefox products in wpt we go through the run.py
script from upstream. Therefore we need to initialise the logger in
that module as well as the one in wptrunner now that it's been
properly converted to use logging rather than print statements.
In addition, we change the run.py module to allow passing in logging
defaults for the case where nothing is specified on the command line
since the behaviour for wpt upstream is different to the behaviour for
gecko.
Differential Revision: https://phabricator.services.mozilla.com/D13547
The fast internal reftest harness doesn't work yet, so in the meantime
default to the functional but slower external harness. Also ensure
that the Ahem font is installed by default.
Depends on D12035
Differential Revision: https://phabricator.services.mozilla.com/D12036
We need to pass in the extra kwargs that wpt run understands but
wptrunner does not. Ideally we could switch to using the run frontend
always and allow users to actually supply these kwargs, although that
might have some unwanted behaviour, and is a larger change. For now
this gets things working again.
Depends on D9081
Differential Revision: https://phabricator.services.mozilla.com/D9082
This replaces a previous Firefox-only method except on Windows 7 where
it seems that we have some issues with the registry.
Differential Revision: https://phabricator.services.mozilla.com/D9091
