Since |mach xpcshell-test| was already defaulting to the 'mach' format, this will be a no-op
for most people. For anyone who had:
[test]
format=tbpl
in their machrc file, this will start working for xpcshell tests.
MozReview-Commit-ID: LwIlh0lefN6
In order for |mach test| and |mach mochitest| to log an overall summary,
every test harness invocation they make needs to use the same structured
logger (otherwise the affected suite will be missing from the summary).
MozReview-Commit-ID: 8LJw7r8SItk
The complexity of the xpcshell test harness has grown over time. I am reluctant
to make it more complex without first trying to simplify it. Here I consolidate
some of the argument passing between functions in an attempt to simplify some
important interfaces. Changes are strictly structural and should result in no
change in behavior.
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
When running xpcshell tests on my i7-6700K, the Windows Defender
MsMpEng.exe process regularly consumes up to 50% CPU - 2 whole cores.
This is because it is canning the thousands of files we create in the
temporary directory. (We create a profile for every xpcshell.exe process.)
This commit changes the behavior of `mach xpcshell-test` to use a temporary
directory under the topobjdir. We encourage people to disable file indexing
and A/V scanning in the source and object directories because this can add
overhead. So by putting temporary files in the object directory, we should
minimize the chances of file scanning slowing down test execution.
On my machine (which has Windows Defender disabled in my source and objdirs),
`mach xpcshell-test` execution time drops from ~13:40 to ~7:30. Seriously.
I'm told we have Windows Defender disabled in automation, so this hack
shouldn't need to be investigated there. i.e. this is a pure local development
win.
MozReview-Commit-ID: BMcSAZ16Yei
This moves test installation for test files out of the monolithic install
manifest for $objdir/_tests, and determines the test and support files
to install based on the object derived from all-tests.json. Additionally,
the files resulting from TEST_HARNESS_FILES are installed, as some tests
will depend on them.
As a result, the time to install tests when invoking the test runner will
scale with the number of tests requested to run rather than the entire set
of tests in the tree, resulting in significantly less overhead.
MozReview-Commit-ID: LeIrUVh1yD4
This removes ambiguity as to which modules are being imported, making
import slightly faster as Python doesn't need to test so many
directories for file presence.
All files should already be using absolute imports because mach command
modules aren't imported to the package they belong to: they instead
belong to the "mach" package. So relative imports shouldn't have been
used.
Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
Make the test paths relative to topsrcdir before passing them to
TestResolver. Also do not passing cwd to TestResolver since it will
filter out tests that do not live under the directory where the mach
command is executed.
Verification steps:
Execute a mach test command from any subdirectory. For example:
$ cd testing/tps/
$ ../../mach xpcshell-test ../../services/fxaccounts/tests/xpcshell/
$ ../../mach test ../../services/fxaccounts/tests/xpcshell/
A subsequent patch will have another mach command dispatch into
`xpcshell-test`. To avoid redundant resolution of test paths, we now
support passing resolved test objects into the mach command.
