We'd like to ensure that both parallel and serial traversal in Stylo are
tested on automation. Since e10s is the future, we've chosen to force
parallel traversal on during e10s tests, and force serial traversal on
during non-e10s tests.
This patch allows the use of the flag '--jscov-dir-prefix' for mochitest plain tests to enable code coverage collection with the JS Debugger. It also enables the mochitest-plain tests for the linux64-jsdcov build platform.
MozReview-Commit-ID: 6RqMEZ1I0D7
This patch provides the implementation that makes it possible to run the devtools test suite with the linux64-jsdcov build to collect js code coverage.
MozReview-Commit-ID: KFmFhKsDq5s
I haven't seen a case of this happening in the wild, but I believe it is possible for a test to dump "TEST-UNEXPECTED-FAIL"
directly to the log and mozharness using the StructuredOutputParser would not pick it up. This patch is just me being extra
careful to flag potential errors like that. I'm not sure they even exist.
This patch also purposefully uses substr to avoid requiring said string to show up at the beginning, which should avoid
certain prefix issues we've run into in the past.
MozReview-Commit-ID: 99n9YizlEDH
This refactors the mozharness configs such that a specific flavor of an overall suite can still use the old
DesktopUnittestOutputParser. This is necessary because mochitest-jetpack still does not use the structured
logger.
MozReview-Commit-ID: E8EpSLH4xt2
Given that we have a universal unpack method now do not keep 'unzip' in method names.
Also adapt arguments to be better understandable.
MozReview-Commit-ID: ClDB5mSVcI2
Given that we have a universal unpack method now do not keep 'unzip' in method names.
Also adapt arguments to be better understandable.
MozReview-Commit-ID: ClDB5mSVcI2
Given that we have a universal unpack method now do not keep 'unzip' in method names.
Also adapt arguments to be better understandable.
MozReview-Commit-ID: ClDB5mSVcI2
We need to be able to run the copydirs logic in desktop_unittest.py independently
of the 'run-tests' action to enable a smooth interactive debugging workflow.
MozReview-Commit-ID: ATftu2NnhQq
This adds the ability to use the command line flag '--jscov-dir-prefix' to collect javascript code coverage from xpcshell tests and output it into the specified directory as a JSON file.
MozReview-Commit-ID: 3MZm73SNChL
This commit teaches the resource monitor in mozharness to emit
Perfherder data for system metrics and step times. This will
allow us to see when the timing or resource characteristics
of jobs in automation changes.
The recorded data includes overall CPU percent usage and I/O.
Each step has its time and CPU percent recorded. There is
certainly more data we could record. However, the immediate
goal of this change is to see if the data provides any benefit.
I'd rather start small and expand reporting once value from
this data is proved.
The wonkiest part of this patch is likely the mechanism to
define the Perfherder "test" names. We don't appear to have
an identifier in mozharness suitable for distinguishing
between job types. e.g. the "desktop_unittest.py" script is
responsible for running a few dozen jobs. So we invent code
for creating an identifier from the script config options.
I /think/ Treeherder will automatically assign the
project/branch, platform, and build type, which is why these
aren't included in the identifier.
MozReview-Commit-ID: HjhtXfxOvzJ
This adds support for web-platform-tests to mach try. It changes the implementation
so that instead of passing paths to manifests, the user passes arbitary paths in the
source tree, and tests under that path are run, with test discovery mainly left to
the harness.
This adds support for web-platform-tests to mach try. It changes the implementation
so that instead of passing paths to manifests, the user passes arbitary paths in the
source tree, and tests under that path are run, with test discovery mainly left to
the harness.
This adds support for web-platform-tests to mach try. It changes the implementation
so that instead of passing paths to manifests, the user passes arbitary paths in the
source tree, and tests under that path are run, with test discovery mainly left to
the harness.
The config files under testing/config/mozharness were created so that certain mozharness options
such as test harness arguments could ride the trees, simplifying a lot of logic in mozharness.
But now that mozharness itself is in-tree, these configs no longer serve any purpose. Instead
they are merged into the main configs at testing/mozharness/configs.
