This sets verbose=True (used by the mach formatter) and compact=False (used by tbplformatter) if
only a single test is specified with either |mach test| or |mach mochitest|.
This way all TEST_STATUS and log lines will be visible when developers are writing a new test.
MozReview-Commit-ID: 2nuKs9DLnx0
Pass --appname org.mozilla.geckoview.test to 'mach mochitest' or
'mach reftest'. This runs the tests without e10s currently.
MozReview-Commit-ID: 7TIvA3zRCw2
The suite name is no part of the suite_start message and can be printed via a
mozlog formatter. We shouldn't be printing this directly to stdout from |mach
mochitest|.
MozReview-Commit-ID: TOuqfEF14
The suite name is no part of the suite_start message and can be printed via a
mozlog formatter. We shouldn't be printing this directly to stdout from |mach
mochitest|.
MozReview-Commit-ID: TOuqfEF14
The suite name is no part of the suite_start message and can be printed via a
mozlog formatter. We shouldn't be printing this directly to stdout from |mach
mochitest|.
MozReview-Commit-ID: TOuqfEF14
When 'summary_on_shutdown' is True (which is the case for |mach test| and |mach
mochitest|), BaseSummaryFormatters will save the summary information until the
'shutdown' action is received at the end of the logger's lifetime.
Summary information will no longer be dumped on 'suite_end'.
MozReview-Commit-ID: HKtVr5PxfOy
In order for |mach test| and |mach mochitest| to log an overall summary,
every test harness invocation they make needs to use the same structured
logger (otherwise the affected suite will be missing from the summary).
MozReview-Commit-ID: 8LJw7r8SItk
The TestMetadata and TestResolver classes aren't technically part of the build
system. The only connection is that they consume some build system output.
The next patch in this series is going to be merging in a bunch of other test
resolving logic from other parts of the tree. Moving this out first allows us
to keep that extra logic out of mozbuild.
MozReview-Commit-ID: 1eq4SjFVCyW
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.
r=haik,froydnj
MozReview-Commit-ID: KNvAoUs5Ati
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.
r=haik,froydnj
MozReview-Commit-ID: KNvAoUs5Ati
./mach robocop calls through to mochitest_options.py to validate the passed command line options, so it's not necessary to define a default Robocop APK path from two different places.
MozReview-Commit-ID: 8CryDqOKDBF
Information in the ALL_FLAVORS dict is needed by interactive loaners (in the mach_test_package_commands.py
file). Because the normal mach_commands.py file doesn't get copied to the tests.zip, this commit refactors
ALL_FLAVORS into mochitest_options.py (which is copied to tests.zip) to avoid duplicating it. A side
benefit of moving ALL_FLAVORS to mochitest_options.py, is that mochitest_options.py itself can make use of
this dict. This means we no longer need to redefine the --flavor argument in the mach command.
The __init__.py file is added to turn the testing/mochitest directory into a python module. This allows
things like mach_commands.py to do things like 'from mochitest import runtests'. Mach commands are able to
find this module because the 'testing' directory is already added to sys.path in the mach bootstrap.
In the future, having mochitest as an importable module should help with running it from the srcdir. So this
is a change we should start making anyway. Unfortunately, we still need to import the main runtests.py file
from the objdir, as lots of things depend on the SCRIPT_DIR variable being in the objdir. We could probably
fix this with minimal work, but that is scope bloat for this bug.
MozReview-Commit-ID: KtWCk91bX0K
CLOSED TREE
Backed out changeset 1f3f88337227 (bug 1312739)
Backed out changeset ecb1d15e8075 (bug 1312739)
Backed out changeset b2adce340421 (bug 1312739)
Information in the ALL_FLAVORS dict is needed by interactive loaners (in the mach_test_package_commands.py
file). Because the normal mach_commands.py file doesn't get copied to the tests.zip, this commit refactors
ALL_FLAVORS into mochitest_options.py (which is copied to tests.zip) to avoid duplicating it. A side
benefit of moving ALL_FLAVORS to mochitest_options.py, is that mochitest_options.py itself can make use of
this dict. This means we no longer need to redefine the --flavor argument in the mach command.
The __init__.py file is added to turn the testing/mochitest directory into a python module. This allows
things like mach_commands.py to do things like 'from mochitest import runtests'. Mach commands are able to
find this module because the 'testing' directory is already added to sys.path in the mach bootstrap.
In the future, having mochitest as an importable module should help with running it from the srcdir. So this
is a change we should start making anyway. Unfortunately, we still need to import the main runtests.py file
from the objdir, as lots of things depend on the SCRIPT_DIR variable being in the objdir. We could probably
fix this with minimal work, but that is scope bloat for this bug.
MozReview-Commit-ID: KtWCk91bX0K
Automatically install websocketprocessbridge_requirements.txt if we detect the 'media' subsuite will be run.
We need to modify sys.executable so the virtualenv python gets used when starting up the
websocketproccessbridge. We probably should use multiprocessing instead, but that is out of scope for this
bug.
MozReview-Commit-ID: 3lOutbcSzIY
This causes consumers managing defaults themselves to fail to find a default
subsuite for tests, because the manifestparser will have provided a blank
default value by the time they incorporate defaults into a test definition.
This patch removes the provided defaults and updates a number of places assuming
the 'subsuite' field is always present.
MozReview-Commit-ID: 1jPy52VmEPr
This causes consumers managing defaults themselves to fail to find a default
subsuite for tests, because the manifestparser will have provided a blank
default value by the time they incorporate defaults into a test definition.
This patch removes the provided defaults and updates a number of places assuming
the 'subsuite' field is always present.
MozReview-Commit-ID: 1jPy52VmEPr
This also adds E402 (no imports at top of file) to the global ignore list. The
other error codes added were previously ignored by default, but now that we
have started a custom ignore list, need to be listed explicitly.
MozReview-Commit-ID: RtMuVEX6i5
This accomplishes three things:
1) Easier to use CLI when running without the benefit of testing/mochitest/mach_commands.py
2) Guarantees these arguments are mutually exclusive
3) Simplifies a bunch of logic in the test harness
The primary motivation for this change is to slightly improve the UX when running mochitest
from a taskcluster interactive loaner. However, this is more of a bandaid solution that was
easy to implement before the proper fix in bug 1293259 can be landed.
MozReview-Commit-ID: IeHBGrJ0Sji
This fixes a regression from bug 1288827. It happened because I moved the logic that finds
the application path a little later on in the test harness. But there was an instance where
it was being used in the android mach command before that point.
As it turned out, we don't really *need* that value there. This patch grabs the same value
from build_obj.substs which is already an argument to the function.
MozReview-Commit-ID: 3IsI4VzEIIF
Because it is now possible for options.app to get set after 'parse_args' time, we need to make sure
the argument validation happens later. To accomplish this we pass in the parser instance to
'run_test_harness' and do parser.validate there. This unfortunately requires some minor uses of
global to accomplish easily due to how mach handles parsers.
MozReview-Commit-ID: s3Js1aZlSE
Because it is now possible for options.app to get set after 'parse_args' time, we need to make sure
the argument validation happens later. To accomplish this we pass in the parser instance to
'run_test_harness' and do parser.validate there. This unfortunately requires some minor uses of
global to accomplish easily due to how mach handles parsers.
MozReview-Commit-ID: s3Js1aZlSE
This moves test installation for test files out of the monolithic install
manifest for $objdir/_tests, and determines the test and support files
to install based on the object derived from all-tests.json. Additionally,
the files resulting from TEST_HARNESS_FILES are installed, as some tests
will depend on them.
As a result, the time to install tests when invoking the test runner will
scale with the number of tests requested to run rather than the entire set
of tests in the tree, resulting in significantly less overhead.
MozReview-Commit-ID: LeIrUVh1yD4
