Commit Graph

61 Commits

Author SHA1 Message Date
Alex Gaynor
478321d9d3 Bug 1294641 - whitelist reads from the .app directory in the macOS sandbox r=froydnj,haik
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.

r=haik,froydnj

MozReview-Commit-ID: KNvAoUs5Ati
2017-04-07 14:53:19 -04:00
Sebastian Hengst
155e26d497 Backed out changeset 225683fed1d1 (bug 1294641) for breaking crashtests, reftests and Windows builds. r=backout on a CLOSED TREE 2017-04-17 16:29:52 +02:00
Alex Gaynor
d7af07ecbe Bug 1294641 - whitelist reads from the .app directory in the macOS sandbox r=froydnj,haik
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.

r=haik,froydnj

MozReview-Commit-ID: KNvAoUs5Ati
2017-04-07 14:53:19 -04:00
Geoff Brown
b72a74a2a6 Bug 1340584 - Remove devicemanagerSUT and references to sutagent; r=wlach
sutagent is no longer built or used; devicemanagerSUT is completely
unused. After this change, devicemanagerADB is the only implementation of
devicemanager, and test harness options like --dm_trans are eliminated.
2017-03-21 12:20:01 -06:00
Deepa
bb63c1ff3d Bug 1280572 - Add layout/tools/reftest to flake8 linter r=ahal
MozReview-Commit-ID: Cqllz9BeV7X
2017-02-15 01:20:56 +05:30
Andrew Halberstadt
b6864170d2 Bug 1330772 - Remove all b2g related code from reftests, r=jmaher
MozReview-Commit-ID: HY1XbcliTNA
2017-01-12 16:37:37 -05:00
Nick Alexander
ce8407ec28 Bug 1321408 - Move ENABLE_MARIONETTE to python configure. r=chmanchester
This patch tries to do three things:

1) Replace the ENABLE_MARIONETTE entrypoint with --enable-marionette.

2) Fold the default value -- forced on unless building for target OS
Android or building with toolkit gonk -- into the flag, rather than
embedding that condition in the tree.

3) Stop using AC_DEFINE and instead use only AC_SUBST, so that no
compiled code needs to be rebuilt if the flag is flipped locally.
n.b., each installer/Makefile.in knows that ENABLE_MARIONETTE is set
(in order to set -DENABLE_MARIONETTE=1 for
*/installer/package-manifest.in) due to it being an AC_SUBST.

MozReview-Commit-ID: AkkmybyP1uI
2016-12-16 15:49:14 -08:00
Ryan VanderMeulen
69c29674a5 Bug 1263230 - Remove in-tree references to IPC/OOP reftest modes. r=ahal 2016-11-07 08:59:49 -05:00
Andrew Halberstadt
431727cd2d Bug 1301431 - Pass in adb_path to devicemanager if specified in Android reftests, r=gbrown
When using an interactive loaner, we compute an explicit path to the adb binary. Passing it in
directly to mozdevice seems a bit cleaner than modifying the $PATH. This also does a minor
refactor around how the DM instances are instantiated.

MozReview-Commit-ID: 5HMfm64wmK2
2016-09-13 09:07:05 -04:00
Andrew Halberstadt
15edcff113 Bug 1301431 - Change reftest harness to accept Namespace object instead of kwargs, r=armenzg
Reftests hackily create a second argument parser and set defaults on it to get around the fact
that the mach command sends the harness a dict rather than a Namespace object. This is bad because:

1. It's much less hacky to just create a Namespace object directly (rather than making a second parser)
2. Most other mach commands actually *do* return a Namespace object, reftest is the odd one out here.

So this patch makes the reftest mach commands convert to the Namespace object, which allows us to get
rid of the hacks in the reftest harness. This also does some light refactoring of the entry points to
the reftest harness so make it more consistent with mochitest and xpcshell.

MozReview-Commit-ID: 5HMfm64wmK2
2016-09-13 09:26:09 -04:00
Andrew Halberstadt
4a72e48caf Bug 1301431 - Stop using imp to import reftest module on Android, r=gbrown
Android reftests still use os.chdir and imp to import reftest files. But there's a
_setup_objdir method (used by b2g) that does a similar thing instead. I decided to
try getting Android to use this as well and it seems to work. This is just a cleanup.

MozReview-Commit-ID: 5HMfm64wmK2
2016-09-13 09:26:04 -04:00
Andrew Halberstadt
89985d276b Bug 1292660 - Fix |mach mochitest| kwargs['app'] is None error on android, r=gbrown
This fixes a regression from bug 1288827. It happened because I moved the logic that finds
the application path a little later on in the test harness. But there was an instance where
it was being used in the android mach command before that point.

As it turned out, we don't really *need* that value there. This patch grabs the same value
from build_obj.substs which is already an argument to the function.

MozReview-Commit-ID: 3IsI4VzEIIF
2016-08-05 16:40:08 -04:00
Krishna
29315be1bc Bug 1276560 - Package hyphenation-dictionaries separately for reftests. r=sebastian
MozReview-Commit-ID: IRvIqGEfc1m
2016-07-05 01:00:15 +08:00
Andrew Halberstadt
42ab46036a Bug 1271448 - Stop saving reftest.log for |mach reftest| perf improvements, r=jgraham
In bug 1271035 gps identified disk I/O as a major cause for reftest slowness.
He was able to fix most of it, but the new highest source of I/O in reftest is
reftest.log.

To note, we were only saving reftest.log when running via mach, so this won't
impact automation in any way:
https://hg.mozilla.org/mozilla-central/file/043082cb7bd8/layout/tools/reftest/mach_commands.py#l204

I don't know why we are doing this given that the same output goes to stdout.
And as dholbert pointed out, since bug 1034290 landed, that log contains raw
structured logs, which are not at all useful for debugging. Given that it is
no longer useful and causes slowness, we should stop saving it.

If anyone wishes to keep saving to a log, they can use:
./mach reftest --log-tbpl reftest.log

If they wish this to be the default behaviour they can make a machrc (or
.machrc) in either topsrcdir, ~/.mozbuild or $MACHRC. Then add:
[alias]
reftest = reftest --log-tbpl reftest.log

MozReview-Commit-ID: A3e2X7qF90H
2016-05-09 17:53:04 -04:00
Geoff Brown
daef4e59e5 Bug 1249681 - Set utilityPath for Android mach reftests; r=bdahl 2016-02-19 16:10:22 -07:00
Andrew Halberstadt
a8bca414a2 Bug 1247054 - Fix |mach reftest| on Android, r=gbrown
There is an ImportError on Android, as well as a log related
regression from the structured log patch once that is fixed.

MozReview-Commit-ID: KxSEotr38qO
2016-02-09 16:19:44 -05:00
Andrew Halberstadt
94aeaa27a9 Bug 1034290 - Use structured log output for test results in reftest, r=jmaher
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.

MozReview-Commit-ID: G3ZLkMRl6p7
2016-02-05 15:44:20 -05:00
Phil Ringnalda
c5159ef2e2 Back out d1779fe421c3 (bug 1034290) for UnicodeDecodeErrors in things run by the reftest harness
CLOSED TREE
2016-02-05 08:18:47 -08:00
Andrew Halberstadt
515a1ec282 Bug 1034290 - Use structured log output for test results in reftest, r=jmaher
Structured logs bring many benefits. We can stop parsing the logs for magic strings, we
can modify the format without breaking things, and we can stream results into systems like
ActiveData. The structured logs originate primarily in reftest.js. StructuredLog.jsm is
used to generate the JSON-based log stream. Finally OutputHandler in the python harness
reads structured output from stdout, and formats it into human readable form.
2015-10-29 15:01:44 -04:00
Geoff Brown
b96201780b Bug 1241907 - Grant runtime permissions before running browser tests; r=jmaher 2016-02-01 15:52:34 -07:00
Andrew Halberstadt
6b276f552b Bug 1228636 - Add mach support for running reftests on mulet, r=jgriffin
Usage:
mach reftest --profile path/to/gaia/profile path/to/test/dir
2016-01-04 16:27:37 -05:00
Geoffrey Sneddon
9258c1ee2a Bug 1231261 - Append plugins to extra profile files instead of overwriting. r=dbaron 2015-12-10 13:34:00 +01:00
Andrew Halberstadt
29793413d8 Bug 1226017 - Drop support for b2g desktop in reftests, r=jgriffin 2015-11-27 13:56:43 -05:00
Geoff Brown
15e8158272 Bug 1197716 - On Android, include fonts in test profile; r=jmaher 2015-11-06 12:01:33 -07:00
Geoff Brown
d168d63a53 Bug 1087791 - Follow-up on 340c1df41b69 (pushed wrong version of patch); r=nalexander 2015-10-14 16:31:19 -06:00
Geoff Brown
7cac099d42 Bug 1087791 - Add |mach {reftest,crashtest,jstestbrowser}| for mobile/android. r=nalexander 2015-10-14 15:20:20 -06:00
James Graham
4cc532bd15 Bug 1193223 - Add reftest support to mach test, r=chmanchester 2015-09-17 10:50:57 +01:00
James Graham
ea97d39711 Bug 1181516 - Allow reftests to take paths to multiple directories containing tests on the command line, r=jmaher
This makes reftest command line arguments behave more like other test suites,
so we can use a simple unified syntax for e.g. |mach try|. The patch also
reworks the command line argument parsing to use argparse rather than optparse,
and causes mach to reuse the same parser as the suite.
2015-09-17 10:50:55 +01:00
Gregory Szorc
d993a444fa Bug 1176642 - Use absolute_import in mach_commands.py files; r=glandium
This removes ambiguity as to which modules are being imported, making
import slightly faster as Python doesn't need to test so many
directories for file presence.

All files should already be using absolute imports because mach command
modules aren't imported to the package they belong to: they instead
belong to the "mach" package. So relative imports shouldn't have been
used.
2015-06-21 17:39:09 -07:00
Mike Hommey
2e51d2ef87 Bug 1147283 - Replace mozpack.path with mozpath. r=mshal
Back when mozpack.path was added, it was used as:

  import mozpack.path
  mozpack.path.func()

Nowadays, the common idiom is:

  import mozpack.path as mozpath
  mozpath.func()

because it's shorter.

$ git grep mozpath\\. | wc -l
423
$ git grep mozpack.path\\. | wc -l
123

This change was done with:
$ git grep -l mozpack.path\\. | xargs sed -i 's/mozpack\.path\./mozpath./g'
$ git grep -l 'import mozpack.path$' | xargs sed -i 's/import mozpack.path$/\0 as mozpath/'
$ (pat='import mozpack.path as mozpath'; git grep -l "$pat" | xargs sed -i "1,/$pat/b;/$pat/d")
2015-03-27 08:13:16 +09:00
Nathan Froyd
050846a825 Bug 1097125 - part 2 - make |mach reftest| and related commands support single files; r=ted.mielczarek 2014-11-11 12:20:33 -05:00
Nathan Froyd
1c6fc4e0bb Bug 1097125 - part 1 - add filtering support to |mach reftest-{remote,b2g-desktop}|; r=ted.mielczarek 2014-11-11 12:16:53 -05:00
Nathan Froyd
286bc4e8b2 Bug 1097125 - part 0 - fix cut-and-pasted comment; r=ted.mielczarek 2014-11-11 12:12:40 -05:00
Aaron Klotz
631c0301b0 Bug 1070042: Fix --debugger parameter in reftest/crashtest; r=ted 2014-09-19 14:49:34 -06:00
L. David Baron
15236e2850 Bug 1052907 - Make --this-chunk and --total-chunks arguments to mach reftest-remote work by coercing them to ints, which they need to be. r=ahal 2014-08-13 15:39:03 -07:00
Sylvestre Ledru
0650e71d3a Bug 1050715 - Update the description of the various tests targed proposed by './mach help' r=jmaher 2014-08-09 15:37:29 +02:00
L. David Baron
bf0fc28e51 Bug 1031635 - Add --debugger-args option to reftest mach commands. r=gps
This allows useful argument combinations such as:
  --debugger valgrind
  --debugger-args "--tool=memcheck --leak-check=no --trace-children=yes --num-callers=50"
2014-07-16 20:10:10 -07:00
Jason Orendorff
ed3716187c Bug 889898 - Add mach jstestbrowser. r=gps.
Relanding without changes, as this couldn't possibly have caused the failures
that made us back it out.
2014-06-20 12:01:03 -05:00
Andrew Halberstadt
df364bfcf8 Bug 997244 - Move emulator.py out of marionette and into mozrunner, r=wlach,mdas,jgriffin 2014-06-19 14:17:26 -04:00
Markus Stange
ab3ec31ff1 Bug 1023464 - Add support for the --setpref argument to mach reftest. r=gps 2014-06-11 14:44:42 +02:00
Carsten "Tomcat" Book
8b2bfaa408 merge b2g-inbound to mozilla-central 2014-05-26 15:26:11 +02:00
Andrew Halberstadt
504dfe799f Bug 922680 - Enable out of process b2g emulator reftests, r=jgriffin 2014-05-23 15:55:54 -04:00
Kartikaya Gupta
27aa09980e Bug 1015369 - Remove some dead code. r=ahal 2014-05-23 17:07:02 -04:00
Kartikaya Gupta
1f5b9f33b9 Bug 1015369 - Add a flag to enable oop mode to b2g-desktop reftests. r=ahal 2014-05-23 17:07:01 -04:00
Vicamo Yang
23091844af Bug 996443: carry emulator arch automatically for mach commands. r=ahal 2014-05-01 00:57:39 +08:00
Robert O'Callahan
458929c0fc Bug 998206. Add --shuffle option to runreftests.py and 'mach reftest'. r=dbaron 2014-03-03 18:13:01 +13:00
Andrew Halberstadt
f8c42229d9 Bug 994711 - Expose --this-chunk and --total-chunks to reftest mach targets, r=jgriffin, DONTBUILD, a=NPOTB 2014-04-14 16:51:46 -04:00
Vicamo Yang
d96efb1aa4 Bug 975442: recognize as emulator if device name begins with 'emulator'. r=jgriffin 2014-04-09 10:14:07 +08:00
Bill McCloskey
0e8ddd36a3 Bug 986653 - Add --e10s option to reftests (r=ted) 2014-03-18 16:31:28 -07:00
Andrew Halberstadt
0516ee69f0 Bug 948070 - Create a mach target for b2g desktop reftests, r=jgriffin, DONTBUILD, a=NPOTB 2014-01-30 08:42:18 -05:00