Commit Graph

228 Commits

Author SHA1 Message Date
Geoff Brown
e9b8814077 Bug 1449767 - Avoid error after robocop tests when no screenshots produced; r=bc
Screenshots are usually only taken when a test fails.
2018-03-29 07:33:40 -06:00
Geoff Brown
9b5730b454 Bug 1449762 - Fix argument to get_logcat() in robocop and android mochitest harnesses; r=bc 2018-03-29 07:33:38 -06:00
Geoff Brown
5ffdc3ffa0 Bug 1448697 - Try to use force-stop instead of kill to end remote applications; r=bc
This is important for running browser tests on some non-privileged devices, where
kill may not be permitted.
2018-03-26 12:21:46 -06:00
Geoff Brown
875a118aab Bug 1440714 - Convert Android browser test harnesses to adb.py; r=bc
This affects Android robocop, mochitest (all flavors) and reftests (all flavors).
2018-03-23 18:06:27 -06:00
Brendan Dahl
c79c2a2502 Bug 1444973 - Remove browser-test-overlay.xul. r=ahal
On desktop use sub script loader during bootstrap instead of the overlay.
On android use a component that listens for browser windows to be created
and then use a sub script loader.

MozReview-Commit-ID: Knj5OdVNEMA
2018-03-13 17:33:33 -07:00
Gurzau Raul
4a3c75a0d4 Backed out changeset deb3484ed429 (bug 1444973) for remoteautomation.py failures on Android a=backout 2018-03-21 16:53:11 +02:00
Brendan Dahl
e8cecdf2de Bug 1444973 - Remove browser-test-overlay.xul. r=ahal
On desktop use sub script loader during bootstrap instead of the overlay.
On android use a component that listens for browser windows to be created
and then use a sub script loader.

MozReview-Commit-ID: Knj5OdVNEMA
2018-03-13 17:33:33 -07:00
Andrew Halberstadt
cc1f360813 Bug 1432683 - [mochitest] Remove 'log' argument from the main options object, r=gbrown
This was causing problems for the test-verify mode. Sometimes the 'log' option
can contain a class instance and then 'verify' attempts to deepcopy that (which
fails).

Since the 'log' option is only used in the Mochitest constructor, it's probably
simplest to just remove it from the main options object right at the start.

MozReview-Commit-ID: 9UQAYxr2Zvm
2018-03-12 10:55:27 -04:00
Geoff Brown
4fe5478138 Bug 1440674 - Simplify some code in remoteautomation.py; r=jmaher
remove RemoteAutomation._product and product options;
remove some unused code paths;
remove "blobber" references;
remove incorrect and/or unhelpful comments;
s/_devicemanager/_dm/ (shorter, improve readability)
2018-02-23 09:47:57 -07:00
James Willcox
3d8aa28f03 No bug, followup for linting failures r=me
MozReview-Commit-ID: At5TO40eKhJ
2018-02-22 09:52:14 -06:00
James Willcox
5084519bd4 Bug 1291387 - Make mochitest and reftest work against TestRunnerActivity r=gbrown,jchen
Pass --appname org.mozilla.geckoview.test to 'mach mochitest' or
'mach reftest'. This runs the tests without e10s currently.

MozReview-Commit-ID: 7TIvA3zRCw2
2018-02-22 09:18:37 -06:00
Geoff Brown
c23f357282 Bug 1393326 - Enable runByManifest for Android mochitests and run Android mochitest-chrome in more chunks; r=jmaher 2018-02-12 09:05:59 -07:00
Geoff Brown
1413626ffe Bug 1435624 - In Android mochitest-chrome, never push tests twice; r=jmaher 2018-02-07 13:33:44 -07:00
Geoff Brown
8baf0845ac Bug 1435433 - Print master test summary after android mochitests when runByManifest; r=jmaher 2018-02-07 13:33:43 -07:00
Andrew Halberstadt
3353b79e9c Bug 1434430 - [flake8] Fix blank 'except' statements r=rwood
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.

This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`.  If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.

Of course, being specific is often better than a catch all.

MozReview-Commit-ID: FKx80MLO4RN
2018-01-31 14:32:08 -05:00
Andrew Halberstadt
d29f3e4728 Bug 1209463 - [test] Use a common logger across suites from |mach test| and other test commands, r=gbrown
In order for |mach test| and |mach mochitest| to log an overall summary,
every test harness invocation they make needs to use the same structured
logger (otherwise the affected suite will be missing from the summary).

MozReview-Commit-ID: 8LJw7r8SItk
2017-11-23 13:04:19 -05:00
Mike Hommey
6841c321d6 Bug 1420355 - Statically link DMD. r=njn 2017-11-28 08:10:48 +09:00
Mike Hommey
e9612fc379 Bug 1420355 - Don't initialize DMD if the DMD environment variable is not given. r=njn
This makes things slightly more inconvenient (having to set two
environment variables instead of one for the simplest case) until a few
patches down the line, when DMD is statically linked, at which point it
will get down to one environment variable every time.
2017-11-28 08:10:07 +09:00
Csoregi Natalia
e4966430b1 Backed out 7 changesets (bug 1420355) for mass failures on OS X and Android. r=backout on a CLOSED TREE
Backed out changeset a7ed89e13a4c (bug 1420355)
Backed out changeset fd6702e6e0a0 (bug 1420355)
Backed out changeset 0479dda078a2 (bug 1420355)
Backed out changeset e69357ccca9e (bug 1420355)
Backed out changeset 3742a4b69ba2 (bug 1420355)
Backed out changeset 451cd087922f (bug 1420355)
Backed out changeset d80b5c4e1dd0 (bug 1420355)
2017-11-29 03:08:46 +02:00
Mike Hommey
0a7b28a47e Bug 1420355 - Statically link DMD. r=njn 2017-11-28 08:10:48 +09:00
Mike Hommey
f88f56aa1d Bug 1420355 - Don't initialize DMD if the DMD environment variable is not given. r=njn
This makes things slightly more inconvenient (having to set two
environment variables instead of one for the simplest case) until a few
patches down the line, when DMD is statically linked, at which point it
will get down to one environment variable every time.
2017-11-28 08:10:07 +09:00
Geoff Brown
d14dbf4e70 Bug 1409062 - Update mochitest and reftest harnesses for --verify on Android; r=jmaher 2017-10-17 08:00:52 -06:00
Sebastian Hengst
68a92cc953 Backed out changeset dbe60b600dc0 (bug 1397853) for failining linter py-compat at testing/mochitest/runtests.py:2236. r=backout 2017-09-28 14:58:36 +02:00
Steve Armand
bee8bf3989 Bug 1397853 - Enable py2 and py3 linter on testing/mochitest. r=ahal
MozReview-Commit-ID: EhkNhyFHDox
2017-09-24 21:28:25 -04:00
Geoff Brown
6c70e6a351 Bug 1388883 - Ensure a fresh necko cache for each mochitest and reftest run; r=bc 2017-08-16 06:55:49 -06:00
Andrew Halberstadt
1f67d4f389 Bug 1328830 - Switch mochitest's --run-by-dir feature to --run-by-manifest, r=gbrown
MozReview-Commit-ID: LWYa3Sk1uyW
2017-07-20 12:10:15 -04:00
Geoff Brown
e20368dcbe Bug 1366129 - Improve cleanup of android mochitest environment before starting tests; r=jmaher 2017-05-25 07:32:46 -06:00
Shing Lyu
31f3c06eb7 Bug 1359288 - Enable Android reftest to resume from crashes. r=jmaher
MozReview-Commit-ID: EqTOdEwfTWi
2017-05-10 10:38:48 +08:00
Geoff Brown
f6246c5baa Bug 1340233 - Take more care when killing browser before starting Android tests; r=jmaher
The devicemanager killProcess() is updated to use force-stop first, then
use kill if force-stop does not work.

Browser test harnesses are updated to check if killProcess() worked, and
warn if it failed.
2017-04-12 09:15:53 -06:00
Andrew Halberstadt
5372c38ce5 Bug 1344346 - [mochitest] Log unstructured output as 'process_output', r=jgraham
Mochitest currently converts unstructured logs (e.g output from gecko) to 'info' messages. But
this means those messages won't be validated against mozharness' error logs. This change first
gets unstructured messages logged as process_output, and also ensures the StructuredOutputParser
in mozharness checks process_output messages against the error list.

MozReview-Commit-ID: KPTQnulwzyK
2017-03-03 16:28:22 -05:00
Iris Hsiao
235550c59e Backed out 3 changesets (bug 1344346) for mochitest shutdownleaks
Backed out changeset 71e3443cdc43 (bug 1344346)
Backed out changeset cd8993e4d0dd (bug 1344346)
Backed out changeset 8e03e57d44aa (bug 1344346)
2017-03-10 10:44:13 +08:00
Andrew Halberstadt
3adca9bea8 Bug 1344346 - [mochitest] Log unstructured output as 'process_output', r=jgraham
Mochitest currently converts unstructured logs (e.g output from gecko) to 'info' messages. But
this means those messages won't be validated against mozharness' error logs. This change first
gets unstructured messages logged as process_output, and also ensures the StructuredOutputParser
in mozharness checks process_output messages against the error list.

MozReview-Commit-ID: KPTQnulwzyK
2017-03-03 16:28:22 -05:00
Sebastian Hengst
3fe6c9ef3e Backed out changeset e6376870fa41 (bug 1344346) for failing at least web-platform-tests and robocop tests. r=backout 2017-03-09 18:36:39 +01:00
Andrew Halberstadt
9858dfa60f Bug 1344346 - [mochitest] Log unstructured output as 'process_output', r=jgraham
Mochitest currently converts unstructured logs (e.g output from gecko) to 'info' messages. But
this means those messages won't be validated against mozharness' error logs. This change first
gets unstructured messages logged as process_output, and also ensures the StructuredOutputParser
in mozharness checks process_output messages against the error list.

MozReview-Commit-ID: KPTQnulwzyK
2017-03-03 16:28:22 -05:00
Chris Peterson
acad849bd8 Bug 1330496 - Part 2: Remove mochitest use of MOZ_WIN_INHERIT_STD_HANDLES_PRE_VISTA. r=jmaher
MozReview-Commit-ID: LX8F53hVySp
2017-01-10 23:47:32 -08:00
Andrew Halberstadt
4a195c2f56 Bug 1020516 - Pass options.quiet into mochitest MessageLogger via constructor, r=ted
For some reason we have a 'buffering' parameter in the MessageLogger constructor, but
then rather than using this, the mochitest harness modifies state after instantiation.
Using the constructor is easier to understand, and simplifies some of the logic in the
next couple of patches.

MozReview-Commit-ID: Jkd9hOlmiGZ
2016-11-02 13:42:57 -04:00
Andrew Halberstadt
ef275026dc Bug 1308202 - Merge MochitestBase and MochitestDesktop into a single class, r=jmaher
MozReview-Commit-ID: 737L08LhLDp
2016-10-06 10:16:06 -04:00
Andrew Halberstadt
0b5479bf7c Bug 1308202 - Remove all b2g and mulet related code from testing/mochitest, r=jmaher
MozReview-Commit-ID: BfIJLp6oGHO
2016-10-06 10:05:27 -04:00
Prabhjyot Singh Sodhi
55ca3ed6d4 Bug 1280571 - Add testing/mochitest to the flake8 linter, r=ahal
MozReview-Commit-ID: INBZaZzcPmF
2016-09-28 08:28:04 -04:00
Wes Kocher
18fb5792ef Backed out changeset e1d144423657 (bug 1280571) for making leaks hard to diagnose a=backout CLOSED TREE 2016-09-21 16:56:50 -07:00
Prabhjyot Singh Sodhi
ade7a9a0d9 Bug 1280571 - Add testing/mochitest to the flake8 linter, r=ahal
This also adds E402 (no imports at top of file) to the global ignore list. The
other error codes added were previously ignored by default, but now that we
have started a custom ignore list, need to be listed explicitly.

MozReview-Commit-ID: RtMuVEX6i5
2016-09-21 08:49:46 -04:00
Andrew Halberstadt
9e6b28f4ea Bug 1293295 - Replace all mochitest 'flavor' options with a single --flavor argument, r=jmaher
This accomplishes three things:

1) Easier to use CLI when running without the benefit of testing/mochitest/mach_commands.py
2) Guarantees these arguments are mutually exclusive
3) Simplifies a bunch of logic in the test harness

The primary motivation for this change is to slightly improve the UX when running mochitest
from a taskcluster interactive loaner. However, this is more of a bandaid solution that was
easy to implement before the proper fix in bug 1293259 can be landed.

MozReview-Commit-ID: IeHBGrJ0Sji
2016-08-08 11:48:49 -04:00
Andrew Halberstadt
cfc6df1ca7 Bug 1288827 - Allow running mochitest from test-package without specifying --appname, r=armenzg
Because it is now possible for options.app to get set after 'parse_args' time, we need to make sure
the argument validation happens later. To accomplish this we pass in the parser instance to
'run_test_harness' and do parser.validate there. This unfortunately requires some minor uses of
global to accomplish easily due to how mach handles parsers.

MozReview-Commit-ID: s3Js1aZlSE
2016-07-20 11:13:30 -04:00
Geoff Brown
f08c875540 Bug 1285663 - Optimize profile pushes and do not chmod files on /sdcard; r=bc 2016-07-14 10:36:31 -06:00
Honza Bambas
24d2808e57 Bug 1248565 - Make test infra use MOZ_LOG_FILE and MOZ_LOG_MODULES. r=jmaher 2016-03-11 08:27:00 -05:00
Geoff Brown
440b6cdd9c Bug 1252868 - Do not install superfluous addons for Android tests; r=jmaher 2016-03-02 09:23:42 -07:00
Andrew Halberstadt
5302f3c79d Bug 1247639 - Fix uncaught exception when running |mach mochitest| on Android, r=gbrown
MozReview-Commit-ID: 1s8x4zIhZSS
2016-02-11 16:09:45 -05:00
Andrew Halberstadt
9dcfc81d19 Bug 1231784 - Install specialpowers and mochikit extensions at runtime via AddonManager.loadTemporaryAddon(), r=jgriffin
MozReview-Commit-ID: 9I56x6Vnbf7
2016-01-25 09:55:57 -05:00
Wes Kocher
c4797ad496 Backed out 3 changesets (bug 1231784) for breaking 10.10 opt jetpack tests
Backed out changeset 653df21ca861 (bug 1231784)
Backed out changeset 0aac20bc3619 (bug 1231784)
Backed out changeset edcbd7c0aa36 (bug 1231784)
2016-02-03 17:38:12 -08:00
Andrew Halberstadt
f84ba669d4 Bug 1231784 - Install specialpowers and mochikit extensions at runtime via AddonManager.loadTemporaryAddon(), r=jgriffin 2016-01-25 09:55:57 -05:00