The mozharness_test job was using the wrong artifact path for native-engine.
Native-engine also enforces that artifact directories exist, so include some
mkdir calls to keep it happy.
MozReview-Commit-ID: Evw34gzzLev
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
This formats the marionette-harness python tests to be a regular |mach python-test| suite. Though
we add subsuite=marionette, this is just for automation purposes. The new preferred way to run the
marionette harness tests locally is:
./mach python-test testing/marionette
They will also run if running the full suite.
The mozbase packages.txt file modifies mozlog to use 'setup.py' instead of 'pth'. The reason for
this is that the marionette-harness tests use the pytest_mozlog pytest plugin for formatting
their results (converts pytest format into something resembling the standard tbpl logging format).
In order for this plugin to get picked up however, mozlog's setup.py file needs to be processed.
MozReview-Commit-ID: Ata99evHxbd
This is a minor refactor that aims to always attempt to set up 'mach' after running the 'run-mozharness' script.
We put it in a 'finally' block so we'll even do this if the user presses Ctrl-C or there was an exception
in the test harness. Importantly, this will be set up regardless of whether the user chooses "Option 1" or
"Option 2" at the wizard prompt.
The reason for this change is mostly 'might as well'. If it can save some users confusion, then it is worthwhile.
MozReview-Commit-ID: Dx3rV17FOoJ
Previously the run-wizard script would add a command to source the virtualenv in ~/.bashrc after
mozharness finished setting things up. This is fragile, assumes people are using bash, etc. Plus
it appeared to intermittently fail for some users.
Instead, this activates the virtualenv directly from individual mach commands that need it. This
guarantees we will always be using the virtualenv if required (and won't be using it if not). The
'activate_this.py' script is invoked the same way that we do it for in-tree mach commands:
https://dxr.mozilla.org/mozilla-central/rev/9c06e744b1befb3a2e2fdac7414ce18220774a1d/python/mozbuild/mozbuild/virtualenv.py#456
MozReview-Commit-ID: CfcoiVJXQTl
We add support for Mac OSX tests in Taskcluster by support
taskcluster-worker macosx engine. We also add a new configuration
blob-upload-branch needed by mochitest-other.
MozReview-Commit-ID: 5ZaYYRueRVr
We add support for Mac OSX tests in Taskcluster by support
taskcluster-worker macosx engine. We also add a new configuration
blob-upload-branch needed by mochitest-other.
MozReview-Commit-ID: CxT3BP3sydG
332a08725ed0 changed the store path logic in a quick and crude manner.
The code could lead to multiple shared stores existing on a cache
if checkouts were in different parent directories.
This commit refactors the code to explicitly declare a path to the
shared hg store. This restores the behavior of ensuring there is only
a single shared store per cache.
MozReview-Commit-ID: 19Aa1QVrVQ8
We were seeing issues with the Mercurial working directory not being
pristine. While I can't reproduce this, I have a hunch it is due to
mixing and matching stores and checkouts in TaskCluster. For example,
if a worker supports running concurrent tasks and 2 tasks arrive at
the same time, the caches for the store and checkout may look like:
(store0, checkout0)
(store1, checkout1)
However, the next task may get:
(store1, checkout0)
This may confuse Mercurial.
This commit eliminates the "hg-shared" cache and places the shared
stores as a sibling directory of the checkout.
MozReview-Commit-ID: 8SzyS6wWf9C
Currently, build tasks produce a zip file artifact containing the
content of testing/mozharness. This zip file is downloaded and applied
by any test using mozharness (which is most of them).
Now that some tests have a source checkout, we don't need to download
a mozharness zip because we can just use the source checkout.
This commit teaches test-ubuntu.sh to accept a custom path to
mozharness files. If that path is defined, we skip downloading the
mozharness zip.
The taskgraph has been updated to pass the path of mozharness in the
checkout to test-ubuntu.sh when a source checkout is available.
MozReview-Commit-ID: 2P17Wx1oytI
Mozharness scripts all live in "mozharness/scripts." So, we don't need
to redundantly define this path. Let's define the script as the relative
path within "mozharness/scripts" instead.
IMO this cosmetic improvement can stand on its own. However, the real
reason for this change is to pave the way for running mozharness from
a source checkout instead of downloading a zip archive of it.
MozReview-Commit-ID: FUnMiLlCcGK
The only difference in these files was the order that pulseaudio is
started and whether compiz is started. We rename test-ubuntu1604.sh
to test-ubuntu.sh, add some distro release detection, and add
some conditional branches so it works on both Ubuntu 12.04 and 16.04.
MozReview-Commit-ID: CaSfuDxss3d
Currently, build tasks produce a zip file artifact containing the
content of testing/mozharness. This zip file is downloaded and applied
by any test using mozharness (which is most of them).
Now that some tests have a source checkout, we don't need to download
a mozharness zip because we can just use the source checkout.
This commit teaches test-ubuntu.sh to accept a custom path to
mozharness files. If that path is defined, we skip downloading the
mozharness zip.
The taskgraph has been updated to pass the path of mozharness in the
checkout to test-ubuntu.sh when a source checkout is available.
MozReview-Commit-ID: 2P17Wx1oytI
Mozharness scripts all live in "mozharness/scripts." So, we don't need
to redundantly define this path. Let's define the script as the relative
path within "mozharness/scripts" instead.
IMO this cosmetic improvement can stand on its own. However, the real
reason for this change is to pave the way for running mozharness from
a source checkout instead of downloading a zip archive of it.
MozReview-Commit-ID: FUnMiLlCcGK
The only difference in these files was the order that pulseaudio is
started and whether compiz is started. We rename test-ubuntu1604.sh
to test-ubuntu.sh, add some distro release detection, and add
some conditional branches so it works on both Ubuntu 12.04 and 16.04.
MozReview-Commit-ID: CaSfuDxss3d
Robust checkout is the preferred method to clone a mercurial repository. This should
speed up the cloning process a bit and reduce storage size.
MozReview-Commit-ID: 92rcwMwRLYN
Sometimes xvfb will not start up with the current retry/delay settings. This will
attempt to retry more and delay for longer to ensure xvfb has started up. Common
pieces of this have been factored out into a recipe that all docker images can schare
that need this functionality.
MozReview-Commit-ID: BTXkJkBWLZX
Sometimes xvfb will not start up with the current retry/delay settings. This will
attempt to retry more and delay for longer to ensure xvfb has started up. Common
pieces of this have been factored out into a recipe that all docker images can schare
that need this functionality.
MozReview-Commit-ID: 2ww0eT3cIt6
Before, test.sh (duplicated between the desktop-test and
desktop1604-test images) was dropping permissions, creating a workspace,
and executing test-linux.sh. This is functionality now provided by
run-task.
So, convert the test tasks to use run-task.
One thing run-task isn't doing is created the workspace. So this
functionality has been moved into test-ubuntu1204.sh and
test-ubuntu1604.sh.
Since the test.sh files are no longer used after this change, they have
been deleted. The desktop-test image no longer has any files in the
bin/ directory, so the Dockerfile entry to copy those files has been
removed.
MozReview-Commit-ID: 1BiskrMs6xW
Before, test.sh (duplicated between the desktop-test and
desktop1604-test images) was dropping permissions, creating a workspace,
and executing test-linux.sh. This is functionality now provided by
run-task.
So, convert the test tasks to use run-task.
One thing run-task isn't doing is created the workspace. So this
functionality has been moved into test-ubuntu1204.sh and
test-ubuntu1604.sh.
Since the test.sh files are no longer used after this change, they have
been deleted. The desktop-test image no longer has any files in the
bin/ directory, so the Dockerfile entry to copy those files has been
removed.
MozReview-Commit-ID: 1BiskrMs6xW
There are some packages like 'requests' that are bundled in the mozharness venv, but not in the test
package. It would be easy to manually add these to sys.path in the mach bootstrap script, but it's
much nicer to simply activate this virtualenv in the first place.
MozReview-Commit-ID: 8xeJEIgUbLj
Normally we start Xvfb as a background task, then run the tests from the
same script. However, in interactive mode we were starting Xvfb, the script
would exit, and then we would potentially run tests later on from another
script. Unforunately this meant that Xvfb was dying with the first script
and tests would fail.
This patch runs Xvfb in a screen session so that it will still be available
later on when running an interactive shell.
MozReview-Commit-ID: EduVyglo2BG
This fixes a race condition between the 'test-linux.sh' process and the 'taskcluster-interactive-shell'
process in interactive tasks.
MozReview-Commit-ID: GhqKpq5pAtj
When running an interactive worker (aka one-click loaner), developers have the option of setting
up the mozharness environment without running tests. When they do this, we should automatically
symlink the mach binary found in the tests.zip to their path. This way developers don't need to
go searching for $HOME/workspace/build/tests/mach to run their tests.
MozReview-Commit-ID: 1JKPYSsYKlN
The run-wizard binary (used by interactive workers) will likely need to
change relatively frequently. Therefore, it should be baked directly into
the docker image. This patch instead downloads it from the appropriate
commit on hg.mozilla.org, only when needed.
MozReview-Commit-ID: 70hlloywCSj
