Most entries in virtualenv_packages.txt that are .pth or packages.txt
are currently in SEARCH_PATHS in mach_bootstrap. The ones that are
missing would make sense in SEARCH_PATHS. None of non-.pth or
packages.txt entries, however, are in SEARCH_PATHS and don't make sense
there.
On the other hand, virtualenv_packages.txt misses a lot of things that
are in SEARCH_PATHS, all of which should be there. One exception:
xpcom/idl-parser, which causes problems due to the xpidl package
containing an xpidl module, which causes problems with the in-tree
scripts using it. Plus, it needs a cache directory, which is messy, so
it's preferable to keep it away from the virtualenv. It turns out it was
added to mach_bootstrap.py in bug 893976 for a command that was since
then removed (bug 1244736), so we can get away with removing it.
So instead of keeping those two separate lists out of sync, we replace
the SEARCH_PATHS list from mach_bootstrap with one that is derived at
runtime from the contents of virtualenv_packages.txt.
And since a .pth can't fail to install in the virtualenv, it makes no
sense to have psutil.pth defined as optional, which allows it to end up
in the mach_bootstrap search path automatically.
Finally, because we do have overlapping module names in the tree (e.g.
runtests), and mach_bootstrap's SEARCH_PATHS had a guaranteed order,
we change the order of the virtualenv_packages.txt file to match what
used to be in mach_bootstrap, and make all the pth entries use the same
file name so that the order is more guaranteed in the virtualenv too.
usage: mach [global arguments] awsy-test [command arguments]
mach awsy-test runs the in-tree version of the Are We Slim Yet
(AWSY) tests.
awsy-test is implemented as a marionette test and marionette
test arguments also apply although they are not necessary
since reasonable defaults will be chosen.
The AWSY specific arguments can be found in the Command
Arguments for AWSY section below.
awsy-test will automatically download the tp5n.zip talos
pageset from tooltool and install it under
topobjdir/_tests/awsy/html. You can specify your own page set
by specifying --web-root and --page-manifest.
The results of the test will be placed in the results
directory specified by the --results argument.
Command Arguments for AWSY:
--web-root WEBROOTDIR
Path to web server root directory. If not specified,
defaults to topobjdir/_tests/awsy/html.
--page-manifest PAGEMANIFEST
Path to page manifest text file containing a list of
urls to test. The urls must be served from localhost.
If not specified, defaults to
page_load_test/tp5b/tp5n.manifest under the web root.
--results RESULTSDIR Path to results directory. If not specified, defaults
to the parent directory of the web root.
--quick Set --entities=3, --iterations=1, --per-tab-pause=1,
--settle-wait-time=1 for a quick test. Overrides any
explicit argument settings.
--entities ENTITIES Number of urls to load. Defaults to the total number
of urls.
--max-tabs MAXTABS Maximum number of tabs to open. Defaults to 30.
--iterations ITERATIONS
Number of times to run through the test suite.
Defaults to 5.
--per-tab-pause PERTABPAUSE
Seconds to wait in between opening tabs. Defaults to
10.
--settle-wait-time SETTLEWAITTIME
Seconds to wait for things to settled down. Defaults
to 30.
This imports two modules from mozregression in the tree to do so. They
are imported from current trunk on github, rather than the version we
were getting from pypi.
Note we take six from testing/web-platform/tests/tools/six) instead of
moving it to python/six because it's there by coming from a copy of
https://github.com/w3c/wpt-tools, which contains it as a submodule, and
moving it would make updates there harder.
This generalizes the monkey-patching of the main module added
in bug 914563 to allow multiprocessing to be used from config.status
(which triggers the same bug as when it's used with `mach` as main).
MozReview-Commit-ID: AdOdpKzmbsp
Provide a single mach command to automatically generate the static
database of CSS properties that devtools uses for the inspector
and various editors.
MozReview-Commit-ID: 8E2jwxF0KbM
Vendor in Pytest (2.9.2) and its requirement Py (1.4.31),
so that it can be used for e.g. the Marionette
harness unit tests and a pytest plugin for mozlog.
Copy pytest and py package directories (extracted from
tars on Pip) into `mozilla-central/python/`, removing
some support files (e.g. changelog, docs, tests).
Add both `.pth` entries to `virtualenv_packages.txt`.
Add both paths to `SEARCH_PATHS` in `mach_bootstrap.py`.
MozReview-Commit-ID: IOTCOUxX8R9
It used to be there, but was removed in bug 1151124 because what I
thought was the only use of pymake in mach was removed in that bug.
Which, it turns out, was not the case. So restore it, so that mach
empty-makefiles works again.
The wizard has been ported to the version-control-tools repository
and in-tree consumers have been switched to consume it from there. This
code is now dead. Kill it.
References to the now-defunct code have been removed/updated.
MozReview-Commit-ID: 5fpCXdNIp8L
This begins the consolidation of `mach mercurial-setup` into
`mach bootstrap`. The first step is to move the content of the
mach_commands.py file into the bootstrapper's.
I'm not crazy about adding the sys.path entry for tools/mercurial.
I intend to clean this up later.
MozReview-Commit-ID: Cq56wPG8sO1
I never really liked this. Other people had even more visceral
reactions. Let's get rid of it.
The code for touching a file when it runs has also been removed because
the only thing it was used for was the nagging feature.
MozReview-Commit-ID: ERUVkEYgkzx
The `taskgraph` package generates TaskCluster task graphs based on collections
of task "kinds". Initially, there is only one kind, the "legacy" kind, which
reads the YAML files from `testing/taskcluster/tasks` to generate the task
graph.
Try syntax is implemented by filtering the tasks in the taskgraph after it has
been created, then extending the result to include any prerequisite tasks.
A collection of `mach taskgraph` subcommands are provided for developers to
extend or debug the task-graph generation process.
MozReview-Commit-ID: 1TJCns4XxZ8
Runtime configs have been implemented for awhile, but disabled. This patch
enables configuration. Config files will be loaded in the following order
(later files override earlier ones):
1a. $MACHRC
1b. $MOZBUILD_STATE_PATH/machrc (if $MACHRC is unset)
2. topsrcdir/machrc
3. CLI via --settings
Note: .machrc may be used instead of machrc if desired.
MozReview-Commit-ID: IntONAZLGML
This spins up a separate process to submit telemetry data rather than
blocking the execution the current mach command.
Although the initial Python process needs to wait for the second process
to complete prior to exiting, it releases control of the console once it
finishes executing Python code, so from the user's perspective, mahc command
completion is not blocked by submitting telemetry data.
MozReview-Commit-ID: FlKDYd6rNPc
This change moves testing/marionette/client to testing/marionette/harness,
and testing/marionette/driver to testing/marionette/client. This parent
directory name change reflects the true purpose of these Python packages.
MozReview-Commit-ID: B0KrGzDTLlQ
Previously on first `mach` run on a system, we'd display a prompt with a
20s countdown timer after which the default state directory
(~/.mozbuild) was created. Users had to wait 20s or ctrl+c and
make the directory by hand. Either way, they'd have to re-invoke
mach to run whatever command they were trying to run.
This was annoying.
This commit makes the following changes:
1) The countdown timer is replaced with "press RETURN/ENTER to continue"
2) The requirement to re-invoke mach has been removed
On first run, people now see a message telling them to press
RETURN/ENTER. Then the directory is created and whatever mach command
they requested to run is run.
While I was here, I also changed the permissions on the newly created
directory to match what it was a few lines above. Without, we're relying
on umask, which may be permissive. UNIX convention is to use a more
restrictive umask insider user directories. So this feels like the right
behavior.
MozReview-Commit-ID: IodN13xAJ8P
This adds a simple schema for build telemetry data. We can make it more
restrictive once we have a better feeling for what kind of data we want
to submit.
This also moves more common data about the system to the telemetry handler.
We leave psutil derivied information in the resource usage data as not every
system will have psutil installed.
MozReview-Commit-ID: CFRq1Ow6AOf
When I modified ehsan's patch in bug 904572 to switch to a build backend
instead of a mach command, I forgot to remove the line added to
mach_bootstrap.py to pick the mach command, which now doesn't exist.
Pymake's clinetoargv is very specific to pymake's use case, yet has been abused
as a replacement for shlex because shlex doesn't handle things properly for our
use cases.
Using pymake's clinetoargv, however, has shortcomings, and we're better off
importing its code in mozbuild, simplifying it a little, and using that
instead.
Plus, less dependencies on pymake will help kill it for good some day.
