You can still run them on a --disable-stylo build, as long as that works
(presumably not for long).
I think I haven't missed anything, but please double-check.
MozReview-Commit-ID: 3BIAEjgTLo5
The end goal here is to be able to use |mach try fuzzy <path>| with tests that
belong to a subsuite. To do this, we need a unique 'task_regex' value for each
subsuite so that we can map a test path back to a set of tasks.
This removes the TEST_FLAVORS dict (which was mostly just a redefinition of the
data in TEST_SUITES), and instead provides two new private mappings:
<flavor> -> suite definition
(<flavor>, <subsuite>) -> suite definition
To retrieve a suite definition given a flavor/subsuite, consumers can now call
get_suite_definition.
MozReview-Commit-ID: 2pe1v1IHUVy
This enables the syntax like:
./mach try fuzzy dom/indexedDB
This will open up the fzf interface like normal, except only tasks
that have tests under dom/indexedDB will be selectable (and there
will only be one chunk per configuration).
This can be combined with -q/--query like normal:
./mach try fuzzy dom/indexedDB -q "!pgo !cov !asan"
When the tasks get scheduled, only the tests under the specified
path(s) will run within the harness.
MozReview-Commit-ID: IHRXXi5mB4G
This sets the MOZHARNESS_TEST_PATHS environment variables for all tasks.
When specifying paths, this will cause many test tasks to only run the
tests under that directory as opposed to the normal default.
MozReview-Commit-ID: IHRXXi5mB4G
This changes templates so they return an entire context dict instead of
only returning context based on their name. For example, now the 'path'
template can set context for 'env'.
A side effect of this is that there is no longer a 1-to-1 mapping of templates
in tryselect and taskgraph.
MozReview-Commit-ID: IHRXXi5mB4G
This makes use of pytest's generation feature. To add a new
template test, just add a new entry containing the input and
expected output to the dict in test_templates.py
MozReview-Commit-ID: 4qMefYHMjAp
This adds some basic documentation for |mach try| and its various subcommands.
This was a bit hastily made for the Austin all-hands, but at least provides a
place to link to and can be improved upon in the future.
MozReview-Commit-ID: 8N6LZO5kTlL
This speeds up taskgraph generation by ~6 seconds on my machine. Future
improvements are also planned for 'fast' mode.
MozReview-Commit-ID: CLORvLXuV8y
This is a new module that will provide a place to store some common
abstractions around the 'blessings' module. The main entrypoint is:
from mozterm import Terminal
term = Terminal()
If blessings is available, this will return a blessings.Terminal()
object. If it isn't available, or something went wrong on import,
this will return a NullTerminal() object, which is a drop-in
replacement that does no formatting.
MozReview-Commit-ID: 6c63svm4tM5
`mach try` pushes the repository containing the current directory. When this is
a comm-central checkout, the taskcluster configuration should also come from that
repository.
MozReview-Commit-ID: KWbNAe4jrHT
This allows rebuilding all selected tasks. This defines an upper limit of
20 rebuilds per push. If more are needed, developers can either change it
in code or push multiple times.
MozReview-Commit-ID: I0XtMP5yEEq
The code in |mach test| for test resolving, should get merged with the TestResolver
class in moztest.resolve. This way it can be shared with other modules and we'll
have a single canonical place for all our test resolving logic.
MozReview-Commit-ID: IHRXXi5mB4G
The TestMetadata and TestResolver classes aren't technically part of the build
system. The only connection is that they consume some build system output.
The next patch in this series is going to be merging in a bunch of other test
resolving logic from other parts of the tree. Moving this out first allows us
to keep that extra logic out of mozbuild.
MozReview-Commit-ID: 1eq4SjFVCyW
Currently the prompts don't make it clear enough that running fzf will mess with your
shell settings. This means users could install it without realizing, forget and get
confused later on.
Rather than try to address this, it's simpler to always skip the shell extensions as
|mach try fuzzy| doesn't need them anyway. The extensions are useful, but are better
installed via something like |mach bootstrap| which can be tackled in a separate bug.
MozReview-Commit-ID: 2kx7UGO5LJ0
We parse the output of several version control commands in |mach try|, yet
redirect stderr to stdout. This is causing issues for people whenever the
vcs outputs some kind of warning.
MozReview-Commit-ID: F0L56at0MYS
This allows subparsers more control over which sets of arguments they need to
implement. For example, it doesn't make sense for the 'empty' selector to
accept the preset arguments. Now it can opt-out of those and only implement
the 'push' arguments.
MozReview-Commit-ID: GOfjcFtlfDD
find_paths_and_tags() can instantiate a BuildReader to read moz.build
metadata. We switch `mach test` to call this to avoid some redundant
code.
To facilitate `mach try`'s use case, we add test flavors to the
result of find_paths_and_metadata() and change the function name to
reflect its more generic purpose.
MozReview-Commit-ID: Lcz05W8g2dt
This will make the return value more easily extensible and will help
consumers know what the data structures represent.
MozReview-Commit-ID: DaeYsqfMW37
This adds all the common arguments to |mach try empty|, including preset
arguments which will just be ignored if specified. The commit message can be
changed with:
./mach try empty -m DONTBUILD
MozReview-Commit-ID: 5RYfLBOIX1g
This was preventing us from running ./mach try fuzzy from a subdirectory of the
topsrcdir. This also fixes taskgraph/utils/verify.py to find the docs directory
based off of topsrcdir instead of cwd. This was needed as cwd was being set at
import time.
MozReview-Commit-ID: CgQqD6bQ5q4
Running |./mach try -j <job>| may fail if there are changes in the local
working copy because the command will attempt to provide test paths based
on those changes and subsequently require platforms to be specified on the
basis of those paths. This commit makes this auto-detection only run when
a particular option is passed so this doesn't interfere with the common
case of simply running a selected job on try.
MozReview-Commit-ID: F3RBgDAYi27
This will allow us to inspect Try pushes to see how many are using
`mach try`.
The commit message for `mach try fuzzy` already advertised its usage.
We tweak the syntax a little so it isn't redundant.
MozReview-Commit-ID: 6kmYRKp0ffF
This adds a new morph template for modifying a task's env and the corresponding glue
to specify it from |mach try fuzzy|. It can be used like:
./mach try fuzzy --env FOO=1 --env BAR=baz
This will simply set those environment variables in *all* tasks. We could add the
ability to only specify it for a subset of tasks in the future, but that seems like
a hard problem that probably isn't worth it.
MozReview-Commit-ID: C4sokv886PU
To use this, create a ~/.mozbuild/machrc file and add:
[try]
default = fuzzy
Now, running |mach try| without a subcommand will default to the fuzzy selector. The syntax selector can still be
run with |mach try syntax|.
MozReview-Commit-ID: Ai6f8cW3Swq
This fixes a regression from bug 1384593. I was going to add a test for this, but it would be
a better use of time to fix bug 1185599 and add a test there. We already have vcs tests in other
parts of the tree so consolidating them into a single module is likely easier than standing up
another one-off vcs tester.
MozReview-Commit-ID: E51Tb1qC9Wb
This also prints the last known query used in the commit message. This won't be accurate as
users can delete + enter multiple queries in a single session, but will give users a slightly
better idea of what was scheduled by glancing at the commit message.
MozReview-Commit-ID: 93FbEmMvd9t
The main motivation behind this change is to make testing easier, so e.g:
./mach try fuzzy --no-push
and
./mach try syntax --no-push
both work the same way.
MozReview-Commit-ID: LmjA3Kq7xKN
This uses the new templating mechanism in taskgraph to schedule artifact builds when using
|mach try fuzzy|. Like |mach try syntax|, this will automatically be enabled if
--enable-artifact-builds is set in the mozconfig. The --artifact/--no-artifact arguments
can be used to override the default.
MozReview-Commit-ID: J8HVZzOt4mX
This adds --full to switch to using the full_task_set instead of the target_task_set. The full set has
around ~7000 tasks while the target set for mozilla-central has ~2300, so I think leaving the target as
default makes sense (as it is the 95% use case).
This also adds the ability to specify a custom parameters.yml file via -p/--parameters. This just gets
forwarded to the taskgraph for generation.
MozReview-Commit-ID: Esjvkh1p0Yw
This try selector works as follows:
1. Generate target tasks (similar to ./mach taskgraph target)
2. Pipe all tasks to fzf (a fuzzy finding binary, this will be bootstrapped if necessary)
3. Allow user to make selection
4. Save selected tasks to 'try_task_config.json'. This is a new try scheduling
mechanism built into taskcluster (see bug 1380306).
5. Use `hg push-to-try` (or git-cinnabar) to push the added file to try. This
will use a temporary commit, so no trace of 'try_task_config.json' should be
left over after use.
If you get messages like STOP! No try syntax found, you need to update version-control-tools:
./mach mercurial-setup --update
MozReview-Commit-ID: 4xHwZ9fATLv
