Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
With --chunk-by-runtime enabled, test runtime data collected from automation is used to try and make all chunks take the same amount of time. So far only data for mochitest browser-chrome is added.
Add a `tags` attribute to a test or DEFAULT section in a manifest:
[test_foo]
tags = foo
Then run all tests with a given tag by passing in `--tag foo` to a supported test harness. So far mochitest, xpcshell and marionette are supported.
With this chunking strategy, the runtimes of tests are taken into account, such that each chunk
takes roughly the same amount of time to finish. Tests belonging to the same manifest will not get
split up.
The algorithm works by sorting every manifest from slowest to fastest. Each manifest is popped off
and its tests are added to the fastest chunk to date until no manifests are left. Total runtimes of
the chunks are re-calculated after every addition.
A filter is a callable that accepts an iterable of tests and a dictionary of values (e.g mozinfo.info) and returns an iterable of tests. Note filtering can mean modifying tests in addition to removing them. For example, this implements a "timeout-if" tag in the manifest:
from manifestparser import expression
import mozinfo
def timeout_if(tests, values):
for test in tests:
if 'timeout-if' in test:
timeout, condition = test['timeout-if'].split(',', 1)
if expression.parse(condition, **values):
test['timeout'] = timeout
yield test
tests = mp.active_tests(filters=[timeout_if], **mozinfo.info)
The path of the including manifest is included in test objects whenever possible to aide identification of tests in cases the same test file is included by multiple manifests.
