this patch:
* forces always passing '-r' when pushing after a migration run
* removes '--new-branch' from beta_to_release push. we don't generate new branches on releases anymore because of release promotion!
* has migration runs use a unique share dir for each repo.
this can be used as a stop gap. maybe even a more permanent solution so we don't need to add complexity to robustcheckout for an edgecase like this.
MozReview-Commit-ID: HXY5vDI1pIt
Enables flake8 linting! To start, only these directories are actually linted:
- python/mozlint
- tools/lint
To enable new directories, add them to the 'include' directive at the bottom of:
tools/lint/flake8.lint
Edit topsrcdir/.flake8 to modify global configuration. Add a new .flake8 to a
subdirectory to override the global. The current configuration is more or less
just the default and we should tweak it to our needs.
MozReview-Commit-ID: iXbToRhm3b
This adds flake8 dependencies. Note that ubuntu 16.04 repos include pip > 8.0
which has peep merged into it, so there's no need to install peep separately.
I also ran into a locale issue which was causing a UnicodeDecodeError anytime
python tried to print a unicode character. The "locale-gen/dpkg-reconfigure locales"
fix the problem.
MozReview-Commit-ID: DQhYV6Emklw
MozReview-Commit-ID: oDGtQ2Vnq4
Marionette harness has undergone lots of changes, so I decided to take current sources from M-C and do the copy + changes patches again with my own changes to get a Session harness.
MozReview-Commit-ID: 9e6HNAaF0Yo
In case of in-process restarts it can happen that the new process gets forked into a new process group.
When that happens we loose the capability to kill the process. To prevent a hang when joining the output
reader threads in wait(), we simply skip that call by passing-through the IO error.
This commit teaches the resource monitor in mozharness to emit
Perfherder data for system metrics and step times. This will
allow us to see when the timing or resource characteristics
of jobs in automation changes.
The recorded data includes overall CPU percent usage and I/O.
Each step has its time and CPU percent recorded. There is
certainly more data we could record. However, the immediate
goal of this change is to see if the data provides any benefit.
I'd rather start small and expand reporting once value from
this data is proved.
The wonkiest part of this patch is likely the mechanism to
define the Perfherder "test" names. We don't appear to have
an identifier in mozharness suitable for distinguishing
between job types. e.g. the "desktop_unittest.py" script is
responsible for running a few dozen jobs. So we invent code
for creating an identifier from the script config options.
I /think/ Treeherder will automatically assign the
project/branch, platform, and build type, which is why these
aren't included in the identifier.
MozReview-Commit-ID: HjhtXfxOvzJ
Having the latest schema available seems like a good thing. This is a
direct copy of schemas/performance-artifact.json from
https://github.com/mozilla/treeherder.git at commit
7bed1b22ceb01e3e71536fa1c4ecd14ddc87e803.
MozReview-Commit-ID: JQC4CeW6szM
Currently, only Talos accesses this file. An uncoming commit will add
a non-Talos consumer. Enable all mozharness consumers to access the
file by including it in the mozharness directory (previously it was
part of the Talos test archive).
MozReview-Commit-ID: ADlCj9E5BwC
Sometimes the IO completion port doesn't shutdown child processes. When this happens,
mozprocess will attempt to force kill the child processes manually. However, there is
a bug here which causes the OSError to get raised.
Although this fixes that bug, the original issue(s) which prevented the IOC port
from signaling shutdown remain and are still undiagnosed.
MozReview-Commit-ID: L3DQPW0Is5v
We can set MOZPROCESS_DEBUG to help debug windows process code. However on try
it is unreadable as there are multiple things using mozprocess, and each process
has multiple threads. It's impossible to tell which log message comes from where.
This improves the debug logs a bit by always specifying the PID and thread name.
There are a few other drive-by cleanups in this thread. The only one of note is
removing a python 2.5 only code path.
MozReview-Commit-ID: L3DQPW0Is5v
Enables flake8 linting! To start, only these directories are actually linted:
- python/mozlint
- tools/lint
To enable new directories, add them to the 'include' directive at the bottom of:
tools/lint/flake8.lint
Edit topsrcdir/.flake8 to modify global configuration. Add a new .flake8 to a
subdirectory to override the global. The current configuration is more or less
just the default and we should tweak it to our needs.
MozReview-Commit-ID: iXbToRhm3b
This adds flake8 dependencies. Note that ubuntu 16.04 repos include pip > 8.0
which has peep merged into it, so there's no need to install peep separately.
I also ran into a locale issue which was causing a UnicodeDecodeError anytime
python tried to print a unicode character. The "locale-gen/dpkg-reconfigure locales"
fix the problem.
MozReview-Commit-ID: DQhYV6Emklw
This removes the UnknowOperationError that was being thrown and returns the
property on the element that has been requested.
MozReview-Commit-ID: 2WCnBfdmit5
The ES job uses an image based on the official node image. While this was convenient
for eslint, it is a bit less convenient for other things. I want to use this image
for all lint jobs, and switching the base to a generic ubuntu image seems a bit cleaner.
I chose 16.04 for no good reason other than it is the most recent, and we might as well.
Node v4.4.5 and taskcluster-vcs have been uploaded to tooltool.
MozReview-Commit-ID: GngYeH98zO1
I'm not especially happy with this method, but the DMD tests are the only
other tests I can find doing anything like this, and I don't have a better
solution than doing it the same way.
MozReview-Commit-ID: JlclyKoUhRf
