The only tricky piece here is that the resulting toolchain archive is
private, and uses a newly allocated Task Cluster scope
(queue:get-artifact:project/gecko/android-sdk/*) to restrict access to
the archive. All SCM levels (1, 2, 3) have been given the new scope:
see https://tools.taskcluster.net/auth/roles/moz-tree:level:1 and
friends.
MozReview-Commit-ID: CcDqDOHODpe
The only tricky piece here is that the resulting toolchain archive is
private, and uses a newly allocated Task Cluster scope
(queue:get-artifact:project/gecko/android-sdk/*) to restrict access to
the archive. All SCM levels (1, 2, 3) have been given the new scope:
see https://tools.taskcluster.net/auth/roles/moz-tree:level:1 and
friends.
MozReview-Commit-ID: CcDqDOHODpe
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
I also snuck in some last-minute assertions and minor fixes into this patch:
- don't stop reporting for a callee if we've seen it already (or rather, make the reachable set local to a root rather than global to all roots). This slows down runs with hundreds of hazards, but results in every problematic root being reported, for a more accurate count.
- annotate away some thread assertions
- special-case annotation for bug 1400435 since it's a whole family of hazards
CentOS 6 is pinned to glibc 2.12, but newer Android build-tools (like
aapt) require glibc 2.14. It's not possible to safely upgrade CentOS
6 distributions to glibc 2.14.
CentOS 7 is pinned to glibc 2.17, which is new enough for newer
Android build-tools. However, I had great difficulty bringing forward
our existing centos:6 Docker image to centos:7. In particular,
installing recent enough Mercurial, git, Python, and pip versions was
difficult enough that I elected to not pursue this approach.
Instead, I've elected to follow glandium's suggestion from
https://bugzilla.mozilla.org/show_bug.cgi?id=1370119#c5: base on
Debian with snapshots.debian.org for reproducibility.
The most significant changes here:
- using Debian's snapshots repository
- using Python and related tools provided by Debian and baked into the
build image
- using the JDK and JRE provided by Debian and baked into the build
image, rather than versions from tooltool (or eventually a toolchain
build)
Moving the builds over to use this image will follow in the patches
ahead.
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: 554IPMRWgzK
We're about to ban files in Docker volumes so they behave almost
identically to caches (which start empty).
We move the install of nexus.xml from Docker image time to
task time. This also means that changes to nexus.xml don't result
in having to rebuild the Docker image.
MozReview-Commit-ID: JIjeJN4mt2
The old process ran "before" and "after" steps as root. The
mozharness script doesn't run as root, which required some small
changes to not run Sonatype Nexus as root. Everything else is a
straight-forward move of the scripts out of the `android-gradle-build`
image and into `taskcluster/scripts`.
MozReview-Commit-ID: CqnNI33OKmb
Allow an extra heap write hazard introduced by enabling stylo
in default builds until it can be addressed. See bug 1384625.
MozReview-Commit-ID: 2N3z6FVHa0G
Bug 1374940 adds a MOZ_TOOLCHAINS environment variable with a list of
path@task-id strings, where task-id is corresponding to the (possibly
optimized) toolchain job, and path corresponding to the
toolchain-artifact defined for that toolchain job.
We want to use that to pull artifacts instead of tooltool packages.
This is a quick work around for not blocking the progress of Bug 1380133.
We should definely investigate the real root cause sooner than later.
MozReview-Commit-ID: 8X1FH6f2GyN
This is a quick work around for not blocking the progress of Bug 1380133.
We should definely investigate the real root cause sooner than later.
MozReview-Commit-ID: 8X1FH6f2GyN
We're about to make MOZ_AUTOMATION more strict about things like having
a source checkout.
The whole point of build-sm-package.sh is to verify that SpiderMonkey
can be built outside of Mozilla's source repo and automation from a
standalone package. Since the presence of MOZ_AUTOMATION can influence
so much behavior in the build system, unset it so that the job
tests a !Mozilla environment more accurately.
MozReview-Commit-ID: EMfyLKfY0uU
We're about to make MOZ_AUTOMATION more strict about things like having
a source checkout.
The whole point of build-sm-package.sh is to verify that SpiderMonkey
can be built outside of Mozilla's source repo and automation from a
standalone package. Since the presence of MOZ_AUTOMATION can influence
so much behavior in the build system, unset it so that the job
tests a !Mozilla environment more accurately.
MozReview-Commit-ID: EMfyLKfY0uU
Using /home/worker is the build directory has a 30% talos performance
loss, because test machines has a /home mount directory.
MozReview-Commit-ID: zehcGJrUQX
Hazard jobs use a specific tooltool-manifest field in their definition.
Since there is no post-processing happening on those definitions, and
since generalizing it would require adding the field to a bunch of
validation schemas, and the same code to various transforms, it's just
simpler to move to use environment variable definitions here too.
Eventually, tooltool manifests won't be necessary anyways, and those
environment variables will go away.
A quick fix for hazard bustage by increase the NUM_ALLOWED_WRITE_HAZARDS
from 3 to 7 is pushed in bug 1348173 comment 37. In this bug, we shall do
the actual fix and restore the NUM_ALLOWED_WRITE_HAZARDS.
The -moz-border-*-colors bindings trigger errors because they're using
outparams (nsStyleBorder) which further manipulate its member (mBorderColors)
which is a double raw pointers. Since we don't have the ability to
whitelist the indirect access to mBorderColors[x] list, we can only add
them to the ignoreContents for now.
We might be able to move these bindings to the whitelist of the above
treatAsSafeArgument function, if we could refactor mBorderColors to use
nsTArray directly.
MozReview-Commit-ID: 2cQz58K2A10
