Commit Graph

2133 Commits

Author SHA1 Message Date
Chris Manchester
8fd2a8e93c Bug 1360360 - Package screenshot programs during artifact builds. r=mshal
MozReview-Commit-ID: ExYrFbObOrG
2017-04-27 14:03:19 -07:00
J. Ryan Stinnett
4a04812483 Bug 1359501 - Accept artifact builds from beta. r=chmanchester
MozReview-Commit-ID: 5a2NIi6U7n7
2017-04-25 12:12:16 -05:00
Tom Ritter
7f3e0b6468 Bug 1353541 Fix rustc in MinGW build r=froydnj,ted
rustc generates .lib files for its libraries when compiling for Windows
(even using MinGW on Linux). But MinGW expects .a files. So we add in
rust-specific prefix and suffixes so MinGW builds can find the libs that
rustc generates. (And the RUST_LIB- variables default to the same vales
as the LIB_ variables otherwise.)

MozReview-Commit-ID: ClsA0YuJaxh
2017-04-26 12:08:59 -05:00
Cameron McCormack
916c5f6589 Bug 1257451 - Make |mach run| use --debugger-args instead of --debugparams, like the mochitest/reftest commands do. r=gps
MozReview-Commit-ID: ChB9KjNKbHO
2017-04-21 13:17:31 +10:00
Mike Hommey
d51030340d Bug 1357323 - Remove support for gonk in the build system. r=gps
Everything depending on the widget being gonk can go away, as well as
everything depending on MOZ_AUDIO_CHANNEL_MANAGER, which was only
defined on gonk builds under b2g/ (which goes away in bug 1357326).
2017-04-18 16:56:09 +09:00
Mike Hommey
dab330c9a7 Bug 1357326 - Remove things relying on the b2g* MOZ_BUILD_APP in the build system. r=gps 2017-04-18 18:39:53 +09:00
Mike Hommey
559588e2bc Bug 1356937 - Avoid mach artifact toolchain failing to get taskcluster artifacts when they are not directly under public/. r=gps 2017-04-17 09:47:56 +09:00
Chris Manchester
f2477a98e4 Bug 1247567 - Include per-source flags in the CompileDB. r=glandium
MozReview-Commit-ID: Ivm4DeG8z94
2017-04-14 16:02:25 -07:00
Mike Hommey
c041d72727 Bug 1356976 - Don't make mach artifact toolchain return an error code when there is nothing to download. r=gps
There can be cases where there is simply nothing to download, especially
with the --for-job argument. So we just stop erroring out when nothing
was downloaded.

However, if the user explicitly requested a particular file(s) via the
command line and there is nothing to download, we still emit an error
code.
2017-04-17 16:02:25 +09:00
Mike Hommey
0829e42061 Bug 1356934 - Don't assume the path to the taskcluster config is relative to topsrcdir. r=gps 2017-04-17 09:05:13 +09:00
Alex Gaynor
478321d9d3 Bug 1294641 - whitelist reads from the .app directory in the macOS sandbox r=froydnj,haik
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.

r=haik,froydnj

MozReview-Commit-ID: KNvAoUs5Ati
2017-04-07 14:53:19 -04:00
Sebastian Hengst
155e26d497 Backed out changeset 225683fed1d1 (bug 1294641) for breaking crashtests, reftests and Windows builds. r=backout on a CLOSED TREE 2017-04-17 16:29:52 +02:00
Alex Gaynor
d7af07ecbe Bug 1294641 - whitelist reads from the .app directory in the macOS sandbox r=froydnj,haik
This patch does a few things:
a) Adds the resources location from the .app directory to the read whitelist
b) When it's a non-packaged build, mach run (and various mach tests) set an environment variable for the repo location which we allow reads from.

r=haik,froydnj

MozReview-Commit-ID: KNvAoUs5Ati
2017-04-07 14:53:19 -04:00
Mike Hommey
2ff2f6029e Bug 1356524 - Add a mach artifact toolchain option to get artifacts from taskcluster builds. r=gps
Now that we have automated build jobs that produce toolchains, we want
to avoid the burden of uploading them to tooltool and then update the
tooltool manifests. But we don't have build jobs for all the possible
toolchains, so we allow `mach artifact toolchain` to get a mix of
tooltool and taskcluster artifacts.

For taskcluster artifacts, we can give a list of job names (conveniently
automatically normalized to begin with 'toolchain-' and end with '/opt')
for which the artifacts will be downloaded, in place of any tooltool
package with the same name (if a tooltool manifest is given).

The taskcluster artifacts that we download are the ones matching the
contents of the tree the command is run with, per the resources declared
for the corresponding toolchain build job (in
taskcluster/ci/toolchain*.yml)

So for example, a linux64 build could call the following command:
  mach artifact toolchain --tooltool-manifest \
    browser/config/tooltool-manifests/linux64/releng.manifest \
    --from-build linux64-gcc

and get the right gcc corresponding to the build-gcc script in tree,
along with the other non-gcc files from the tooltool manifest.

Things are however planned to be even more convenient, but some commands
can already benefit from this form (even without a tooltool manifest).
See e.g. bug 1328454.
2017-04-14 11:34:53 +09:00
Mike Hommey
5f1224fd91 Bug 1356524 - Only add Authorization header when sending requests to the tooltool url. r=gps
We're going to potentially use the same download manager for tooltool
and taskcluster artifacts, and we don't want to send the tooltool
authentication header to the taskcluster requests.
2017-04-14 10:28:42 +09:00
Gregory Szorc
c3f0bf3ccd Bug 1334525 - Handle tuple types and WPT manifests correctly; r=chmanchester
beb43155b7a6 changed WPT items to be 3-tuples instead of 2-tuples.
This broke test_defaults_for_path().

MozReview-Commit-ID: 7M0RcQ7bOIU
2017-04-12 15:15:36 -07:00
Mike Hommey
a3fde2761b Bug 1356142 - Use mozbuild.util.mkdir instead of os.makedirs for the artifact cache directory. r=gps
At the same time, make the artifact cache directory not indexed, and
move the directory creation to the base classes that actually use it.
2017-04-13 16:35:20 +09:00
Mike Hommey
e3198c5052 Bug 1355731 - Add a mach command wrapping tooltool. r=gps
The ultimate goal is to have a generic command that pulls relevant
toolchains from either tooltool or taskcluster artifacts.

This introduces the command and makes it work to allow to wrap tooltool
in most places where it's used currently, with the ability to replace
tooltool_wrapper.sh as well.
2017-04-12 17:42:17 +09:00
Mike Hommey
ee7e64fee7 Bug 1355731 - For artifacts cache, use the file name from the url if it looks like a hash digest. r=gps 2017-04-12 16:26:20 +09:00
Mike Hommey
187e7f6ebc Bug 1355731 - Move tooltool to mozbuild.action. r=gps 2017-02-22 17:20:41 +09:00
Wes Kocher
74eee0ab05 Merge m-c to autoland, a=merge 2017-04-13 17:35:34 -07:00
Wes Kocher
e18b7d684a Merge inbound to central, a=merge 2017-04-13 17:24:01 -07:00
Mike Hommey
0921909d80 Bug 1356064 - Remove the mach artifact print-cache subcommand. r=nalexander 2017-04-13 09:50:57 +09:00
Mike Hommey
d6d63bd867 Bug 1356064 - Remove the mach artifact last subcommand. r=nalexander
It has been broken for a while (bug 1341205).
2017-04-13 09:47:27 +09:00
Gregory Szorc
9922a34b43 Bug 1355630 - Inline specialized version of relpath(); r=chmanchester
Profiling revealed that mozpath.relpath() accounted for a lot of CPU
time when operating on an input of ~42,000 paths.

Due to the nature of the paths we're operating on, we don't need the
full power of mozpath.relpath() here. Instead, we can implement a
specialized version that works given already normalized paths and the
knowledge that context paths must be ancestors of the current path
being examined.

This change drops execution time of a mach command feeding ~42,000
paths to this function from ~90s to ~24s. On an input with 9131 paths,
execution time dropped from ~8.8s to ~3.7s.

MozReview-Commit-ID: EGLiJa10Zj2
2017-04-11 16:46:55 -07:00
Gregory Szorc
852c4554dc Bug 1355630 - Memoize tests_defaults_for_path call; r=chmanchester
Calling self.test_defaults_for_path() from files_info() with tens
of thousands of paths resulted in a CPU explosion in various path
normalization functions. I don't think it was so much the complexity
of the operations as much as the volume.

For an input with 9131 elements, this reduces execution time of a
mach command from ~25.7s to ~8.8s. With ~42,000 inputs, execution time
drops from <it took too long and I gave up> to ~90s.

MozReview-Commit-ID: pjQQByi2Bc
2017-04-11 16:24:42 -07:00
Gregory Szorc
e8781978ff Bug 1355630 - Use a set for membership testing; r=chmanchester
With ~42,000 entries in relpaths, this change drops execution time
of this loop from ~23s to ~0.01s.

MozReview-Commit-ID: Afm245tjWUQ
2017-04-11 15:25:13 -07:00
Nathan Froyd
d58a810176 Bug 1353810 - add a --enable-rust-debug option; r=chmanchester
For people working on Rust code, compiling in debug mode (Cargo's "dev"
profile) is convenient: debug assertions are turned on, optimization is
turned off, and parallel compilation inside of rustc itself can be
used.  These things make the build faster and the debugging experience
more pleasant.

To obtain that currently, one needs to --enable-debug at the Gecko
toplevel, which turns on debug assertions for the entire browser, which
makes things run unreasonably slowly.  So it would be desirable to be
able to turn *off* debug mode for the entirety of the browser, but turn
on debug mode for the Rust code only.

Hence this added switch, --enable-rust-debug, which does what it
suggests and defaults to the value of --enable-debug.  For our own
sanity and because we judge it a non-existent use case, we do not
support --enable-debug --disable-rust-debug.
2017-04-12 21:49:25 -04:00
Petr Sumbera
3d759d1f3d Bug 1354510 - Firefox build should recognize Solaris. r=glandium 2017-04-11 00:01:32 -07:00
Chris Manchester
dd7cb62d32 Bug 1337986 - Dump symbols during the compile tier. r=ted
This commit moves symbol dumping to the compile tier, to be run via "syms"
targets. Tracking files are used for the sake of incremental builds, because
dump_syms may genearate multiple outputs whose paths are not known ahead of
time.

Minimal changes to symbolstore.py are made here. More extensive
simplifications will be made in a future commit on the basis of symbolstore.py
handling one file at a time.

MozReview-Commit-ID: 3mOP8A6Y7iM
2017-02-10 08:34:08 -08:00
Chris Manchester
4d3d7f49e9 Bug 1337986 - Generate "syms" targets for directories containing programs or shared libraries. r=glandium
This creates "syms" targets that depend on the corresponding "target" for
directories containing shared libraries or programs. These targets are added
to the main compile graph in automation, and can be invoked through a special
"symbols" target. A future commit will use these targets to dump symbols for
shared libraries and programs during the compile tier.

MozReview-Commit-ID: KLuvmqsK4Zj
2017-04-10 10:27:49 -07:00
Simon Sapin
f3649eba80 Bug 1352815 - "codegen-units = 4" for Rust code in debug mode r=froydnj
MozReview-Commit-ID: IIFVmijY4B3
2017-04-02 14:17:44 +02:00
Michael Layzell
c7a9c27962 Bug 1320179 - Part 2: Add the nserror rust crate and generate NS_ERROR codes, r=froydnj
MozReview-Commit-ID: FFU6WfEqev
2017-04-06 18:37:25 -04:00
Ralph Giles
7d7a07b3c6 Bug 1354221 - Remove dead code in _process_rust_library. r=froydnj
These variables are ignored in favour or reading the same values
from the passed in libdef object.

MozReview-Commit-ID: 8Xkkd68clNN
2017-04-06 10:59:46 -07:00
Ting-Yu Chou
f8fcde2d04 Bug 1333003 part 5 - Include ASan runtime dll in common.tests.zip. r=ted
MozReview-Commit-ID: AzPdxaWwJn4
2017-03-10 12:09:17 +08:00
Chris AtLee
24bb152a2d Bug 1351071: Get rid of pre-generated startup cache r=glandium
MozReview-Commit-ID: BcWcqEKwGBv
2017-04-04 09:26:25 -04:00
Justin Wood
3782f35120 Bug 1185666 - Move DMG unpack logic to a python script, support linux. r=mshal
MozReview-Commit-ID: inKT2BWof4
2017-03-28 21:46:19 -04:00
Mike Hommey
7a0cf9b94e Bug 1343718 - Limit artifact downloads by size rather than by number. r=chmanchester
This adds a unit test for the expected behavior, and adds the modules
from mach_bootstrap.py that are missing in the virtualenv for the test
to run.
2017-04-12 16:17:52 +09:00
Mike Hommey
17549a9462 Bug 1343718 - Don't display download progress when we don't know the download size in advance. r=chmanchester
In most cases, the HTTP response for the download will contain the
content-length, but if some error happens (e.g. authentication error),
there might not be one, and the download fails with a TypeError for a
division by None, instead of failing with a more friendly error message
about the HTTP error.
2017-03-01 11:16:27 +09:00
Mike Hommey
dea1eb82f6 Bug 1343718 - Avoid requiring an objdir when simply importing mozbuild.artifacts. r=chmanchester 2017-03-01 11:15:15 +09:00
Chris Manchester
cae182e6aa Bug 1349640 - Upload a mapping for headers in dist/include for the benefit of code coverage builds. r=mshal
MozReview-Commit-ID: 5q9I5S1QOt9
2017-03-29 11:43:16 -07:00
Mike Shal
2198612479 Bug 1347576 - Add a 'mach repackage' command, with OSX dmg support; r=chmanchester
This is the initial support of 'mach repackage', which can take an
existing tarball and create a DMG on either an OSX host or on a Linux
host with cross-OSX tools. Configure is needed in order to find the
tools necessary to create the DMG. On a Linux cross-compiled environment
with tooltool, this can be as simple as:

export MKFSHFS=$topsrcdir/hfsplus-tools/newfs_hfs
export DMG_TOOL=$topsrcdir/dmg/dmg
export HFS_TOOL=$topsrcdir/dmg/hfsplus
ac_add_options --disable-compile-environment

MozReview-Commit-ID: 6t2rlXpwUvu
2017-03-10 18:36:23 -05:00
Dan Minor
fa7422512c Bug 964133 - Add mach command for webrtc.org unit tests; r=chmanchester
MozReview-Commit-ID: HJ1eNdaqIQM
2017-01-26 13:23:44 -05:00
Wes Kocher
3095b422dc Backed out 7 changesets (bug 1333003) for windows asan failures a=backout
Backed out changeset 3d2b2eeda8d3 (bug 1333003)
Backed out changeset 400d409ba4ca (bug 1333003)
Backed out changeset 1ba027abdfc9 (bug 1333003)
Backed out changeset 70114135bd8c (bug 1333003)
Backed out changeset 5715b15e33c0 (bug 1333003)
Backed out changeset 375e952bd738 (bug 1333003)
Backed out changeset d5d4112599f2 (bug 1333003)

MozReview-Commit-ID: DZUHJTdjX7V
2017-03-23 11:01:44 -07:00
Carsten "Tomcat" Book
97bcba5d10 Merge mozilla-central to autoland 2017-03-23 13:59:20 +01:00
Ting-Yu Chou
9fb02d37da Bug 1333003 part 5 - Include ASan runtime dll in common.tests.zip. r=ted
MozReview-Commit-ID: AzPdxaWwJn4
2017-03-10 12:09:17 +08:00
Steve Fink
1bd38ff6a5 Bug 1330746 - Disable terminal-style output when running within emacs, r=qdot
MozReview-Commit-ID: CgjgL91alUa
2017-03-22 14:03:53 -07:00
Carsten "Tomcat" Book
13c33830e3 Backed out changeset 42cf5ddabc8a (bug 1333003) 2017-03-23 10:38:09 +01:00
Ting-Yu Chou
9ce8f712c2 Bug 1333003 part 5 - Include ASan runtime dll in common.tests.zip. r=ted
MozReview-Commit-ID: AzPdxaWwJn4
2017-03-10 12:09:17 +08:00
Edmund Wong
503acfebbf Bug 1345422 - Do not hardcode the sfx package and installer path. r=mshal 2017-03-22 08:52:35 +08:00