Since we've upgraded to clang 9, clang-format changed and now uses dynamic libraries
for the clang tooling lib that it leverages.
Differential Revision: https://phabricator.services.mozilla.com/D47265
Otherwise we wind up with clownshoes like:
```
froydnj@hawkeye:/opt/build/froydnj/tmp/clang$ ls -l lib/libstdc++.so*
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6
-rwxr-xr-x 1 froydnj froydnj 11633720 Aug 6 20:44 lib/libstdc++.so.6.0.24
```
and have duplicate copies of shared libraries in our toolchain packages,
which are not exactly small.
Differential Revision: https://phabricator.services.mozilla.com/D41368
What this means is that the sources for clang/llvm are downloaded
separately from the toolchain build (which also means we finally only
download a given version of clang once for all platforms).
In turn, this means the build-clang.py script needs to start with an
existing llvm-project tree, and we choose to make build-clang.py expect
that it's run from the llvm-project root directory.
This also means we don't need to download git for the windows toolchain
task.
Differential Revision: https://phabricator.services.mozilla.com/D40402
Bug 1546136 wrapped subprocess execution output to capture cmake's, but
at the detriment of other output, and hiding everything unless an error
occurs.
So instead, we only capture the output when the called process is cmake,
and even when it is cmake, we don't pipe stderr at all (since we only
care about cmake's stdout) and we print out stdout as it comes in rather
than later. We then later check the output for hints at the more useful
cmake logs and dump them.
While here, add verbosity to ninja output (which gives the command
lines, rather than generic "Building foo.o" output).
Differential Revision: https://phabricator.services.mozilla.com/D40142
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
Our current OS X builds use `--target=x86_64-darwin11` (which
corresponds to OS X 10.7). This target is problematic for two reasons:
* We're actually targeting for OS X 10.9 (`MACOSX_DEPLOYMENT_TARGET`);
* It's slightly different from the default Rust target.
Let's address these problems in reverse order: differences from the Rust
target are bad, because the `--target` we provide to `clang` and the
Rust target find their way into LLVM bitcode files and the linker will
refuse to link together bitcode files that have incompatible targets.
Why are the two incompatible? The current `--target` doesn't have a
"vendor" in triple-speak, whereas the Rust one has "apple" as the
vendor (`x86_64-apple-darwin`) We therefore need to change the
`--target` we pass to `clang` to have a vendor of "apple".
This need is behind the {init,toolchain}.configure changes,
but it has ramifications elsewhere, because `clang` looks for
`--target`-prefixed build tools. So we have to change the `--target`
for cctools to get the right tool prefixes and we have to change the
`--target` for building clang ourselves so that *those* builds can find
the newly renamed cctools.
Once we've done, that's really enough; we don't *need to address the
first problem: While the `--target` might be `x86_64-apple-darwin11`,
both `clang` and `rustc` will dynamically choose the target triple that
eventually lands in LLVM bitcode files based on
`MACOSX_DEPLOYMENT_TARGET`, which we set in all builds. But the current
target is slightly misleading, and the cctools don't need to be prefixed
with a particular Darwin version, since they work for all Darwin
targets. Let's just drop the "11" from the `--target` and eliminate a
little bit of confusion.
Differential Revision: https://phabricator.services.mozilla.com/D31128
This change enables us to build compiler-rt and related
libraries (e.g. address sanitizer, etc.) for whatever targets we like,
assuming that we have an accessible sysroot for the target on the build
machine.
Depends on D28404
Differential Revision: https://phabricator.services.mozilla.com/D28405
CMake errors can be pretty opaque, especially if CMake is being run
inside the Ninja build process. Let's try to surface those errors to
make problems easier to debug.
Differential Revision: https://phabricator.services.mozilla.com/D28360
It seems better to set switches enabling runtime libraries and switches
enabling runtime libraries to build in different places, as future
changes might only enable runtime libraries for certain targets, but not
need any special switches for building.
Depends on D27594
Differential Revision: https://phabricator.services.mozilla.com/D27595
`android_targets` here is a dict, not a sequence, and while `iter` on a
dict object implicitly means `dict.iterkeys()`, that's not really
obvious. We should instead be explicit about what we're doing here.
Depends on D27593
Differential Revision: https://phabricator.services.mozilla.com/D27594
The setup for compiler-rt is currently done before the stage 2 build,
which happens to be the final stage for our android runtime libraries
build. But we may also want to build runtime libraries on 3-stage
bootstrap builds, in which case we don't want compiler-rt to be active
for the second stage. Move the setup into build_one_stage so that the
setup is controllable by is_final_stage, which is set in all the place
that we care about.
Differential Revision: https://phabricator.services.mozilla.com/D27592
As of clang 8, llvm-config doesn't return all flags clang was built
with, and omits some flags that do impact the libclang ABI,
-stdlib=libc++ being one of them (it might well be the only one).
Building clang with LLVM_ENABLE_LIBCXX=ON does build it with
-stdlib=libc++, but is unrelated to whether or not libc++ is built and
shipped with clang, which still happens without it.
So while versions older than clang 8 are not really affected, it doesn't
hurt to build clang without -stdlib=libc++ (especially when it
currently only applies to the clang used to cross build android with
PGO, not even the other android cross builds), in preparation for
switching to clang 8.
Differential Revision: https://phabricator.services.mozilla.com/D25031
We want our clang bootstrap to use the GCC headers we're building with,
not whatever sysroot it happens to find on the server we're building on.
The -gcc-toolchain argument we specify when building clang will also be
picked up by llvm-config, so we need to strip it out when building the
plugin. Otherwise, we will get peculiar failures about not being able to
find C++ header files.
Differential Revision: https://phabricator.services.mozilla.com/D22880
We want our clang bootstrap to use the GCC headers we're building with,
not whatever sysroot it happens to find on the server we're building on.
The -gcc-toolchain argument we specify when building clang will also be
picked up by llvm-config, so we need to strip it out when building the
plugin. Otherwise, we will get peculiar failures about not being able to
find C++ header files.
Depends on D22879
Differential Revision: https://phabricator.services.mozilla.com/D22880
This matches more closely cross toolchains prefixes (as can be seen in
e.g. media/libvpx/libvpx/README for x86_64-darwin*-gcc), and leaves it
to the build system to figure out the right --target to pass to clang on
its own.
Differential Revision: https://phabricator.services.mozilla.com/D14376
The cctools-port linker links against libraries from clang (for LTO),
which have different SONAMEs depending on the clang version. Which means
the linker needs to be used along the same version of clang it was built
against. Thus we also make it depend on linux64-clang-7.
But changing the dependency is not enough, cf. bug 1471905, so also
touch its build script, which it turns out, we need to do anyways
because llvm-dsymutil was renamed to dsymutil.
Relatedly, all toolchains that are built using cctools-port need to use
linux64-clang-7 too.
Building compiler-rt 7 with the OSX 10.11 SDK fails because of some
newer APIs being used in compiler-rt for xray, but this is not a feature
we use, so disable that.
Differential Revision: https://phabricator.services.mozilla.com/D6766
The cctools-port linker links against libraries from clang (for LTO),
which have different SONAMEs depending on the clang version. Which means
the linker needs to be used along the same version of clang it was built
against. Thus we also make it depend on linux64-clang-7.
But changing the dependency is not enough, cf. bug 1471905, so also
touch its build script, which it turns out, we need to do anyways
because llvm-dsymutil was renamed to dsymutil.
Relatedly, all toolchains that are built using cctools-port need to use
linux64-clang-7 too.
Building compiler-rt 7 with the OSX 10.11 SDK fails because of some
newer APIs being used in compiler-rt for xray, but this is not a feature
we use, so disable that.
Differential Revision: https://phabricator.services.mozilla.com/D6766
With libLLVM being a shared library exporting many symbols, all internal
calls using those symbols default to go through the PLT, which is
unnecessary (and costly) overhead. Using -Bsymbolic-functions makes
internal calls go directly to the right place without going through the
PLT.
Differential Revision: https://phabricator.services.mozilla.com/D7029
