Bug 1492037 - Build clang with LLVM as a shared library. r=froydnj

Doing so changes the size of the compressed toolchain archive from ~280M
to ~120M, and the decompressed size from ~1500M to ~675M. This will
reduce the overhead of decompression during builds.

As we ship llvm-symbolizer as part of ASan builds, we do need it to
still statically link against LLVM, which we do with a small patch.

With LLVM as a shared library, libLTO, which is used by cctools-port for
the linker, is dynamically linked to LLVM, and the cctools-port
configure script fails to link against libLTO. So we add a -rpath-link
to make it find the LLVM library. This happens to force a rebuild of
cctools-port, but for future cases where we might need a rebuild because
of some clang changes, we add a comment to ease the process, and avoid
a newer cctools-port taking the cache spot of an older one.

Ideally, mac cctools-port would need something similar, but it needs a
mac libLTO.dylib, which is not there anyways (and the mac cctools-port
thus already didn't support LTO).

Also, with LLVM built as a shared library, all its symbols are exported
with a LLVM_x.y version. Combined with -static-libstdc++ that is used
during the clang build, this causes problems (see
https://bugzilla.mozilla.org/show_bug.cgi?id=1492037#c7). But it turns
out things have evolved since -static-libstdc++ has been added to the
clang build script, and things work without now, so remove it (as well
as -static-libgcc).

Differential Revision: https://phabricator.services.mozilla.com/D6117

build/build-clang/build-clang.py

@@ -10,6 +10,7 @@ import subprocess
 import platform
 import json
 import argparse
+import fnmatch
 import glob
 import errno
 import re
@@ -222,6 +223,9 @@ def build_one_stage(cc, cxx, asm, ld, ar, ranlib, libtool,
         if is_windows():
             cmake_args.insert(-1, "-DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=ON")
             cmake_args.insert(-1, "-DLLVM_USE_CRT_RELEASE=MT")
+        else:
+            # libllvm as a shared library is not supported on Windows
+            cmake_args += ["-DLLVM_LINK_LLVM_DYLIB=ON"]
         if ranlib is not None:
             cmake_args += ["-DCMAKE_RANLIB=%s" % slashify_path(ranlib)]
         if libtool is not None:
@@ -366,11 +370,17 @@ def prune_final_dir_for_clang_tidy(final_dir):
 
     # Keep include/ intact.
 
-    # In lib/, only keep lib/clang/N.M.O/include.
+    # In lib/, only keep lib/clang/N.M.O/include and the LLVM shared library.
     re_ver_num = re.compile(r"^\d+\.\d+\.\d+$", re.I)
     for f in glob.glob("%s/lib/*" % final_dir):
-        if os.path.basename(f) != "clang":
-            delete(f)
+        name = os.path.basename(f)
+        if name == "clang":
+            continue
+        if is_darwin() and name == 'libLLVM.dylib':
+            continue
+        if is_linux() and fnmatch.fnmatch(name, 'libLLVM*.so'):
+            continue
+        delete(f)
     for f in glob.glob("%s/lib/clang/*" % final_dir):
         if re_ver_num.search(os.path.basename(f)) is None:
             delete(f)
@@ -600,11 +610,11 @@ if __name__ == "__main__":
         extra_asmflags = []
         extra_ldflags = []
     elif is_linux():
-        extra_cflags = ["-static-libgcc"]
-        extra_cxxflags = ["-static-libgcc", "-static-libstdc++"]
+        extra_cflags = []
+        extra_cxxflags = []
         extra_cflags2 = ["-fPIC"]
         # Silence clang's warnings about arguments not being used in compilation.
-        extra_cxxflags2 = ["-fPIC", '-Qunused-arguments', "-static-libstdc++"]
+        extra_cxxflags2 = ["-fPIC", '-Qunused-arguments']
         extra_asmflags = []
         extra_ldflags = []