There are a lot of choices and moving pieces in this commit. I elected
to include the mechanics and the target use case in the same commit so
that readers can compare and contrast the implementation and final
expression in one review window.
- Initially, I wanted to make the {AB_CD} substitutions in
LOCALIZED_FILES and not in LOCALIZED_GENERATED_FILES. However, I ran
into conceptual blockers doing this. Fundamentally, LOCALIZED_FILES
is FINAL_TARGET_FILES, and my use case should _not_ be putting files
anywhere near dist/bin. In addition, LOCALIZED_FILES
(FINAL_TARGET_FILES) is handled using manifests, which would need to
grow locale-aware functionality to handle this. That's not desirable.
In addition, if we use manifests, then we lose the powerful locality
of |mach build mobile/android{/base}| re-generating changed
locale-dependent resources. This is similar to how the build system
plumbs dist/idl manifest processing throughout the build: we're
repairing local workflows after moving work into a global process.
For these reasons, this doesn't support {AB_CD} in LOCALIZED_FILES.
- There is even another layer of complexity! There are two axes
involved with these files: AB_CD controls localization and the Make
target controls destination. For the record, it is:
regular builds - AB_CD unset
multi-locale builds - AB_CD set
single-locale repacks - AB_CD set
For the record, the existing logic (before any changes) is:
regular builds - Make target is `libs` in mobile/android/base/locales
multi-locale builds - Make target is `chrome-%` in mobile/android/base/locales
single-locale repacks - Make target is `libs` in mobile/android/base/locales
This commit adds targets for both destinations, and uses Make
chrome-%:: and libs:: magic to control what is invoked in the various
situations. Tricky!
- I added MERGE_RELATIVE_FILES in order to be able to follow-up this
patch with more patches that will get rid of
m/a/base/locales/{moz.build,Makefile.in} altogether, and fold this work
into m/a/base. As it stands, we're already reaching from
m/a/base/locales all the way out to
mobile/locales/.../region.properties, so the existing code doesn't
follow the layout expected between mozilla-central and
l10n-central/$(AB_CD). But that'll impedance will get worse as we
improve the build system dependencies, not better, so we should grow
support for localized resources that aren't exactly as expected.
- I chose to follow Python's syntax for string substitutions. I
would have preferred to mark files that should be localized with a
leading '%'... but I took that for filesystem absolute paths in
moz.build files already. I also considered @AB_CD@ to echo the
preprocessor, but didn't want to open the door to an expecation that
_all_ preprocessor DEFINEs will work in the way {AB_CD} does.
- The generate_*py script changes required a bit of a hack to "turn
off" locale dependent resources. This would have been nicer if we had
marked localized resources with '%'... but we didn't. See the
--fallback flag. The real reason this is needed is that we're doing
work which is more like the work of compare-locales (merging
locale-dependent resources) at build-time rather than repack time. I
don't know why that's the case -- probably when we (I) implemented it,
compare-locales and the whole l10n process was entirely opaque. It's
not worth changing it now, so we use this --fallback flag approach.
- I didn't get to tup support. This should gently fail without
breaking tup builds: any {AB_CD} substitutions just won't be
expanded. I haven't a clue how this should work in tup in the future
(or, more generally, how to make any sense of repacks without
declaring the full set of expected locales at configure time.)
- strings.xml can't be a LOCALIZED_PP_FILES, since we need to
customize the output location based on AB_rCD, and since we need a
little more flexibility than PP_FILES gives for our inputs.
MozReview-Commit-ID: MyfIkNSEzt
Starting with Rust 1.24, the default codegen-units limit is 16,
with jobserver control to avoid overprovisioning. Remove our
previous fixed limit of 4 threads for debug builds.
For release, retain codegen-units=1 to make sure we get the
most complete optimization results.
Thanks to Simon Sapin for the suggestion.
MozReview-Commit-ID: FmYF4DcmBvt
I choose to clean a bunch of ANDROID_* moz.build cruft here, too,
since it's just passing dependencies between moz.build and
Makefile.in. The replacement for all of this is to just use
GENERATED_FILES in moz.build, but it'll still take some work to get to
that. (Why does this stuff exist? GENERATED_FILES didn't exist and
was resisted when I built this stuff.)
MozReview-Commit-ID: D3GJqJNL0Ih
The last APK produced using the ANDROID_APK_* moz.build/Makefile.in
mechanism was Robocop, so we can get rid of these now.
MozReview-Commit-ID: 9b08ZvvOAoC
This formatter is useful for triaging paths when enabling new linters or
expanding existing ones. It works well with the -n/--no-filter option.
For example, if I wanted to look for candidates of new directories to enable
the codespell linter on, I could run:
./mach lint -l codespell -nf summary
This will print something like:
accessible: 429
dom: 142
layout: 15
testing: 53
etc..
If desired, you can also specify a depth by setting MOZLINT_SUMMARY_DEPTH. A
depth of 2 means results will be aggregated further down, e.g:
accesible/build: 129
accesible/ipc: 300
dom/indexedDB: 100
dom/workers: 42
etc..
The depth is always relative to the common path prefix of all results, so
running:
./mach lint -l codespell -nf python/mozbuild
Would expand all the directories under python/mozbuild (not topsrdir).
MozReview-Commit-ID: OiihLTpULA
Two things have changed. One, Brew's java package became Java 9,
which doesn't work for building on Android. Two, Brew's cask system
also changed, requiring some small updates.
In order to actually use the install java toolchain, we need to use
the --with-java-bin-path configure option, which required some small
tweaks to the suggested mozconfigs.
MozReview-Commit-ID: JlZpdqaOkp0
The invalid variable test for #if{,n}def was only checking that the
first character in the variable was alphanumeric or underscore, not
the other characters.
More generally, preprocessor instructions were also cut out such that
whitespaces before and after arguments were part of the arguments.
Subtly, some legitimate strings end with what, in ISO-8859-1, is
considered as whitespaces, and because the preprocessor largely works
on byte strings (str), and because the regexps are using re.U, those
characters (e.g. 0xa0) that can legitimately appear in byte strings of
UTF-8 encoding, are treated as whitespaces. So we remove the re.U from
the instruction regexp, so that only plain ascii whitespaces only are
stripped out.
There's one place in layout/tools/reftest/manifest.jsm that was using
a broken pattern, making the test never true, which, once fixed, unveils
broken tests, so the branch that was never used is removed.
This allows linters to define a 'setup' method which will automatically be
called by |mach lint| before running the linter. Users can also explicitly run
these methods (without doing any actual linting) by running |mach lint --setup|.
MozReview-Commit-ID: 74aY1pfsaX1
This is a new issue that gets linted with flake8 3.5.0. Basically you should
never use a blank except: statement.
This will catch all exceptions, including KeyboardInterrupt and SystemExit
(which is likely not intended). If a catch all is needed, use
`except: Exception`. If you *really* mean to also catch KeyboardInterrupt et
al, use `except: BaseException`.
Of course, being specific is often better than a catch all.
MozReview-Commit-ID: FKx80MLO4RN
Depending on the compiler you use when --enable-stdcxx-compat, the
compiler can know about different libstdc++.so libraries that are not
suitable for your target. This will manifest as an assertion in the
current libstdcxx.py file. And then, when you change the assertion to
actually print out useful information, you will see things like:
/bin/ld: skipping incompatible /builds/worker/workspace/build/src/clang/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.9.4/../../../libstdc++.so when searching for -lstdc++
/bin/ld: skipping incompatible /builds/worker/workspace/build/src/clang/bin/../lib/libstdc++.so when searching for -lstdc++
which libstdcxx.py misinterprets as candidates for libstdc++.so.
This patch attempts to remedy both situations by providing a more
informative error message when things go sideways and also filtering out
error messages from the linker. You could argue that perhaps
--enable-stdcxx-compat shouldn't be getting set for such builds, but
this change seems reasonable enough on its own.
