Previously, we raised errors attempting to register a mach sub-command
if the parent command was not defined on the Registrar. This broke B2G
because its mach command Registrar imposes restrictions on which
commands it exposes. Commands there were not getting registered on the
Registrar, leading mach to give a false positive that the parent command
was never defined.
We change the verification logic to take present but unregistered
commands into consideration and to skip registering sub-commands if the
parent isn't present in the Registrar.
Now that we have proper moz.build objects for GENERATED_FILES, we can
add 'script' flags and 'args' flags in moz.build for select
GENERATED_FILES. We restrict 'args' to being filenames for ease of
implementing checks for file existence, and many (all?) of the examples
of file generation throughout the tree don't need arbitrary strings or
Python data.
This patch is mostly useful for being able to see these changes
independently of the major changes to GENERATED_FILES. We are going to
need proper moz.build objects for GENERATED_FILES when we add the
ability to define scripts and arguments for them, so we might as well do
that first.
With the previous changes, we can now tell Visual Studio about the
actual unified files, rather than the files that are #include'd in them.
I believe this is much closer to what Visual Studio wants to see, and
enables things like Intellisense to work properly.
CommonBackend is where the writing of the unified files belong, since
that's an operation common to all backends. A special hook into
subclasses is used to enable subclass-specific processing of
UnifiedSources.
UnifiedSources will be processed outside of consume_object, so we need
some way of accessing the backend file for an object outside of
consume_object as well.
UnifiedSources, not the recursivemake backend, should be responsible for
figuring out what unified files to generate, and what those unified
files should contain.
Generating the list of idl deps to generate an xpt from its dependency list
makes us give all _previous_ dependencies, inherited from the .deps makefiles.
This leads to removed files being listed on xpidl-process.py command line, and
the command subsequently failing.
Instead, use generated lists of idl dependencies. At the same time, lighten the
generated Makefile further by not emitting xpt dependencies on their containing
directory, and instead generating it from the $xpt_files list.
This brings down the Makefile size from 100k to 38k.
The previous debugger was setting a breakpoint in the mach dispatcher.
This required users to step into the main function to be called into.
Using pdb.runcall(), the debugger starts at the first line in the
executed command, which is far more useful.
Similar to the changes made for IPDL files, this commit moves all of the
non-makefile related logic for WebIDL files out of the recursive make
backend and into the common build backend. Derivative backends that
would like to do interesting things with WebIDL files now need to
implement _handle_webidl_build, which takes more parameters, but should
ideally require less duplication of logic.
After a bunch of tiny changes, we're finally ready to make real
progress. We can now move the grouping of the generated IDPL C++ files
and the actual writing of the unified files for them into the common
build backend. Derivative backends now only have to concern themselves
with adding the particular logic that compiling those files requires.
We'll need to write out unified files for multiple backends, not just
the recursive make one. Put that logic someplace where all build
backends can access it.
_add_unified_build_rules shouldn't be in the business of determining how
to group files into their unified files. That logic belongs in the
caller of _add_unified_build_rules. Once that's done, the logic for
determining how to group files can migrate out of the recursive make
bakend.
Nothing about writing unified files is specific to the recursive make
backend, and if we want to write the unified files for IPDL and WebIDL
files, we'll need this functionality available in the common build
backend.
RecursiveMakeBackend._group_unified_files doesn't contain any
functionality specific to the recursive make backend. We would also
like to move the unification of generated IPDL and WebIDL source files
into the common build backend. Moving _group_unified_files into the
common build backend would be the logical place for it, but the frontend
should also be able to handle unifying files so that backends don't have
to duplicate logic for UNIFIED_FILES. Therefore, we choose to move it
to mozbuild.util as its final resting place.
This handles:
list.0=A
list.1=B
list.sublist.0=C
so that
list=>[A, B]
list.sublist=>[C]
and
dict=default
dict.key1=A
dict.key2=B
so that
dict=>{key1:A, key2:B}
dict=>default
This does two things. First, it aligns the brew formula name
(AndroidNdk) with the brew file name (android-ndk.rb). Second, it
makes sure that we actually find the android-ndk.rb file. I think
|mach bootstrap| always worked, due to a felicity where the working
directory always contained android-ndk.rb; but |python bootstrap.py|
failed because android-ndk.rb was downloaded to a temporary location
that was not included in the |brew install| command.
Writing the unified files is another thing that will have to be moved
out of recursivemake.py eventually. And it doesn't belong inline amidst
makefile rules and variables. Move its logic to a separate function as
well.
_add_unified_build_rules does quite a lot of work besides adding
makefile rules and variables. The divying up of source files into
unified files is one part of that, so move it out into its own function.
When we eventually move that computation out of recursivemake.py, this
refactoring will make it easier to verify that's what we've done.
terminal.py had an ambiguous |import logging| that was importing
mach.logging from Sphinx. We fix it.
There was also a poorly formed link in the mach commands documentation.
We fix it.
Python API documentation requires the ability to import modules. So, we
set up a virtualenv in our Sphinx environment so module loading works.
This solution isn't perfect: a number of modules fail to import when run
under sphinx-build.
Previously, code for staging the Sphinx documentation from moz.build
metadata lived in a mach command and in the moztreedocs module. This
patch moves the invocation to the Sphinx extension.
When the code is part of the Sphinx extension, it will run when executed
with sphinx-build. This is a prerequisite to getting RTD working, since
sphinx-build is the only supported entrypoint for generating
documentation there.
With this patch, we can now invoke sphinx-build to build the
documentation. The `mach build-docs` command is no longer needed.
The recursivemake backend knows how to do several things with the IPDL
sources:
1) Determine the C++ sources that will be generated from given IPDL
sources.
2) Write out all the makefile rules and variables for said sources.
The first part isn't unique to the recursivemake backend; other backends
would eventually like to know what C++ sources come from IPDL source
files for easier cross-referencing purposes, etc. Let's take a first
cut at moving things into CommonBackend. (This may not be the best
interface, since it relies on consume_finished being invoked, and not
all backends call CommonBackend.consume_finished. Still, it's a start.)
