Previously, we uploaded the main Firefox tree docs to /.
In reality, there are multiple Sphinx projects in the repo. In
addition, it is sometimes desirable to access docs for an older
version of Firefox.
In this commit, we add support for specifying the S3 key prefix
for uploads. Then we change the upload code to upload to multiple
locations:
* <project>/latest (always)
* <project>/<version> (if a version is defined in the Sphinx config)
* / (for the main Sphinx docs project)
For the Firefox docs, ``version`` corresponds to a sanitized value from
``milestone.txt``. Currently, it resolves to ``57.0``.
While we're here, we add support for declaring an alternate project
name in the Sphinx conf.py file. If ``moz_project_name`` is defined,
we use that as the project name. For Firefox, we set it to ``main``.
This means our paths (local and uploaded) are now ``main`` instead of
``Mozilla_Source_Tree_Docs``. That's much more pleasant.
MozReview-Commit-ID: 8Gl6l2m6uU4
We now have an --upload flag to control whether upload is performed.
We don't inline it because we want to maintain a "firewall" between
regular docs and all the extra packages and imports needed for S3.
MozReview-Commit-ID: DVKhsS545gp
Instead of doing the file finding inside s3_upload(), the function now
takes the output of distribution_files().
The new code is much simpler.
MozReview-Commit-ID: 43i2Alvyu5i
The upload now uses MOZ_SCM_LEVEL to determine which secret and bucket to
upload to, so it can potentially run at any level.
This also modifies task descriptions to allow {level} in scopes, and updates
try syntax to allow `-j doc-upload` even though run-on-tasks says it doesn't
run on try by default.
MozReview-Commit-ID: Dm27TGPa7IM
This uses credentials stored in the Taskcluster secret service. The task should
only run on mozilla-central to avoid confusion between branches.
MozReview-Commit-ID: 31XfTg0sCht
This uses credentials stored in the Taskcluster secret service. The task should
only run on mozilla-central to avoid confusion between branches.
MozReview-Commit-ID: 31XfTg0sCht
Also removes InstallManifestNoSymlinks which can be more simply expressed by
passing link_policy='copy' to InstallManifest.populate_registry.
MozReview-Commit-ID: Bkjc2hIub4A
Back when the class was written, for the packaging code, it made sense
that the default was True. But now that it's used all over the place,
and that the vast majority of uses are with find_executables=False, it
makes more sense for that to be the default.
RTD is complaining that it can't find concurrent.futures. We solve this
problem on RTD by adjusting sys.path in the Sphinx config file to add
paths to the vendored packages.
Sadly, there isn't really an easy way to test this before landing. But
it should work.
MozReview-Commit-ID: FrsLrbbOQWt
Now, running |mach doc <path/to/project>| will generate the sphinx based docs of the project and open them
in the default browser. Mulitple doc paths can be supplied at a time. E.g:
./mach doc testing/mozbase
The Read the Docs environment doesn't have SHELL defined. This causes
mach.mixin.process's import to fail, as it insists on finding an active
shell. While we could fix mach.mixin.process to not raise if the
variable is not set, this is a bit more work and has wider build system
and mach implications. So we employ a quick hack instead.
DONTBUILD (NPOTB)
This removes ambiguity as to which modules are being imported, making
import slightly faster as Python doesn't need to test so many
directories for file presence.
All files should already be using absolute imports because mach command
modules aren't imported to the package they belong to: they instead
belong to the "mach" package. So relative imports shouldn't have been
used.
Python API documentation requires the ability to import modules. So, we
set up a virtualenv in our Sphinx environment so module loading works.
This solution isn't perfect: a number of modules fail to import when run
under sphinx-build.
Previously, code for staging the Sphinx documentation from moz.build
metadata lived in a mach command and in the moztreedocs module. This
patch moves the invocation to the Sphinx extension.
When the code is part of the Sphinx extension, it will run when executed
with sphinx-build. This is a prerequisite to getting RTD working, since
sphinx-build is the only supported entrypoint for generating
documentation there.
With this patch, we can now invoke sphinx-build to build the
documentation. The `mach build-docs` command is no longer needed.
In order for Sphinx documentation to work with Read The Docs, we need
the code for scanning the build config for Sphinx documentation to live
in an importable module. This patch moves some code from the
|build-docs| mach command into an importable module.
Ever since Sphinx variable reading operates at the AST level (bug
1071012), we don't technically need a fully configured environment in
order to generate the documentation!
This patch stubs out the config environment object with a fake one that
provides the only needed context to generate the Sphinx docs. This
allows us to build the Sphinx docs on a fresh clone of the tree with no
configure and with no object directory.
Read the Docs has a lovely Sphinx theme that beats the pants off the
built-in and default theme. And since it looks like MDN's Sphinx theme
is dead in the water, this gets us a nice UI win until the MDN theme
comes back from the dead.
The in-tree Sphinx docs have been broken since bug 1041941 because
processing moz.build files outside their context doesn't work.
Specifically, templates aren't loaded (because this information usually
comes from a parent moz.build file). A new execution mode is needed.
I tried to implement a proper execution mode. However, I kept running
into walls. While we should strive for a proper execution mode, this can
be a follow-up, tracked in bug 1058359.
This patch implements extraction of Sphinx variables from ast walking.
It is extremely low-level and definitely a one-off. But it solves the
problem at hand: |mach build-docs| will work after this patch is
applied.
