This patch addresses the following issues which prevent building the
documentation on a Windows system:
1. check_jsdoc attempts to launch a file by the name of jsdoc using
subprocess.check_output(). But, the file jsdoc is a Bash script,
so it can't be launched this way on Windows. We should find the
correct file name to use instead (likely jsdoc.cmd).
2. The Windows CopyFile API will fail if the destination directory does
not exist. The procedure that creates the staging directory for Sphinx
attempts to use this API before any directories are created, so most calls
to it fail and most files don't get staged. These errors are ignored, so
the result is that Sphinx succeeds in generating a set of documentation
that is mostly empty.
3. Several instances where manually constructed paths that contain both back
and forward slashes are compared to or otherwise used with normalized paths,
resulting in incorrect behaviors.
Differential Revision: https://phabricator.services.mozilla.com/D54549
This was changing the project from "main" -> ("main",) which was causing the DocUp
task to upload to urls like:
"('main',)/62.0/_sources/..."
MozReview-Commit-ID: 1bL9nqiAEFE
We no longer store the docs under a project name (since all the docs are now
built using the root conf.py). This mean the name and version are only used for
packaging and uploading, which typically is only used in CI.
This allows us to lazy load the package name and version, so we only read the
conf.py when we need to.
MozReview-Commit-ID: DV5Jxrbskoh
Previously, running |mach doc <subtree>| would use whatever conf.py file
happened to live in the subtree. For example, running:
./mach doc tools/lint
Would build with tools/lint/docs/conf.py. This is bad because it means the
generated docs will look different from the docs that eventually will be
published to firefox-source-docs.mozilla.com.
This patch makes sure we always use tools/docs/conf.py for building, even when
only generating a subtree.
Furthermore, this sets things up such that when you modify a file, only the
subtree containing the modified file will be re-generated. This cuts down
rebuild times from ~2 minutes to ~20 seconds.
There is one caveat. When rebuilding a subtree, the index of other trees will
be overwritten in that particular subtree. I couldn't figure out anyway around
this. This tradeoff for *much* faster rebuild times seems worth it.
MozReview-Commit-ID: Ly88mvHKpo7
These two functions are typically only used by CI for packaging/uploading the
documentation. This is a minor re-organiztion for clarity.
MozReview-Commit-ID: 62UhQhSSkOs
We no longer store the docs under a project name (since all the docs are now
built using the root conf.py). This mean the name and version are only used for
packaging and uploading, which typically is only used in CI.
This allows us to lazy load the package name and version, so we only read the
conf.py when we need to.
MozReview-Commit-ID: DV5Jxrbskoh
Previously, running |mach doc <subtree>| would use whatever conf.py file
happened to live in the subtree. For example, running:
./mach doc tools/lint
Would build with tools/lint/docs/conf.py. This is bad because it means the
generated docs will look different from the docs that eventually will be
published to firefox-source-docs.mozilla.com.
This patch makes sure we always use tools/docs/conf.py for building, even when
only generating a subtree.
Furthermore, this sets things up such that when you modify a file, only the
subtree containing the modified file will be re-generated. This cuts down
rebuild times from ~2 minutes to ~20 seconds.
There is one caveat. When rebuilding a subtree, the index of other trees will
be overwritten in that particular subtree. I couldn't figure out anyway around
this. This tradeoff for *much* faster rebuild times seems worth it.
MozReview-Commit-ID: Ly88mvHKpo7
These two functions are typically only used by CI for packaging/uploading the
documentation. This is a minor re-organiztion for clarity.
MozReview-Commit-ID: 62UhQhSSkOs
This changes the default to opening a livereload webserver after doc generation
(as opposed to opening the index file). Any changes to the specified path will
result in a rebuild and refresh of the browser.
For example, if you run:
./mach doc tools/lint
The linting docs will be built, served and opened in a browser. Modifying any
file under 'tools/lint/docs' will refresh the browser with your changes.
To disable this behaviour and simply open the index file, you can pass in
'--no-serve'. The '--no-open' flag will continue to work (both with http and
the file system).
One caveat to this patch is that when generating the root docs (by running
|mach doc|), we don't watch all possible doc paths (just the root one under
'tools/docs/'). This will probably be fixed in the follow-up bug 1454640.
MozReview-Commit-ID: FQecuePM0zZ
This removes the ability to specify multiple doc paths at the same time with
|mach doc|. We will be changing the default from opening index files to serving
the documentation with a webserver. Supporting multiple doc roots would mean
spinning up multiple servers in different threads.
This would add a lot of complexity for a feature which I don't think is very
useful. It's very rare that one would need to edit more than one doc location
at the same time. And if this is ever needed, the developer can just build the
entire doctree (by running |mach doc|) or run |mach doc <path>| in multiple
different terminals.
MozReview-Commit-ID: GXEZJSgLpgF
Some requirements.txt are very large and result in a lot of package already
installed messages. Would be nice to hide this.
MozReview-Commit-ID: FQecuePM0zZ
Previously we weren't explicitly installing sphinx. Instead, the 'sphinx-js'
package had a dependency on 'sphinx<2.0'. This caused errors when sphinx
released their backwards incompatible version 1.7.
This patch pins sphinx==1.6.7 and adds all other dependencies to the same
requirements.txt (with hashes).
Upgrading to sphinx==1.7 will happen in a follow-up.
MozReview-Commit-ID: 28fKI7T4vfa
Previously, we uploaded the main Firefox tree docs to /.
In reality, there are multiple Sphinx projects in the repo. In
addition, it is sometimes desirable to access docs for an older
version of Firefox.
In this commit, we add support for specifying the S3 key prefix
for uploads. Then we change the upload code to upload to multiple
locations:
* <project>/latest (always)
* <project>/<version> (if a version is defined in the Sphinx config)
* / (for the main Sphinx docs project)
For the Firefox docs, ``version`` corresponds to a sanitized value from
``milestone.txt``. Currently, it resolves to ``57.0``.
While we're here, we add support for declaring an alternate project
name in the Sphinx conf.py file. If ``moz_project_name`` is defined,
we use that as the project name. For Firefox, we set it to ``main``.
This means our paths (local and uploaded) are now ``main`` instead of
``Mozilla_Source_Tree_Docs``. That's much more pleasant.
MozReview-Commit-ID: 8Gl6l2m6uU4
We now have an --upload flag to control whether upload is performed.
We don't inline it because we want to maintain a "firewall" between
regular docs and all the extra packages and imports needed for S3.
MozReview-Commit-ID: DVKhsS545gp
Instead of doing the file finding inside s3_upload(), the function now
takes the output of distribution_files().
The new code is much simpler.
MozReview-Commit-ID: 43i2Alvyu5i
The upload now uses MOZ_SCM_LEVEL to determine which secret and bucket to
upload to, so it can potentially run at any level.
This also modifies task descriptions to allow {level} in scopes, and updates
try syntax to allow `-j doc-upload` even though run-on-tasks says it doesn't
run on try by default.
MozReview-Commit-ID: Dm27TGPa7IM
This uses credentials stored in the Taskcluster secret service. The task should
only run on mozilla-central to avoid confusion between branches.
MozReview-Commit-ID: 31XfTg0sCht
This uses credentials stored in the Taskcluster secret service. The task should
only run on mozilla-central to avoid confusion between branches.
MozReview-Commit-ID: 31XfTg0sCht
Now, running |mach doc <path/to/project>| will generate the sphinx based docs of the project and open them
in the default browser. Mulitple doc paths can be supplied at a time. E.g:
./mach doc testing/mozbase
