Commit Graph

4 Commits

Author SHA1 Message Date
Gregory Szorc
52b7f4901f Bug 1390693 - Use thread pool for S3 uploads; r=dustin
This reduces the time taken to upload the Firefox docs from ~30s to
~5s (per invocation).

MozReview-Commit-ID: DxOrvxvVn42
2017-08-24 12:38:01 -07:00
Gregory Szorc
42620b1f15 Bug 1390693 - Upload docs to project and version specific locations; r=dustin
Previously, we uploaded the main Firefox tree docs to /.

In reality, there are multiple Sphinx projects in the repo. In
addition, it is sometimes desirable to access docs for an older
version of Firefox.

In this commit, we add support for specifying the S3 key prefix
for uploads. Then we change the upload code to upload to multiple
locations:

* <project>/latest (always)
* <project>/<version> (if a version is defined in the Sphinx config)
* / (for the main Sphinx docs project)

For the Firefox docs, ``version`` corresponds to a sanitized value from
``milestone.txt``. Currently, it resolves to ``57.0``.

While we're here, we add support for declaring an alternate project
name in the Sphinx conf.py file. If ``moz_project_name`` is defined,
we use that as the project name. For Firefox, we set it to ``main``.
This means our paths (local and uploaded) are now ``main`` instead of
``Mozilla_Source_Tree_Docs``. That's much more pleasant.

MozReview-Commit-ID: 8Gl6l2m6uU4
2017-08-24 11:12:21 -07:00
Gregory Szorc
9441405713 Bug 1390693 - Use distribution_files() for finding files to upload; r=dustin
Instead of doing the file finding inside s3_upload(), the function now
takes the output of distribution_files().

The new code is much simpler.

MozReview-Commit-ID: 43i2Alvyu5i
2017-08-24 11:07:10 -07:00
Gregory Szorc
aa4cabec87 Bug 1390693 - Move S3 upload logic to standalone module; r=dustin
We try to keep mach_commands.py files as minimal as possible.

MozReview-Commit-ID: I4kvZtDjqGd
2017-08-24 10:49:28 -07:00