Bug 1946210: add firefox source docs for update verify r=releng-reviewers,ahal DONTBUILD
These docs replace the mostly dated README. The first half of that has been entirely rewritten to reflect the current state of things. The second half ("Running it locally") has been preserved in full.
Differential Revision: https://phabricator.services.mozilla.com/D236937
This commit is contained in:
Ben Hearsum
@@ -83,6 +83,7 @@ categories:
|
||||
- testing-rust-code
|
||||
release_doc:
|
||||
- update-infrastructure
|
||||
- tools/update-verify
|
||||
l10n_doc:
|
||||
- intl
|
||||
- l10n
|
||||
|
||||
@@ -65,6 +65,8 @@ SPHINX_TREES["code-coverage"] = "code-coverage/docs"
|
||||
|
||||
SPHINX_TREES["profiler"] = "profiler/docs"
|
||||
|
||||
SPHINX_TREES["update-verify"] = "update-verify/docs"
|
||||
|
||||
with Files("tryselect/docs/**"):
|
||||
SCHEDULES.exclusive = ["docs"]
|
||||
|
||||
|
||||
@@ -1,118 +0,0 @@
|
||||
Mozilla Build Verification Scripts
|
||||
==================================
|
||||
|
||||
Contents
|
||||
--------
|
||||
|
||||
updates -> AUS and update verification
|
||||
|
||||
l10n -> l10n vs. en-US verification
|
||||
|
||||
common -> useful utility scripts
|
||||
|
||||
Update Verification
|
||||
-------------------
|
||||
|
||||
`verify.sh`
|
||||
|
||||
> Does a low-level check of all advertised MAR files. Expects to have a
|
||||
> file named all-locales, but does not (yet) handle platform exceptions, so
|
||||
> these should be removed from the locales file.
|
||||
>
|
||||
> Prints errors on both STDOUT and STDIN, the intention is to run the
|
||||
> script with STDOUT redirected to an output log. If there is not output
|
||||
> on the console and an exit code of 0 then all tests pass; otherwise one
|
||||
> or more tests failed.
|
||||
>
|
||||
> Does the following:
|
||||
>
|
||||
> 1) download update.xml from AUS for a particular release
|
||||
> 2) download the partial and full mar advertised
|
||||
> 3) check that the partial and full match the advertised size and sha1sum
|
||||
> 4) downloads the latest release, and an older release
|
||||
> 5) applies MAR to the older release, and compares the two releases.
|
||||
>
|
||||
> Step 5 is repeated for both the complete and partial MAR.
|
||||
>
|
||||
> Expects to have an updates.cfg file, describing all releases to try updating
|
||||
> from.
|
||||
|
||||
Valid Platforms for AUS
|
||||
-----------------------
|
||||
- Linux_x86-gcc3
|
||||
- Darwin_Universal-gcc3
|
||||
- Linux_x86-gcc3
|
||||
- WINNT_x86-msvc
|
||||
- Darwin_ppc-gcc3
|
||||
|
||||
---
|
||||
Running it locally
|
||||
==================
|
||||
|
||||
Requirements:
|
||||
-------------
|
||||
|
||||
- [Docker](https://docs.docker.com/get-docker/)
|
||||
- [optional | Mac] zstd (`brew install zst`)
|
||||
|
||||
Docker Image
|
||||
------------
|
||||
|
||||
1. [Ship-it](https://shipit.mozilla-releng.net/recent) holds the latest builds.
|
||||
1. Clicking on "Ship task" of latest build will open the task group in
|
||||
Taskcluster.
|
||||
1. On the "Name contains" lookup box, search for `release-update-verify-firefox`
|
||||
and open a `update-verify` task
|
||||
1. Make note of the `CHANNEL` under Payload. ie: `beta-localtest`
|
||||
1. Click "See more" under Task Details and open the `docker-image-update-verify`
|
||||
task.
|
||||
|
||||
Download the image artifact from *docker-image-update-verify* task and load it
|
||||
manually
|
||||
```
|
||||
zstd -d image.tar.zst
|
||||
docker image load -i image.tar
|
||||
```
|
||||
|
||||
**OR**
|
||||
|
||||
Load docker image using mach and a task
|
||||
```
|
||||
# Replace TASK-ID with the ID of a docker-image-update-verify task
|
||||
./mach taskcluster-load-image --task-id=<TASK-ID>
|
||||
```
|
||||
|
||||
Update Verify Config
|
||||
--------------------
|
||||
|
||||
1. Open Taskcluster Task Group
|
||||
1. Search for `update-verify-config` and open the task
|
||||
1. Under Artifacts, download `update-verify.cfg` file
|
||||
|
||||
Run Docker
|
||||
----------
|
||||
|
||||
To run the container interactively:
|
||||
> Replace `<MOZ DIRECTORY>` with gecko repository path on local host <br />
|
||||
> Replace `<UVC PATH>` with path to `update-verify.cfg` file on local host.
|
||||
ie.: `~/Downloads/update-verify.cfg`
|
||||
> Replace `<CHANNEL>` with value from `update-verify` task (Docker steps)
|
||||
|
||||
```
|
||||
docker run \
|
||||
-it \
|
||||
--rm \
|
||||
-e CHANNEL=beta-localtest \
|
||||
-e MOZ_FETCHES_DIR=/builds/worker/fetches \
|
||||
-e MOZBUILD_STATE_PATH=/builds/worker/.mozbuild \
|
||||
-v <UVC PATH>:/builds/worker/fetches/update-verify.cfg
|
||||
-v <MOZ DIRECTORY>:/builds/worker/checkouts/gecko \
|
||||
-w /builds/worker/checkouts/gecko \
|
||||
update-verify
|
||||
```
|
||||
> Note that `MOZ_FETCHES_DIR` here is different from what is used in production.
|
||||
|
||||
`total-chunks` and `this-chunk` refer to the number of lines in `update-verify.cfg`
|
||||
```
|
||||
./tools/update-verify/scripts/chunked-verify.sh --total-chunks=228 --this-chunk=4
|
||||
```
|
||||
31
tools/update-verify/docs/index.rst
Normal file
31
tools/update-verify/docs/index.rst
Normal file
@@ -0,0 +1,31 @@
|
||||
=============
|
||||
Update Verify
|
||||
=============
|
||||
Update verify is test that runs before each Firefox Release (excluding Nightlies) are shipped. Its main purpose is to ensure that users who receive the release through an update MAR (`Mozilla ARchive <https://wiki.mozilla.org/Software_Update:MAR>`__) end up in the same place as a fresh install would get them. This helps us to ensure that `partial MARs <https://firefox-source-docs.mozilla.org/taskcluster/partials.html>`__ work in future updates, and that code signatures are valid regardless of how a user arrived at a new version.
|
||||
|
||||
Note that the object under test here is the update MAR files that are prepared as part of a Firefox release -- and *nothing* else. Although certain other parts of the update system are currently used as part of these tests, the following things are specifically *not* considered under test:
|
||||
|
||||
* The state of update rules in `Balrog <https://mozilla-balrog.readthedocs.io/en/latest/index.html>`__
|
||||
* The `updater` binary itself
|
||||
* The :ref:`Application Update <Application Update>` component
|
||||
|
||||
Both Balrog and the Linux updater binary are currently used as part of running update verify tests, but there are no guarantees they will continue to be in the future.
|
||||
|
||||
------------
|
||||
How it works
|
||||
------------
|
||||
At a high level, update verify simulates what happens when an update MAR is applied on a user machine. The result of that is compared against the full installer for the same version. If there are any differences (aside from a `small set of expected and known OK ones <https://searchfox.org/mozilla-central/source/tools/update-verify/release/compare-directories.py#25-99>`__), the test fails.
|
||||
|
||||
This test is run for all older builds (back to the `last watershed update <https://searchfox.org/mozilla-central/source/taskcluster/kinds/release-update-verify-config/kind.yml#53-59>`__ on all platforms. This means that we apply the same MARs over and over again, to different older versions of Firefox.
|
||||
|
||||
All tests are performed on Linux, with the `updater` from the older version's Linux package. For example, when testing 128.0 -> 129.0 mac updates, we will apply a 129.0 MAR to a 128.0 mac build with the 128.0 Linux updater binary.
|
||||
|
||||
With these details in mind, this is what the process looks like for testing a 128.0 -> 129.0 macOS en-US complete MAR:
|
||||
|
||||
* Download the 128.0 en-US Linux tarball and unpack it
|
||||
* Download the 128.0 en-US DMG and unpack it into the `source` directory
|
||||
* Download the 129.0 en-US DMG and unpack it into the `target` directory
|
||||
* Download the 129.0 en-US macOS complete MAR
|
||||
* Run the `updater` binary from the unpacked Linux build to apply it to the `source` directory
|
||||
* Diff the `source` and `target` directories
|
||||
* Compare the result against `expected differences <https://searchfox.org/mozilla-central/source/tools/update-verify/release/compare-directories.py#25-99>`__ to determine pass or fail
|
||||
Reference in New Issue
Block a user