diff --git a/docs/config.yml b/docs/config.yml index 4dd59235227d..c9da2ea8608b 100644 --- a/docs/config.yml +++ b/docs/config.yml @@ -83,6 +83,7 @@ categories: - testing-rust-code release_doc: - update-infrastructure + - tools/update-verify l10n_doc: - intl - l10n diff --git a/tools/moz.build b/tools/moz.build index 0475e6384297..175097f642f7 100644 --- a/tools/moz.build +++ b/tools/moz.build @@ -65,6 +65,8 @@ SPHINX_TREES["code-coverage"] = "code-coverage/docs" SPHINX_TREES["profiler"] = "profiler/docs" +SPHINX_TREES["update-verify"] = "update-verify/docs" + with Files("tryselect/docs/**"): SCHEDULES.exclusive = ["docs"] diff --git a/tools/update-verify/README.md b/tools/update-verify/README.md deleted file mode 100644 index 14eb2a5f9aa9..000000000000 --- a/tools/update-verify/README.md +++ /dev/null @@ -1,118 +0,0 @@ -Mozilla Build Verification Scripts -================================== - -Contents --------- - -updates -> AUS and update verification - -l10n -> l10n vs. en-US verification - -common -> useful utility scripts - -Update Verification -------------------- - -`verify.sh` - -> Does a low-level check of all advertised MAR files. Expects to have a -> file named all-locales, but does not (yet) handle platform exceptions, so -> these should be removed from the locales file. -> -> Prints errors on both STDOUT and STDIN, the intention is to run the -> script with STDOUT redirected to an output log. If there is not output -> on the console and an exit code of 0 then all tests pass; otherwise one -> or more tests failed. -> -> Does the following: -> -> 1) download update.xml from AUS for a particular release -> 2) download the partial and full mar advertised -> 3) check that the partial and full match the advertised size and sha1sum -> 4) downloads the latest release, and an older release -> 5) applies MAR to the older release, and compares the two releases. -> -> Step 5 is repeated for both the complete and partial MAR. -> -> Expects to have an updates.cfg file, describing all releases to try updating -> from. - -Valid Platforms for AUS ------------------------ -- Linux_x86-gcc3 -- Darwin_Universal-gcc3 -- Linux_x86-gcc3 -- WINNT_x86-msvc -- Darwin_ppc-gcc3 - ---- -Running it locally -================== - -Requirements: -------------- - -- [Docker](https://docs.docker.com/get-docker/) -- [optional | Mac] zstd (`brew install zst`) - -Docker Image ------------- - -1. [Ship-it](https://shipit.mozilla-releng.net/recent) holds the latest builds. -1. Clicking on "Ship task" of latest build will open the task group in -Taskcluster. -1. On the "Name contains" lookup box, search for `release-update-verify-firefox` -and open a `update-verify` task -1. Make note of the `CHANNEL` under Payload. ie: `beta-localtest` -1. Click "See more" under Task Details and open the `docker-image-update-verify` -task. - -Download the image artifact from *docker-image-update-verify* task and load it -manually -``` -zstd -d image.tar.zst -docker image load -i image.tar -``` - -**OR** - -Load docker image using mach and a task -``` -# Replace TASK-ID with the ID of a docker-image-update-verify task -./mach taskcluster-load-image --task-id= -``` - -Update Verify Config --------------------- - -1. Open Taskcluster Task Group -1. Search for `update-verify-config` and open the task -1. Under Artifacts, download `update-verify.cfg` file - -Run Docker ----------- - -To run the container interactively: -> Replace `` with gecko repository path on local host
-> Replace `` with path to `update-verify.cfg` file on local host. -ie.: `~/Downloads/update-verify.cfg` -> Replace `` with value from `update-verify` task (Docker steps) - -``` -docker run \ - -it \ - --rm \ - -e CHANNEL=beta-localtest \ - -e MOZ_FETCHES_DIR=/builds/worker/fetches \ - -e MOZBUILD_STATE_PATH=/builds/worker/.mozbuild \ - -v :/builds/worker/fetches/update-verify.cfg - -v :/builds/worker/checkouts/gecko \ - -w /builds/worker/checkouts/gecko \ - update-verify -``` -> Note that `MOZ_FETCHES_DIR` here is different from what is used in production. - -`total-chunks` and `this-chunk` refer to the number of lines in `update-verify.cfg` -``` -./tools/update-verify/scripts/chunked-verify.sh --total-chunks=228 --this-chunk=4 -``` diff --git a/tools/update-verify/docs/index.rst b/tools/update-verify/docs/index.rst new file mode 100644 index 000000000000..7c7c74b83510 --- /dev/null +++ b/tools/update-verify/docs/index.rst @@ -0,0 +1,31 @@ +============= +Update Verify +============= +Update verify is test that runs before each Firefox Release (excluding Nightlies) are shipped. Its main purpose is to ensure that users who receive the release through an update MAR (`Mozilla ARchive `__) end up in the same place as a fresh install would get them. This helps us to ensure that `partial MARs `__ work in future updates, and that code signatures are valid regardless of how a user arrived at a new version. + +Note that the object under test here is the update MAR files that are prepared as part of a Firefox release -- and *nothing* else. Although certain other parts of the update system are currently used as part of these tests, the following things are specifically *not* considered under test: + +* The state of update rules in `Balrog `__ +* The `updater` binary itself +* The :ref:`Application Update ` component + +Both Balrog and the Linux updater binary are currently used as part of running update verify tests, but there are no guarantees they will continue to be in the future. + +------------ +How it works +------------ +At a high level, update verify simulates what happens when an update MAR is applied on a user machine. The result of that is compared against the full installer for the same version. If there are any differences (aside from a `small set of expected and known OK ones `__), the test fails. + +This test is run for all older builds (back to the `last watershed update `__ on all platforms. This means that we apply the same MARs over and over again, to different older versions of Firefox. + +All tests are performed on Linux, with the `updater` from the older version's Linux package. For example, when testing 128.0 -> 129.0 mac updates, we will apply a 129.0 MAR to a 128.0 mac build with the 128.0 Linux updater binary. + +With these details in mind, this is what the process looks like for testing a 128.0 -> 129.0 macOS en-US complete MAR: + +* Download the 128.0 en-US Linux tarball and unpack it +* Download the 128.0 en-US DMG and unpack it into the `source` directory +* Download the 129.0 en-US DMG and unpack it into the `target` directory +* Download the 129.0 en-US macOS complete MAR +* Run the `updater` binary from the unpacked Linux build to apply it to the `source` directory +* Diff the `source` and `target` directories +* Compare the result against `expected differences `__ to determine pass or fail