Bug 1893288 - Update the docs for translations for 2024; r=translations-reviewers,nordzilla
Differential Revision: https://phabricator.services.mozilla.com/D208541
This commit is contained in:
Greg Tatum
@@ -1,8 +1,6 @@
|
||||
# Firefox Translations
|
||||
# Translations
|
||||
|
||||
Firefox Translations is a project initiative to give Firefox the ability to translate web content from one language to another language as first-class functionality in the browser.
|
||||
|
||||
This project is based initially on the [Firefox Translations WebExtension].
|
||||
Translations in Firefox are done through machine learning models that are downloaded and run locally on the user's device.
|
||||
|
||||
## Resources
|
||||
|
||||
@@ -13,5 +11,3 @@ This project is based initially on the [Firefox Translations WebExtension].
|
||||
|
||||
resources/*
|
||||
```
|
||||
|
||||
[Firefox Translations WebExtension]: https://github.com/mozilla/firefox-translations
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Overview
|
||||
# Overview of Translations
|
||||
|
||||
The following is a high-level overview of the technologies associated with Firefox Translations.
|
||||
The following is a high-level overview of the technologies associated with Translations.
|
||||
|
||||
- [Supported Platforms](#supported-platforms)
|
||||
- [Language Translations](#language-translation)
|
||||
@@ -11,7 +11,7 @@ The following is a high-level overview of the technologies associated with Firef
|
||||
- [Technology](#technology-1)
|
||||
- [Models](#models-1)
|
||||
- [Remote Settings](#remote-settings)
|
||||
- [Enabling Firefox Translations](#enabling-firefox-translations)
|
||||
- [Enabling Translations](#enabling-translations)
|
||||
- [Translating Web Pages](#translating-web-pages)
|
||||
- [about:translations](#abouttranslations)
|
||||
|
||||
@@ -23,31 +23,31 @@ The following is a high-level overview of the technologies associated with Firef
|
||||
<input type="checkbox" style="pointer-events: none;"><b>iOS</b><br>
|
||||
|
||||
```{note}
|
||||
- Firefox Translations is available only on devices support [SSE4.1] due to required [SIMD] calculations in [WASM].
|
||||
- Translations is available only on devices support [SSE4.1] due to required [SIMD] calculations in [WASM].
|
||||
```
|
||||
|
||||
|
||||
---
|
||||
## Language Translation
|
||||
|
||||
Firefox Translations utilizes trained machine-learning models that run locally on client
|
||||
Translations utilizes trained machine-learning models that run locally on client
|
||||
architecture to translate web content from one language to another.
|
||||
|
||||
### Technology
|
||||
|
||||
Firefox Translations utilizes a [WASM] version of the [Bergamot] library to translate from
|
||||
Translations utilizes a [WASM] version of the [Bergamot] library to translate from
|
||||
one language to another. [Bergamot] is powered by [Marian].
|
||||
|
||||
### Models
|
||||
|
||||
[Bergamot] translation models are single-direction, one-to-one models trained to translate from one language
|
||||
to one other language (e.g. **`en ⟶ es`**). When Firefox Translations determines a source language and a target language
|
||||
to one other language (e.g. **`en ⟶ es`**). When Translations determines a source language and a target language
|
||||
it utilizes a model specific to this language pair to translate from one to the other.
|
||||
|
||||
### Pivot Translations
|
||||
|
||||
In the event that there is no model to translate directly from a source language and a target language,
|
||||
Firefox Translations will attempt to satisfy a transitive translation path and will perform a multi-step
|
||||
Translations will attempt to satisfy a transitive translation path and will perform a multi-step
|
||||
translation from the source language to the target language.
|
||||
|
||||
|
||||
@@ -64,23 +64,23 @@ translation from the source language to the target language.
|
||||
> <input type="checkbox" style="pointer-events: none;" checked><b>`es ⟶ en ⟶ fr`</b><br>
|
||||
|
||||
In this example, no direct model exists for **`es ⟶ fr`**, but a transitive dependency is satisfied by the two
|
||||
models for **`es ⟶ en`** and **`en ⟶ fr`**. Firefox Translations will pivot on the **`en`** language by first
|
||||
models for **`es ⟶ en`** and **`en ⟶ fr`**. Translations will pivot on the **`en`** language by first
|
||||
translating from **`es`** to **`en`** and then from **`en`** to **`fr`**.
|
||||
```
|
||||
```{note}
|
||||
- Firefox Translations will not pivot more than once.
|
||||
- Translations will not pivot more than once.
|
||||
- At present, only **`en`** is used as a pivot language.
|
||||
```
|
||||
|
||||
---
|
||||
## Language Identification
|
||||
|
||||
Firefox Translations utilizes trained machine-learning models that run locally on client
|
||||
Translations utilizes trained machine-learning models that run locally on client
|
||||
architecture to identify content as being written in a detected language.
|
||||
|
||||
### Technology
|
||||
|
||||
Firefox Translations utilizes a [CLD2] language detector to identify in which language content is written.
|
||||
Translations utilizes a [CLD2] language detector to identify in which language content is written.
|
||||
|
||||
### Models
|
||||
|
||||
@@ -92,36 +92,20 @@ No models are currently used for language identification, since [CLD2] exists in
|
||||
Remote Settings is not currently used for language identification, since [CLD2] exists in the Firefox source tree.
|
||||
|
||||
---
|
||||
## Using Firefox Translations
|
||||
## Using Translations
|
||||
|
||||
The following documentation describes a high-level overview of using Firefox Translations.
|
||||
|
||||
```{note}
|
||||
- Firefox Translations is actively under development and is currently available only in [Firefox Nightly].
|
||||
```
|
||||
|
||||
### Enabling Firefox Translations
|
||||
|
||||
Firefox Translations functionality can be enabled by modifying the [translations preferences] in **`about:config`**.
|
||||
|
||||
These configurations are likely to change as the project develops which is why this documentation links to them
|
||||
in the source code rather than defining them.
|
||||
|
||||
At a time when the preferences are more stable, they can be documented here more clearly.
|
||||
The following documentation describes a high-level overview of using Translations.
|
||||
|
||||
### Translating Web Pages
|
||||
|
||||
Once Firefox Translations is enabled, Firefox will analyze each web page to determine if it is translatable
|
||||
via the available translations models.
|
||||
Firefox analyzes each web page to determine if it is translatable via the available translations models.
|
||||
|
||||
If the web page is translatable, then a translations icon will appear in the URL bar of the browser, allowing
|
||||
the user to initiate the available translation process.
|
||||
|
||||
### about:translations
|
||||
|
||||
When Firefox Translations is enabled, a page called **`about:translations`** becomes available in the browser.
|
||||
|
||||
This is a test page where there user can select a source language and a target language by typing content into
|
||||
**`about:translations`** is a test page where there user can select a source language and a target language by typing content into
|
||||
the source-language text box and seeing the translated text in the target-language text box.
|
||||
|
||||
```{note}
|
||||
@@ -136,7 +120,6 @@ It is, however, useful and fun, so it is documented here.
|
||||
<!-- Hyperlinks -->
|
||||
[Bergamot]: https://browser.mt/
|
||||
[CLD2]: https://github.com/CLD2Owners/cld2
|
||||
[Firefox Nightly]: https://www.mozilla.org/en-US/firefox/channel/desktop/
|
||||
[Marian]: https://aclanthology.org/P18-4020/
|
||||
[Remote Settings]: https://remote-settings.readthedocs.io/en/latest/
|
||||
[SIMD]: https://en.wikipedia.org/wiki/Single_instruction,_multiple_data
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Contributing
|
||||
# Contributing to Translations
|
||||
|
||||
The following content goes more in-depth than the [Overview](./01_overview.md) section
|
||||
to provide helpful information regarding contributing to Firefox Translations.
|
||||
to provide helpful information regarding contributing to Translations.
|
||||
|
||||
- [Source Code](#source-code)
|
||||
- [Architecture](#architecture)
|
||||
@@ -18,9 +18,10 @@ to provide helpful information regarding contributing to Firefox Translations.
|
||||
---
|
||||
## Source Code
|
||||
|
||||
The primary source code for Firefox Translations lives in the following directory:
|
||||
The primary source code for Translations lives in the following directories:
|
||||
|
||||
> **[toolkit/components/translations]**
|
||||
> **[browser/components/translations]**
|
||||
|
||||
---
|
||||
## Architecture
|
||||
@@ -43,10 +44,10 @@ The machine-learning models and [WASM] binaries are all hosted in Remote Setting
|
||||
|
||||
### Admin Dashboards
|
||||
|
||||
In order to get access to Firefox Translations content in the Remote Settings admin dashboards, you will need to request
|
||||
In order to get access to Translations content in the Remote Settings admin dashboards, you will need to request
|
||||
access in the Remote Settings component on [Bugzilla].
|
||||
|
||||
Once you have access to Firefox Translations content in Remote Settings, you will be able to view it in the admin dashboards:
|
||||
Once you have access to Translations content in Remote Settings, you will be able to view it in the admin dashboards:
|
||||
|
||||
**Dev**<br>
|
||||
> [https://settings.dev.mozaws.net/v0/admin](https://settings.dev.mozaws.net/v1/admin)
|
||||
@@ -70,11 +71,11 @@ the latest [remote-settings-devtools] Firefox extension.
|
||||
|
||||
### Versioning
|
||||
|
||||
Firefox Translations uses semantic versioning for all of its records via the **`version`** property.
|
||||
Translations uses semantic versioning for all of its records via the **`version`** property.
|
||||
|
||||
#### Non-breaking Changes
|
||||
|
||||
Firefox Translations code will always retrieve the maximum compatible version of each record from Remote Settings.
|
||||
Translations code will always retrieve the maximum compatible version of each record from Remote Settings.
|
||||
If two records exist with different versions, (e.g. **`1.0`** and **`1.1`**) then only the version **`1.1`** record
|
||||
will be considered.
|
||||
|
||||
@@ -84,16 +85,16 @@ in both backward-compatible and forward-compatible ways. These can be released t
|
||||
|
||||
#### Breaking Changes
|
||||
|
||||
Breaking changes for Firefox Translations are a bit more tricky. These are changes that make older-version records
|
||||
Breaking changes for Translations are a bit more tricky. These are changes that make older-version records
|
||||
incompatible with the current Firefox source code and/or [WASM] runtimes.
|
||||
|
||||
While a breaking change will result in a change of the semver number (e.g. **`1.1 ⟶ 2.0`**), this alone is not
|
||||
sufficient. Since Firefox Translations always attempts to use the maximum compatible version, only bumping this number
|
||||
sufficient. Since Translations always attempts to use the maximum compatible version, only bumping this number
|
||||
would result in older versions of Firefox attempting to use a newer-version record that is no longer compatible with the
|
||||
Firefox source code or [WASM] runtimes.
|
||||
|
||||
To handle these changes, Firefox Translations utilizes Remote Settings [Filter Expressions] to make certain records
|
||||
available to only particular releases of Firefox. This will allow Firefox Translations to make different sets of Remote Settings records available to different versions
|
||||
To handle these changes, Translations utilizes Remote Settings [Filter Expressions] to make certain records
|
||||
available to only particular releases of Firefox. This will allow Translations to make different sets of Remote Settings records available to different versions
|
||||
of Firefox.
|
||||
|
||||
```{admonition} Example
|
||||
@@ -118,7 +119,7 @@ This means that these records will only be available in Firefox versions greater
|
||||
|
||||
```
|
||||
|
||||
Tying breaking changes to releases in this way frees up Firefox Translations to make changes as large as entirely
|
||||
Tying breaking changes to releases in this way frees up Translations to make changes as large as entirely
|
||||
switching one third-party library for another in the compiled source code, while allowing older versions of Firefox to continue utilizing the old library and allowing newer versions of Firefox to utilize the new library.
|
||||
|
||||
---
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# The Bergamot Translator
|
||||
|
||||
The [Bergamot Translator](https://github.com/browsermt/bergamot-translator) is the translations engine used to power Firefox translations. The project configures a fork of [Marian NMT](https://marian-nmt.github.io/) that enables translations through a Wasm API.
|
||||
The [Bergamot Translator](https://github.com/browsermt/bergamot-translator) is the translations engine used to power Firefox's translations. The project configures a fork of [Marian NMT](https://marian-nmt.github.io/) that enables translations through a Wasm API.
|
||||
|
||||
Bergamot adds a few additional pieces of code on top of the Marian code, which includes HTML alignments (matching up source and target tags in a translation) and sentence iteration. It provides the [Wasm API](https://github.com/browsermt/bergamot-translator/tree/main/wasm) that Firefox uses in its own translation implementation. The Bergamot Translator uses a forked copy of the Marian NMT package in order to provide support for quantized translation models.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user