Commit Graph

65 Commits

Author SHA1 Message Date
Nihanth Subramanya
db1e6bc9f8 Bug 1626083 - Rate-limit DoH heuristics runs. r=necko-reviewers,johannh,dragana
This prevents clients on volatile networks from running heuristics repeatedly,
which has two consequences: performance hit due to the DNS lookups performed by
heuristics, as well as flooding telemetry since we record an event each time.

With this patch, when a heuristics run is first triggered, we run it as usual
but start a timer in the background. Any runs triggered before the timer fires
are coalesced into a count. When the timer fires, if the count exceeded a limit,
we restart the timer and also the count. If the count was within the limit, we
run heuristics just once if it was triggered at least once while throttling.

To make this testable deterministically, I had to make a few subtle changes,
e.g. to not await calls to runHeuristics.

I also took the opportunity to replace BrowserTestUtils.waitForCondition with
the more canonical copy that lives in TestUtils, and to put the confirmationNS
pref into the prefs object so it doesn't need to be explicitly cleaned up.

Differential Revision: https://phabricator.services.mozilla.com/D110358
2021-04-12 16:48:42 +00:00
Nihanth Subramanya
8af2f914c0 Bug 1704158 - [DoH] Turn on provider steering by default. r=necko-reviewers,valentin
Differential Revision: https://phabricator.services.mozilla.com/D111464
2021-04-12 08:00:53 +00:00
Nihanth Subramanya
b93b678023 Bug 1703989 - Add unit test for DoH parental controls heuristic. r=necko-reviewers,valentin
Differential Revision: https://phabricator.services.mozilla.com/D111352
2021-04-09 11:25:58 +00:00
Valentin Gosu
086f34b47e Bug 1658278 - Split TRR's CONFIRM_TRYING into CONFIRM_TRYING_OK and CONFIRM_TRYING_FAILED r=necko-reviewers,nhnt11,dragana,preferences-reviewers
- Adds CONFIRM_TRYING_FAILED confirmation state. We use this state when we retry confirmation after confirmation fails.
- Rename CONFIRM_TRYING to CONFIRM_TRYING_OK. We use this state when we try confirmation but no confirmation failure has happened.
- Rename CONFIRM_INIT to CONFIRM_OFF. We use this state whenever there is an event that would disable TRR - such as a TRR mode change.
- Add CONFIRM_DISABLED confirmation state. We use this state in mode3 or when confirmationNS=="skip"
- To potentially allow us to have the same behaviour as after Bug 1689113, specifically the we might be able to report TRRService::Enabled = true when retrying and the state is CONFIRM_TRYING_FAILED we added `network.trr.attempt-when-retrying-confirmation`
- After a large number of TRR failures occurs, we immediately trigger another confirmation and go into CONFIRM_TRYING_OK. This allows us to cope with a temporary increase in network latency that is smaller than 6s.
- We no longer trigger confirmation for nsIRequest::TRR_FIRST_MODE when the resolver mode is not TRR_FIRST. This allows us to simplify the code.
- test_trr_proxy.js now calls trr_test_setup() after it sets up the pac script to avoid confirmation causing non-local connections in tests.
- Moves all the confirmation state handing into HandleConfirmationEvent

Differential Revision: https://phabricator.services.mozilla.com/D107666
2021-03-19 17:51:26 +00:00
Nihanth Subramanya
ccd9db9859 Bug 1693527 - When migrating old TRR mode, don't clear network.trr.mode if its value is 5. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D105625
2021-02-18 14:34:03 +00:00
Valentin Gosu
10a100023b Bug 1677501 - Add nsIX509CertDB.asyncHasThirdPartyRoots and use it in DoHHeuristics.jsm r=keeler,Gijs,nhnt11
This allows us to avoid calling any NSSCertificateDB methods on the main
thread or allocating memory for xpconnect wrappers of cert objects.

Differential Revision: https://phabricator.services.mozilla.com/D97970
2020-12-01 14:42:36 +00:00
Valentin Gosu
f9de71fb5a Bug 1674328 - Make DNS requests use TRR_DISABLED_MODE instead of using RESOLVE_DISABLE_TRR flag r=dragana,necko-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D95263
2020-11-02 10:30:31 +00:00
Ricky Stewart
31b8ca9716 Bug 1654103: Standardize on Black for Python code in mozilla-central.
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-26 18:34:53 +00:00
Bogdan Tara
612312a64c Backed out 10 changesets (bug 1654103, bug 1672023, bug 1518999) for PanZoomControllerTest.touchEventForResult gv-junit failures CLOSED TREE
Backed out changeset ff3fb0b4a512 (bug 1672023)
Backed out changeset e7834b600201 (bug 1654103)
Backed out changeset 807893ca8069 (bug 1518999)
Backed out changeset 13e6b92440e9 (bug 1518999)
Backed out changeset 8b2ac5a6c98a (bug 1518999)
Backed out changeset 575748295752 (bug 1518999)
Backed out changeset 65f07ce7b39b (bug 1518999)
Backed out changeset 4bb80556158d (bug 1518999)
Backed out changeset 8ac8461d7bd7 (bug 1518999)
Backed out changeset e8ba13ee17f5 (bug 1518999)
2020-10-24 03:36:18 +03:00
Ricky Stewart
065d2eb893 Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Make some ad-hoc manual updates to `testing/marionette/client/setup.py`, `testing/marionette/harness/setup.py`, and `testing/firefox-ui/harness/setup.py`, which have hard-coded regexes that break after the reformat.

5. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-23 20:40:42 +00:00
Dorel Luca
95b44c982f Backed out changeset 7558c8821a07 (bug 1654103) for multiple failures. CLOSED TREE 2020-10-22 03:51:06 +03:00
Ricky Stewart
43baed3c18 Bug 1654103: Standardize on Black for Python code in mozilla-central. r=remote-protocol-reviewers,marionette-reviewers,webdriver-reviewers,perftest-reviewers,devtools-backward-compat-reviewers,jgilbert,preferences-reviewers,sylvestre,maja_zf,webcompat-reviewers,denschub,ntim,whimboo,sparky
Allow-list all Python code in tree for use with the black linter, and re-format all code in-tree accordingly.

To produce this patch I did all of the following:

1. Make changes to tools/lint/black.yml to remove include: stanza and update list of source extensions.

2. Run ./mach lint --linter black --fix

3. Make some ad-hoc manual updates to python/mozbuild/mozbuild/test/configure/test_configure.py -- it has some hard-coded line numbers that the reformat breaks.

4. Add a set of exclusions to black.yml. These will be deleted in a follow-up bug (1672023).

# ignore-this-changeset

Differential Revision: https://phabricator.services.mozilla.com/D94045
2020-10-21 21:27:27 +00:00
Nihanth Subramanya
222a631782 Bug 1668104 - [DoH] Set clearModeOnShutdown pref value explicitly in tests. r=valentin
Depends on D91827

Differential Revision: https://phabricator.services.mozilla.com/D92747
2020-10-07 11:23:00 +00:00
Valentin Gosu
b231428366 Bug 1664874 - Do heuristics DNS lookups at the same time r=nhnt11
Differential Revision: https://phabricator.services.mozilla.com/D92431
2020-10-05 15:35:45 +00:00
Nihanth Subramanya
57e1b450bd Bug 1662430 - Allow a pref to toggle whether to clear doh-rollout.mode at shutdown. r=valentin,preferences-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D90856
2020-09-24 18:06:37 +00:00
Razvan Maries
c141744ed8 Backed out changeset bb81093f7e32 (bug 1662430) for perma failures on browser_rollback.js, browser_connection_dnsoverhttps.js and browser_policyOverride.js. CLOSED TREE 2020-09-24 20:57:28 +03:00
Nihanth Subramanya
e77ebbcea4 Bug 1662430 - Allow a pref to toggle whether to clear doh-rollout.mode at shutdown. r=valentin,preferences-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D90856
2020-09-24 12:09:52 +00:00
Dorel Luca
f6e46379cc Backed out changeset fccbbed6fc57 (bug 1662430) for Browser-chrome failures in preferences/tests/browser_connection_dnsoverhttps.js. CLOSED TREE 2020-09-24 15:04:39 +03:00
Nihanth Subramanya
2e9c8604a0 Bug 1662430 - Allow a pref to toggle whether to clear doh-rollout.mode at shutdown. r=valentin,preferences-reviewers
Differential Revision: https://phabricator.services.mozilla.com/D90856
2020-09-24 09:10:11 +00:00
Nihanth Subramanya
ab0ca1d5ce Bug 1666676 - Use fully-qualified names for all lookups performed by DoH heuristics. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D91091
2020-09-23 16:37:19 +00:00
Narcis Beleuzu
ddc7d5072a Backed out changeset 7d0db31be648 (bug 1666676) for bc failure on browser_cleanFlow.js . CLOSED TREE 2020-09-23 16:49:51 +03:00
Nihanth Subramanya
29d809f6f4 Bug 1664870 - Create a data access layer module for DoH config. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D90952
2020-09-23 07:38:05 +00:00
Nihanth Subramanya
82a49913fe Bug 1666676 - Use fully-qualified names for all lookups performed by DoH heuristics. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D91091
2020-09-23 06:36:10 +00:00
Mark Banner
15403a5af5 Bug 1666524 - Fix used of unknown idl property in browser_platformDetection.js. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D91019
2020-09-22 16:23:13 +00:00
Razvan Maries
cf070445d7 Backed out changeset 7e73d7a3ddd2 (bug 1662430) for perma failures on browser_connection_dnsoverhttps.js. CLOSED TREE 2020-09-22 19:19:03 +03:00
Nihanth Subramanya
72775bac07 Bug 1662430 - Allow a pref to toggle whether to clear doh-rollout.mode at shutdown. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D90856
2020-09-22 07:34:33 +00:00
Nihanth Subramanya
c8de569d8b Bug 1666263 - Clean up doh-rollout.profileCreationThreshold pref. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D90861
2020-09-21 14:23:31 +00:00
Valentin Gosu
76eebd4651 Bug 1657582 - Add nsIDNSAddrRecord interface r=necko-reviewers,geckoview-reviewers,snorp,mixedpuppy,dragana
This interface extends nsIDNSRecord and makes the DNS code more extensible
by allowing us to support more record types.
This change does require the consumer to be aware of the type they requested
and to QueryInterface to either nsIDNSAddrRecord for regular IP lookups,
or to nsIDNSByTypeRecord for other kinds of lookups.

Differential Revision: https://phabricator.services.mozilla.com/D86177
2020-08-12 10:49:39 +00:00
Valentin Gosu
a3c770dc4b Bug 1657582 - Add nsIDNSResolverInfo interface r=necko-reviewers,geckoview-reviewers,snorp,mixedpuppy,extension-reviewers,dragana
This patch adds the nsIDNSResolverInfo interface which is used to hold
information about the resolver to be used in a DNS resolution.

We use this to merge all of the *WithTRRServer resolve functions into one.
Passing a resolver info will use that object when appropriate. No resolver
info means that we default to using the system resolver, or the default TRR
resolver.

This patch also converts the RESOLVE_TYPE_* flags into a cenum and adds
the resolveType as a parameter to asyncResolve thus removing the need
to have asyncResolveByType methods.

Differential Revision: https://phabricator.services.mozilla.com/D86176
2020-08-12 08:25:14 +00:00
Bogdan Tara
d19650bd2d Backed out 2 changesets (bug 1657582) for test_DNSLookup.js failures CLOSED TREE
Backed out changeset 784122a5f5ab (bug 1657582)
Backed out changeset 0f17312b01ad (bug 1657582)
2020-08-12 11:18:06 +03:00
Valentin Gosu
ad5a386fd3 Bug 1657582 - Add nsIDNSAddrRecord interface r=necko-reviewers,geckoview-reviewers,snorp,mixedpuppy,dragana
This interface extends nsIDNSRecord and makes the DNS code more extensible
by allowing us to support more record types.
This change does require the consumer to be aware of the type they requested
and to QueryInterface to either nsIDNSAddrRecord for regular IP lookups,
or to nsIDNSByTypeRecord for other kinds of lookups.

Differential Revision: https://phabricator.services.mozilla.com/D86177
2020-08-12 01:35:10 +00:00
Valentin Gosu
adf8b12bbe Bug 1657582 - Add nsIDNSResolverInfo interface r=necko-reviewers,geckoview-reviewers,snorp,mixedpuppy,extension-reviewers,dragana
This patch adds the nsIDNSResolverInfo interface which is used to hold
information about the resolver to be used in a DNS resolution.

We use this to merge all of the *WithTRRServer resolve functions into one.
Passing a resolver info will use that object when appropriate. No resolver
info means that we default to using the system resolver, or the default TRR
resolver.

This patch also converts the RESOLVE_TYPE_* flags into a cenum and adds
the resolveType as a parameter to asyncResolve thus removing the need
to have asyncResolveByType methods.

Differential Revision: https://phabricator.services.mozilla.com/D86176
2020-08-12 01:00:39 +00:00
Valentin Gosu
82a7631e75 Bug 1657897 - Add test for platform indication DoH heuristic r=nhnt11
Differential Revision: https://phabricator.services.mozilla.com/D86371
2020-08-11 13:27:07 +00:00
Valentin Gosu
18f1bba6f8 Bug 1657897 - Convert TRR split horizon mitigations to a doh-rollout heuristic r=nhnt11
Differential Revision: https://phabricator.services.mozilla.com/D86366
2020-08-11 13:26:19 +00:00
Nihanth Subramanya
a4fe98a3bc Bug 1654520 - Concat the networkID to the clientID and SHA256 the result before including in telemetry. r=mossop
Differential Revision: https://phabricator.services.mozilla.com/D84859
2020-07-24 20:23:33 +00:00
Nihanth Subramanya
30d9f38cce Bug 1654520 - [DoH] Debounce heuristics; Include networkID and captiveState in heuristics telemetry event. r=dragana
Differential Revision: https://phabricator.services.mozilla.com/D84648
2020-07-24 20:23:33 +00:00
Nihanth Subramanya
31d1f2c017 Bug 1654714 - [DoH] Group heuristics results into three categories to reduce extra key count in telemetry. r=Gijs,tdsmith
Differential Revision: https://phabricator.services.mozilla.com/D84719
2020-07-24 20:22:25 +00:00
Nihanth Subramanya
d075d41102 Bug 1652510 - Revert DoH heuristics telemetry event category name to "doh". r=tdsmith,Gijs
Differential Revision: https://phabricator.services.mozilla.com/D83357
2020-07-13 23:42:27 +00:00
Kris Maglione
9bddbcfdd1 Bug 1649221: Update ChromeUtils.generateQI callers to pass strings. r=mccr8,remote-protocol-reviewers,marionette-reviewers,perftest-reviewers,webcompat-reviewers,geckoview-reviewers,preferences-reviewers,agi,whimboo,Bebe,twisniewski
Differential Revision: https://phabricator.services.mozilla.com/D81594
2020-07-10 23:58:28 +00:00
Nihanth Subramanya
7310ce733e Bug 1603779 - Part 4: Mass rename prefs in tests to remove DOH_ suffix. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D82094
2020-07-10 15:15:41 +00:00
Nihanth Subramanya
da1d78775e Bug 1603779 - Part 3: Update tests. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D82090
2020-07-10 15:15:02 +00:00
Nihanth Subramanya
5a66b9772e Bug 1603779 - Part 2: Move doh-rollout mochitests into browser/components/doh/test/. r=Gijs
Differential Revision: https://phabricator.services.mozilla.com/D82088
2020-07-10 15:14:16 +00:00
Nihanth Subramanya
1cc6dc4f57 Bug 1603779 - Part 1: Implement DoHController.jsm and DoHHeuristics.jsm and minimalize doh-rollout extension. r=valentin
Differential Revision: https://phabricator.services.mozilla.com/D78598
2020-07-10 15:13:48 +00:00
Kershaw Chang
7ad0dd19d4 Bug 1641496 - P6: Reenable some tests r=dragana,extension-reviewers,mixedpuppy
Differential Revision: https://phabricator.services.mozilla.com/D77277
2020-06-04 22:56:38 +00:00
Csoregi Natalia
77821df454 Backed out 7 changesets (bug 1641496) for failures on test_trr.js. CLOSED TREE
Backed out changeset 1c927e612dda (bug 1641496)
Backed out changeset 0f1ba77e3942 (bug 1641496)
Backed out changeset a626e86ce5c9 (bug 1641496)
Backed out changeset 196381953eac (bug 1641496)
Backed out changeset 70b0af678bfd (bug 1641496)
Backed out changeset bf41bd8ca89c (bug 1641496)
Backed out changeset 745da0e1cb80 (bug 1641496)
2020-06-05 01:39:19 +03:00
Kershaw Chang
8f7ebf8ee1 Bug 1641496 - P6: Reenable some tests r=dragana,extension-reviewers,mixedpuppy
Differential Revision: https://phabricator.services.mozilla.com/D77277
2020-06-04 16:53:23 +00:00
Kershaw Chang
90e7faad9f Bug 1602832 - P3: Skip failed tests r=dragana
Differential Revision: https://phabricator.services.mozilla.com/D67448
2020-05-26 14:30:26 +00:00
Butkovits Atila
34d69c0668 Backed out 6 changesets (bug 1602832) for causing perma failure at test_trr_case_sensitivity.js. CLOSED TREE
Backed out changeset 0e9c378df995 (bug 1602832)
Backed out changeset 534fedb3836e (bug 1602832)
Backed out changeset db012c05d64e (bug 1602832)
Backed out changeset ca0c207dca21 (bug 1602832)
Backed out changeset bea1f3aeea49 (bug 1602832)
Backed out changeset 2d54acd78124 (bug 1602832)
2020-05-26 13:09:07 +03:00
Kershaw Chang
75da29ed57 Bug 1602832 - P3: Skip failed tests r=dragana
Differential Revision: https://phabricator.services.mozilla.com/D67448
2020-05-26 08:35:32 +00:00
Coroiu Cristina
86104594c6 Backed out 5 changesets (bug 1602832) for browser-chrome failures at toolkit/mozapps/extensions/test/xpinstall/browser_doorhanger_installs.js on a CLOSED TREE
Backed out changeset 059a7f44d1a9 (bug 1602832)
Backed out changeset 2f3cc391b48a (bug 1602832)
Backed out changeset 24d1ce1b0ac9 (bug 1602832)
Backed out changeset 5ea85726cc48 (bug 1602832)
Backed out changeset ee00e846104e (bug 1602832)
2020-05-19 23:05:26 +03:00