Commit Graph

1266 Commits

Author SHA1 Message Date
Cosmin Sabou
3c35ccef25 Backed out changeset 5950c9d63c3b (bug 1090497) for build bustages on several files. CLOSED TREE 2018-08-02 19:59:53 +03:00
Masatoshi Kimura
581bf032ba Bug 1090497 - Re-enable warnings as errors on clang-cl. r=froydnj 2018-07-31 22:10:07 +09:00
Francois Marier
e4b702d126 Bug 1461515 - Make TP test fail if it uses the wrong list. r=dimi
I added a test case in one of the tracking protection tests which will
fail if the annotation list gets pulled into the TP ones.

I also removed unnecessary prefs that were being set in the test.

Differential Revision: https://phabricator.services.mozilla.com/D2486
2018-08-01 11:53:14 +00:00
Francois Marier
986e68a21c Bug 1461515 - Fix and expand tracking annotation test. r=dimi
Here's a summary of things that were wrong about this test:

1. It was setting urlclassifier.trackingTable only to be overwritten
   later by addTestTrackers().
2. It was using an http event which fires before the classification has
   been done.
3. It didn't disable tailing, which interferes with lowering the priority of
   XHRs.
4. It was not testing that non-annotated or whitelisted resources would not
   have their priority lowered.

I added more test cases both to ensure that the correct list
(urlclassifier.trackingAnnotationTable) is used but also to ensure that
whitelisted or non-blacklisted URLs preserve the normal priority (point #4 above).

I found that XHRs do not get their priority lowered because of this flag:

  https://searchfox.org/mozilla-central/rev/d47c829065767b6f36d29303d650bffb7c4f94a3/netwerk/base/nsChannelClassifier.cpp#221

which gets set here:

  https://searchfox.org/mozilla-central/rev/d47c829065767b6f36d29303d650bffb7c4f94a3/dom/xhr/XMLHttpRequestMainThread.cpp#2548

and so I had to disable tailing in the test (point #3 above).

There was also a problem where the test was resetting the prefs too early
because we were not actually waiting for the classification to finish.

We would wait for the following event: http-on-opening-request

  https://searchfox.org/mozilla-central/rev/d47c829065767b6f36d29303d650bffb7c4f94a3/netwerk/protocol/http/nsIHttpProtocolHandler.idl#85

whereas maybe a more appropriate one would be http-on-before-connect:

  https://searchfox.org/mozilla-central/rev/d47c829065767b6f36d29303d650bffb7c4f94a3/netwerk/protocol/http/nsIHttpProtocolHandler.idl#103

since that is triggerred after annotations (point #2 above):

  https://searchfox.org/mozilla-central/rev/d47c829065767b6f36d29303d650bffb7c4f94a3/netwerk/protocol/http/nsHttpChannel.cpp#6614

Differential Revision: https://phabricator.services.mozilla.com/D2485
2018-08-01 11:52:03 +00:00
Francois Marier
6d9c2d145e Bug 1461515 - Split tracking annotations from tracking protection. r=dimi
This makes it possible to use different lists for tracking protection
and for the features that rely on tracking annotations.

Differential Revision: https://phabricator.services.mozilla.com/D2484
2018-08-01 20:45:41 +00:00
Kris Maglione
e97c582e3f Bug 1463291: Move docShell getter from Document to Window. r=bz
DocShells are associated with outer DOM Windows, rather than Documents, so
having the getter on the document is a bit odd to begin with. But it's also
considerably less convenient, since most of the times when we want a docShell
from JS, we're dealing most directly with a window, and have to detour through
the document to get it.

MozReview-Commit-ID: LUj1H9nG3QL
2018-05-21 16:58:23 -07:00
Kris Maglione
9ba14892f2 Bug 1463016: Part 5 - Add domWindow property to DocShellTreeItem and update callers to use it. r=nika
MozReview-Commit-ID: FRRAdxLHRtG
2018-05-20 18:10:16 -07:00
Narcis Beleuzu
749f83cc65 Backed out 2 changesets (bug 1463016, bug 1463291) for geckoview failures
Backed out changeset fcfb99baa0f0 (bug 1463291)
Backed out changeset 0d69b4fb1ed4 (bug 1463016)
2018-07-29 03:55:23 +03:00
Kris Maglione
6dcf46ad68 Bug 1463291: Move docShell getter from Document to Window. r=bz
DocShells are associated with outer DOM Windows, rather than Documents, so
having the getter on the document is a bit odd to begin with. But it's also
considerably less convenient, since most of the times when we want a docShell
from JS, we're dealing most directly with a window, and have to detour through
the document to get it.

MozReview-Commit-ID: LUj1H9nG3QL
2018-05-21 16:58:23 -07:00
Kris Maglione
86fbdd6653 Bug 1463016: Part 5 - Add domWindow property to DocShellTreeItem and update callers to use it. r=nika
MozReview-Commit-ID: FRRAdxLHRtG
2018-05-20 18:10:16 -07:00
Dimi Lee
ccd7628134 Bug 1407123 - Wait for the update task to be finished before running next test case. r=francois
The test_partialUpdateV4() test case doesn't wait for the update task
to be finished. It checks the status in the HTTP server side and then calls
run_next_test(). However, when XPCShell test is done, it will trigger
the shutdown process and hence interrupt the ongoing update task.

This cause the xpcshell test receives an error since the update is
interrupted and returns an error like NS_ERROR_UC_UPDATE_SHUTDOWNING.

This patch also fixes a javascript error that we didn't stop the httpd
server when cleanup.

Differential Revision: https://phabricator.services.mozilla.com/D2360
2018-07-26 18:26:19 +00:00
dlee
9479f45ea5 Bug 1453038 - Interrupt update when firefox is shutting down. r=francois
URL Classifier has a mUpdateInterrupted flag to abort an ongoing
update in several checkpoints, but we didn't use this while shutting down.

Set this flag in PreShutdown() to avoid url-classifier worker thread or
update thread takes too long to finish an update.

Differential Revision: https://phabricator.services.mozilla.com/D2157
2018-07-16 16:48:53 +00:00
Francois Marier
9dd9a89c3f Bug 1362761 - Make WritePrefixes() more readable. r=dimi
Add assertions to highlight what the various data structures
should look like.

Also assert to ensure that mIndexPrefixes is always the same
length as mIndexDeltas and avoid writing the prefixes to disk
if that's not the case.

Do a single fallible allocation before we create the indexStarts
array instead of checking on each AppendElement() to simplify
the loop and emphasize the number of elements that the array
will receive (indexSize + 1). Remove the last element since we
don't actually end up writing it to disk.

MozReview-Commit-ID: HIg7ZmgaL7x

Differential Revision: https://phabricator.services.mozilla.com/D2063
2018-07-11 09:10:31 +00:00
Francois Marier
6f7879d6d8 Bug 1362761 - Safer Clean() and IsEmpty() handling in PrefixSet. r=dimi
This simplifies the logic around clearing the prefix set and also adds
the clearing of the mIndexDeltasChecksum which should have been done
as part of 3a00711bb0e6.

Additionally, the checks for whether or not the prefix set is empty
include some sanity-checking asserts.

Finally, mTotalPrefixes could be out of sync with mIndexPrefixes
and mIndexDeltas if LoadPrefixes() or MakePrefixSet() fail so we
now only update it once all elements have been added successfully.

There is now a release assert to catch grossly out-of-sync (or
corrupt) values of mTotalPrefixes.

MozReview-Commit-ID: BSbyD2dGsUY

Differential Revision: https://phabricator.services.mozilla.com/D2062
2018-07-11 12:40:34 +00:00
Francois Marier
8e84aa521c Bug 1362761 - Add more specific warnings in case of file corruption. r=dimi
MozReview-Commit-ID: KsgcQWLGulH

Differential Revision: https://phabricator.services.mozilla.com/D2061
2018-07-11 08:58:15 +00:00
Francois Marier
70a9e6972b Bug 1362761 - Force file and streams to use smart pointers. r=dimi
MozReview-Commit-ID: GscB9PaaN02

Differential Revision: https://phabricator.services.mozilla.com/D2060
2018-07-12 22:19:40 +00:00
Francois Marier
a0ae6bcf01 Bug 1362761 - Make prefix and chunk sets as const as possible. r=dimi
MozReview-Commit-ID: JdnNOxnBAgC

Differential Revision: https://phabricator.services.mozilla.com/D2058
2018-07-11 12:52:37 +00:00
dlee
c05217e7e3 Bug 1469412 - Convert nsIUrlClassifierCacheInfo* objects to smart pointers. r=francois
Differential Revision: https://phabricator.services.mozilla.com/D2048
2018-07-10 17:12:42 +00:00
Kris Maglione
8766ceb571 Bug 1472523: Part 4 - Avoid unnecessary domain string duplication in preference observers. r=njn
MozReview-Commit-ID: EMCgMRTDqDn
2018-07-04 14:52:48 -07:00
Chris Peterson
f7ceeaf5cf Bug 1469769 - Part 6: Replace non-failing NS_NOTREACHED with MOZ_ASSERT_UNREACHABLE. r=froydnj
This patch is an automatic replacement of s/NS_NOTREACHED/MOZ_ASSERT_UNREACHABLE/. Reindenting long lines and whitespace fixups follow in patch 6b.

MozReview-Commit-ID: 5UQVHElSpCr
2018-06-17 22:43:11 -07:00
Francois Marier
74fecd1eba Bug 1467581 - Replace all nsAutoPtrs with UniquePtrs in Safe Browsing code. r=gcp
This should not change anything since we were using nsAutoPtrs properly,
but we may as well clean this up given the recent move to smart pointers
for everything.

MozReview-Commit-ID: FWS54SYNiBm
2018-06-18 13:54:59 -07:00
Francois Marier
e52a614324 Bug 1467581 - Do less work in Reset() if Close() has been called. r=gcp
If Reset() is interleaved with a shutdown, there's no point in
finishing up and we may as well bail early.

MozReview-Commit-ID: Lhm6NfAEgSj
2018-06-07 17:45:46 -07:00
Francois Marier
3c7e7d1f0a Bug 1467581 - Remove the use of default captures in closures. r=gcp
Explicitly specify the arguments to copy to avoid making a copy of
a dangling `this` pointer.

Convert nsUrlClassifierDBService::mClassifier to a RefPtr since
the update closure might need to continue to access its members
after it's been released by the main thread.

MozReview-Commit-ID: CPio3n9MmsK
2018-06-07 17:22:32 -07:00
Mark Banner
9af64d461a Bug 1456762 - Remove now unnecessary import-globals-from statements in html files. r=mossop
Depends on D1643

MozReview-Commit-ID: 84M50HDg0wl

Differential Revision: https://phabricator.services.mozilla.com/D1644
2018-06-16 08:21:33 +00:00
Joel Maher
d0823df1ef Bug 1405428 - skip-if = verify on mochitests which do not pass test-verify. r=gbrown 2018-06-10 05:01:47 -04:00
arthur.iakab
d27db38974 Merge inbound to mozilla-central a=merge 2018-06-08 12:55:49 +03:00
Francois Marier
910891cfd7 Bug 1434206 - Keep LookupResult objects in smart pointers. r=gcp
Replace raw pointers to LookupResult with RefPtrs and eplace the
nsAutoPtr objects + raw pointers params with UniquePtrs.

Also remove unnecessarily paranoid OOM checks when creating single
LookupResult objects since those are pretty small.

MozReview-Commit-ID: G85RNnAat6H
2018-06-05 13:15:03 -07:00
Francois Marier
e4cf2350e6 Bug 1452445 - Fix update error observer in unit test. r=gcp
We should only observe for update errors while we are expecting
a successful update.

MozReview-Commit-ID: 3grGhmxqhIX
2018-05-29 18:06:49 -07:00
Francois Marier
8a6df0adea Bug 1434206 - Make DBSserviceWorkerProxy as const as possible. r=gcp
MozReview-Commit-ID: KPzzmb1Jehi
2018-05-24 16:40:35 -07:00
Francois Marier
04f2b020d6 Bug 1434206 - Keep CacheResult objects in smart pointers. r=gcp
Some of the objects were kept in UniquePtr and nsAutoPtr but that
seemed unnecessary complexity given that we can simply use RefPtr
everywhere.

It's also possible to make all of the CacheResult arrays const
since we don't ever modify the elements once they are added.

MozReview-Commit-ID: 5OlcbkQLrGb
2018-06-01 15:49:14 -07:00
Francois Marier
57c9c8f684 Bug 1434206 - Clarify when the lookupcache arrays should be cleared. r=gcp
MozReview-Commit-ID: 7smtnCC0MGT
2018-05-21 15:58:12 -07:00
Francois Marier
ec97d8a79b Bug 1434206 - Make LookupCache objects const as much as possible. r=gcp
MozReview-Commit-ID: AqC6NUh6ifm
2018-05-21 15:11:01 -07:00
Francois Marier
9be228a9d4 Bug 1434206 - Keep LookupCache objects in smart pointers. r=gcp
The existing mix of UniquePtr and raw pointers is confusing when
trying to figure out the exact lifetime of these objects.

MozReview-Commit-ID: Br4S7BXEFKs
2018-05-16 19:13:48 -07:00
Francois Marier
2b1135b69b Bug 1434206 - Add const to functions and members that can take it. r=gcp
MozReview-Commit-ID: D8IQoLZkFaA
2018-05-16 15:39:33 -07:00
Francois Marier
470d7a550c Bug 1434206 - Replace a pointer with a reference. r=gcp
This will prevent our holding on to this pointer incorrectly in the
future.

MozReview-Commit-ID: H8ueIOK1qAK
2018-05-15 17:18:18 -07:00
Francois Marier
b910cbbb34 Bug 1434206 - Make TableUpdate objects const as much as possible. r=gcp
I tried to make TableUpdateArray point to const TableUpdate objects
everywhere but there were two problems:

- HashStore::ApplyUpdate() triggers a few Merge() calls which include
  sorting the underlying TableUpdate object first.

- LookupCacheV4::ApplyUpdate() calls TableUpdateV4::NewChecksum() when the
  checksum is missing and that sets mChecksum.

MozReview-Commit-ID: LIhJcoxo7e7
2018-05-11 16:02:37 -07:00
Francois Marier
ef8ea55aca Bug 1434206 - Keep TableUpdate objects in smart pointers. r=gcp
Manually keeping tabs on the lifetime of these objects is a pain
and is the likely source of some of our crashes. I suspect we might
also be leaking memory.

This change creates an explicit copy of the main array into the
update thread to avoid using a non-thread-safe shared data
structure. This is a shallow copy. Only the pointers to the
TableUpdates are copied, which means one pointer per list (e.g. 5
in total for google4 in a new profile).

MozReview-Commit-ID: 221d6GkKt0M
2018-06-01 15:48:48 -07:00
Francois Marier
6f76ad1921 Bug 1434206 - Assert that gethash processing happens on the right thread. r=gcp
MozReview-Commit-ID: FbF8LzCZ3XO
2018-05-11 11:47:26 -07:00
Francois Marier
978f983eec Bug 1434206 - Don't cache gethash response if we failed to apply results. r=gcp
MozReview-Commit-ID: APJ1qhofU3t
2018-06-01 15:33:30 -07:00
Francois Marier
6d1e9d5082 Bug 1434206 - Use a TableUpdateV2 param in ApplyUpdate(). r=gcp
HashStore::ApplyUpdate() is a V2-only function and so we can be
explicit about that and remove unnecessary casts.

Add a new update error code for when we fail to cast a TableUpdate
object to the expected protocol version.

MozReview-Commit-ID: 65BBwiZJw6J
2018-05-10 16:06:49 -07:00
Francois Marier
f0a2dbac78 Bug 1434206 - Clear the current table when protocol parser is done. r=gcp
Add assertions in the functions that don't already test for
mTableUpdate just to be extra-safe.

MozReview-Commit-ID: 8R67SLSgj23
2018-05-10 17:58:01 -07:00
Francois Marier
ffcbff3049 Bug 1434206 - Add const to members and functions that can take it. r=gcp
MozReview-Commit-ID: B2aaQTttPAV
2018-05-16 15:26:14 -07:00
Francois Marier
0dc53a8b9d Bug 1434206 - Remove unused and undefined functions. r=gcp
mProtocolV2 is still used to skip the caching of misses on V4:

https://searchfox.org/mozilla-central/rev/d4b9e50875ad7e5d20f2fee6a53418315f6dfcc0/toolkit/components/url-classifier/nsUrlClassifierDBService.cpp#1353-1357

MozReview-Commit-ID: 2cHh9JiZuHh
2018-05-28 14:39:32 -07:00
Francois Marier
f9429798b0 Bug 1414051 - Move ThreatHit protobuf details to the body. r=gcp
Mark that channel as anonymous in order to prevent any cookies
from being set here and potentially tie these proxied requests
to list updates which will not go through the proxy.

Also fix a return code in the case that httpChannel fails to QI.

MozReview-Commit-ID: BaRyYRlUbsB
2018-02-19 16:18:37 -08:00
Miko Mynttinen
3df26bc8a5 Bug 1465060 - Part 1: Fix warnings for std::move() use r=froydnj
MozReview-Commit-ID: HpdFXqQdIOO
2018-06-01 17:59:07 +02:00
arthur.iakab
9588ba6d78 Backed out 2 changesets (bug 1465060) for build bustages on security/sandbox/linux/reporter/SandboxReporter.cpp
Backed out changeset 7c8905b6b226 (bug 1465060)
Backed out changeset 10446073eca8 (bug 1465060)
2018-06-03 19:25:41 +03:00
Miko Mynttinen
5afe37aefc Bug 1465060 - Part 1: Fix warnings for std::move() use r=froydnj
MozReview-Commit-ID: HpdFXqQdIOO
2018-06-01 17:59:07 +02:00
Emilio Cobos Álvarez
6100dee429 Bug 1466168: Remove mozilla::Forward in favor of std::forward. r=froydnj
Same approach as the other bug, mostly replacing automatically by removing
'using mozilla::Forward;' and then:

  s/mozilla::Forward/std::forward/
  s/Forward</std::forward</

The only file that required manual fixup was TestTreeTraversal.cpp, which had
a class called TestNodeForward with template parameters :)

MozReview-Commit-ID: A88qFG5AccP
2018-06-02 09:33:26 +02:00
Emilio Cobos Álvarez
4b8b5e1717 Bug 1465585: Switch from mozilla::Move to std::move. r=froydnj
This was done automatically replacing:

  s/mozilla::Move/std::move/
  s/ Move(/ std::move(/
  s/(Move(/(std::move(/

Removing the 'using mozilla::Move;' lines.

And then with a few manual fixups, see the bug for the split series..

MozReview-Commit-ID: Jxze3adipUh
2018-06-01 10:45:27 +02:00
Kris Maglione
187318b0b7 Bug 1462937: Update callers to use nsIFile::GetDirectoryEntries as a nsIDirectoryEnumerator. r=froydnj
MozReview-Commit-ID: Iv4T1MVAF5
2018-05-19 20:17:45 -07:00