SafeBrowsing V4 protocol use SHA-256 as the checksum to check integrity
of update data and also the integrity of prefix files.
SafeBrowsing V2 HashStore use MD5 as the checksum to check integrity of
.sbstore
Since we are going to use CRC32 as the integrity check of V4 prefix files,
I think rename V4 "checksum" to SHA256 can improve readability.
Differential Revision: https://phabricator.services.mozilla.com/D21460
In Bug 1453038, |mUpdateInterrupted| is set in Classifer::Close() which is
called by PreShutdown to abort an ongoing update. That doesn't handle
all the cases.
The SafeBrowsing update involves two threads, worker thread, and update
thread. Update thread takes care of most of the update work, when it finishes
its task, it posts a task back to the worker thread to apply the updated database
and also do some cleanup stuff. Then the update is complete.
The fix in Bug 1453038 doesn't abort an update if the woker thread is doing
the job. This is because the |mUpdateInterrupted| flag is set in the
worker thread. The PreShutdown event which eventually sets the flag has to
wait until the worker thread's current task is done.
In this patch:
1. Check nsUrlClassifierDBService::ShutdownHasStarted() to abort shutdown.
This is set by main thread so both worker thread and update thread can
be interrupted now.
2. mIsClosed is now replaced by the mUpdateInterrupted. The semantics of
mUpdateInterrupted is now changed to abort update for any internal APIs
which should cause an update to abort.
3. Remove |mUpdateInterrupted| and |ShutdownHasStarted()| checks and
unify with |ShouldAbort()|
Differential Revision: https://phabricator.services.mozilla.com/D12229
A directory-based file copy without checkpoint to abort may take lots
of time to finish. This cause an issue that if firefox is shutting down
and try to close an ongoing update thread, main-thread may be blocked
for a long time.
This patch adds a wrapper for copying an entire directory, within this
wrapper, we use file-based copy and add checkpoints to let update thread
has a chance to abort.
Differential Revision: https://phabricator.services.mozilla.com/D3414
A directory-based file copy without checkpoint to abort may take lots
of time to finish. This cause an issue that if firefox is shutting down
and try to close an ongoing update thread, main-thread may be blocked
for a long time.
This patch adds a wrapper for copying an entire directory, within this
wrapper, we use file-based copy and add checkpoints to let update thread
has a chance to abort.
Differential Revision: https://phabricator.services.mozilla.com/D3414
This makes it possible to use different lists for tracking protection
and for the features that rely on tracking annotations.
Differential Revision: https://phabricator.services.mozilla.com/D2484
URL Classifier has a mUpdateInterrupted flag to abort an ongoing
update in several checkpoints, but we didn't use this while shutting down.
Set this flag in PreShutdown() to avoid url-classifier worker thread or
update thread takes too long to finish an update.
Differential Revision: https://phabricator.services.mozilla.com/D2157
Explicitly specify the arguments to copy to avoid making a copy of
a dangling `this` pointer.
Convert nsUrlClassifierDBService::mClassifier to a RefPtr since
the update closure might need to continue to access its members
after it's been released by the main thread.
MozReview-Commit-ID: CPio3n9MmsK
Replace raw pointers to LookupResult with RefPtrs and eplace the
nsAutoPtr objects + raw pointers params with UniquePtrs.
Also remove unnecessarily paranoid OOM checks when creating single
LookupResult objects since those are pretty small.
MozReview-Commit-ID: G85RNnAat6H
The existing mix of UniquePtr and raw pointers is confusing when
trying to figure out the exact lifetime of these objects.
MozReview-Commit-ID: Br4S7BXEFKs
I tried to make TableUpdateArray point to const TableUpdate objects
everywhere but there were two problems:
- HashStore::ApplyUpdate() triggers a few Merge() calls which include
sorting the underlying TableUpdate object first.
- LookupCacheV4::ApplyUpdate() calls TableUpdateV4::NewChecksum() when the
checksum is missing and that sets mChecksum.
MozReview-Commit-ID: LIhJcoxo7e7
Manually keeping tabs on the lifetime of these objects is a pain
and is the likely source of some of our crashes. I suspect we might
also be leaking memory.
This change creates an explicit copy of the main array into the
update thread to avoid using a non-thread-safe shared data
structure. This is a shallow copy. Only the pointers to the
TableUpdates are copied, which means one pointer per list (e.g. 5
in total for google4 in a new profile).
MozReview-Commit-ID: 221d6GkKt0M
HashStore::ApplyUpdate() is a V2-only function and so we can be
explicit about that and remove unnecessary casts.
Add a new update error code for when we fail to cast a TableUpdate
object to the expected protocol version.
MozReview-Commit-ID: 65BBwiZJw6J
RegenActiveTables() relies on mPrimed being set correctly and so
the V4 lookup cache should behave the same way as the V2 one.
The V2 lookup cache on the other hand was unnecessarily setting
mPrimed to true twice.
MozReview-Commit-ID: LwNdI9DTqZ7
Passing a value of -1 to nsCString::Truncate() converts that
value to a large integer and leads to an unnecessary 4GB
memory allocation.
MozReview-Commit-ID: Icm5iUsEgA6
This fixes usages of `Find`, `RFind` and the equality operator that kind of
work right now but will break with the proper type checking of a templatized
version of the string classes.
For `Find` and `RFind` it appears that `nsCString::(R)Find("foo", 0)` calls
were being coerced to the `Find(char*, bool, int, int)` versions. The intent was
probably to just start searching from position zero.
For the equality operator, the type of nullptr is nullptr_t rather than
char(16_t)* so we'd need to add an operator overload that takes nullptr_t. In
this case just using `IsVoid` is probably more appropriate.
We have a minimum requirement of VS 2015 for Windows builds, which supports
the z length modifier for format specifiers. So we don't need SizePrintfMacros.h
any more, and can just use %zu and friends directly everywhere.
MozReview-Commit-ID: 6s78RvPFMzv
After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.
Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.
This patch will copy cache data from old lookup cache to new lookup
cache while update.
MozReview-Commit-ID: L0WpiHOGIGm
