corysanin/tubestation
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Bug 1353956 - P4. Add header and CRC32 checksum to SafeBrowsing V4 prefix files. r=gcp

Browse Source 
After this patch, we may have the following files in SafeBrowsing
directory:
- (v2) .sbstore  : Store V2 chunkdata, for update, MD5 integrity check
                   while load
- (v2) .pset     : Store V2 prefixset, for lookup, load upon startup, no
                  integrity check
- (v4) .metadata : Store V4 state, for update, no integrity check
- (v4) .vlpset   : Store V4 prefixset, for lookup, load upon startup,
                   CRC32 integrity check
- (v4) .pset     : V4 prefix set before this patch, should be removed

The magic string is also added to ".vlpset" header so we can add
a telemetry to see if sanity check is good enough for prefix set
integrity check (The telemetry is not yet added). If yes, we can remove
the CRC32 in the future for even better performance.

Differential Revision: https://phabricator.services.mozilla.com/D21463
This commit is contained in:
dlee
2019-03-07 14:41:25 +00:00
parent e1ed95ebdf
commit c975e636dd
9 changed files with 296 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
toolkit/components/url-classifier/LookupCache.cpp
View File

@@ -31,9 +31,6 @@
// returned from the gethash server. They are not serialized,
// only cached until the next update.
// Name of the persistent PrefixSet storage
#define PREFIXSET_SUFFIX ".pset"
#define V2_CACHE_DURATION_SEC (15 * 60)
// MOZ_LOG=UrlClassifierDbService:5
@@ -122,7 +119,7 @@ nsresult LookupCache::WriteFile() {
nsresult rv = mStoreDirectory->Clone(getter_AddRefs(psFile));
NS_ENSURE_SUCCESS(rv, rv);
rv = psFile->AppendNative(mTableName + NS_LITERAL_CSTRING(PREFIXSET_SUFFIX));
rv = psFile->AppendNative(mTableName + GetPrefixSetSuffix());
NS_ENSURE_SUCCESS(rv, rv);
rv = StoreToFile(psFile);
@@ -441,7 +438,7 @@ nsresult LookupCache::LoadPrefixSet() {
nsresult rv = mStoreDirectory->Clone(getter_AddRefs(psFile));
NS_ENSURE_SUCCESS(rv, rv);
rv = psFile->AppendNative(mTableName + NS_LITERAL_CSTRING(PREFIXSET_SUFFIX));
rv = psFile->AppendNative(mTableName + GetPrefixSetSuffix());
NS_ENSURE_SUCCESS(rv, rv);
bool exists;
@@ -726,6 +723,10 @@ size_t LookupCacheV2::SizeOfPrefixSet() const {
return mPrefixSet->SizeOfIncludingThis(moz_malloc_size_of);
}
nsCString LookupCacheV2::GetPrefixSetSuffix() const {
return NS_LITERAL_CSTRING(".pset");
}
#ifdef DEBUG
template <class T>
static void EnsureSorted(T* aArray) {