Commit Graph

52 Commits

Author SHA1 Message Date
Thomas Nguyen
71a001d62e Bug 1394031 - Remove mCryptoHash members of nsUrlClassifierDBServiceWorker and ProtocolParser
The usage of cryptoHash consists of a complete set of Init, Update, and Finish, there's
no reason to keep it around

MozReview-Commit-ID: 7bT9IsWEM5m
2017-09-14 10:51:55 +08:00
DimiL
02e313d384 Bug 1359299 - V4 caches in LookupCache need to be copied around in copy constructor. r=hchang
MozReview-Commit-ID: AjzUUmQKiPW
2017-06-06 14:16:57 +08:00
Ryan VanderMeulen
c121499332 Backed out changeset c0b940487708 (bug 1359299) for causing intermittent Windows safebrowsing crashes. 2017-05-24 09:11:04 -04:00
DimiL
ecf67ffc51 Bug 1359299 - Copy fullhash cache when update. r=hchang
After adopting the new thread model for safebrowsing, we will create a new
lookup cache for update so we can still check lookup cache at the same time.

Prefix set, completions will be generated when we open the new lookup cache
but it won't include cache, so we will loss cache after that.

This patch will copy cache data from old lookup cache to new lookup
cache while update.

MozReview-Commit-ID: L0WpiHOGIGm
2017-05-23 09:19:06 +08:00
DimiL
c8f160ac4e Bug 1360480 - about:url-classifier: Cache information. r=francois
MozReview-Commit-ID: 4YXtb2KPgwL
2017-05-17 10:32:33 +08:00
DimiL
769f299a32 Bug 1333328 - Remove mTableFreshness and gFreshnessGuarantee. r=francois
Functions and Preference related to mTableFreshness and gFreshnessGuarantee
could be removed since we will no longer require them.

MozReview-Commit-ID: IAa0UHLSQ56
2017-05-04 11:13:21 +08:00
DimiL
a0b8501692 Bug 1333328 - Refactor cache miss handling mechanism for V2. r=francois
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.

But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.

An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).
2017-05-04 09:38:14 +08:00
Michael Layzell
9e9bb6861f Bug 1339537 - Part 6: Pass std::function values tree by const reference instead of by value, r=ehsan
MozReview-Commit-ID: PVAqU2DPs2
2017-04-27 12:44:57 -04:00
Thomas Nguyen
87c03bcdea Bug 1353853 - Cache preferences when doing channel classify r=francois
We will cache all preferences which will be read during classifing channel
- Store them into static variables nsUrlClassifierDBService
- Use a singleton class to manage/update preferrences in nsChannelClassifier

MozReview-Commit-ID: GvyBI3rVpYh
2017-04-18 17:00:27 +08:00
Henry Chang
a629121fce Bug 1354968 - Avoid concurrent access of mTableRefreshness. r=francois
mTableRefreshness, a non-thread-safe object, might be accessed on worker thread
and update thread cocurrently. To solve this issue, on update thread we only
insert data to mNewTableRefreshness and merge to mTableRefreshness on
the worker thread later.

MozReview-Commit-ID: 9WgoeYfWVfK
2017-04-11 01:02:42 +08:00
DimiL
c1a8a088db Bug 1311935 - P1. Make ActiveTables() work for safebrowsing v4. r=francois
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.

Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.

Fix this issue by checking subdirectory recursively in ScanStoreDir.

MozReview-Commit-ID: I6pa6e4bFND
2017-04-07 14:30:11 +08:00
Iris Hsiao
cd018fd494 Backed out 4 changesets (bug 1311935) for causing assertion crash by developer's request
Backed out changeset 27e624cd9479 (bug 1311935)
Backed out changeset 4c0381ab0990 (bug 1311935)
Backed out changeset 73587838ef16 (bug 1311935)
Backed out changeset a5a6c0f79733 (bug 1311935)
2017-04-11 11:04:54 +08:00
DimiL
c4fc0215be Bug 1311935 - P1. Make ActiveTables() work for safebrowsing v4. r=francois
This patch fixes that Classifier::ActiveTables doesn't return v4 tables.

Classifier::mActiveTablesCache is generated by scanning safebrowsing directory.
We use Classifier::ScanStoreDir to do the work, but it will ignore subdirectory.
Since v4 tables are stored in subdirectory 'google4', mActiveTablesCache doesn't
include v4 tables.

Fix this issue by checking subdirectory recursively in ScanStoreDir.

MozReview-Commit-ID: I6pa6e4bFND
2017-04-07 14:30:11 +08:00
Henry Chang
4543c61a48 Bug 1339050 - Asynchronously apply safebrowsing DB update. r=francois,gcp
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.

In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)

As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.

MozReview-Commit-ID: DXZvA2eFKlc
2017-04-06 07:07:56 +08:00
Henry Chang
75d7890222 Bug 1339760 - Split update process to background/foreground and run background on update thread **synchronously**. r=francois,gcp
MozReview-Commit-ID: J0phPC1nWsf
2017-02-22 17:25:26 +08:00
Henry Chang
73fc8b0a2d Bug 1342397 - Properly reset stuff when failing to create LookupCache for update. r=gcp
MozReview-Commit-ID: 1ecI2lFIj2U
2017-02-25 00:22:39 +08:00
Thomas Nguyen
2b6e2edd11 Bug 1297962 - Add noise data when sending v4 gethash request r=francois
MozReview-Commit-ID: GbyvX7wcg8c
* * *
[mq]: 1297962_review

MozReview-Commit-ID: 1U2T0wq778R
2017-02-24 10:22:12 +08:00
Henry Chang
c16ea9af44 Bug 1338970 - Introduce 2-step DB update scheme. (background update then swap in) r=francois,gcp
MozReview-Commit-ID: 4C9N0tuSso0
2017-02-21 14:18:29 +08:00
dimi
9cac186808 Bug 1328821 - hash completion request for v4 should not depend on table freshness. r=francois,henry
MozReview-Commit-ID: EIjDrnj1I4S
2017-01-17 08:33:08 +08:00
Henry Chang
acfb7674fb Bug 1319286 - Cache nsIUrlClassifierDBService.getTables result until next update. r=francois.
MozReview-Commit-ID: ItjTQNzCVED
2016-11-22 10:39:58 +08:00
Henry Chang
7f94b8915f Bug 1315097 - Build the provider dictionary on the main thread to be used everywhere. r=francois,gcp
MozReview-Commit-ID: Ft1deSNKuVB
2016-11-04 17:54:05 +08:00
Henry Chang
891cee9c33 Bug 1310142 - Preserve backup databases and raw table updates for diagnostic. r=francois
MozReview-Commit-ID: GeJoBrhuTgA
2016-10-14 17:58:18 +08:00
DimiL
e841cd1751 Bug 1305780 - P1. Implement the update fail scheme for v4. r=gcp
MozReview-Commit-ID: LeVpVIUdmjc
2016-10-19 12:52:05 +08:00
Henry Chang
fff35e78ca Bug 1305484 - Save/load state and checksum to/from disk rather than prefs. r=dimi,francois
MozReview-Commit-ID: 4gmmrI9wY4c
2016-10-13 15:22:08 +08:00
Dimi Lee
e6d6ca8983 Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg
2016-09-19 11:51:01 +08:00
Phil Ringnalda
81e623a5b4 Backed out 5 changesets (bug 1305801) for ASan gtest bustage
Backed out changeset 0c95d5dec6d9 (bug 1305801)
Backed out changeset bca0e706dbc5 (bug 1305801)
Backed out changeset def8da367beb (bug 1305801)
Backed out changeset 56ceae52d847 (bug 1305801)
Backed out changeset 14457cc4c325 (bug 1305801)
2016-10-03 22:14:49 -07:00
Dimi Lee
340a321260 Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg
2016-09-19 11:51:01 +08:00
Henry Chang
176430c017 Bug 1254763 - Part 1: Use per-provider directory for V4 databases. r=dimi,gcp
MozReview-Commit-ID: KZGWIcKAQyW
2016-09-02 12:00:00 +08:00
dimi
32c27de82f Bug 1037560 - P1. Safebrowsing pleasereset resets all tables. r=gcp
MozReview-Commit-ID: C5GkBBDAhn3
2016-08-23 08:24:07 +08:00
dimi
ac0dea5510 Bug 1254766 - Stop caching Safe Browsing completions to disk. r=gcp
MozReview-Commit-ID: 8Qnc8yymgDL
2016-08-11 08:17:39 +08:00
dimi
f5ae7d93c7 Bug 1287024 - Remove unused mHashStores variable in Classifier.cpp. r=francois
MozReview-Commit-ID: 2EIDaRHKW8Q
2016-07-15 12:16:26 +08:00
Gian-Carlo Pascutto
761749e88b Bug 1175562 - Persist last update time for SafeBrowsing. r=francois 2015-10-06 13:53:07 +02:00
Birunthan Mohanathas
a29151dc87 Bug 1182996 - Fix and add missing namespace comments. rs=ehsan
The bulk of this commit was generated by running:

  run-clang-tidy.py \
    -checks='-*,llvm-namespace-comment' \
    -header-filter=^/.../mozilla-central/.* \
    -fix
2015-07-13 08:25:42 -07:00
Monica Chew
1814023cb1 Bug 1120499: Proxy DoLocalLookup to the worker thread (r=gcp) 2015-01-13 17:09:13 -08:00
Monica Chew
67679b28f7 Bug 1108009 - Make synchronous interface nsIURIClassifier.ClassifyLocal. r=gcp 2014-12-18 10:18:09 -08:00
Monica Chew
796c721622 Bug 985623: Force url classifier clients to specify which tables to lookup, add a pref to skip hash completion checks (r=gcp) 2014-03-20 14:25:35 -07:00
Gian-Carlo Pascutto
cc689ffbd1 Bug 825891 - Remove the code for per-client randomization in the url-classifier. r=dcamp 2013-01-08 16:43:33 +01:00
Daniel Holbert
040ae4eb23 Bug 825627, part 1: Make nsUrlClassifierDBService.cpp's "gethashNoise" & "aCount" variables as unsigned, since counts are unsigned by definition. r=gcp 2013-01-02 09:05:19 -08:00
Saurabh Anand
15f0c5068c Bug 818817 - Fix some compiler warnings, r=Ms2ger 2012-12-09 22:53:19 +05:30
Gian-Carlo Pascutto
1293a5c009 Bug 727370 - Recover correctly from a Reset during an update. r=dcamp 2012-09-28 18:31:19 +02:00
Gian-Carlo Pascutto
a04ed76212 Bug 727370 - Make SafeBrowsing updates atomic transactions. r=dcamp 2012-09-28 18:31:18 +02:00
Ehsan Akhgari
243c878d26 Bug 579517 - Part 1: Automated conversion of NSPR numeric types to stdint types in Gecko; r=bsmedberg
This patch was generated by a script.  Here's the source of the script for
future reference:

function convert() {
echo "Converting $1 to $2..."
find . ! -wholename "*nsprpub*" \
       ! -wholename "*security/nss*" \
       ! -wholename "*/.hg*" \
       ! -wholename "obj-ff-dbg*" \
       ! -name nsXPCOMCID.h \
       ! -name prtypes.h \
         -type f \
      \( -iname "*.cpp" \
         -o -iname "*.h" \
         -o -iname "*.c" \
         -o -iname "*.cc" \
         -o -iname "*.idl" \
         -o -iname "*.ipdl" \
         -o -iname "*.ipdlh" \
         -o -iname "*.mm" \) | \
    xargs -n 1 sed -i -e "s/\b$1\b/$2/g"
}

convert PRInt8 int8_t
convert PRUint8 uint8_t
convert PRInt16 int16_t
convert PRUint16 uint16_t
convert PRInt32 int32_t
convert PRUint32 uint32_t
convert PRInt64 int64_t
convert PRUint64 uint64_t

convert PRIntn int
convert PRUintn unsigned

convert PRSize size_t

convert PROffset32 int32_t
convert PROffset64 int64_t

convert PRPtrdiff ptrdiff_t

convert PRFloat64 double
2012-08-22 11:56:38 -04:00
Gian-Carlo Pascutto
f59dcebe9c Bug 673470 - Update the new SafeBrowsing code to MPL 2.0. r=dcamp 2012-08-15 09:10:56 +02:00
Gian-Carlo Pascutto
6b5d1b65c2 Bug 768871 - Provide an option to disable per-client randomization. r=dcamp 2012-08-15 09:07:10 +02:00
Gian-Carlo Pascutto
6cb825e03c Bug 673470 - Optimize input buffer size. Cache active tables. r=dcamp 2012-08-15 09:04:31 +02:00
Gian-Carlo Pascutto
1c0ff29874 Bug 673470 - Replace the sqlite safeb store with a flat file. r=dcamp 2012-08-15 09:04:19 +02:00
Gian-Carlo Pascutto
10acc9e06e Backout 173f90d397a8 (Bug 673470). rs=dcamp a=mfinkle 2012-04-20 07:46:47 +02:00
Gian-Carlo Pascutto
e5fefd0538 Backout db52b4916cde (Bug 673470). rs=dcamp a=mfinkle 2012-04-20 07:46:46 +02:00
Gian-Carlo Pascutto
b7cc115c04 Bug 673470 - Optimize input buffer size. Cache active tables. r=dcamp 2012-02-03 08:32:32 +01:00
Gian-Carlo Pascutto
724fb285e2 Bug 673470 - Replace the SQLite SafeBrowsing store with an optimized store. r=dcamp 2011-12-06 19:03:05 +01:00