Commit Graph

48 Commits

Author SHA1 Message Date
Iris Hsiao
cd018fd494 Backed out 4 changesets (bug 1311935) for causing assertion crash by developer's request
Backed out changeset 27e624cd9479 (bug 1311935)
Backed out changeset 4c0381ab0990 (bug 1311935)
Backed out changeset 73587838ef16 (bug 1311935)
Backed out changeset a5a6c0f79733 (bug 1311935)
2017-04-11 11:04:54 +08:00
dimi
3a7526678a Bug 1311935 - P3. Implement safebrowsing v4 caching logic. r=francois
LookupCacheV4::Has implements safebrowsing v4 caching logic.
1. Check if fullhash match any prefix in local database:
  - If not, the URL is safe.
2. Check if prefix is in the cache(prefix is always the first 4-byte of
   the fullhash, Bug 1323953):
  - If not, send fullhash request
3. Check if fullhash is in the positive cache:
  - If fullhash is found and it is not expired, the URL is not safe.
  - If fullhash is found and it is expired, send fullhash request.
4. If fullhash is not found, check negative cache expired time:
  - If negative cache time is not expired, the URL is safe.
  - If negative cache time is expired, send fullhash request.

MozReview-Commit-ID: GRX7CP8ig49
2017-04-10 14:21:08 +08:00
dimi
5856f89d1f Bug 1311935 - P2. Process fullHashes.find response. r=francois
This patch includes following changes:

1. nsUrlClassifierHashCompleter.js
   nsUrlClassifierHashCompleter.idl
   - Add completionV4 interface for hashCompleter to pass response data to
     DB service.
   - Process response data includes negative cache duration, matched full
     hashes and cache duration for each match. Full matches are passed through
     nsIFullHashMatch interface.
   - Change _requests.responses from array contains matched fullhashes to
     dictionary so that it can store additional information likes negative cache
     duration.
2. nsUrlClassifierDBService.cpp
   - Implement CompletionV4 interface, store response data to CacheResultV4
     object. Expired duration to expired time is handled here.
   - Add CacheResultToTableUpdate function to convert V2 & V4 cache result
     to TableUpdate object.
3. LookupCache.h
   - Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
     response data in CompletionV2 and CompletionV4.
4. HashStore.h
   - Add API and member variable in TableUpdateV4 to store response data.
     TableUpdate object is used by DB service to pass update data or gethash
     response to Classifier, so we need to extend TableUpdateV4 to be able
     to store fullHashes.find response.
6. Entry.h
   - Define the structure about how we cache fullHashes.find response.

MozReview-Commit-ID: KgR1NASl7GC
2017-04-10 16:20:09 +08:00
Henry Chang
4543c61a48 Bug 1339050 - Asynchronously apply safebrowsing DB update. r=francois,gcp
A new function Classifier::AsyncApplyUpdates() is implemented for async update.
Besides, all public Classifier interfaces become "worker thread only" and
we remove DBServiceWorker::ApplyUpdatesBackground/Foreground.

In DBServiceWorker::FinishUpdate, instead of calling Classifier::ApplyUpdates,
we call Classifier::AsyncApplyUpdates and install a callback for notifying
the update observer when update is finished. The callback will occur on the
caller thread (i.e. worker thread.)

As for the shutdown issue, when the main thread is notified to shut down,
we at first *synchronously* dispatch an event to the worker thread to
shut down the update thread. After getting synchronized with all other
threads, we send last two events "CancelUpdate" and "CloseDb" to notify
dangling update (i.e. BeginUpdate is called but FinishUpdate isn't)
and do cleanup work.

MozReview-Commit-ID: DXZvA2eFKlc
2017-04-06 07:07:56 +08:00
Dimi Lee
5375a9f0e5 Bug 1311931 - Add telemetry to measure full match rate for v2 and v4. r=francois
MozReview-Commit-ID: H9jAR82rgDh
2017-02-23 23:07:13 +08:00
dimi
5750b4baba Bug 1336909 - Restrict URLCLASSIFIER_PREFIX_MATCH to profiles that have working V4. r=francois
MozReview-Commit-ID: L3lKgiohalH
2017-02-08 15:18:35 +08:00
Dimi Lee
45dd73df16 Bug 1333257 - Only cache V2 misses when doing Safe Browsing lookups. r=francois
MozReview-Commit-ID: 6kvM6z5OnPw
2017-01-26 11:36:52 +08:00
dimi
9cac186808 Bug 1328821 - hash completion request for v4 should not depend on table freshness. r=francois,henry
MozReview-Commit-ID: EIjDrnj1I4S
2017-01-17 08:33:08 +08:00
Henry Chang
b9d5f5080f Bug 1312339 - LookupResult to support variable length partial hash. r=francois
MozReview-Commit-ID: DKwNCNKJAW
2016-12-16 14:34:32 +08:00
Henry Chang
7f94b8915f Bug 1315097 - Build the provider dictionary on the main thread to be used everywhere. r=francois,gcp
MozReview-Commit-ID: Ft1deSNKuVB
2016-11-04 17:54:05 +08:00
DimiL
e841cd1751 Bug 1305780 - P1. Implement the update fail scheme for v4. r=gcp
MozReview-Commit-ID: LeVpVIUdmjc
2016-10-19 12:52:05 +08:00
Dimi Lee
e49ab99ec5 Bug 1308606 - Crash in mozilla::safebrowsing::Classifier::UpdateHashStore. r=francois
MozReview-Commit-ID: FIl5cPFzGbl
2016-10-08 20:42:43 +08:00
Dimi Lee
d316530abf Bug 1305801 - Part 5: Support SafeBrowsing v4 partial update. r=gcp
MozReview-Commit-ID: 7OEWLaZbotS
2016-10-04 09:14:39 +08:00
Dimi Lee
e6d6ca8983 Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg
2016-09-19 11:51:01 +08:00
Phil Ringnalda
81e623a5b4 Backed out 5 changesets (bug 1305801) for ASan gtest bustage
Backed out changeset 0c95d5dec6d9 (bug 1305801)
Backed out changeset bca0e706dbc5 (bug 1305801)
Backed out changeset def8da367beb (bug 1305801)
Backed out changeset 56ceae52d847 (bug 1305801)
Backed out changeset 14457cc4c325 (bug 1305801)
2016-10-03 22:14:49 -07:00
Dimi Lee
daec409fa0 Bug 1305801 - Part 5: Support SafeBrowsing v4 partial update. r=gcp
MozReview-Commit-ID: 7OEWLaZbotS
2016-10-04 09:14:39 +08:00
Dimi Lee
340a321260 Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg
2016-09-19 11:51:01 +08:00
Henry Chang
8844a50a30 Bug 1254763 - Part 2: Test cases. r=gcp
MozReview-Commit-ID: JDjV3WUfgWW
2016-09-02 12:02:40 +08:00
Henry Chang
176430c017 Bug 1254763 - Part 1: Use per-provider directory for V4 databases. r=dimi,gcp
MozReview-Commit-ID: KZGWIcKAQyW
2016-09-02 12:00:00 +08:00
dimi
ac0dea5510 Bug 1254766 - Stop caching Safe Browsing completions to disk. r=gcp
MozReview-Commit-ID: 8Qnc8yymgDL
2016-08-11 08:17:39 +08:00
Francois Marier
2223c3af6b Bug 1164518 - Avoid unnecessary DB updates when caching Safe Browsing results. r=gcp
MozReview-Commit-ID: HYNaTdCRohL
2016-03-04 12:33:20 -08:00
Gian-Carlo Pascutto
615c26249c Bug 1100338 - Do not consider noise for Completion matches. Remove dead code. r=francois 2015-07-06 04:24:00 +02:00
Nathan Froyd
4e6d8f6705 Bug 1207245 - part 6 - rename nsRefPtr<T> to RefPtr<T>; r=ehsan; a=Tomcat
The bulk of this commit was generated with a script, executed at the top
level of a typical source code checkout.  The only non-machine-generated
part was modifying MFBT's moz.build to reflect the new naming.

CLOSED TREE makes big refactorings like this a piece of cake.

 # The main substitution.
find . -name '*.cpp' -o -name '*.cc' -o -name '*.h' -o -name '*.mm' -o -name '*.idl'| \
    xargs perl -p -i -e '
 s/nsRefPtr\.h/RefPtr\.h/g; # handle includes
 s/nsRefPtr ?</RefPtr</g;   # handle declarations and variables
'

 # Handle a special friend declaration in gfx/layers/AtomicRefCountedWithFinalize.h.
perl -p -i -e 's/::nsRefPtr;/::RefPtr;/' gfx/layers/AtomicRefCountedWithFinalize.h

 # Handle nsRefPtr.h itself, a couple places that define constructors
 # from nsRefPtr, and code generators specially.  We do this here, rather
 # than indiscriminantly s/nsRefPtr/RefPtr/, because that would rename
 # things like nsRefPtrHashtable.
perl -p -i -e 's/nsRefPtr/RefPtr/g' \
     mfbt/nsRefPtr.h \
     xpcom/glue/nsCOMPtr.h \
     xpcom/base/OwningNonNull.h \
     ipc/ipdl/ipdl/lower.py \
     ipc/ipdl/ipdl/builtin.py \
     dom/bindings/Codegen.py \
     python/lldbutils/lldbutils/utils.py

 # In our indiscriminate substitution above, we renamed
 # nsRefPtrGetterAddRefs, the class behind getter_AddRefs.  Fix that up.
find . -name '*.cpp' -o -name '*.h' -o -name '*.idl' | \
    xargs perl -p -i -e 's/nsRefPtrGetterAddRefs/RefPtrGetterAddRefs/g'

if [ -d .git ]; then
    git mv mfbt/nsRefPtr.h mfbt/RefPtr.h
else
    hg mv mfbt/nsRefPtr.h mfbt/RefPtr.h
fi
2015-10-18 01:24:48 -04:00
Bobby Holley
710865df63 Bug 1188696 - Hoist nsRefPtr.h into MFBT. r=froydnj 2015-07-29 10:44:59 -07:00
Birunthan Mohanathas
a29151dc87 Bug 1182996 - Fix and add missing namespace comments. rs=ehsan
The bulk of this commit was generated by running:

  run-clang-tidy.py \
    -checks='-*,llvm-namespace-comment' \
    -header-filter=^/.../mozilla-central/.* \
    -fix
2015-07-13 08:25:42 -07:00
Emanuel Hoogeveen
b30abdc582 Bug 905127 - Part 1 - Make some functions from nsNetUtil not inline. r=jduell 2015-07-07 04:17:00 +02:00
Eric Rahm
ddc8ac650b Bug 1165518 - Part 2: Replace prlog.h with Logging.h. rs=froydnj 2015-05-19 11:15:34 -07:00
Eric Rahm
78a96dc27f Bug 1164556 - Part 1: Remove instances of #ifdef PR_LOGGING in toolkit. r=froydnj
PR_LOGGING is now always defined, we can remove #ifdefs checking for it.
2015-05-14 10:13:23 -07:00
Gian-Carlo Pascutto
265601caa5 Bug 1102687 - Do transient Classifier/HashStore object allocations on the stack. r=dmajor 2014-11-24 08:38:43 +01:00
Gian-Carlo Pascutto
fe253223fe Bug 1102687 - Avoid copying and allocating 3 times in GetPrefixes. r=dmajor 2014-11-24 08:38:29 +01:00
Gian-Carlo Pascutto
cc689ffbd1 Bug 825891 - Remove the code for per-client randomization in the url-classifier. r=dcamp 2013-01-08 16:43:33 +01:00
Gian-Carlo Pascutto
0aba543f82 Bug 806422 - Do not cache Complete's across a SafeBrowsing update. r=dcamp 2012-11-06 15:38:16 +01:00
Isaac Aggrey
50d6240e45 Bug 794510: Part 2: Use more stdint types instead of protypes.h types; r=ehsan 2012-10-11 18:38:04 -05:00
Gian-Carlo Pascutto
a04ed76212 Bug 727370 - Make SafeBrowsing updates atomic transactions. r=dcamp 2012-09-28 18:31:18 +02:00
Ehsan Akhgari
243c878d26 Bug 579517 - Part 1: Automated conversion of NSPR numeric types to stdint types in Gecko; r=bsmedberg
This patch was generated by a script.  Here's the source of the script for
future reference:

function convert() {
echo "Converting $1 to $2..."
find . ! -wholename "*nsprpub*" \
       ! -wholename "*security/nss*" \
       ! -wholename "*/.hg*" \
       ! -wholename "obj-ff-dbg*" \
       ! -name nsXPCOMCID.h \
       ! -name prtypes.h \
         -type f \
      \( -iname "*.cpp" \
         -o -iname "*.h" \
         -o -iname "*.c" \
         -o -iname "*.cc" \
         -o -iname "*.idl" \
         -o -iname "*.ipdl" \
         -o -iname "*.ipdlh" \
         -o -iname "*.mm" \) | \
    xargs -n 1 sed -i -e "s/\b$1\b/$2/g"
}

convert PRInt8 int8_t
convert PRUint8 uint8_t
convert PRInt16 int16_t
convert PRUint16 uint16_t
convert PRInt32 int32_t
convert PRUint32 uint32_t
convert PRInt64 int64_t
convert PRUint64 uint64_t

convert PRIntn int
convert PRUintn unsigned

convert PRSize size_t

convert PROffset32 int32_t
convert PROffset64 int64_t

convert PRPtrdiff ptrdiff_t

convert PRFloat64 double
2012-08-22 11:56:38 -04:00
Gian-Carlo Pascutto
f59dcebe9c Bug 673470 - Update the new SafeBrowsing code to MPL 2.0. r=dcamp 2012-08-15 09:10:56 +02:00
Gian-Carlo Pascutto
420a4b2679 Bug 673470 - Fix wrong order of arguments to KeyedHash function. r=dcamp 2012-08-15 09:08:23 +02:00
Gian-Carlo Pascutto
6b5d1b65c2 Bug 768871 - Provide an option to disable per-client randomization. r=dcamp 2012-08-15 09:07:10 +02:00
Gian-Carlo Pascutto
6301b08636 Bug 726002 - Clear some big nsTArrays as early as possible in updates. r=dcamp 2012-08-15 09:05:18 +02:00
Gian-Carlo Pascutto
1c0ff29874 Bug 673470 - Replace the sqlite safeb store with a flat file. r=dcamp 2012-08-15 09:04:19 +02:00
Gian-Carlo Pascutto
10acc9e06e Backout 173f90d397a8 (Bug 673470). rs=dcamp a=mfinkle 2012-04-20 07:46:47 +02:00
Gian-Carlo Pascutto
f27f8b25c1 Backout 35bf0d62cc30 (Bug 726002). rs=dcamp a=mfinkle 2012-04-20 07:46:46 +02:00
Gian-Carlo Pascutto
a6f2f9cd49 Bug 726002 - Clear some big nsTArrays as early as possible in updates. r=dcamp 2012-02-13 17:10:22 +01:00
Gian-Carlo Pascutto
6688e97a43 Backout e0e2cc5570ac (Bug 726002) due to broken error handling. 2012-02-13 17:04:46 +01:00
Gian-Carlo Pascutto
d465e43455 Bug 726002 - Clear some big nsTArrays as early as possible in updates. r=dcamp 2012-02-13 13:17:48 +01:00
Gian-Carlo Pascutto
724fb285e2 Bug 673470 - Replace the SQLite SafeBrowsing store with an optimized store. r=dcamp 2011-12-06 19:03:05 +01:00
Gian-Carlo Pascutto
056c10da3c Backout 44a0dc4fb9ff (Bug 673470) for Talos regression. 2012-02-01 20:10:02 +01:00
Gian-Carlo Pascutto
3b9f32d92d Bug 673470 - Replace the SQLite SafeBrowsing store with an optimized store. r=dcamp 2011-12-06 19:03:05 +01:00