Commit Graph

50 Commits

Author SHA1 Message Date
Francois Marier
d682ea8c54 Bug 1385461 - Move URL Classifier protobuf files to match other components. r=dimi
MozReview-Commit-ID: 5N1iOpocVd8
2017-10-12 17:27:36 -07:00
Thomas Nguyen
41442f215a Bug 1378680 - Refactor usage fallible alloc in url-classifier r=francois
MozReview-Commit-ID: HHG1ByeIwZv
2017-07-20 16:20:27 +08:00
DimiL
a0b8501692 Bug 1333328 - Refactor cache miss handling mechanism for V2. r=francois
In this patch, we will make Safebrowsing V2 caching use the same algorithm as V4.
So we remove "mMissCache" for negative caching and TableFresness check for
positive caching.

But Safebrowsing V2 doesn't contain negative/positive cache duration information in
gethash response. So we hard-code a fixed value, 15 minutes, as cache duration.
In this way, we can sync the mechanism we handle caching for V2 and V4.

An extra effort for V2 here is that we need to manually record prefixes misses
because we won't get any response for those prefixes(implemented in
nsUrlClassifierLookupCallback::CacheMisses).
2017-05-04 09:38:14 +08:00
dimi
b19db734bc Bug 1311933 - P1. Use integer as the key of safebrowsing cache. r=francois
In Bug 1323953, we always send 4-bytes prefix for completion and the prefix is also
used as the key to store cache result from gethash request.
Since it is always 4-bytes, we could convert it to integer for simplicity.

MozReview-Commit-ID: Lkvrg0wvX5Z
2017-04-11 16:07:26 +08:00
dimi
5b1a0ff5b5 Bug 1311935 - P2. Process fullHashes.find response. r=francois
This patch includes following changes:

1. nsUrlClassifierHashCompleter.js
   nsUrlClassifierHashCompleter.idl
   - Add completionV4 interface for hashCompleter to pass response data to
     DB service.
   - Process response data includes negative cache duration, matched full
     hashes and cache duration for each match. Full matches are passed through
     nsIFullHashMatch interface.
   - Change _requests.responses from array contains matched fullhashes to
     dictionary so that it can store additional information likes negative cache
     duration.
2. nsUrlClassifierDBService.cpp
   - Implement CompletionV4 interface, store response data to CacheResultV4
     object. Expired duration to expired time is handled here.
   - Add CacheResultToTableUpdate function to convert V2 & V4 cache result
     to TableUpdate object.
3. LookupCache.h
   - Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
     response data in CompletionV2 and CompletionV4.
4. HashStore.h
   - Add API and member variable in TableUpdateV4 to store response data.
     TableUpdate object is used by DB service to pass update data or gethash
     response to Classifier, so we need to extend TableUpdateV4 to be able
     to store fullHashes.find response.
6. Entry.h
   - Define the structure about how we cache fullHashes.find response.

MozReview-Commit-ID: FV4yAl2SAc6
2017-04-11 11:50:48 +08:00
Iris Hsiao
cd018fd494 Backed out 4 changesets (bug 1311935) for causing assertion crash by developer's request
Backed out changeset 27e624cd9479 (bug 1311935)
Backed out changeset 4c0381ab0990 (bug 1311935)
Backed out changeset 73587838ef16 (bug 1311935)
Backed out changeset a5a6c0f79733 (bug 1311935)
2017-04-11 11:04:54 +08:00
dimi
5856f89d1f Bug 1311935 - P2. Process fullHashes.find response. r=francois
This patch includes following changes:

1. nsUrlClassifierHashCompleter.js
   nsUrlClassifierHashCompleter.idl
   - Add completionV4 interface for hashCompleter to pass response data to
     DB service.
   - Process response data includes negative cache duration, matched full
     hashes and cache duration for each match. Full matches are passed through
     nsIFullHashMatch interface.
   - Change _requests.responses from array contains matched fullhashes to
     dictionary so that it can store additional information likes negative cache
     duration.
2. nsUrlClassifierDBService.cpp
   - Implement CompletionV4 interface, store response data to CacheResultV4
     object. Expired duration to expired time is handled here.
   - Add CacheResultToTableUpdate function to convert V2 & V4 cache result
     to TableUpdate object.
3. LookupCache.h
   - Extend CacheResult to CacheResultV2 and CacheResultV4 so we can store
     response data in CompletionV2 and CompletionV4.
4. HashStore.h
   - Add API and member variable in TableUpdateV4 to store response data.
     TableUpdate object is used by DB service to pass update data or gethash
     response to Classifier, so we need to extend TableUpdateV4 to be able
     to store fullHashes.find response.
6. Entry.h
   - Define the structure about how we cache fullHashes.find response.

MozReview-Commit-ID: KgR1NASl7GC
2017-04-10 16:20:09 +08:00
Henry Chang
7f94b8915f Bug 1315097 - Build the provider dictionary on the main thread to be used everywhere. r=francois,gcp
MozReview-Commit-ID: Ft1deSNKuVB
2016-11-04 17:54:05 +08:00
Henry Chang
fff35e78ca Bug 1305484 - Save/load state and checksum to/from disk rather than prefs. r=dimi,francois
MozReview-Commit-ID: 4gmmrI9wY4c
2016-10-13 15:22:08 +08:00
Dimi Lee
d316530abf Bug 1305801 - Part 5: Support SafeBrowsing v4 partial update. r=gcp
MozReview-Commit-ID: 7OEWLaZbotS
2016-10-04 09:14:39 +08:00
Dimi Lee
e6d6ca8983 Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg
2016-09-19 11:51:01 +08:00
Phil Ringnalda
81e623a5b4 Backed out 5 changesets (bug 1305801) for ASan gtest bustage
Backed out changeset 0c95d5dec6d9 (bug 1305801)
Backed out changeset bca0e706dbc5 (bug 1305801)
Backed out changeset def8da367beb (bug 1305801)
Backed out changeset 56ceae52d847 (bug 1305801)
Backed out changeset 14457cc4c325 (bug 1305801)
2016-10-03 22:14:49 -07:00
Dimi Lee
daec409fa0 Bug 1305801 - Part 5: Support SafeBrowsing v4 partial update. r=gcp
MozReview-Commit-ID: 7OEWLaZbotS
2016-10-04 09:14:39 +08:00
Dimi Lee
340a321260 Bug 1305801 - Part 4: Store variable-length prefix to disk. r=francois, r=gcp
MozReview-Commit-ID: BMTGtgMuQdg
2016-09-19 11:51:01 +08:00
Henry Chang
8844a50a30 Bug 1254763 - Part 2: Test cases. r=gcp
MozReview-Commit-ID: JDjV3WUfgWW
2016-09-02 12:02:40 +08:00
Henry Chang
176430c017 Bug 1254763 - Part 1: Use per-provider directory for V4 databases. r=dimi,gcp
MozReview-Commit-ID: KZGWIcKAQyW
2016-09-02 12:00:00 +08:00
Henry Chang
a677ac7afb Bug 1284204 - Parse SafeBrowsing V4 updates to TableUpdateV4. r=francois.
MozReview-Commit-ID: 88AEIbosxKl
2016-08-12 11:55:48 +08:00
dimi
ac0dea5510 Bug 1254766 - Stop caching Safe Browsing completions to disk. r=gcp
MozReview-Commit-ID: 8Qnc8yymgDL
2016-08-11 08:17:39 +08:00
Nicholas Nethercote
26b6e79d96 Bug 1267550 (part 2) - Rename MOZ_WARN_UNUSED_RESULT as MOZ_MUST_USE. r=froydnj.
It's an annotation that is used a lot, and should be used even more, so a
shorter name is better.

MozReview-Commit-ID: 1VS4Dney4WX
2016-04-27 14:16:50 +10:00
Birunthan Mohanathas
a29151dc87 Bug 1182996 - Fix and add missing namespace comments. rs=ehsan
The bulk of this commit was generated by running:

  run-clang-tidy.py \
    -checks='-*,llvm-namespace-comment' \
    -header-filter=^/.../mozilla-central/.* \
    -fix
2015-07-13 08:25:42 -07:00
Jim Blandy
2bb4503427 Bug 1145051: Use MOZ_WARN_UNUSED_RESULT instead of NS_WARN_UNUSED_RESULT. r=glandium 2015-03-19 00:46:40 -07:00
Gian-Carlo Pascutto
127c8c9adf Bug 1134885 - ChunkSets should be fallible. r=mmc 2015-02-25 14:05:17 +01:00
Carsten "Tomcat" Book
ca7e3ed5a6 Backed out changeset bc7ca0aabb24 (bug 1134885) for bustage on a CLOSED TREE 2015-02-25 09:05:47 +01:00
Gian-Carlo Pascutto
e700403b56 Bug 1134885 - ChunkSets should be fallible. r=mmc 2015-02-25 08:31:11 +01:00
Ehsan Akhgari
d282557bca Bug 1061061 - Fix more bad implicit constructors in misc. code; r=bsmedberg 2014-09-02 18:24:24 -04:00
Monica Chew
ddde716c72 Bug 904607: Add protocol parser for -digest256 lists (r=gcp). 2013-09-06 17:12:33 -07:00
Daniel Holbert
44c602ba9a Bug 826052: Make HashStore explicitly reject files larger than UINT32_MAX bytes, to avoid precision issues when we pass the file-length as a uint32_t function argument. (Also, skip a redundant file-size lookup.) r=gcp 2013-01-03 00:38:24 -08:00
Saurabh Anand
15f0c5068c Bug 818817 - Fix some compiler warnings, r=Ms2ger 2012-12-09 22:53:19 +05:30
Gian-Carlo Pascutto
0aba543f82 Bug 806422 - Do not cache Complete's across a SafeBrowsing update. r=dcamp 2012-11-06 15:38:16 +01:00
Isaac Aggrey
50d6240e45 Bug 794510: Part 2: Use more stdint types instead of protypes.h types; r=ehsan 2012-10-11 18:38:04 -05:00
Gian-Carlo Pascutto
bce21c1450 Bug 727370 - Small cleanups in HashStore. r=dcamp 2012-09-28 18:31:19 +02:00
Gian-Carlo Pascutto
48c6f466f8 Bug 727370 - Improve HashStore checksumming performance. r=dcamp 2012-09-28 18:31:19 +02:00
Gian-Carlo Pascutto
a04ed76212 Bug 727370 - Make SafeBrowsing updates atomic transactions. r=dcamp 2012-09-28 18:31:18 +02:00
Randell Jesup
3b4961b603 Bug 773151: Convert nsCAutoString->nsAutoCString CLOSED TREE r=bsmedberg 2012-09-01 22:35:17 -04:00
Ehsan Akhgari
243c878d26 Bug 579517 - Part 1: Automated conversion of NSPR numeric types to stdint types in Gecko; r=bsmedberg
This patch was generated by a script.  Here's the source of the script for
future reference:

function convert() {
echo "Converting $1 to $2..."
find . ! -wholename "*nsprpub*" \
       ! -wholename "*security/nss*" \
       ! -wholename "*/.hg*" \
       ! -wholename "obj-ff-dbg*" \
       ! -name nsXPCOMCID.h \
       ! -name prtypes.h \
         -type f \
      \( -iname "*.cpp" \
         -o -iname "*.h" \
         -o -iname "*.c" \
         -o -iname "*.cc" \
         -o -iname "*.idl" \
         -o -iname "*.ipdl" \
         -o -iname "*.ipdlh" \
         -o -iname "*.mm" \) | \
    xargs -n 1 sed -i -e "s/\b$1\b/$2/g"
}

convert PRInt8 int8_t
convert PRUint8 uint8_t
convert PRInt16 int16_t
convert PRUint16 uint16_t
convert PRInt32 int32_t
convert PRUint32 uint32_t
convert PRInt64 int64_t
convert PRUint64 uint64_t

convert PRIntn int
convert PRUintn unsigned

convert PRSize size_t

convert PROffset32 int32_t
convert PROffset64 int64_t

convert PRPtrdiff ptrdiff_t

convert PRFloat64 double
2012-08-22 11:56:38 -04:00
Gian-Carlo Pascutto
f59dcebe9c Bug 673470 - Update the new SafeBrowsing code to MPL 2.0. r=dcamp 2012-08-15 09:10:56 +02:00
Gian-Carlo Pascutto
9edc9e38c8 Bug 673470 - Fix inconsistent variable naming. r=dcamp 2012-08-15 09:07:36 +02:00
Gian-Carlo Pascutto
6301b08636 Bug 726002 - Clear some big nsTArrays as early as possible in updates. r=dcamp 2012-08-15 09:05:18 +02:00
Gian-Carlo Pascutto
6cb825e03c Bug 673470 - Optimize input buffer size. Cache active tables. r=dcamp 2012-08-15 09:04:31 +02:00
Gian-Carlo Pascutto
1c0ff29874 Bug 673470 - Replace the sqlite safeb store with a flat file. r=dcamp 2012-08-15 09:04:19 +02:00
Gian-Carlo Pascutto
10acc9e06e Backout 173f90d397a8 (Bug 673470). rs=dcamp a=mfinkle 2012-04-20 07:46:47 +02:00
Gian-Carlo Pascutto
e5fefd0538 Backout db52b4916cde (Bug 673470). rs=dcamp a=mfinkle 2012-04-20 07:46:46 +02:00
Gian-Carlo Pascutto
f27f8b25c1 Backout 35bf0d62cc30 (Bug 726002). rs=dcamp a=mfinkle 2012-04-20 07:46:46 +02:00
Gian-Carlo Pascutto
a6f2f9cd49 Bug 726002 - Clear some big nsTArrays as early as possible in updates. r=dcamp 2012-02-13 17:10:22 +01:00
Gian-Carlo Pascutto
6688e97a43 Backout e0e2cc5570ac (Bug 726002) due to broken error handling. 2012-02-13 17:04:46 +01:00
Gian-Carlo Pascutto
d465e43455 Bug 726002 - Clear some big nsTArrays as early as possible in updates. r=dcamp 2012-02-13 13:17:48 +01:00
Gian-Carlo Pascutto
b7cc115c04 Bug 673470 - Optimize input buffer size. Cache active tables. r=dcamp 2012-02-03 08:32:32 +01:00
Gian-Carlo Pascutto
724fb285e2 Bug 673470 - Replace the SQLite SafeBrowsing store with an optimized store. r=dcamp 2011-12-06 19:03:05 +01:00
Gian-Carlo Pascutto
056c10da3c Backout 44a0dc4fb9ff (Bug 673470) for Talos regression. 2012-02-01 20:10:02 +01:00
Gian-Carlo Pascutto
3b9f32d92d Bug 673470 - Replace the SQLite SafeBrowsing store with an optimized store. r=dcamp 2011-12-06 19:03:05 +01:00