This patch changes our preference look-up behavior to first check the dynamic
hashtable, and then fall back to the shared map.
In order for this to work, we need to make several other changes as well:
- Attempts to modify a preference that only exists in the shared table
requires that we copy it to the dynamic table, and change the value of the
new entry.
- Attempts to clear a user preference with no default value, but which also
exists in the shared map, requires that we keep an entry in the dynamic
table to mask the shared entry. To make this work, we change the type of
these entries to None, and ignore them during look-ups and iteration.
- Iteration needs to take both hashtables into consideration. The
serialization iterator for changed preferences only needs to care about
dynamic values, so it remains unchanged. Most of the others need to use
PrefsIter() instead.
MozReview-Commit-ID: 9PWmSZxoC9Z
Since lookups in the snapshotted hashtable are currently O(log n) rather than
O(1), they're expected to be slightly more expensive than the previous
purely-dynamic lookups.
In general, I expect this not to be a major issue. The main concern is pref
cache lookups while initializing the database. Initialization in the parent
process will still always use only a dynamic hashtable, so the performance
characteristics there won't change.
In the child process, though, we'll still need to notify observers of
preferences in the snapshot when they have user values, which could require
any number of lookups at startup (though in practice probably will not).
This patch solves that problem by caching the wrapper for the current known
preference value when dispatching callbacks, and optimizing lookups for that
value when it is present.
MozReview-Commit-ID: 2CAn0rM0bE9
For memory efficiency in content processes, we need to be able to store
changed preferences in a separate dynamic hashtable when their values don't
match the snapshot values.
That makes iteration over the full set of preferences somewhat more
complicated, since not only do we need to iterate over two tables, but we also
need to ignore preferences in the snapshot table if they also exist in the
dynamic hashtable.
This patch solves that problem by adding an iterator helper which iterates
over values in both tables, and skips values in the static table if they also
exist in the dynamic table.
In order to support completely deleting preferences that exist in the base
table, it also ignores all dynamic entries with the None type, so that they
can completely mask deleted base table values.
MozReview-Commit-ID: LCIwyPJMByj
The in-memory format of shared-memory preferences is significantly different
from the format used by dynamic preferences, which means that we need
different classes to access their properties.
Virtual classes would be a potential solution to this problem, but I don't
think the performance characteristics would be acceptable for preferences
code. And since the wrapper classes used for static prefs are temporary, they
would add the additional snag of figuring out how to keep a valid pointer
alive.
So, instead, this patch adds a wrapper class that can access either type of
preference, based on known type flags in a Variant. It also moves some of the
logic for deciding which preference value to return to the wrapper, so that it
doesn't need to be duplicated for each representation.
MozReview-Commit-ID: LameIIbYcD3
This is based on the SharedStringMap that's currently used for shared memory
string bundles.
When the parent process is ready to launch its first content process, it
creates a snapshot of the current state of the preference database, maps that
as read-only, and shares it with each content process. Look-ups in the
snapshotted map are done entirely using data in the shared memory region. It
doesn't require any additional per-process state data.
MozReview-Commit-ID: BdTUhak7dmS
Once the majority of preferences are stored in a read-only shared map, it
won't be possible to modify the access counts in their entries. Which means we
need a separate map for the access counts.
Fortunately, this code is only active in debug builds, so it shouldn't affect
release users. And the net impact on memory usage of this patchset will still
be decidedly downward.
MozReview-Commit-ID: EuLXlMQJP1M
This reduced the additional string duplication that we currently do every time
we add a preference observer.
It changes the string that we store in the observer objects to be absolute,
rather than relative to the branch, but keeps the semantics the same, by
resolving the full preference name in the places we were previously matching
by relative string.
This actually has the effect of simplifying a lot of code, since the absolute
preference name is usually what we want.
MozReview-Commit-ID: 10WjHb0tNGB
Most preference callbacks use literal strings for their domain filters, which
means that there's no need to make copies of them at all. Currently, however,
every preference observer node makes a separate heap-allocated copy of its
domain string.
This patch switches the domain string storage to nsCString instances, which
dramatically reduces the amount of unnecessary copies, at the expense of
making the callback nodes slightly larger.
MozReview-Commit-ID: 8NA3t2JS2UI
This removes our ability to detect when an unnecessary override is occurring,
but it's necessary for Thunderbird to work.
MozReview-Commit-ID: GZYLnEEVuvd
Currently VarCache prefs are setup in two parts:
- The vanilla pref part, installed via a data file such as all.js, or via an
API call.
- The VarCache variable part, setup by an Add*VarCache() call.
Both parts are needed for the pref to actually operate as a proper VarCache
pref. (There are various prefs for which we do one but not the other unless a
certain condition is met.)
This patch introduces a new way of doing things. There is a new file,
modules/libpref/init/StaticPrefList.h, which defines prefs like this:
> VARCACHE_PREF(
> "layout.accessiblecaret.width",
> layout_accessiblecaret_width,
> float, 34.0
> )
This replaces both the existing parts.
The preprocessor is used to generate multiple things from this single
definition:
- A global variable (the VarCache itself).
- A getter for that global variable.
- A call to an init function that unconditionally installs the pref in the
prefs hash table at startup.
C++ files can include the new StaticPrefs.h file to access the getter.
Rust code cannot use the getter, but can access the global variable directly
via structs.rs. This is similar to how things currently work for Rust code.
Non-VarCache prefs can also be declared in StaticPrefList.h, using PREF instead
of the VARCACHE_PREF.
The new approach has the following advantages.
+ It eliminates the duplication (in all.js and the Add*VarCache() call) of the
pref name and default value, preventing potential mismatches. (This is a real
problem in practice!)
+ There is now a single initialization point for these VarCache prefs.
+ This avoids need to find a place to insert the Add*VarCache() calls, which
are currently spread all over the place.
+ It also eliminates the common pattern whereby these calls are wrapped in a
execute-once block protected by a static boolean (see bug 1346224).
+ It's no longer possible to have a VarCache pref for which only one of the
pieces has been setup.
+ It encapsulates the VarCache global variable, so there is no need to declare
it separately.
+ VarCache reads are done via a getter (e.g. StaticPrefs::foo_bar_baz())
instead of a raw global variable read.
+ This makes it clearer that you're reading a pref value, and easier to
search for uses.
+ This prevents accidental writes to the global variable.
+ This prevents accidental mistyping of the pref name.
+ This provides a single chokepoint in the code for such accesses, which make
adding checking and instrumentation feasible.
+ It subsumes MediaPrefs, and will allow that class to be removed. (gfxPrefs is
a harder lift, unfortunately.)
+ Once all VarCache prefs are migrated to the new approach, the VarCache
mechanism will be better encapsulated, with fewer details publicly visible.
+ (Future work) This will allow the pref names to be stored statically, saving
memory in every process.
The main downside of the new approach is that all of these prefs are in a
single header that is included in quite a few places, so any changes to this
header will cause a fair amount of recompilation.
Another minor downside is that all VarCache prefs are defined and visible from
start-up. For test-only prefs like network.predictor.doing-tests, having them
show in about:config isn't particularly useful.
The patch also moves three network VarCache prefs to the new mechanism as a
basic demonstration. (And note the inconsistencies in the multiple initial
values that were provided for
network.auth.subresource-img-cross-origin-http-auth-allow!) There will be
numerous follow-up bugs to convert the remaining VarCache prefs.
MozReview-Commit-ID: 9ABNpOR16uW
* * *
[mq]: fixup
MozReview-Commit-ID: 6ToT9dQjIAq
Currently all pref initialization is done from file, but soon we will also be
initializing prefs from code compiled into the binary. The new name encompasses
both cases.
MozReview-Commit-ID: 5g0jfjHTvnE
All prefs that need to be sent to a new content process are now put into the
shared memory segment, and they are identified by the pref name instead of an
index into a list. The old IPC used at process startup (in XPCOMInitData) is
removed.
Benefits:
- It removes the need for the early prefs list
(dom/ipc/ContentProcesses.{h,cpp}) and the associated checking, which is ugly
and often trips people up (e.g. bug 1432979, bug 1439406).
- Using prefnames instead of indices fixes some fragility (fixing bug 1419432).
- It fixes the problem of early prefs being installed as unlocked default
values even if they are locked and/or have user values.
MozReview-Commit-ID: FRIzHF8Tjd
These statistics will be used by browser tests to analyze frequently accessed
preferences so that we can recommend using preference observers instead.
MozReview-Commit-ID: 9uZnwmjZL4U
These statistics will be used by browser tests to analyze frequently accessed
preferences so that we can recommend using preference observers instead.
MozReview-Commit-ID: 9uZnwmjZL4U
This patch doesn't change the functionality, it just splits out the code into
separate functions that are easier to read.
MozReview-Commit-ID: Gx05YCxGgve
This patch replaces the large -intPrefs/-boolPrefs/-stringPrefs flags with
a short-lived, anonymous, shared memory segment that is used to pass the early
prefs.
Removing the bloat from the command line is nice, but more important is the
fact that this will let us pass more prefs at content process start-up, which
will allow us to remove the early/late prefs split (bug 1436911).
Although this mechanism is only used for prefs, it's conceivable that it could
be used for other data that must be received very early by children, and for
which the command line isn't ideal.
Notable details:
- Much of the patch deals with the various platform-specific ways of passing
handles/fds to children.
- Linux and Mac: we use a fixed fd (8) in combination with the new
GeckoChildProcessHost::AddFdToRemap() function (which ensures the child
won't close the fd).
- Android: like Linux and Mac, but the handles get passed via "parcels" and
we use the new SetPrefsFd() function instead of the fixed fd.
- Windows: there is no need to duplicate the handle because Windows handles
are system-wide. But we do use the new
GeckoChildProcessHost::AddHandleToShare() function to add it to the list of
inheritable handles. We also ensure that list is processed on all paths
(MOZ_SANDBOX with sandbox, MOZ_SANDBOX without sandbox, non-MOZ_SANDBOX) so
that the handles are marked as inheritable. The handle is passed via the
-prefsHandle flag.
The -prefsLen flag is used on all platforms to indicate the size of the
shared memory segment.
- The patch also moves the serialization/deserialization of the prefs in/out of
the shared memory into libpref, which is a better spot for it. (This means
Preferences::MustSendToContentProcesses() can be removed.)
MozReview-Commit-ID: 8fREEBiYFvc
Sticky prefs are already specifiable with `sticky_pref`, but this is a more
general attribute mechanism. The ability to specify a locked pref in the data
file is new.
The patch also adds nsIPrefService.readDefaultPrefsFromFile, to match the
existing nsIPrefService.readUserPrefsFromFile method, and converts a number of
the existing tests to use it.
MozReview-Commit-ID: 9LLMBJVZfg7
It optimizes Preferences::IsLocked(), but that function is called fewer than
200 times while starting the browser and opening a range of tabs.
MozReview-Commit-ID: 5q0zS8TSwdu
The following table shows the effect of this change:
> old 64-bit new 64-bit old 32-bit new 32-bit
> sizeof(CallbackNode) 40 32 20 16
> size when heap allocated 48 32 32 16
This reduces memory usage by about 15--40 KB per process.
MozReview-Commit-ID: 4gXgGI3SiJz
This shows that the objects themselves are accounting for about 60% of the
callback memory on 64-bit, and the domains are about 40%.
MozReview-Commit-ID: JndlyIvlrGs
Before Firefox 58 we collected extended collection from users on nightly,
aurora, and beta. Then we had to change things (see bug 1406391).
In doing so, we accidentally stopped receiving data from "release candidate"
beta builds. This patch resumes that collection by detecting an RC build as
having a MOZ_UPDATE_CHANNEL of "release", but an app.update.channel of "beta"
MozReview-Commit-ID: 3EzzDtQj8Kw
This was first suggested 17 years ago!
The error recovery works by just scanning forward for the next ';' token.
This change allows a lot of the gtest tests to be combined.
MozReview-Commit-ID: CbZ2OFtdIxf
The meaning of "possibly-changed" is provided by the big comment above
MustSendToContentProcesses.
On a new profile this reduces the number of prefs sent like so:
- Command-line: 222 --> 3
- IPC: 3129 --> 130
On an older profile:
- Command-line: 222 --> 3
- IPC: 3165 --> 180
MozReview-Commit-ID: DcgedhXhZd8
This has two advantages. First, it reduces memory usage, as per the following
calculation.
64-bit:
- Old sizes:
- sizeof(Pref) = 32
- New sizes:
- sizeof(PrefEntry) = 16
- sizeof(Pref) = 32
- Change:
- -16 per empty slot in the hash table
- +16 per used slot
- A win if less than half the table slots are used
32-bit
- Old sizes:
- sizeof(Pref) = 20
- New sizes:
- sizeof(PrefEntry) = 8
- sizeof(Pref) = 16
- Change:
- -12 per empty slot in the hash table
- +4 per used slot in the hash table
- A win if table is < 75% full
Table size:
- The table is currently less than half full: ~3100 used out of 8192 slots.
- The table is always <= 75% full, because that's the max load factor (for
non-gigantic tables).
- Therefore it's a win for both cases.
Old sizes, chrome process, 64-bit:
> 718,712 B (00.36%) -- preferences
> +--262,176 B (00.13%) -- hash-table
> +--197,384 B (00.10%) -- callbacks
> +--114,688 B (00.06%) -- pref-name-arena
> +---92,240 B (00.05%) -- root-branches
> +---30,456 B (00.02%) -- string-values
> +---21,688 B (00.01%) -- cache-data
> +-------80 B (00.00%) -- misc
New sizes, chrome process, 64-bit:
> 672,568 B (00.41%) -- preferences
> +--181,160 B (00.11%) -- callbacks
> +--131,104 B (00.08%) -- hash-table # smaller
> +--114,688 B (00.07%) -- pref-name-arena
> +--101,152 B (00.06%) -- pref-values # new
> +---92,240 B (00.06%) -- root-branches
> +---30,456 B (00.02%) -- string-values
> +---21,688 B (00.01%) -- cache-data
> +-------80 B (00.00%) -- misc
Old sizes, smallest content process, 64-bit:
> 500,712 B (02.89%) -- preferences
> +--262,176 B (01.51%) -- hash-table
> +--114,688 B (00.66%) -- pref-name-arena
> +---62,520 B (00.36%) -- callbacks
> +---30,456 B (00.18%) -- string-values
> +---17,832 B (00.10%) -- cache-data
> +---12,960 B (00.07%) -- root-branches
> +-------80 B (00.00%) -- misc
New sizes, smallest content process, 64-bit:
> 470,792 B (02.70%) -- preferences
> +--131,104 B (00.75%) -- hash-table # smaller
> +--114,688 B (00.66%) -- pref-name-arena
> +--101,152 B (00.58%) -- pref-values # new
> +---62,520 B (00.36%) -- callbacks
> +---30,456 B (00.17%) -- string-values
> +---17,832 B (00.10%) -- cache-data
> +---12,960 B (00.07%) -- root-branches
> +-------80 B (00.00%) -- misc
The "hash-table" values drop by more than the size of the new "pref-values"
value.
On 64-bit, this reduces memory usage per process by 30--40 KB. On 32-bit, the
number is slightly more.
The second major advantage of this change is flexibility -- it opens up the
possibility of different Pref objects being stored in different way. For
example, static Prefs could be stared statically, letting them be shared
between processes so long as they don't change (see bug 1437168).
MozReview-Commit-ID: KmgbJaoOQ1J
This construct is nicer than NS_INTERFACE_MAP_BEGIN and assures the
reader there's no weirdness in the QI implementation. This change does
mean that PGO doesn't get an opportunity to measure the frequency of
which interfaces are QI'd most often. I think this is probably an OK
tradeoff to make, given the prevalence of NS_IMPL_QUERY_INTERFACE
elsewhere in the codebase.
The one thing we have to ensure with this change is that the ambiguous
QI to nsISupports uses the proper class after the change. The
NS_IMPL_QUERY_INTERFACE macro chooses the first interface listed to
disambiguate the cast to nsISupports.
This lets us have a proper constructor for Pref, which is nice.
The patch also adds a missing case to PrefTypeToString(), and reorders the
fields in Pref to be more sensible.
MozReview-Commit-ID: A01ULF4q08O
They currently fail to pass on `aKind`, always getting the user value (when
possible). There are three callsites that are affected:
- nsSHistory::Startup, docshell/shistory/nsSHistory.cpp.
- FeatureState::SetDefaultFromPref(), in gfx/config/gfxFeature.cpp.
- gfxPlatform::InitOMTPConfig(), in gfx/thebes/gfxPlatform.cpp.
The patch also adds a gtest that would have failed prior to the fix.
MozReview-Commit-ID: L0U1XQmPUFc
This patch rearranges these functions so that nsPrefBranch::GetPrefType() calls
into Preferences::GetType(), because all other nsPrefBranch methods depend on
Preferences methods.
The patch also removes the `aKind` argument from GetType(), because it has no
effect -- a pref only has one type, regardless of whether it has a default
value, a user value, or both.
MozReview-Commit-ID: J3vxFPaP8S3
