This reduced the additional string duplication that we currently do every time
we add a preference observer.
It changes the string that we store in the observer objects to be absolute,
rather than relative to the branch, but keeps the semantics the same, by
resolving the full preference name in the places we were previously matching
by relative string.
This actually has the effect of simplifying a lot of code, since the absolute
preference name is usually what we want.
MozReview-Commit-ID: 10WjHb0tNGB
Most preference callbacks use literal strings for their domain filters, which
means that there's no need to make copies of them at all. Currently, however,
every preference observer node makes a separate heap-allocated copy of its
domain string.
This patch switches the domain string storage to nsCString instances, which
dramatically reduces the amount of unnecessary copies, at the expense of
making the callback nodes slightly larger.
MozReview-Commit-ID: 8NA3t2JS2UI
This removes our ability to detect when an unnecessary override is occurring,
but it's necessary for Thunderbird to work.
MozReview-Commit-ID: GZYLnEEVuvd
Currently VarCache prefs are setup in two parts:
- The vanilla pref part, installed via a data file such as all.js, or via an
API call.
- The VarCache variable part, setup by an Add*VarCache() call.
Both parts are needed for the pref to actually operate as a proper VarCache
pref. (There are various prefs for which we do one but not the other unless a
certain condition is met.)
This patch introduces a new way of doing things. There is a new file,
modules/libpref/init/StaticPrefList.h, which defines prefs like this:
> VARCACHE_PREF(
> "layout.accessiblecaret.width",
> layout_accessiblecaret_width,
> float, 34.0
> )
This replaces both the existing parts.
The preprocessor is used to generate multiple things from this single
definition:
- A global variable (the VarCache itself).
- A getter for that global variable.
- A call to an init function that unconditionally installs the pref in the
prefs hash table at startup.
C++ files can include the new StaticPrefs.h file to access the getter.
Rust code cannot use the getter, but can access the global variable directly
via structs.rs. This is similar to how things currently work for Rust code.
Non-VarCache prefs can also be declared in StaticPrefList.h, using PREF instead
of the VARCACHE_PREF.
The new approach has the following advantages.
+ It eliminates the duplication (in all.js and the Add*VarCache() call) of the
pref name and default value, preventing potential mismatches. (This is a real
problem in practice!)
+ There is now a single initialization point for these VarCache prefs.
+ This avoids need to find a place to insert the Add*VarCache() calls, which
are currently spread all over the place.
+ It also eliminates the common pattern whereby these calls are wrapped in a
execute-once block protected by a static boolean (see bug 1346224).
+ It's no longer possible to have a VarCache pref for which only one of the
pieces has been setup.
+ It encapsulates the VarCache global variable, so there is no need to declare
it separately.
+ VarCache reads are done via a getter (e.g. StaticPrefs::foo_bar_baz())
instead of a raw global variable read.
+ This makes it clearer that you're reading a pref value, and easier to
search for uses.
+ This prevents accidental writes to the global variable.
+ This prevents accidental mistyping of the pref name.
+ This provides a single chokepoint in the code for such accesses, which make
adding checking and instrumentation feasible.
+ It subsumes MediaPrefs, and will allow that class to be removed. (gfxPrefs is
a harder lift, unfortunately.)
+ Once all VarCache prefs are migrated to the new approach, the VarCache
mechanism will be better encapsulated, with fewer details publicly visible.
+ (Future work) This will allow the pref names to be stored statically, saving
memory in every process.
The main downside of the new approach is that all of these prefs are in a
single header that is included in quite a few places, so any changes to this
header will cause a fair amount of recompilation.
Another minor downside is that all VarCache prefs are defined and visible from
start-up. For test-only prefs like network.predictor.doing-tests, having them
show in about:config isn't particularly useful.
The patch also moves three network VarCache prefs to the new mechanism as a
basic demonstration. (And note the inconsistencies in the multiple initial
values that were provided for
network.auth.subresource-img-cross-origin-http-auth-allow!) There will be
numerous follow-up bugs to convert the remaining VarCache prefs.
MozReview-Commit-ID: 9ABNpOR16uW
* * *
[mq]: fixup
MozReview-Commit-ID: 6ToT9dQjIAq
Currently all pref initialization is done from file, but soon we will also be
initializing prefs from code compiled into the binary. The new name encompasses
both cases.
MozReview-Commit-ID: 5g0jfjHTvnE
All prefs that need to be sent to a new content process are now put into the
shared memory segment, and they are identified by the pref name instead of an
index into a list. The old IPC used at process startup (in XPCOMInitData) is
removed.
Benefits:
- It removes the need for the early prefs list
(dom/ipc/ContentProcesses.{h,cpp}) and the associated checking, which is ugly
and often trips people up (e.g. bug 1432979, bug 1439406).
- Using prefnames instead of indices fixes some fragility (fixing bug 1419432).
- It fixes the problem of early prefs being installed as unlocked default
values even if they are locked and/or have user values.
MozReview-Commit-ID: FRIzHF8Tjd
These statistics will be used by browser tests to analyze frequently accessed
preferences so that we can recommend using preference observers instead.
MozReview-Commit-ID: 9uZnwmjZL4U
These statistics will be used by browser tests to analyze frequently accessed
preferences so that we can recommend using preference observers instead.
MozReview-Commit-ID: 9uZnwmjZL4U
This patch doesn't change the functionality, it just splits out the code into
separate functions that are easier to read.
MozReview-Commit-ID: Gx05YCxGgve
This patch replaces the large -intPrefs/-boolPrefs/-stringPrefs flags with
a short-lived, anonymous, shared memory segment that is used to pass the early
prefs.
Removing the bloat from the command line is nice, but more important is the
fact that this will let us pass more prefs at content process start-up, which
will allow us to remove the early/late prefs split (bug 1436911).
Although this mechanism is only used for prefs, it's conceivable that it could
be used for other data that must be received very early by children, and for
which the command line isn't ideal.
Notable details:
- Much of the patch deals with the various platform-specific ways of passing
handles/fds to children.
- Linux and Mac: we use a fixed fd (8) in combination with the new
GeckoChildProcessHost::AddFdToRemap() function (which ensures the child
won't close the fd).
- Android: like Linux and Mac, but the handles get passed via "parcels" and
we use the new SetPrefsFd() function instead of the fixed fd.
- Windows: there is no need to duplicate the handle because Windows handles
are system-wide. But we do use the new
GeckoChildProcessHost::AddHandleToShare() function to add it to the list of
inheritable handles. We also ensure that list is processed on all paths
(MOZ_SANDBOX with sandbox, MOZ_SANDBOX without sandbox, non-MOZ_SANDBOX) so
that the handles are marked as inheritable. The handle is passed via the
-prefsHandle flag.
The -prefsLen flag is used on all platforms to indicate the size of the
shared memory segment.
- The patch also moves the serialization/deserialization of the prefs in/out of
the shared memory into libpref, which is a better spot for it. (This means
Preferences::MustSendToContentProcesses() can be removed.)
MozReview-Commit-ID: 8fREEBiYFvc
Sticky prefs are already specifiable with `sticky_pref`, but this is a more
general attribute mechanism. The ability to specify a locked pref in the data
file is new.
The patch also adds nsIPrefService.readDefaultPrefsFromFile, to match the
existing nsIPrefService.readUserPrefsFromFile method, and converts a number of
the existing tests to use it.
MozReview-Commit-ID: 9LLMBJVZfg7
It optimizes Preferences::IsLocked(), but that function is called fewer than
200 times while starting the browser and opening a range of tabs.
MozReview-Commit-ID: 5q0zS8TSwdu
The following table shows the effect of this change:
> old 64-bit new 64-bit old 32-bit new 32-bit
> sizeof(CallbackNode) 40 32 20 16
> size when heap allocated 48 32 32 16
This reduces memory usage by about 15--40 KB per process.
MozReview-Commit-ID: 4gXgGI3SiJz
This shows that the objects themselves are accounting for about 60% of the
callback memory on 64-bit, and the domains are about 40%.
MozReview-Commit-ID: JndlyIvlrGs
Before Firefox 58 we collected extended collection from users on nightly,
aurora, and beta. Then we had to change things (see bug 1406391).
In doing so, we accidentally stopped receiving data from "release candidate"
beta builds. This patch resumes that collection by detecting an RC build as
having a MOZ_UPDATE_CHANNEL of "release", but an app.update.channel of "beta"
MozReview-Commit-ID: 3EzzDtQj8Kw
This was first suggested 17 years ago!
The error recovery works by just scanning forward for the next ';' token.
This change allows a lot of the gtest tests to be combined.
MozReview-Commit-ID: CbZ2OFtdIxf
The meaning of "possibly-changed" is provided by the big comment above
MustSendToContentProcesses.
On a new profile this reduces the number of prefs sent like so:
- Command-line: 222 --> 3
- IPC: 3129 --> 130
On an older profile:
- Command-line: 222 --> 3
- IPC: 3165 --> 180
MozReview-Commit-ID: DcgedhXhZd8
This has two advantages. First, it reduces memory usage, as per the following
calculation.
64-bit:
- Old sizes:
- sizeof(Pref) = 32
- New sizes:
- sizeof(PrefEntry) = 16
- sizeof(Pref) = 32
- Change:
- -16 per empty slot in the hash table
- +16 per used slot
- A win if less than half the table slots are used
32-bit
- Old sizes:
- sizeof(Pref) = 20
- New sizes:
- sizeof(PrefEntry) = 8
- sizeof(Pref) = 16
- Change:
- -12 per empty slot in the hash table
- +4 per used slot in the hash table
- A win if table is < 75% full
Table size:
- The table is currently less than half full: ~3100 used out of 8192 slots.
- The table is always <= 75% full, because that's the max load factor (for
non-gigantic tables).
- Therefore it's a win for both cases.
Old sizes, chrome process, 64-bit:
> 718,712 B (00.36%) -- preferences
> +--262,176 B (00.13%) -- hash-table
> +--197,384 B (00.10%) -- callbacks
> +--114,688 B (00.06%) -- pref-name-arena
> +---92,240 B (00.05%) -- root-branches
> +---30,456 B (00.02%) -- string-values
> +---21,688 B (00.01%) -- cache-data
> +-------80 B (00.00%) -- misc
New sizes, chrome process, 64-bit:
> 672,568 B (00.41%) -- preferences
> +--181,160 B (00.11%) -- callbacks
> +--131,104 B (00.08%) -- hash-table # smaller
> +--114,688 B (00.07%) -- pref-name-arena
> +--101,152 B (00.06%) -- pref-values # new
> +---92,240 B (00.06%) -- root-branches
> +---30,456 B (00.02%) -- string-values
> +---21,688 B (00.01%) -- cache-data
> +-------80 B (00.00%) -- misc
Old sizes, smallest content process, 64-bit:
> 500,712 B (02.89%) -- preferences
> +--262,176 B (01.51%) -- hash-table
> +--114,688 B (00.66%) -- pref-name-arena
> +---62,520 B (00.36%) -- callbacks
> +---30,456 B (00.18%) -- string-values
> +---17,832 B (00.10%) -- cache-data
> +---12,960 B (00.07%) -- root-branches
> +-------80 B (00.00%) -- misc
New sizes, smallest content process, 64-bit:
> 470,792 B (02.70%) -- preferences
> +--131,104 B (00.75%) -- hash-table # smaller
> +--114,688 B (00.66%) -- pref-name-arena
> +--101,152 B (00.58%) -- pref-values # new
> +---62,520 B (00.36%) -- callbacks
> +---30,456 B (00.17%) -- string-values
> +---17,832 B (00.10%) -- cache-data
> +---12,960 B (00.07%) -- root-branches
> +-------80 B (00.00%) -- misc
The "hash-table" values drop by more than the size of the new "pref-values"
value.
On 64-bit, this reduces memory usage per process by 30--40 KB. On 32-bit, the
number is slightly more.
The second major advantage of this change is flexibility -- it opens up the
possibility of different Pref objects being stored in different way. For
example, static Prefs could be stared statically, letting them be shared
between processes so long as they don't change (see bug 1437168).
MozReview-Commit-ID: KmgbJaoOQ1J
This construct is nicer than NS_INTERFACE_MAP_BEGIN and assures the
reader there's no weirdness in the QI implementation. This change does
mean that PGO doesn't get an opportunity to measure the frequency of
which interfaces are QI'd most often. I think this is probably an OK
tradeoff to make, given the prevalence of NS_IMPL_QUERY_INTERFACE
elsewhere in the codebase.
The one thing we have to ensure with this change is that the ambiguous
QI to nsISupports uses the proper class after the change. The
NS_IMPL_QUERY_INTERFACE macro chooses the first interface listed to
disambiguate the cast to nsISupports.
This lets us have a proper constructor for Pref, which is nice.
The patch also adds a missing case to PrefTypeToString(), and reorders the
fields in Pref to be more sensible.
MozReview-Commit-ID: A01ULF4q08O
They currently fail to pass on `aKind`, always getting the user value (when
possible). There are three callsites that are affected:
- nsSHistory::Startup, docshell/shistory/nsSHistory.cpp.
- FeatureState::SetDefaultFromPref(), in gfx/config/gfxFeature.cpp.
- gfxPlatform::InitOMTPConfig(), in gfx/thebes/gfxPlatform.cpp.
The patch also adds a gtest that would have failed prior to the fix.
MozReview-Commit-ID: L0U1XQmPUFc
This patch rearranges these functions so that nsPrefBranch::GetPrefType() calls
into Preferences::GetType(), because all other nsPrefBranch methods depend on
Preferences methods.
The patch also removes the `aKind` argument from GetType(), because it has no
effect -- a pref only has one type, regardless of whether it has a default
value, a user value, or both.
MozReview-Commit-ID: J3vxFPaP8S3
Specifically:
- Make the warning about editing in all-caps;
- Make it clear that about:config is a browser thing;
- Add a mention of the user.js file;
- Use C++ comments, because I prefer them to C comments and I am the module
owner :)
MozReview-Commit-ID: 9GXS5nNHywO
The prefs parser has two significant problems.
- It doesn't separate tokenizing from parsing.
- It is implemented as a loop around a big switch on a "current state"
variable.
As a result, it is hard to understand and modify, slower than it could be, and
in obscure cases (involving comments and whitespace) it fails to parse what
should be valid input.
This patch replaces it with a recursive descent parser (albeit one without any
recursion!) that has separate tokenization. The new parser is easier to
understand and modify, more correct, and has better error messages. It doesn't
do error recovery, but that would be much easier to add than in the old parser.
The new parser also runs about 1.9x faster than the existing parser. (As
measured by parsing greprefs.js's contents from memory 1000 times in
succession, omitting the prefs hash table construction. If the table
construction is included, it's about 1.6x faster.)
The new parser is slightly stricter than the old parser in a few ways.
- Disconcertingly, the old parser allowed arbitrary junk between prefs
(including at the start and end of the prefs file) so long as that junk
didn't include any of the following chars: '/', '#', 'u', 's', 'p'. I.e.
lines like these:
!foo@bar&pref("prefname", true);
ticky_pref("prefname", true); // missing 's' at start
User_pref("prefname", true); // should be 'u' at start
would all be treated the same as this:
pref("prefname", true);
The new parser disallows such junk because it isn't necessary and seems like
an unintentional botch by the old parser.
- The old parser allowed character 0x1a (SUB) between tokens and treated it
like '\n'.
The new parser does not allow this character. SUB was used to indicate
end-of-file (*not* end-of-line) in some old operating systems such as MS-DOS,
but this doesn't seem necessary today.
- The old parser tolerated (with a warning) invalid escape sequences within
string literals -- such as "\q" (not a valid escape) and "\x1" and "\u12"
(both of which have insufficient hex digits) -- accepting them literally.
The new parser does not tolerate invalid escape sequences because it doesn't
seem necessary and would complicate things.
- The old parser tolerated character 0x00 (NUL) within string literals; this is
dangerous because C++ code that manipulates string values with embedded NULs
will almost certainly consider those chars as end-of-string markers.
The new parser treats NUL chars as end-of-file, to avoid this danger and
because it facilitates a significant optimization (described within the
code).
- The old parser allowed integer literals to overflow, silently wrapping them.
The new parser treats integer overflow as a parse error. This seems better,
and it caught existing overflows of places.database.lastMaintenance, in
testing/profiles/prefs_general.js (bug 1424030) and
testing/talos/talos/config.py (bug 1434813).
The first of these changes meant that a couple of existing prefs with ";;" at
the end had to be changed (done in the preceding patch).
The minor increase in strictness shouldn't be a problem for default pref files
such as greprefs.js within the application (which we can modify), nor for
app-written prefs files such as prefs.js. It could affect user-written prefs
files such as user.js; the experience above suggests that integer overflow and
";;" are the most likely problems in practice. In my opinion, the risk here is
acceptable.
The new parser also does a better job of tracking line numbers because it (a)
treats "\r\n" sequences as a single end-of-line marker, and (a) pays attention
to end-of-line sequences within string literals.
Finally, the patch adds thorough tests of both valid and invalid syntax.
MozReview-Commit-ID: JD3beOQl4AJ
The prefs parser has two significant problems.
- It doesn't separate tokenizing from parsing.
- It is implemented as a loop around a big switch on a "current state"
variable.
As a result, it is hard to understand and modify, slower than it could be, and
in obscure cases (involving comments and whitespace) it fails to parse what
should be valid input.
This patch replaces it with a recursive descent parser (albeit one without any
recursion!) that has separate tokenization. The new parser is easier to
understand and modify, more correct, and has better error messages. It doesn't
do error recovery, but that would be much easier to add than in the old parser.
The new parser also runs about 1.9x faster than the existing parser. (As
measured by parsing greprefs.js's contents from memory 1000 times in
succession, omitting the prefs hash table construction. If the table
construction is included, it's about 1.6x faster.)
The new parser is slightly stricter than the old parser in a few ways.
- Disconcertingly, the old parser allowed arbitrary junk between prefs
(including at the start and end of the prefs file) so long as that junk
didn't include any of the following chars: '/', '#', 'u', 's', 'p'. I.e.
lines like these:
!foo@bar&pref("prefname", true);
ticky_pref("prefname", true); // missing 's' at start
User_pref("prefname", true); // should be 'u' at start
would all be treated the same as this:
pref("prefname", true);
The new parser disallows such junk because it isn't necessary and seems like
an unintentional botch by the old parser.
- The old parser allowed character 0x1a (SUB) between tokens and treated it
like '\n'.
The new parser does not allow this character. SUB was used to indicate
end-of-file (*not* end-of-line) in some old operating systems such as MS-DOS,
but this doesn't seem necessary today.
- The old parser tolerated (with a warning) invalid escape sequences within
string literals -- such as "\q" (not a valid escape) and "\x1" and "\u12"
(both of which have insufficient hex digits) -- accepting them literally.
The new parser does not tolerate invalid escape sequences because it doesn't
seem necessary and would complicate things.
- The old parser tolerated character 0x00 (NUL) within string literals; this is
dangerous because C++ code that manipulates string values with embedded NULs
will almost certainly consider those chars as end-of-string markers.
The new parser treats NUL chars as end-of-file, to avoid this danger and
because it facilitates a significant optimization (described within the
code).
- The old parser allowed integer literals to overflow, silently wrapping them.
The new parser treats integer overflow as a parse error. This seems better,
and it caught an existing overflow in testing/profiles/prefs_general.js, for
places.database.lastMaintenance (see bug 1424030).
The first of these changes meant that a couple of existing prefs with ";;" at
the end had to be changed (done in the preceding patch).
The minor increase in strictness shouldn't be a problem for default pref files
such as greprefs.js within the application (which we can modify), nor for
app-written prefs files such as prefs.js. It could affect user-written prefs
files such as user.js; the experience above suggests that ";;" is the most
likely problem in practice. In my opinion, the risk here is acceptable.
The new parser also does a better job of tracking line numbers because it (a)
treats "\r\n" sequences as a single end-of-line marker, and (a) pays attention
to end-of-line sequences within string literals.
Finally, the patch adds thorough tests of both valid and invalid syntax.
MozReview-Commit-ID: 8EYWH7KxGG
* * *
[mq]: win-fix
MozReview-Commit-ID: 91Bxjfghqfw
