LSNG already uses some QuotaManager APIs to achieve that origin directories are
not created if they don't exist during datastore preparation, but the feature
is not easy to use and it's also not generalized enough for use in other quota
clients. Besides that, the way how it's currently done in LSNG complicates
removal of QuotaManager::EnsureTemporaryOriginIsInitializedInternal calls from
LSNG. This patch is about generalizing of the feature, making it available to
all quota clients.
Differential Revision: https://phabricator.services.mozilla.com/D195551
Currently, the cached value is updated before creation of the metadata file
which can lead to a mismatch between the metadata file and the cached value if
creation of the metadata file fails. It will be safer to update the cached
value only after successful creation of the metadata file.
Differential Revision: https://phabricator.services.mozilla.com/D198670
Currently, we call QuotaManager::PersistOrigin even when the origin was already
persisted. QuotaManager::PersistOrigin won't try to update the value again,
but it will have to do a hash lookup. We can easilly avoid this by calling
QuotaManager::PersistOrigin only when the value changed in the metadata file.
Differential Revision: https://phabricator.services.mozilla.com/D198668
If temporary storage is already initialized, we can use the cached value and
avoid reading it from the metadata file (which was used for loading the cached
value).
Differential Revision: https://phabricator.services.mozilla.com/D198664
LSNG already uses some QuotaManager APIs to achieve that origin directories are
not created if they don't exist during datastore preparation, but the feature
is not easy to use and it's also not generalized enough for use in other quota
clients. Besides that, the way how it's currently done in LSNG complicates
removal of QuotaManager::EnsureTemporaryOriginIsInitializedInternal calls from
LSNG. This patch is about generalizing of the feature, making it available to
all quota clients.
Differential Revision: https://phabricator.services.mozilla.com/D195551
Currently, the cached value is updated before creation of the metadata file
which can lead to a mismatch between the metadata file and the cached value if
creation of the metadata file fails. It will be safer to update the cached
value only after successful creation of the metadata file.
Differential Revision: https://phabricator.services.mozilla.com/D198670
Currently, we call QuotaManager::PersistOrigin even when the origin was already
persisted. QuotaManager::PersistOrigin won't try to update the value again,
but it will have to do a hash lookup. We can easilly avoid this by calling
QuotaManager::PersistOrigin only when the value changed in the metadata file.
Differential Revision: https://phabricator.services.mozilla.com/D198668
If temporary storage is already initialized, we can use the cached value and
avoid reading it from the metadata file (which was used for loading the cached
value).
Differential Revision: https://phabricator.services.mozilla.com/D198664
The new category is needed for implementing an optimization in
QuotaManager::InitializePersistentOrigin,
QuotaManager::InitializeTemporaryOrigin and
QuotaManager::OpenClientDirectory to avoid creating a new origin initialization
operations when a given origin is already initialized.
Differential Revision: https://phabricator.services.mozilla.com/D195543
Origin initialization tracking on the owning thread is needed for covering
origin initialization in QuotaManager::OpenClientDirectory.
Differential Revision: https://phabricator.services.mozilla.com/D195535
Client storages can be currently reset by using the optional argument of
nsIQuotaManagerService::ResetStoragesForPrincipal. However, supporting
resetting storages for both a principal and a client is not ideal for several
reasons. Resetting of origins will have to unset some flags which is
problematic when a client type is specified and resetting of clients will
eventually need to unset some other flags as well.
Differential Revision: https://phabricator.services.mozilla.com/D195508
Client storages can be currently cleared by using the optional argument of
nsIQuotaManagerService::ClearStoragesForPrincipal. However, supporting clearing
storages for both a principal and a client is not ideal for several reasons.
Clearing of origins will have to unset some flags which is problematic when
a client type is specified and clearing of clients will eventually need to
unset some other flags as well.
Differential Revision: https://phabricator.services.mozilla.com/D195403
QuotaManager::OpenStorageDirectory already makes sure that temporary storage
is initialized so all these calls are now redundant.
Differential Revision: https://phabricator.services.mozilla.com/D195395
QuotaManager::OpenStorageDirectory can now detect the requested persistence
scope and initialize temporary storage if it's needed.
Differential Revision: https://phabricator.services.mozilla.com/D195393
Temporary storage initialization currently uses a directory lock over all
repositories which blocks any exclusive directory locks for the persistent
repository for no reason.
Differential Revision: https://phabricator.services.mozilla.com/D195389
Temporary storage initialization can finally lock only specific repositories
instead of locking all repositories (actual temporary storage initialization
locking chagnes will be done in a separate patch).
Differential Revision: https://phabricator.services.mozilla.com/D195374
Nullable<PersistenceType> only allows to specify a concrete persistence type or
all persistence types while PersistenceScope is intended to allow specification
of multiple persistence types as well. Support for that will be added in a
separate patch. This patch is about using the new type especially in directory
locks.
Differential Revision: https://phabricator.services.mozilla.com/D195373
Cached origin usage can be currently obtained by passing fromMemory=true to
nsIQuotaManagerService::GetUsageForPrincipal. However, supporting both ways of
getting origin on one operation is not ideal for sevaral reasons. The return
value is a plain integer in the case of cached origin usage and getting cached
origin usage in theory doesn't have to go the QM IO thread. So it would be
better to have a dedicated method for that called
nsIQuotaManagerService::GetCachedUsageForPrincipal.
Differential Revision: https://phabricator.services.mozilla.com/D195360
Currently, getting cached origin usage may trigger temporary storage
initialization which is a problem in tests when we want just compare real and
cached usage without affecting the state of initialization.
This patch changes the behavior to return Nothing if temporary storage is not
initialized.
Consumers may either explicitly make sure that temporary storage is initialized
before getting cached origin usage or they can use the other mode when real
usage is colleced from disk.
Differential Revision: https://phabricator.services.mozilla.com/D194504
Sub actors are still created, but their only purpose is to allow cancellation
of alreaady created requests. Actual results are now returned as asynchronous
responses to the asynchronous messages.
Differential Revision: https://phabricator.services.mozilla.com/D194097
The class which implements the protocol is already refcounted, but the
interaction with IPC still uses manual deallocation.
Differential Revision: https://phabricator.services.mozilla.com/D194042
This is one of the things which are needed for making GetUsageOp and
GetOriginUsageOp independent from the actor. In practise, for changing the main
base class of the operations from QuotaUsageRequestBase to
ResolvableNormalOriginOp.
Differential Revision: https://phabricator.services.mozilla.com/D194011
There are currently EnsurePersistentOriginIsInitializedInternal and
EnsureTemporaryOriginIsInitializedInternal methods which can only be
called on the QuotaManager IO thread. These methods shouldn't be exposed to
quota clients and origin operations. There should be public methods callable
from the PBackground thread returning a MozPromise instead. Such methods will
guarantee that proper directory locking is acquired before persistent origin
initialization or temporary origin initialization is started.
Differential Revision: https://phabricator.services.mozilla.com/D194005
One of the goals of the asynchronous temporary storage initialization is to
call Ensure(Persistent|Temporary)OriginIsInitialized only from
Initialize(Persistent|Temporary)OriginOp. Calling from other places including
quota clients will be disallowed by changing the method to a private method.
The private nature of the method should be emphasized by adding the Internal
suffix.
Differential Revision: https://phabricator.services.mozilla.com/D192150
Counting of clear/shutdown storage operations was only a temporary solution.
We are now approaching a point when we will be able to initialize origins
asynchronously and that can't efficiently work with counting of clear origin
operations because initialization and clearing of origins is parameterized by
persistence type and actual origin. So there can be a clear origin operation
which doesn't block other init origin operation. Couting of clear operations
would be problematic in that case.
Evaluation of existing directory locks should work both for storage and origin
initialization.
Differential Revision: https://phabricator.services.mozilla.com/D193191
Until now, directory locks were dropped when the last strong reference was
removed or after calling Drop explicitly. The dependency on ref-counting makes
it less obvious when directory locks are dropped for real and it's also
difficult to release them asynchronously eventually. This patch removes the
directory lock unregistration from the destructor, so from now on, directory
locks must be always dropped explicitly.
Differential Revision: https://phabricator.services.mozilla.com/D197294
There's currently EnsureTemporaryStorageIsInitializedInternal which can only be
called on the QuotaManager IO thread. That method shouldn't be exposed to quota
clients and origin operations. There should be a public method callable from
the PBackground thread returning a MozPromise instead. Such method will
guarantee that proper directory locking is acquired before temporary storage
initialization is started.
Differential Revision: https://phabricator.services.mozilla.com/D192135
One of the goals of the asynchronous temporary storage initialization is to
call EnsureTemporaryStorageIsInitialized only from InitTemporaryStorageOp.
Calling from other places including quota clients will be disallowed by
changing the method to a private method. The private nature of the method
should be emphasized by adding the Internal suffix.
Differential Revision: https://phabricator.services.mozilla.com/D188332
This patch also adds QuotaManager::StorageInitialized and
QuotaManager::TemporaryStorageInitialized which can be then used by the
QuotaManagerDependencyFixture.
Differential Revision: https://phabricator.services.mozilla.com/D191930
Operations are shown as "pending" in the QM shutdown hang annotations until they are fully executed. Given that executing them often requires acquiring a directory lock first and also involves some thread hopping, we want to understand better, where we hang.
In particular many crashes show `ShutdownStorageOp` as pending but there is no sign in the stack trace that `DoDirectoryWork` ever started. In addition, most if not all of those crashes show either an open IDB connection or an unclosed Cache manager, which might interfere with directory locking here.
The annotation should help to confirm that `ShutdownStorageOp` is endlessly waiting to acquire the directory lock in those cases.
Differential Revision: https://phabricator.services.mozilla.com/D190425
