corysanin/tubestation
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Bug 1869043 track and resolve device changed/running promises in MediaStreamRenderer r=pehrsons

Browse Source 
The primary motivation for MediaStreamRenderer keeping track of and settling
incomplete promises is that, after changes in subsequent patches,
AudioStreamTrack outputs will share CrossGraphReceivers and so dedicated
CrossGraphReceivers will no longer be available to reject incomplete promises
when CrossGraphRecievers are Destroy()ed when an output is removed.

This also reliably keeps promise resolution in order wrt the synchronous
resolution from a setSinkId() call while playback is paused.

When a promise is settled because a subsequent pause or setSinkId() makes
the device change unnecessary, the promise is now resolved instead of
rejected.  The new behavior is consistent with the resolution of a
promise created while playback is paused and with AudioSinkWrapper.  Promise
resolution may be less likely to surprise content script than promise
rejection.

The situation with multiple tracks is somewhat arbitrary.  Settling of the
promise depends on which tracks were present when setSinkId() was called.
GenericPromise::All() in MediaStreamRenderer::SetAudioOutputDevice() would
reject when the first track that existed at setSinkId() was removed or ended.
This patch switches to AllSettled() and resolves when all tracks that existed
at setSinkId() have ended.  When AudioStreamTrack outputs no longer have
dedicated CrossGraphRecievers, removal of tracks will no longer cause the
promise to be settled until no tracks require the device.

Differential Revision: https://phabricator.services.mozilla.com/D198231
This commit is contained in:
Karl Tomlinson
2024-01-15 23:51:19 +00:00
parent 122eba1435
commit e18f1219f6
3 changed files with 194 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
dom/html/HTMLMediaElement.cpp
View File

@@ -794,6 +794,9 @@ class HTMLMediaElement::MediaStreamRenderer
t->AsAudioStreamTrack()->RemoveAudioOutput(mAudioOutputKey);
}
}
// There is no longer an audio output that needs the device so the
// device may not start. Ensure the promise is resolved.
ResolveAudioDevicePromiseIfExists(__func__);
if (mVideoTrack) {
mVideoTrack->AsVideoStreamTrack()->RemoveVideoOutput(mVideoContainer);
@@ -816,16 +819,15 @@ class HTMLMediaElement::MediaStreamRenderer
}
}
RefPtr<GenericPromise::AllPromiseType> SetAudioOutputDevice(
AudioDeviceInfo* aSink) {
RefPtr<GenericPromise> SetAudioOutputDevice(AudioDeviceInfo* aSink) {
MOZ_ASSERT(aSink);
MOZ_ASSERT(mAudioOutputSink != aSink);
mAudioOutputSink = aSink;
if (!mRendering) {
return GenericPromise::AllPromiseType::CreateAndResolve(nsTArray<bool>(),
__func__);
MOZ_ASSERT(mSetAudioDevicePromise.IsEmpty());
return GenericPromise::CreateAndResolve(true, __func__);
}
nsTArray<RefPtr<GenericPromise>> promises;
@@ -838,11 +840,35 @@ class HTMLMediaElement::MediaStreamRenderer
}
if (!promises.Length()) {
// Not active track, save it for later
return GenericPromise::AllPromiseType::CreateAndResolve(nsTArray<bool>(),
__func__);
MOZ_ASSERT(mSetAudioDevicePromise.IsEmpty());
return GenericPromise::CreateAndResolve(true, __func__);
}
return GenericPromise::All(GetCurrentSerialEventTarget(), promises);
// Resolve any existing promise for a previous device so that promises
// resolve in order of setSinkId() invocation.
ResolveAudioDevicePromiseIfExists(__func__);
RefPtr promise = mSetAudioDevicePromise.Ensure(__func__);
GenericPromise::AllSettled(GetCurrentSerialEventTarget(), promises)
->Then(GetMainThreadSerialEventTarget(), __func__,
[self = RefPtr{this},
this](const GenericPromise::AllSettledPromiseType::
ResolveOrRejectValue& aValue) {
// This handler should have been disconnected if
// mSetAudioDevicePromise has been settled.
MOZ_ASSERT(!mSetAudioDevicePromise.IsEmpty());
mDeviceStartedRequest.Complete();
// The AudioStreamTrack::AddAudioOutput() promise is rejected
// either when the track ends or the graph is force shutdown.
// Rejection is treated in the same way as resolution for
// consistency with the synchronous resolution when
// AddAudioOutput() is called on a track that has already
// ended.
mSetAudioDevicePromise.Resolve(true, __func__);
})
->Track(mDeviceStartedRequest);
return promise;
}
void AddTrack(AudioStreamTrack* aTrack) {
@@ -878,6 +904,12 @@ class HTMLMediaElement::MediaStreamRenderer
aTrack->RemoveAudioOutput(mAudioOutputKey);
}
mAudioTracks.RemoveElement(aTrack);
if (mAudioTracks.IsEmpty()) {
// There is no longer an audio output that needs the device so the
// device may not start. Ensure the promise is resolved.
ResolveAudioDevicePromiseIfExists(__func__);
}
}
void RemoveTrack(VideoStreamTrack* aTrack) {
MOZ_DIAGNOSTIC_ASSERT(mVideoTrack == aTrack);
@@ -938,6 +970,11 @@ class HTMLMediaElement::MediaStreamRenderer
graph->CreateSourceTrack(MediaSegment::AUDIO));
}
void ResolveAudioDevicePromiseIfExists(const char* aMethodName) {
mSetAudioDevicePromise.ResolveIfExists(true, aMethodName);
mDeviceStartedRequest.DisconnectIfExists();
}
// True when all tracks are being rendered, i.e., when the media element is
// playing.
bool mRendering = false;
@@ -950,6 +987,13 @@ class HTMLMediaElement::MediaStreamRenderer
// The sink device for all audio tracks.
RefPtr<AudioDeviceInfo> mAudioOutputSink;
// The promise returned from SetAudioOutputDevice() when an output is
// active.
MozPromiseHolder<GenericPromise> mSetAudioDevicePromise;
// Request tracking the promise to indicate when the device passed to
// SetAudioOutputDevice() is running.
MozPromiseRequestHolder<GenericPromise::AllSettledPromiseType>
mDeviceStartedRequest;
// WatchManager for mGraphTime.
WatchManager<MediaStreamRenderer> mWatchManager;
@@ -7582,8 +7626,8 @@ already_AddRefed<Promise> HTMLMediaElement::SetSinkId(const nsAString& aSinkId,
RefPtr<SinkInfoPromise> p =
mMediaStreamRenderer->SetAudioOutputDevice(aInfo)->Then(
AbstractMainThread(), __func__,
[aInfo](const GenericPromise::AllPromiseType::
ResolveOrRejectValue& aValue) {
[aInfo](
const GenericPromise::ResolveOrRejectValue& aValue) {
if (aValue.IsResolve()) {
return SinkInfoPromise::CreateAndResolve(aInfo,
__func__);