Attachment 264297 Details for Bug 150449 – Patch for landing.

[patch] Patch for landing.

media_player_patch_17.txt (text/plain), 118.18 KB, created by Eric Carlson on 2015-10-28 21:36:24 PDT

(hide)

Description:

Filename:

MIME Type:

Creator: Eric Carlson

Created: 2015-10-28 21:36:24 PDT

Size: 118.18 KB

patch

obsolete

>Index: Source/WebCore/ChangeLog
>===================================================================
>--- Source/WebCore/ChangeLog	(revision 191696)
>+++ Source/WebCore/ChangeLog	(working copy)
>@@ -1,3 +1,191 @@
>+2015-10-28  Eric Carlson  <eric.carlson@apple.com>
>+
>+        [MediaStream] Play MediaStream through media element and rendered to canvas
>+        https://bugs.webkit.org/show_bug.cgi?id=150449
>+
>+        Reviewed by Jer Noble.
>+
>+        * Modules/mediastream/MediaStream.cpp:
>+        (WebCore::MediaStream::create): Don't die a recursive death.
>+        (WebCore::MediaStream::MediaStream): setClient -> addObserver. Set private stream's public stream pointer.
>+        (WebCore::MediaStream::~MediaStream): setClient -> addObserver. Clear private stream's public stream pointer.
>+        (WebCore::MediaStream::didAddTrack): Short circuit calling internalAddTrack when the track is unknown.
>+        (WebCore::MediaStream::didRemoveTrack): ASSERT that the track is known.
>+        * Modules/mediastream/MediaStream.h:
>+
>+        * Modules/mediastream/MediaStreamTrack.cpp:
>+        (WebCore::MediaStreamTrack::trackProducingDataChanged): New.
>+        (WebCore::MediaStreamTrack::trackEnabledChanged): New.
>+        * Modules/mediastream/MediaStreamTrack.h:
>+
>+        * Modules/mediastream/RTCPeerConnection.cpp:
>+        (WebCore::RTCPeerConnection::didRemoveRemoteStream): Use the new MediaStreamPrivate::publicStream() method.
>+
>+        * Modules/webaudio/MediaStreamAudioSource.cpp:
>+]        (WebCore::MediaStreamAudioSource::capabilities): capabilities isn't const.
>+        * Modules/webaudio/MediaStreamAudioSource.h:
>+
>+        * Modules/webaudio/MediaStreamAudioSourceNode.cpp:
>+        (WebCore::MediaStreamAudioSourceNode::setFormat): Reformat to make it use early return.
>+
>+        * WebCore.xcodeproj/project.pbxproj: Remove MediaStreamPrivateAVFObjC.mm/h, they are no longer necessary.
>+
>+        * platform/graphics/MediaPlayer.cpp:
>+        (WebCore::buildMediaEnginesVector): Register MediaPlayerPrivateMediaStreamAVFObjC engine.
>+
>+        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
>+        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::MediaPlayerPrivateMediaStreamAVFObjC): Cleanup, add logging.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC): Add
>+          logging, remove private stream observer.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::registerMediaEngine): Cleanup.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::isAvailable): Ditto.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::getSupportedTypes): Return an empty vector.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::load): Use the MediaStreamPrivate passed instead
>+          of creating a new private stream. 
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::cancelLoad): Pause output.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::prepareToPlay): Add logging.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformLayer): m_previewLayer -> m_videoBackgroundLayer.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::setPausedImageVisible): New, show/hide the preview layer and configure
>+          the background layer so we show a still image even though the capture device continues to run. Start/stop clock.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::play): Don't need to start the source, that happens in in load. Just
>+          set flags and hide the paused image.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::pause): Don't stop the source, it may be shared by more than one 
>+          stream/track. Just set the playing flag and show the paused image.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::paused): Fix.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::internalSetVolume): New, called by setMuted and setVolume.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::setVolume): New.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::setMuted): Don't mute the source, it doesn't do what you think.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::hasVideo): Pass-through to stream.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::hasAudio): Ditto.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentMediaTime): Return clock time.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState): New.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::activeStatusChanged): Update readyState.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize): Update intrinsic size, create layers if necessary.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers): Create preview layers.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::characteristicsChanged): Update for changed characteristics.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::seekable): 'seekable' must return an empty TimeRanges object.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::paintCurrentFrameInContext): New.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::scheduleDeferredTask):
>+        (WebCore::mimeTypeCache): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::platformMedia): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::playInternal): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::pauseInternal): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::sizeChanged): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::maxMediaTimeSeekable): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::minMediaTimeSeekable): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::createImageFromSampleBuffer): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::supportsAcceleratedRendering): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::movieLoadType): Deleted.
>+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::extraMemoryCost): Deleted.
>+
>+        Delete MediaStreamPrivateAVFObjC.mm/.h as they aren't necessary. 
>+        * platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.h: Removed.
>+        * platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.mm: Removed.
>+
>+        * platform/mediastream/MediaStreamPrivate.cpp:
>+        (WebCore::MediaStreamPrivate::MediaStreamPrivate): Add track observer!
>+        (WebCore::MediaStreamPrivate::addObserver): New.
>+        (WebCore::MediaStreamPrivate::removeObserver): New.
>+        (WebCore::MediaStreamPrivate::updateActiveState): Remember the first active video track.
>+        (WebCore::MediaStreamPrivate::addTrack): Notify observers.
>+        (WebCore::MediaStreamPrivate::removeTrack): Ditto.
>+        (WebCore::MediaStreamPrivate::startProducingData): Pass-through to tracks.
>+        (WebCore::MediaStreamPrivate::stopProducingData): Ditto.
>+        (WebCore::MediaStreamPrivate::isProducingData): Ditto.
>+        (WebCore::MediaStreamPrivate::hasVideo): Ditto.
>+        (WebCore::MediaStreamPrivate::hasAudio): Ditto.
>+        (WebCore::MediaStreamPrivate::platformLayer): Ditto.
>+        (WebCore::MediaStreamPrivate::paintCurrentFrameInContext): Either pass-through to first active video
>+          track, or paint the context black.
>+        (WebCore::MediaStreamPrivate::currentFrameImage): Pass-through to first active video track.
>+        (WebCore::MediaStreamPrivate::characteristicsChanged): Inform observers.
>+        (WebCore::MediaStreamPrivate::trackMutedChanged):
>+        (WebCore::MediaStreamPrivate::trackStatesChanged):
>+        (WebCore::MediaStreamPrivate::trackEnabledChanged):
>+        (WebCore::MediaStreamPrivate::trackProducingDataChanged):
>+        (WebCore::MediaStreamPrivate::scheduleDeferredTask): New, call a function asynchronously on
>+          the main thread.
>+        * platform/mediastream/MediaStreamPrivate.h:
>+
>+        * platform/mediastream/MediaStreamTrackPrivate.cpp:
>+        (WebCore::MediaStreamTrackPrivate::endTrack):
>+        (WebCore::MediaStreamTrackPrivate::sourceProducingDataChanged): New, inform observers.
>+        (WebCore::MediaStreamTrackPrivate::sourceEnabledChanged): Ditto.
>+        * platform/mediastream/MediaStreamTrackPrivate.h:
>+
>+        * platform/mediastream/RealtimeMediaSource.cpp:
>+        (WebCore::RealtimeMediaSource::isProducingDataDidChange): New, inform observers.
>+        (WebCore::RealtimeMediaSource::setEnabled): Ditto.
>+        (WebCore::RealtimeMediaSource::stop): Call reset();
>+        * platform/mediastream/RealtimeMediaSource.h: Don't declare "capabilities" as const so 
>+          capabilities can be created and initialized lazily.
>+
>+        * platform/mediastream/mac/AVAudioCaptureSource.h:
>+        * platform/mediastream/mac/AVAudioCaptureSource.mm:
>+        (WebCore::AVAudioCaptureSource::initializeCapabilities): New.
>+        (WebCore::AVAudioCaptureSource::addObserver): Hold the lock while calling observers so the list
>+          can't be mutated.
>+        (WebCore::AVAudioCaptureSource::setupCaptureSession): Log and fail if the session won't add the
>+          input or output.
>+        (WebCore::AVAudioCaptureSource::shutdownCaptureSession): Cleanup.
>+        (WebCore::AVAudioCaptureSource::captureOutputDidOutputSampleBufferFromConnection): Don't block
>+          if it isn't possible to acquire the lock. Hold the lock while calling observers.
>+        (WebCore::AVAudioCaptureSource::capabilities): Deleted.
>+
>+        * platform/mediastream/mac/AVCaptureDeviceManager.mm:
>+        (WebCore::refreshCaptureDeviceList): Don't include devices that can't be used.
>+        (WebCore::AVCaptureDeviceManager::bestSourcesForTypeAndConstraints):  Use AVCaptureDeviceManager::sourceWithUID
>+          so constraints are considered.
>+        (WebCore::AVCaptureDeviceManager::sourceWithUID): Don't consider disabled devices. Always
>+          create a new capture device so each track starts out with a unique source.
>+        * platform/mediastream/mac/AVMediaCaptureSource.h:
>+
>+        * platform/mediastream/mac/AVMediaCaptureSource.mm:
>+        (WebCore::globaAudioCaptureSerialQueue):
>+        (WebCore::AVMediaCaptureSource::AVMediaCaptureSource):
>+        (WebCore::AVMediaCaptureSource::~AVMediaCaptureSource): Remove KVO observers.
>+        (WebCore::AVMediaCaptureSource::startProducingData): m_isRunning is changed in captureSessionIsRunningDidChange.
>+        (WebCore::AVMediaCaptureSource::stopProducingData): Ditto.
>+        (WebCore::AVMediaCaptureSource::capabilities): New.
>+        (WebCore::AVMediaCaptureSource::setupSession): Add KVObservers for the properties we care about.
>+        (WebCore::AVMediaCaptureSource::reset): New, cleanup.
>+        (WebCore::AVMediaCaptureSource::captureSessionIsRunningDidChange): Dispatch to the main thread 
>+          to set m_isRunning, call isProducingDataDidChange so observers can find out.
>+        (WebCore::sessionKVOProperties):
>+        (-[WebCoreAVMediaCaptureSourceObserver disconnect]):
>+        (-[WebCoreAVMediaCaptureSourceObserver captureOutput:didOutputSampleBuffer:fromConnection:]):
>+        (-[WebCoreAVMediaCaptureSourceObserver observeValueForKeyPath:ofObject:change:context:]): Respond
>+          to running changes.
>+        (WebCore::AVMediaCaptureSource::captureSessionStoppedRunning): Deleted.
>+        (-[WebCoreAVMediaCaptureSourceObserver captureSessionStoppedRunning:]): Deleted.
>+
>+        * platform/mediastream/mac/AVVideoCaptureSource.h:
>+        * platform/mediastream/mac/AVVideoCaptureSource.mm:
>+        (WebCore::AVVideoCaptureSource::initializeCapabilities): Partial implementation.
>+        (WebCore::AVVideoCaptureSource::setupCaptureSession):  Log and fail if the session won't add the
>+          input or output.
>+        (WebCore::AVVideoCaptureSource::shutdownCaptureSession): Cleanup.
>+        (WebCore::AVVideoCaptureSource::updateFramerate): Renamed from calculateFramerate.
>+        (WebCore::AVVideoCaptureSource::currentFrameImage): Return an Image.
>+        (WebCore::AVVideoCaptureSource::platformLayer): New.
>+        (WebCore::AVVideoCaptureSource::capabilities): Deleted.
>+        (WebCore::AVVideoCaptureSource::calculateFramerate): Deleted.
>+
>+        * platform/mediastream/mac/WebAudioSourceProviderAVFObjC.h:
>+        * platform/mediastream/mac/WebAudioSourceProviderAVFObjC.mm:
>+        (WebCore::WebAudioSourceProviderAVFObjC::~WebAudioSourceProviderAVFObjC): AudioConverterRef is
>+          not a CF/ObjC object so we can't use a RetainPtr<> for it.
>+        (WebCore::WebAudioSourceProviderAVFObjC::provideInput): Ditto.
>+        (WebCore::WebAudioSourceProviderAVFObjC::prepare): Ditto.
>+        (WebCore::WebAudioSourceProviderAVFObjC::unprepare): Ditto.
>+        (WebCore::WebAudioSourceProviderAVFObjC::process): Fix typo in logging.
>+
>+        * platform/mock/MockRealtimeMediaSourceCenter.cpp:
>+        (WebCore::MockSource::capabilities): Update for capabilities change.
>+
>+        * platform/mediastream/openwebrtc/RealtimeMediaSourceOwr.h: Update for capabilities change.
>+
> 2015-10-28  Chris Dumez  <cdumez@apple.com>
> 
>         Assertion failure in WebCore::FrameLoader::stopLoading() running fast/events tests
>Index: Source/WebCore/Modules/mediastream/MediaStream.cpp
>===================================================================
>--- Source/WebCore/Modules/mediastream/MediaStream.cpp	(revision 191696)
>+++ Source/WebCore/Modules/mediastream/MediaStream.cpp	(working copy)
>@@ -40,6 +40,36 @@
> 
> namespace WebCore {
> 
>+static Vector<MediaStream*>& mediaStreams()
>+{
>+    static NeverDestroyed<Vector<MediaStream*>> streams;
>+    return streams;
>+}
>+
>+static void registerMediaStream(MediaStream* stream)
>+{
>+    mediaStreams().append(stream);
>+}
>+
>+static void unRegisterMediaStream(MediaStream* stream)
>+{
>+    Vector<MediaStream*>& allStreams = mediaStreams();
>+    size_t pos = allStreams.find(stream);
>+    if (pos != notFound)
>+        allStreams.remove(pos);
>+}
>+
>+MediaStream* MediaStream::lookUp(const MediaStreamPrivate& privateStream)
>+{
>+    Vector<MediaStream*>& allStreams = mediaStreams();
>+    for (auto& stream : allStreams) {
>+        if (stream->m_private == &privateStream)
>+            return stream;
>+    }
>+
>+    return nullptr;
>+}
>+
> static URLRegistry* s_registry;
> 
> void MediaStream::setRegistry(URLRegistry& registry)
>@@ -58,7 +88,7 @@ MediaStream* MediaStream::lookUp(const U
> 
> Ref<MediaStream> MediaStream::create(ScriptExecutionContext& context)
> {
>-    return MediaStream::create(context);
>+    return MediaStream::create(context, MediaStreamPrivate::create(Vector<RefPtr<MediaStreamTrackPrivate>>()));
> }
> 
> Ref<MediaStream> MediaStream::create(ScriptExecutionContext& context, MediaStream* stream)
>@@ -95,7 +125,8 @@ MediaStream::MediaStream(ScriptExecution
> 
>     m_private = MediaStreamPrivate::create(trackPrivates);
>     m_isActive = m_private->active();
>-    m_private->setClient(this);
>+    m_private->addObserver(*this);
>+    registerMediaStream(this);
> }
> 
> MediaStream::MediaStream(ScriptExecutionContext& context, RefPtr<MediaStreamPrivate>&& streamPrivate)
>@@ -105,7 +136,8 @@ MediaStream::MediaStream(ScriptExecution
>     , m_activityEventTimer(*this, &MediaStream::activityEventTimerFired)
> {
>     ASSERT(m_private);
>-    m_private->setClient(this);
>+    m_private->addObserver(*this);
>+    registerMediaStream(this);
> 
>     for (auto& trackPrivate : m_private->tracks()) {
>         RefPtr<MediaStreamTrack> track = MediaStreamTrack::create(context, *trackPrivate);
>@@ -116,7 +148,10 @@ MediaStream::MediaStream(ScriptExecution
> 
> MediaStream::~MediaStream()
> {
>-    m_private->setClient(nullptr);
>+    unRegisterMediaStream(this);
>+    m_private->removeObserver(*this);
>+    for (auto& track : m_trackSet.values())
>+        track->removeObserver(this);
> }
> 
> RefPtr<MediaStream> MediaStream::clone()
>@@ -199,12 +234,14 @@ void MediaStream::didAddTrack(MediaStrea
>     if (!context)
>         return;
> 
>-    internalAddTrack(MediaStreamTrack::create(*context, trackPrivate), StreamModifier::Platform);
>+    if (!getTrackById(trackPrivate.id()))
>+        internalAddTrack(MediaStreamTrack::create(*context, trackPrivate), StreamModifier::Platform);
> }
> 
> void MediaStream::didRemoveTrack(MediaStreamTrackPrivate& trackPrivate)
> {
>     RefPtr<MediaStreamTrack> track = getTrackById(trackPrivate.id());
>+    ASSERT(track);
>     internalRemoveTrack(WTF::move(track), StreamModifier::Platform);
> }
> 
>@@ -254,6 +291,7 @@ void MediaStream::activityEventTimerFire
>     events.swap(m_scheduledActivityEvents);
> 
>     for (auto& event : events) {
>+        // FIXME: This can't be right, the event type is set from m_isActive in scheduleActiveStateChange! - https://webkit.org/b/150635
>         m_isActive = event->type() == eventNames().activeEvent;
>         dispatchEvent(event.release());
>     }
>Index: Source/WebCore/Modules/mediastream/MediaStream.h
>===================================================================
>--- Source/WebCore/Modules/mediastream/MediaStream.h	(revision 191696)
>+++ Source/WebCore/Modules/mediastream/MediaStream.h	(working copy)
>@@ -44,7 +44,7 @@
> 
> namespace WebCore {
> 
>-class MediaStream final : public URLRegistrable, public ScriptWrappable, public MediaStreamPrivateClient, public EventTargetWithInlineData, public ContextDestructionObserver, public MediaStreamTrack::Observer {
>+class MediaStream final : public URLRegistrable, public ScriptWrappable, public EventTargetWithInlineData, public ContextDestructionObserver, public MediaStreamTrack::Observer, public MediaStreamPrivate::Observer, public RefCounted<MediaStream> {
> public:
>     class Observer {
>     public:
>@@ -55,6 +55,8 @@ public:
>     static void setRegistry(URLRegistry&);
>     static MediaStream* lookUp(const URL&);
> 
>+    static MediaStream* lookUp(const MediaStreamPrivate&);
>+
>     static Ref<MediaStream> create(ScriptExecutionContext&);
>     static Ref<MediaStream> create(ScriptExecutionContext&, MediaStream*);
>     static Ref<MediaStream> create(ScriptExecutionContext&, const MediaStreamTrackVector&);
>@@ -81,8 +83,8 @@ public:
>     virtual EventTargetInterface eventTargetInterface() const final { return MediaStreamEventTargetInterfaceType; }
>     virtual ScriptExecutionContext* scriptExecutionContext() const final { return ContextDestructionObserver::scriptExecutionContext(); }
> 
>-    using RefCounted<MediaStreamPrivateClient>::ref;
>-    using RefCounted<MediaStreamPrivateClient>::deref;
>+    using RefCounted<MediaStream>::ref;
>+    using RefCounted<MediaStream>::deref;
> 
>     // URLRegistrable
>     virtual URLRegistry& registry() const override;
>@@ -107,7 +109,7 @@ private:
>     // MediaStreamTrack::Observer
>     virtual void trackDidEnd() override final;
> 
>-    // MediaStreamPrivateClient
>+    // MediaStreamPrivate::Observer
>     virtual void activeStatusChanged() override final;
>     virtual void didAddTrack(MediaStreamTrackPrivate&) override final;
>     virtual void didRemoveTrack(MediaStreamTrackPrivate&) override final;
>Index: Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp
>===================================================================
>--- Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp	(revision 191696)
>+++ Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp	(working copy)
>@@ -213,6 +213,16 @@ void MediaStreamTrack::trackStatesChange
>     configureTrackRendering();
> }
> 
>+void MediaStreamTrack::trackProducingDataChanged(MediaStreamTrackPrivate&)
>+{
>+    configureTrackRendering();
>+}
>+
>+void MediaStreamTrack::trackEnabledChanged(MediaStreamTrackPrivate&)
>+{
>+    configureTrackRendering();
>+}
>+
> void MediaStreamTrack::configureTrackRendering()
> {
>     // 4.3.1
>Index: Source/WebCore/Modules/mediastream/MediaStreamTrack.h
>===================================================================
>--- Source/WebCore/Modules/mediastream/MediaStreamTrack.h	(revision 191696)
>+++ Source/WebCore/Modules/mediastream/MediaStreamTrack.h	(working copy)
>@@ -118,6 +118,8 @@ private:
>     void trackEnded(MediaStreamTrackPrivate&) override;
>     void trackMutedChanged(MediaStreamTrackPrivate&) override;
>     void trackStatesChanged(MediaStreamTrackPrivate&) override;
>+    void trackProducingDataChanged(MediaStreamTrackPrivate&) override;
>+    void trackEnabledChanged(MediaStreamTrackPrivate&) override;
> 
>     Vector<Observer*> m_observers;
>     Ref<MediaStreamTrackPrivate> m_private;
>Index: Source/WebCore/Modules/mediastream/RTCPeerConnection.cpp
>===================================================================
>--- Source/WebCore/Modules/mediastream/RTCPeerConnection.cpp	(revision 191696)
>+++ Source/WebCore/Modules/mediastream/RTCPeerConnection.cpp	(working copy)
>@@ -649,14 +649,12 @@ void RTCPeerConnection::didAddRemoteStre
> void RTCPeerConnection::didRemoveRemoteStream(MediaStreamPrivate* privateStream)
> {
>     ASSERT(scriptExecutionContext()->isContextThread());
>-    ASSERT(privateStream->client());
>-
>-    // FIXME: this class shouldn't know that the private stream client is a MediaStream!
>-    RefPtr<MediaStream> stream = static_cast<MediaStream*>(privateStream->client());
> 
>     if (m_signalingState == SignalingStateClosed)
>         return;
> 
>+    RefPtr<MediaStream> stream = MediaStream::lookUp(*privateStream);
>+    ASSERT(stream);
>     size_t pos = m_remoteStreams.find(stream);
>     ASSERT(pos != notFound);
>     m_remoteStreams.remove(pos);
>Index: Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp
>===================================================================
>--- Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp	(revision 191696)
>+++ Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp	(working copy)
>@@ -44,7 +44,7 @@ MediaStreamAudioSource::MediaStreamAudio
> {
> }
> 
>-RefPtr<RealtimeMediaSourceCapabilities> MediaStreamAudioSource::capabilities() const
>+RefPtr<RealtimeMediaSourceCapabilities> MediaStreamAudioSource::capabilities()
> {
>     // FIXME: implement this.
>     // https://bugs.webkit.org/show_bug.cgi?id=122430
>Index: Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h
>===================================================================
>--- Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h	(revision 191696)
>+++ Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h	(working copy)
>@@ -47,7 +47,7 @@ public:
> 
>     ~MediaStreamAudioSource() { }
> 
>-    RefPtr<RealtimeMediaSourceCapabilities> capabilities() const override;
>+    RefPtr<RealtimeMediaSourceCapabilities> capabilities() override;
>     const RealtimeMediaSourceStates& states() override;
>     
>     const String& deviceId() const { return m_deviceId; }
>Index: Source/WebCore/Modules/webaudio/MediaStreamAudioSourceNode.cpp
>===================================================================
>--- Source/WebCore/Modules/webaudio/MediaStreamAudioSourceNode.cpp	(revision 191696)
>+++ Source/WebCore/Modules/webaudio/MediaStreamAudioSourceNode.cpp	(working copy)
>@@ -69,37 +69,38 @@ MediaStreamAudioSourceNode::~MediaStream
> void MediaStreamAudioSourceNode::setFormat(size_t numberOfChannels, float sourceSampleRate)
> {
>     float sampleRate = this->sampleRate();
>-    if (numberOfChannels != m_sourceNumberOfChannels || sourceSampleRate != sampleRate) {
>-        // The sample-rate must be equal to the context's sample-rate.
>-        if (!numberOfChannels || numberOfChannels > AudioContext::maxNumberOfChannels() || sourceSampleRate != sampleRate) {
>-            // process() will generate silence for these uninitialized values.
>-            LOG(Media, "MediaStreamAudioSourceNode::setFormat(%u, %f) - unhandled format change", static_cast<unsigned>(numberOfChannels), sourceSampleRate);
>-            m_sourceNumberOfChannels = 0;
>-            return;
>-        }
>-
>-        // Synchronize with process().
>-        std::lock_guard<Lock> lock(m_processMutex);
>-
>-        m_sourceNumberOfChannels = numberOfChannels;
>-        m_sourceSampleRate = sourceSampleRate;
>-
>-        if (sourceSampleRate == sampleRate)
>-            m_multiChannelResampler = nullptr;
>-        else {
>-            double scaleFactor = sourceSampleRate / sampleRate;
>-            m_multiChannelResampler = std::make_unique<MultiChannelResampler>(scaleFactor, numberOfChannels);
>-        }
>-
>-        m_sourceNumberOfChannels = numberOfChannels;
>-
>-        {
>-            // The context must be locked when changing the number of output channels.
>-            AudioContext::AutoLocker contextLocker(*context());
>-
>-            // Do any necesssary re-configuration to the output's number of channels.
>-            output(0)->setNumberOfChannels(numberOfChannels);
>-        }
>+    if (numberOfChannels == m_sourceNumberOfChannels && sourceSampleRate == sampleRate)
>+        return;
>+
>+    // The sample-rate must be equal to the context's sample-rate.
>+    if (!numberOfChannels || numberOfChannels > AudioContext::maxNumberOfChannels() || sourceSampleRate != sampleRate) {
>+        // process() will generate silence for these uninitialized values.
>+        LOG(Media, "MediaStreamAudioSourceNode::setFormat(%u, %f) - unhandled format change", static_cast<unsigned>(numberOfChannels), sourceSampleRate);
>+        m_sourceNumberOfChannels = 0;
>+        return;
>+    }
>+
>+    // Synchronize with process().
>+    std::lock_guard<Lock> lock(m_processMutex);
>+
>+    m_sourceNumberOfChannels = numberOfChannels;
>+    m_sourceSampleRate = sourceSampleRate;
>+
>+    if (sourceSampleRate == sampleRate)
>+        m_multiChannelResampler = nullptr;
>+    else {
>+        double scaleFactor = sourceSampleRate / sampleRate;
>+        m_multiChannelResampler = std::make_unique<MultiChannelResampler>(scaleFactor, numberOfChannels);
>+    }
>+
>+    m_sourceNumberOfChannels = numberOfChannels;
>+
>+    {
>+        // The context must be locked when changing the number of output channels.
>+        AudioContext::AutoLocker contextLocker(*context());
>+
>+        // Do any necesssary re-configuration to the output's number of channels.
>+        output(0)->setNumberOfChannels(numberOfChannels);
>     }
> }
> 
>Index: Source/WebCore/WebCore.xcodeproj/project.pbxproj
>===================================================================
>--- Source/WebCore/WebCore.xcodeproj/project.pbxproj	(revision 191696)
>+++ Source/WebCore/WebCore.xcodeproj/project.pbxproj	(working copy)
>@@ -654,8 +654,6 @@
> 		14FFE31E0AE1963300136BF5 /* HTMLFrameElementBase.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 14FFE31C0AE1963300136BF5 /* HTMLFrameElementBase.cpp */; };
> 		150524F41B712FF600696AA9 /* MediaPlayerPrivateMediaStreamAVFObjC.h in Headers */ = {isa = PBXBuildFile; fileRef = 150524F01B712FF000696AA9 /* MediaPlayerPrivateMediaStreamAVFObjC.h */; };
> 		150524F51B712FF900696AA9 /* MediaPlayerPrivateMediaStreamAVFObjC.mm in Sources */ = {isa = PBXBuildFile; fileRef = 150524F11B712FF000696AA9 /* MediaPlayerPrivateMediaStreamAVFObjC.mm */; };
>-		150524F61B712FFC00696AA9 /* MediaStreamPrivateAVFObjC.h in Headers */ = {isa = PBXBuildFile; fileRef = 150524F21B712FF000696AA9 /* MediaStreamPrivateAVFObjC.h */; };
>-		150524F71B712FFF00696AA9 /* MediaStreamPrivateAVFObjC.mm in Sources */ = {isa = PBXBuildFile; fileRef = 150524F31B712FF000696AA9 /* MediaStreamPrivateAVFObjC.mm */; };
> 		15145B901B3A1CE000662BF7 /* MediaDeviceInfo.h in Headers */ = {isa = PBXBuildFile; fileRef = 15145B8F1B3A1B3E00662BF7 /* MediaDeviceInfo.h */; settings = {ATTRIBUTES = (Private, ); }; };
> 		15739BBA1B42012A00D258C1 /* JSMediaDevices.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 15739BB81B42012200D258C1 /* JSMediaDevices.cpp */; };
> 		15739BBB1B42012D00D258C1 /* JSMediaDevices.h in Headers */ = {isa = PBXBuildFile; fileRef = 15739BB91B42012200D258C1 /* JSMediaDevices.h */; };
>@@ -7931,8 +7929,6 @@
> 		14FFE31C0AE1963300136BF5 /* HTMLFrameElementBase.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HTMLFrameElementBase.cpp; sourceTree = "<group>"; };
> 		150524F01B712FF000696AA9 /* MediaPlayerPrivateMediaStreamAVFObjC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaPlayerPrivateMediaStreamAVFObjC.h; sourceTree = "<group>"; };
> 		150524F11B712FF000696AA9 /* MediaPlayerPrivateMediaStreamAVFObjC.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MediaPlayerPrivateMediaStreamAVFObjC.mm; sourceTree = "<group>"; };
>-		150524F21B712FF000696AA9 /* MediaStreamPrivateAVFObjC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaStreamPrivateAVFObjC.h; sourceTree = "<group>"; };
>-		150524F31B712FF000696AA9 /* MediaStreamPrivateAVFObjC.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MediaStreamPrivateAVFObjC.mm; sourceTree = "<group>"; };
> 		15145B8F1B3A1B3E00662BF7 /* MediaDeviceInfo.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = MediaDeviceInfo.h; sourceTree = "<group>"; };
> 		15145B911B3A1D4C00662BF7 /* MediaDeviceInfo.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = MediaDeviceInfo.idl; sourceTree = "<group>"; };
> 		15739BB81B42012200D258C1 /* JSMediaDevices.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSMediaDevices.cpp; sourceTree = "<group>"; };
>@@ -22750,8 +22746,6 @@
> 				150524F11B712FF000696AA9 /* MediaPlayerPrivateMediaStreamAVFObjC.mm */,
> 				CDC8B5A5180474F70016E685 /* MediaSourcePrivateAVFObjC.h */,
> 				CDC8B5A4180474F70016E685 /* MediaSourcePrivateAVFObjC.mm */,
>-				150524F21B712FF000696AA9 /* MediaStreamPrivateAVFObjC.h */,
>-				150524F31B712FF000696AA9 /* MediaStreamPrivateAVFObjC.mm */,
> 				7A29F57118C69514004D0F81 /* OutOfBandTextTrackPrivateAVF.h */,
> 				CDC8B5A918047FF10016E685 /* SourceBufferPrivateAVFObjC.h */,
> 				CDC8B5A818047FF10016E685 /* SourceBufferPrivateAVFObjC.mm */,
>@@ -26617,7 +26611,6 @@
> 				078E093917D16B2C00420AA1 /* MediaStreamCreationClient.h in Headers */,
> 				078E091617D14D1C00420AA1 /* MediaStreamEvent.h in Headers */,
> 				078E093717D16B2C00420AA1 /* MediaStreamPrivate.h in Headers */,
>-				150524F61B712FFC00696AA9 /* MediaStreamPrivateAVFObjC.h in Headers */,
> 				078E091717D14D1C00420AA1 /* MediaStreamRegistry.h in Headers */,
> 				078E091817D14D1C00420AA1 /* MediaStreamTrack.h in Headers */,
> 				078E091917D14D1C00420AA1 /* MediaStreamTrackEvent.h in Headers */,
>@@ -30298,7 +30291,6 @@
> 				0705851517FB0EEA005F2BCB /* MediaStreamCapabilities.cpp in Sources */,
> 				078E090017D14CEE00420AA1 /* MediaStreamEvent.cpp in Sources */,
> 				0711589117DF6F6600EDFE2B /* MediaStreamPrivate.cpp in Sources */,
>-				150524F71B712FFF00696AA9 /* MediaStreamPrivateAVFObjC.mm in Sources */,
> 				078E090117D14CEE00420AA1 /* MediaStreamRegistry.cpp in Sources */,
> 				078E090217D14CEE00420AA1 /* MediaStreamTrack.cpp in Sources */,
> 				078E090317D14CEE00420AA1 /* MediaStreamTrackEvent.cpp in Sources */,
>Index: Source/WebCore/platform/graphics/MediaPlayer.cpp
>===================================================================
>--- Source/WebCore/platform/graphics/MediaPlayer.cpp	(revision 191696)
>+++ Source/WebCore/platform/graphics/MediaPlayer.cpp	(working copy)
>@@ -72,6 +72,10 @@
> #include "MediaPlayerPrivateMediaSourceAVFObjC.h"
> #endif
> 
>+#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
>+#include "MediaPlayerPrivateMediaStreamAVFObjC.h"
>+#endif
>+
> #endif // PLATFORM(COCOA)
> 
> #if PLATFORM(WIN) && USE(AVFOUNDATION) && !USE(GSTREAMER)
>@@ -190,6 +194,10 @@ static void buildMediaEnginesVector()
>         MediaPlayerPrivateMediaSourceAVFObjC::registerMediaEngine(addMediaEngine);
> #endif
> 
>+#if ENABLE(MEDIA_STREAM)
>+        MediaPlayerPrivateMediaStreamAVFObjC::registerMediaEngine(addMediaEngine);
>+#endif
>+
> #if PLATFORM(WIN)
>         MediaPlayerPrivateAVFoundationCF::registerMediaEngine(addMediaEngine);
> #endif
>Index: Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h
>===================================================================
>--- Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h	(revision 191696)
>+++ Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h	(working copy)
>@@ -29,24 +29,22 @@
> #if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
> 
> #include "MediaPlayerPrivate.h"
>-#include "MediaStreamPrivateAVFObjC.h"
>-#include "SourceBufferPrivateClient.h"
>+#include "MediaStreamPrivate.h"
> #include <wtf/MediaTime.h>
> #include <wtf/Vector.h>
> #include <wtf/WeakPtr.h>
> 
>-OBJC_CLASS AVAsset;
>-OBJC_CLASS AVCaptureVideoPreviewLayer;
> OBJC_CLASS AVSampleBufferAudioRenderer;
>-OBJC_CLASS AVSampleBufferDisplayLayer;
>-OBJC_CLASS AVSampleBufferRenderSynchronizer;
> OBJC_CLASS AVStreamSession;
>-
>-typedef struct OpaqueCMTimebase* CMTimebaseRef;
>+typedef struct opaqueCMSampleBuffer *CMSampleBufferRef;
> 
> namespace WebCore {
> 
>-class MediaPlayerPrivateMediaStreamAVFObjC : public MediaPlayerPrivateInterface {
>+class AVVideoCaptureSource;
>+class Clock;
>+class MediaSourcePrivateClient;
>+
>+class MediaPlayerPrivateMediaStreamAVFObjC : public MediaPlayerPrivateInterface, public MediaStreamPrivate::Observer {
> public:
>     explicit MediaPlayerPrivateMediaStreamAVFObjC(MediaPlayer*);
>     virtual ~MediaPlayerPrivateMediaStreamAVFObjC();
>@@ -58,116 +56,109 @@ public:
>     static void getSupportedTypes(HashSet<String>& types);
>     static MediaPlayer::SupportsType supportsType(const MediaEngineSupportParameters&);
> 
>-    void addDisplayLayer(AVSampleBufferDisplayLayer*);
>-    void removeDisplayLayer(AVSampleBufferDisplayLayer*);
>-
>-    void addAudioRenderer(AVSampleBufferAudioRenderer*);
>-    void removeAudioRenderer(AVSampleBufferAudioRenderer*);
>-
>     MediaPlayer::NetworkState networkState() const override;
>+    void setNetworkState(MediaPlayer::NetworkState);
>     MediaPlayer::ReadyState readyState() const override;
>     void setReadyState(MediaPlayer::ReadyState);
> 
>-    void sizeChanged();
>-    void characteristicsChanged();
>-
>     WeakPtr<MediaPlayerPrivateMediaStreamAVFObjC> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(); }
> 
> private:
>-        // MediaPlayerPrivateInterface
>-        // FIXME(146853): Implement necessary conformations to standard in HTMLMediaElement for MediaStream 
>-    void load(const String&) override { };
>+    // MediaPlayerPrivateInterface
>+
>+    // FIXME(146853): Implement necessary conformations to standard in HTMLMediaElement for MediaStream
>+
>+    void load(const String&) override;
> #if ENABLE(MEDIA_SOURCE)
>-    void load(const String&, MediaSourcePrivateClient*) override { };
>+    void load(const String&, MediaSourcePrivateClient*) override;
> #endif
>-#if ENABLE(MEDIA_STREAM)
>     void load(MediaStreamPrivate&) override;
>-#endif
>     void cancelLoad() override;
> 
>     void prepareToPlay() override;
>-    PlatformMedia platformMedia() const override;
>     PlatformLayer* platformLayer() const override;
> 
>     bool supportsFullscreen() const override { return true; }
> 
>     void play() override;
>-    void playInternal();
>-
>     void pause() override;
>-    void pauseInternal();
>-
>     bool paused() const override;
> 
>     void setVolume(float) override;
>-    bool supportsMuting() const override { return true; }
>+    void internalSetVolume(float, bool);
>     void setMuted(bool) override;
>+    bool supportsMuting() const override { return true; }
> 
>-    bool supportsScanning() const override;
>+    bool supportsScanning() const override { return false; }
> 
>-    FloatSize naturalSize() const override;
>+    FloatSize naturalSize() const override { return m_intrinsicSize; }
> 
>     bool hasVideo() const override;
>     bool hasAudio() const override;
> 
>-    void setVisible(bool) override;
>+    void setVisible(bool) override { /* No-op */ }
> 
>     MediaTime durationMediaTime() const override;
>     MediaTime currentMediaTime() const override;
> 
>-    void seekWithTolerance(const MediaTime&, const MediaTime&, const MediaTime&) override { };
>-    bool seeking() const override;
>-    void setRateDouble(double) override;
>+    bool seeking() const override { return false; }
> 
>     std::unique_ptr<PlatformTimeRanges> seekable() const override;
>-    MediaTime maxMediaTimeSeekable() const override;
>-    MediaTime minMediaTimeSeekable() const override;
>     std::unique_ptr<PlatformTimeRanges> buffered() const override;
> 
>-    bool didLoadingProgress() const override;
>+    bool didLoadingProgress() const override { return m_playing; }
> 
>-    void setSize(const IntSize&) override;
>+    void setSize(const IntSize&) override { /* No-op */ }
> 
>     void paint(GraphicsContext&, const FloatRect&) override;
>     void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&) override;
>+    bool metaDataAvailable() const { return m_mediaStreamPrivate && m_readyState >= MediaPlayer::HaveMetadata; }
> 
>-    bool supportsAcceleratedRendering() const override;
>+    bool supportsAcceleratedRendering() const override { return true; }
> 
>-    MediaPlayer::MovieLoadType movieLoadType() const override;
>+    bool hasSingleSecurityOrigin() const override { return true; }
> 
>-    void prepareForRendering() override;
>+    MediaPlayer::MovieLoadType movieLoadType() const override { return MediaPlayer::LiveStream; }
> 
>     String engineDescription() const override;
> 
>-    String languageOfPrimaryAudioTrack() const override;
>-
>-    size_t extraMemoryCost() const override;
>+    size_t extraMemoryCost() const override { return 0; }
> 
>     bool shouldBePlaying() const;
> 
>-    RetainPtr<CGImageRef> createImageFromSampleBuffer(CMSampleBufferRef);
>+    MediaPlayer::ReadyState currentReadyState();
>+
>+    enum RenderingModeStatus {
>+        RenderingModeUnchanged,
>+        RenderingModeChanged,
>+    };
>+    RenderingModeStatus updateIntrinsicSize(const FloatSize&);
>+
>+    void createPreviewLayers();
>+
>+    void setPausedImageVisible(bool);
>+
>+    void scheduleDeferredTask(std::function<void()>);
> 
>-    friend class MediaStreamPrivateAVFObjC;
>+    // MediaStreamPrivate::Observer
>+    void activeStatusChanged() override;
>+    void characteristicsChanged() override;
> 
>-    MediaPlayer* m_player;
>+    MediaPlayer* m_player { nullptr };
>     WeakPtrFactory<MediaPlayerPrivateMediaStreamAVFObjC> m_weakPtrFactory;
>-    RefPtr<MediaStreamPrivateAVFObjC> m_MediaStreamPrivate;
>-    RetainPtr<AVAsset> m_asset;
>-    RetainPtr<AVCaptureVideoPreviewLayer> m_previewLayer;
>-    RetainPtr<AVSampleBufferDisplayLayer> m_sampleBufferDisplayLayer;
>-    Vector<RetainPtr<AVSampleBufferAudioRenderer>> m_sampleBufferAudioRenderers;
>-    RetainPtr<CGImageRef> m_lastImage;
>-    RetainPtr<id> m_timeJumpedObserver;
>-    RetainPtr<id> m_durationObserver;
>-    RetainPtr<AVStreamSession> m_streamSession;
>-    MediaPlayer::NetworkState m_networkState;
>-    MediaPlayer::ReadyState m_readyState;
>-    MediaTime m_lastSeekTime;
>-    double m_rate;
>-    bool m_playing;
>-    bool m_seeking;
>-    bool m_seekCompleted;
>-    mutable bool m_loadingProgressed;
>+    RefPtr<MediaStreamPrivate> m_mediaStreamPrivate;
>+    mutable RetainPtr<CALayer> m_previewLayer;
>+    mutable RetainPtr<PlatformLayer> m_videoBackgroundLayer;
>+    RetainPtr<CGImageRef> m_pausedImage;
>+    std::unique_ptr<Clock> m_clock;
>+    MediaPlayer::NetworkState m_networkState { MediaPlayer::Empty };
>+    MediaPlayer::ReadyState m_readyState { MediaPlayer::HaveNothing };
>+    FloatSize m_intrinsicSize;
>+    float m_volume { 1 };
>+    bool m_playing { false };
>+    bool m_muted { false };
>+    bool m_waitingForNewFrame {false };
>+    bool m_haveEverPlayed { false };
> };
>     
> }
>Index: Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm
>===================================================================
>--- Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm	(revision 191696)
>+++ Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm	(working copy)
>@@ -28,18 +28,14 @@
> 
> #if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
> 
>+#import "AVAudioCaptureSource.h"
> #import "AVVideoCaptureSource.h"
>-#import "FileSystem.h"
>+#import "Clock.h"
>+#import "GraphicsContext.h"
> #import "Logging.h"
> #import "MediaStreamPrivate.h"
>-#import "MediaStreamPrivateAVFObjC.h"
>-#import "MediaTimeAVFoundation.h"
>-#import "PlatformClockCM.h"
>-#import "WebCoreSystemInterface.h"
>-#import <AVFoundation/AVAsset.h>
>-#import <AVFoundation/AVCaptureVideoPreviewLayer.h>
>-#import <AVFoundation/AVTime.h>
> #import <QuartzCore/CALayer.h>
>+#import <QuartzCore/CATransaction.h>
> #import <objc_runtime.h>
> #import <wtf/MainThread.h>
> #import <wtf/NeverDestroyed.h>
>@@ -50,18 +46,6 @@
> 
> SOFT_LINK_FRAMEWORK_OPTIONAL(AVFoundation)
> 
>-SOFT_LINK_CLASS_OPTIONAL(AVFoundation, AVAsset)
>-SOFT_LINK_CLASS_OPTIONAL(AVFoundation, AVCaptureVideoPreviewLayer)
>-SOFT_LINK_CLASS_OPTIONAL(AVFoundation, AVURLAsset)
>-SOFT_LINK_CLASS_OPTIONAL(AVFoundation, AVStreamSession);
>-
>-#pragma mark -
>-#pragma mark AVStreamSession
>-
>-@interface AVStreamSession : NSObject
>-- (instancetype)initWithStorageDirectoryAtURL:(NSURL *)storageDirectory;
>-@end
>-
> namespace WebCore {
> 
> #pragma mark -
>@@ -70,19 +54,16 @@ namespace WebCore {
> MediaPlayerPrivateMediaStreamAVFObjC::MediaPlayerPrivateMediaStreamAVFObjC(MediaPlayer* player)
>     : m_player(player)
>     , m_weakPtrFactory(this)
>-    , m_networkState(MediaPlayer::Idle)
>-    , m_readyState(MediaPlayer::HaveNothing)
>-    , m_rate(1)
>-    , m_playing(0)
>-    , m_seeking(false)
>-    , m_seekCompleted(true)
>-    , m_loadingProgressed(false)
>+    , m_clock(Clock::create())
> {
>-
>+    LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::MediaPlayerPrivateMediaStreamAVFObjC(%p)", this);
> }
> 
> MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC()
> {
>+    LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC(%p)", this);
>+    if (m_mediaStreamPrivate)
>+        m_mediaStreamPrivate->removeObserver(*this);
> }
> 
> #pragma mark -
>@@ -97,29 +78,13 @@ void MediaPlayerPrivateMediaStreamAVFObj
> 
> bool MediaPlayerPrivateMediaStreamAVFObjC::isAvailable()
> {
>-    return AVFoundationLibrary()
>-        && isCoreMediaFrameworkAvailable();
>-}
>-
>-static const HashSet<String>& mimeTypeCache()
>-{
>-    static NeverDestroyed<HashSet<String>> cache;
>-    static bool typeListInitialized = false;
>-
>-    if (typeListInitialized)
>-        return cache;
>-    typeListInitialized = true;
>-
>-    NSArray *types = [getAVURLAssetClass() audiovisualMIMETypes];
>-    for (NSString *mimeType in types)
>-        cache.get().add(mimeType);
>-
>-    return cache;
>+    return AVFoundationLibrary() && isCoreMediaFrameworkAvailable();
> }
> 
> void MediaPlayerPrivateMediaStreamAVFObjC::getSupportedTypes(HashSet<String>& types)
> {
>-    types = mimeTypeCache();
>+    static NeverDestroyed<HashSet<String>> cache;
>+    types = cache;
> }
> 
> MediaPlayer::SupportsType MediaPlayerPrivateMediaStreamAVFObjC::supportsType(const MediaEngineSupportParameters& parameters)
>@@ -133,146 +98,174 @@ MediaPlayer::SupportsType MediaPlayerPri
> #pragma mark -
> #pragma mark MediaPlayerPrivateInterface Overrides
> 
>+void MediaPlayerPrivateMediaStreamAVFObjC::load(const String&)
>+{
>+    // This media engine only supports MediaStream URLs.
>+    scheduleDeferredTask([this] {
>+        setNetworkState(MediaPlayer::FormatError);
>+    });
>+}
>+
>+#if ENABLE(MEDIA_SOURCE)
>+void MediaPlayerPrivateMediaStreamAVFObjC::load(const String&, MediaSourcePrivateClient*)
>+{
>+    // This media engine only supports MediaStream URLs.
>+    scheduleDeferredTask([this] {
>+        setNetworkState(MediaPlayer::FormatError);
>+    });
>+}
>+#endif
>+
> void MediaPlayerPrivateMediaStreamAVFObjC::load(MediaStreamPrivate& stream)
> {
>     LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::load(%p)", this);
> 
>-    m_MediaStreamPrivate = MediaStreamPrivateAVFObjC::create(*this, stream);
>-    for (auto track : stream.tracks()) {
>-        m_MediaStreamPrivate->addTrack(WTF::move(track));
>-        if (!track->ended()) {
>-            track->source()->startProducingData();
>-            track->setEnabled(true);
>-        }
>-    }
>+    m_mediaStreamPrivate = &stream;
>+    m_mediaStreamPrivate->addObserver(*this);
>+    m_mediaStreamPrivate->startProducingData();
>+
>     m_previewLayer = nullptr;
>-    for (auto& track : m_MediaStreamPrivate->tracks()) {
>-        if (track->type() == RealtimeMediaSource::Type::Video)
>-            m_previewLayer = static_cast<AVVideoCaptureSource*>(track->source())->previewLayer();
>-    }
>-    m_player->client().mediaPlayerRenderingModeChanged(m_player);
>+
>+    scheduleDeferredTask([this] {
>+        setNetworkState(MediaPlayer::Idle);
>+        if (m_readyState < MediaPlayer::ReadyState::HaveMetadata)
>+            this->setReadyState(MediaPlayer::ReadyState::HaveMetadata);
>+    });
> }
> 
> void MediaPlayerPrivateMediaStreamAVFObjC::cancelLoad()
> {
>+    LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::cancelLoad(%p)", this);
>+    pause();
> }
> 
> void MediaPlayerPrivateMediaStreamAVFObjC::prepareToPlay()
> {
>-}
>-
>-PlatformMedia MediaPlayerPrivateMediaStreamAVFObjC::platformMedia() const
>-{
>-    PlatformMedia pm;
>-    pm.type = PlatformMedia::AVFoundationAssetType;
>-    pm.media.avfAsset = m_asset.get();
>-    return pm;
>+    LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::prepareToPlay(%p)", this);
> }
> 
> PlatformLayer* MediaPlayerPrivateMediaStreamAVFObjC::platformLayer() const
> {
>-    if (!m_previewLayer)
>+    if (!m_videoBackgroundLayer)
>         return nullptr;
>-    return static_cast<PlatformLayer*>(m_previewLayer);
>+
>+    return m_videoBackgroundLayer.get();
> }
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::play()
>+void MediaPlayerPrivateMediaStreamAVFObjC::setPausedImageVisible(bool visible)
> {
>-    auto weakThis = createWeakPtr();
>-    callOnMainThread([weakThis] {
>-        if (!weakThis)
>-            return;
>-        weakThis.get()->playInternal();
>-    });
>+    if (!metaDataAvailable() || !m_videoBackgroundLayer)
>+        return;
>+
>+    if (visible == m_previewLayer.get().hidden)
>+        return;
>+
>+    [CATransaction begin];
>+    [CATransaction setAnimationDuration:0];
>+    [CATransaction setDisableActions:YES];
>+
>+    do {
>+        if (visible) {
>+            m_clock->stop();
>+
>+            RefPtr<Image> image = m_mediaStreamPrivate->currentFrameImage();
>+            if (!image)
>+                break;
>+
>+            m_pausedImage = image->getCGImageRef();
>+            if (!m_pausedImage)
>+                break;
>+
>+            m_videoBackgroundLayer.get().contents = (id)m_pausedImage.get();
>+            m_previewLayer.get().hidden = true;
>+            m_waitingForNewFrame = false;
>+        } else {
>+            if (!m_mediaStreamPrivate->isProducingData()) {
>+                m_waitingForNewFrame = true;
>+                break;
>+            }
>+
>+            m_clock->start();
>+            m_waitingForNewFrame = false;
>+            m_previewLayer.get().hidden = false;
>+            m_videoBackgroundLayer.get().contents = nil;
>+            m_pausedImage = nullptr;
>+        }
>+    } while (0);
>+
>+    [CATransaction commit];
> }
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::playInternal()
>+void MediaPlayerPrivateMediaStreamAVFObjC::play()
> {
>-    m_playing = true;
>+    LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::play(%p)", this);
> 
>-    for (auto& track : m_MediaStreamPrivate->tracks())
>-        track->source()->startProducingData();
>+    if (!metaDataAvailable())
>+        return;
>+
>+    m_playing = true;
>+    m_haveEverPlayed = true;
>+    setPausedImageVisible(false);
> }
> 
> void MediaPlayerPrivateMediaStreamAVFObjC::pause()
> {
>-    auto weakThis = createWeakPtr();
>-    callOnMainThread([weakThis] {
>-        if (!weakThis)
>-            return;
>-        weakThis.get()->pauseInternal();
>-    });
>-}
>+    LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::pause(%p)", this);
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::pauseInternal()
>-{
>-    m_playing = false;
>+    if (!metaDataAvailable())
>+        return;
> 
>-    for (auto& track : m_MediaStreamPrivate->tracks())
>-        track->source()->stopProducingData();
>+    setPausedImageVisible(true);
>+    m_playing = false;
> }
> 
> bool MediaPlayerPrivateMediaStreamAVFObjC::paused() const
> {
>-    // Shouldn't be able to pause streams
>-    return false;
>+    return !m_playing;
> }
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::setVolume(float)
>+void MediaPlayerPrivateMediaStreamAVFObjC::internalSetVolume(float volume, bool internal)
> {
>+    if (!internal)
>+        m_volume = volume;
>+
>+    if (!metaDataAvailable())
>+        return;
>+
>+    // FIXME: Set volume once we actually play audio.
> }
> 
>-bool MediaPlayerPrivateMediaStreamAVFObjC::supportsScanning() const
>+void MediaPlayerPrivateMediaStreamAVFObjC::setVolume(float volume)
> {
>-    return true;
>+    internalSetVolume(volume, false);
> }
> 
> void MediaPlayerPrivateMediaStreamAVFObjC::setMuted(bool muted)
> {
>-    for (auto& track : m_MediaStreamPrivate->tracks()) {
>-        if (track->type() == RealtimeMediaSource::Type::Audio)
>-            track->source()->setMuted(muted);
>-    }
>-}
>+    LOG(Media, "MediaPlayerPrivateMediaStreamAVFObjC::setMuted(%p)", this);
> 
>-FloatSize MediaPlayerPrivateMediaStreamAVFObjC::naturalSize() const
>-{
>-    FloatSize floatSize(0, 0);
>-    for (auto& track : m_MediaStreamPrivate->tracks()) {
>-        if (track->type() == RealtimeMediaSource::Type::Video) {
>-            AVVideoCaptureSource* source = (AVVideoCaptureSource*)track->source();
>-            if (!source->stopped() && track->enabled()) {
>-                if (source->width() > floatSize.width())
>-                    floatSize.setWidth(source->width());
>-                if (source->height() > floatSize.height())
>-                    floatSize.setHeight(source->height());
>-            }
>-        }
>-    }
>-    return floatSize;
>+    if (muted == m_muted)
>+        return;
>+
>+    m_muted = muted;
>+    
>+    internalSetVolume(muted ? 0 : m_volume, true);
> }
> 
> bool MediaPlayerPrivateMediaStreamAVFObjC::hasVideo() const
> {
>-    for (auto& track : m_MediaStreamPrivate->tracks()) {
>-        if (track->type() == RealtimeMediaSource::Type::Video)
>-            return true;
>-    }
>-    return false;
>+    if (!metaDataAvailable())
>+        return false;
>+    
>+    return m_mediaStreamPrivate->hasVideo();
> }
> 
> bool MediaPlayerPrivateMediaStreamAVFObjC::hasAudio() const
> {
>-    for (auto& track : m_MediaStreamPrivate->tracks()) {
>-        if (track->type() == RealtimeMediaSource::Type::Audio)
>-            return true;
>-    }
>-    return false;
>-}
>-
>-void MediaPlayerPrivateMediaStreamAVFObjC::setVisible(bool)
>-{
>-    // No-op.
>+    if (!metaDataAvailable())
>+        return false;
>+    
>+    return m_mediaStreamPrivate->hasAudio();
> }
> 
> MediaTime MediaPlayerPrivateMediaStreamAVFObjC::durationMediaTime() const
>@@ -282,19 +275,7 @@ MediaTime MediaPlayerPrivateMediaStreamA
> 
> MediaTime MediaPlayerPrivateMediaStreamAVFObjC::currentMediaTime() const
> {
>-    // FIXME(147125): Must implement this later
>-    return MediaTime::zeroTime();
>-}
>-
>-bool MediaPlayerPrivateMediaStreamAVFObjC::seeking() const
>-{
>-    // MediaStream is unseekable
>-    return false;
>-}
>-
>-void MediaPlayerPrivateMediaStreamAVFObjC::setRateDouble(double)
>-{
>-    // MediaStream is unseekable; therefore, cannot set rate
>+    return MediaTime::createWithDouble(m_clock->currentTime());
> }
> 
> MediaPlayer::NetworkState MediaPlayerPrivateMediaStreamAVFObjC::networkState() const
>@@ -307,85 +288,129 @@ MediaPlayer::ReadyState MediaPlayerPriva
>     return m_readyState;
> }
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::sizeChanged()
>+MediaPlayer::ReadyState MediaPlayerPrivateMediaStreamAVFObjC::currentReadyState()
> {
>-    m_player->sizeChanged();
>-}
>+    if (m_mediaStreamPrivate) {
>+        if (m_mediaStreamPrivate->active())
>+            return MediaPlayer::ReadyState::HaveEnoughData;
> 
>-std::unique_ptr<PlatformTimeRanges> MediaPlayerPrivateMediaStreamAVFObjC::seekable() const
>-{
>-    return std::make_unique<PlatformTimeRanges>(minMediaTimeSeekable(), maxMediaTimeSeekable());
>-}
>+        return MediaPlayer::ReadyState::HaveCurrentData;
>+    }
> 
>-MediaTime MediaPlayerPrivateMediaStreamAVFObjC::maxMediaTimeSeekable() const
>-{
>-    return durationMediaTime();
>+    return MediaPlayer::ReadyState::HaveNothing;
> }
> 
>-MediaTime MediaPlayerPrivateMediaStreamAVFObjC::minMediaTimeSeekable() const
>+void MediaPlayerPrivateMediaStreamAVFObjC::activeStatusChanged()
> {
>-    return startTime();
>-}
>+    MediaPlayer::ReadyState newReadyState = currentReadyState();
> 
>-std::unique_ptr<PlatformTimeRanges> MediaPlayerPrivateMediaStreamAVFObjC::buffered() const
>-{
>-    return std::make_unique<PlatformTimeRanges>();
>+    scheduleDeferredTask([this, newReadyState] {
>+        if (newReadyState != m_readyState)
>+            setReadyState(newReadyState);
>+    });
> }
> 
>-bool MediaPlayerPrivateMediaStreamAVFObjC::didLoadingProgress() const
>+MediaPlayerPrivateMediaStreamAVFObjC::RenderingModeStatus MediaPlayerPrivateMediaStreamAVFObjC::updateIntrinsicSize(const FloatSize& size)
> {
>-    bool loadingProgressed = m_loadingProgressed;
>-    m_loadingProgressed = false;
>-    return loadingProgressed;
>-}
>+    if (size == m_intrinsicSize)
>+        return RenderingModeUnchanged;
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::setSize(const IntSize&)
>-{
>-    // No-op.
>+    m_intrinsicSize = size;
>+
>+    if (m_videoBackgroundLayer || !m_player || !m_player->client().mediaPlayerRenderingCanBeAccelerated(m_player))
>+        return RenderingModeUnchanged;
>+
>+    if (!m_mediaStreamPrivate || !m_mediaStreamPrivate->platformLayer())
>+        return RenderingModeUnchanged;
>+
>+    createPreviewLayers();
>+
>+    return RenderingModeChanged;
> }
> 
>-RetainPtr<CGImageRef> MediaPlayerPrivateMediaStreamAVFObjC::createImageFromSampleBuffer(CMSampleBufferRef sampleBuffer)
>+void MediaPlayerPrivateMediaStreamAVFObjC::createPreviewLayers()
> {
>-    CVPixelBufferRef imageBuffer = static_cast<CVPixelBufferRef>(CMSampleBufferGetImageBuffer(sampleBuffer));
>-    CVPixelBufferLockBaseAddress(imageBuffer, 0);
>-    
>-    void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer);
>-    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);
>-    size_t width = CVPixelBufferGetWidth(imageBuffer);
>-    size_t height = CVPixelBufferGetHeight(imageBuffer);
>-    
>-    RetainPtr<CGColorSpaceRef> colorSpace = adoptCF(CGColorSpaceCreateDeviceRGB());
>-    RetainPtr<CGDataProviderRef> provider = adoptCF(CGDataProviderCreateWithData(NULL, baseAddress, bytesPerRow * height, NULL));
>-    RetainPtr<CGImageRef> quartzImage = adoptCF(CGImageCreate(width, height, 8, 32, bytesPerRow, colorSpace.get(), kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst, provider.get(), NULL, true, kCGRenderingIntentDefault));
>-    
>-    CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
>-    
>-    return quartzImage;
>+    if (m_videoBackgroundLayer)
>+        return;
>+
>+    m_videoBackgroundLayer = adoptNS([[CALayer alloc] init]);
>+#ifndef NDEBUG
>+    m_videoBackgroundLayer.get().name = @"MediaPlayerPrivateMediaStreamAVFObjC preview background layer";
>+#endif
>+
>+    m_previewLayer = m_mediaStreamPrivate->platformLayer();
>+    m_previewLayer.get().contentsGravity = kCAGravityResize;
>+    m_previewLayer.get().anchorPoint = CGPointZero;
>+#if !PLATFORM(IOS)
>+    m_previewLayer.get().autoresizingMask = kCALayerWidthSizable | kCALayerHeightSizable;
>+#endif
>+    if (!m_playing)
>+        m_previewLayer.get().hidden = true;
>+
>+    [m_videoBackgroundLayer addSublayer:m_previewLayer.get()];
> }
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::paint(GraphicsContext& context, const FloatRect& rect)
>+void MediaPlayerPrivateMediaStreamAVFObjC::characteristicsChanged()
> {
>-    paintCurrentFrameInContext(context, rect);
>+    RenderingModeStatus status = RenderingModeUnchanged;
>+    bool sizeChanged = false;
>+
>+    FloatSize intrinsicSize = m_mediaStreamPrivate->intrinsicSize();
>+    if (intrinsicSize.height() != m_intrinsicSize.height() || intrinsicSize.width() != m_intrinsicSize.width()) {
>+        status = updateIntrinsicSize(intrinsicSize);
>+        sizeChanged = true;
>+    }
>+
>+    if (m_waitingForNewFrame && m_mediaStreamPrivate->isProducingData())
>+        setPausedImageVisible(false);
>+
>+    MediaPlayer::ReadyState newReadyState = currentReadyState();
>+    scheduleDeferredTask([this, sizeChanged, newReadyState, status] {
>+        if (newReadyState != m_readyState)
>+            setReadyState(newReadyState);
>+
>+        if (!m_player)
>+            return;
>+
>+        m_player->characteristicChanged();
>+        if (sizeChanged) {
>+            m_player->sizeChanged();
>+            if (status == RenderingModeChanged)
>+                m_player->client().mediaPlayerRenderingModeChanged(m_player);
>+        }
>+    });
> }
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::paintCurrentFrameInContext(GraphicsContext&, const FloatRect&)
>+std::unique_ptr<PlatformTimeRanges> MediaPlayerPrivateMediaStreamAVFObjC::seekable() const
> {
>-    // FIXME(125157): Implement painting.
>+    return std::make_unique<PlatformTimeRanges>();
> }
> 
>-bool MediaPlayerPrivateMediaStreamAVFObjC::supportsAcceleratedRendering() const
>+std::unique_ptr<PlatformTimeRanges> MediaPlayerPrivateMediaStreamAVFObjC::buffered() const
> {
>-    return true;
>+    return std::make_unique<PlatformTimeRanges>();
> }
> 
>-MediaPlayer::MovieLoadType MediaPlayerPrivateMediaStreamAVFObjC::movieLoadType() const
>+void MediaPlayerPrivateMediaStreamAVFObjC::paint(GraphicsContext& context, const FloatRect& rect)
> {
>-    return MediaPlayer::LiveStream;
>+    paintCurrentFrameInContext(context, rect);
> }
> 
>-void MediaPlayerPrivateMediaStreamAVFObjC::prepareForRendering()
>+void MediaPlayerPrivateMediaStreamAVFObjC::paintCurrentFrameInContext(GraphicsContext& context, const FloatRect& rect)
> {
>-    // No-op.
>+    if (!metaDataAvailable() || context.paintingDisabled() || !m_haveEverPlayed)
>+        return;
>+
>+    if (m_playing)
>+        m_mediaStreamPrivate->paintCurrentFrameInContext(context, rect);
>+    else if (m_pausedImage) {
>+        GraphicsContextStateSaver stateSaver(context);
>+        context.translate(rect.x(), rect.y() + rect.height());
>+        context.scale(FloatSize(1, -1));
>+        context.setImageInterpolationQuality(InterpolationLow);
>+        IntRect paintRect(IntPoint(0, 0), IntSize(rect.width(), rect.height()));
>+        CGContextDrawImage(context.platformContext(), CGRectMake(0, 0, paintRect.width(), paintRect.height()), m_pausedImage.get());
>+    }
> }
> 
> String MediaPlayerPrivateMediaStreamAVFObjC::engineDescription() const
>@@ -394,17 +419,6 @@ String MediaPlayerPrivateMediaStreamAVFO
>     return description;
> }
> 
>-String MediaPlayerPrivateMediaStreamAVFObjC::languageOfPrimaryAudioTrack() const
>-{
>-    // FIXME(125158): implement languageOfPrimaryAudioTrack()
>-    return emptyString();
>-}
>-
>-size_t MediaPlayerPrivateMediaStreamAVFObjC::extraMemoryCost() const
>-{
>-    return 0;
>-}
>-
> bool MediaPlayerPrivateMediaStreamAVFObjC::shouldBePlaying() const
> {
>     return m_playing && m_readyState >= MediaPlayer::HaveFutureData;
>@@ -420,6 +434,27 @@ void MediaPlayerPrivateMediaStreamAVFObj
>     m_player->readyStateChanged();
> }
> 
>+void MediaPlayerPrivateMediaStreamAVFObjC::setNetworkState(MediaPlayer::NetworkState networkState)
>+{
>+    if (m_networkState == networkState)
>+        return;
>+
>+    m_networkState = networkState;
>+    m_player->networkStateChanged();
>+}
>+
>+void MediaPlayerPrivateMediaStreamAVFObjC::scheduleDeferredTask(std::function<void()> function)
>+{
>+    ASSERT(function);
>+    auto weakThis = createWeakPtr();
>+    callOnMainThread([weakThis, function] {
>+        if (!weakThis)
>+            return;
>+
>+        function();
>+    });
>+}
>+
> }
> 
> #endif
>Index: Source/WebCore/platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.h
>===================================================================
>--- Source/WebCore/platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.h	(revision 191696)
>+++ Source/WebCore/platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.h	(working copy)
>@@ -1,87 +0,0 @@
>-/*
>- * Copyright (C) 2015 Apple Inc. All rights reserved.
>- *
>- * Redistribution and use in source and binary forms, with or without
>- * modification, are permitted provided that the following conditions
>- * are met:
>- * 1. Redistributions of source code must retain the above copyright
>- *    notice, this list of conditions and the following disclaimer.
>- * 2. Redistributions in binary form must reproduce the above copyright
>- *    notice, this list of conditions and the following disclaimer in the
>- *    documentation and/or other materials provided with the distribution.
>- *
>- * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
>- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
>- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>- * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
>- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
>- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
>- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
>- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
>- * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>- */
>-
>-#ifndef MediaStreamPrivateAVFObjC_h
>-#define MediaStreamPrivateAVFObjC_h
>-
>-#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
>-
>-#include "MediaPlayer.h"
>-#include "MediaStreamPrivate.h"
>-#include <wtf/Deque.h>
>-#include <wtf/HashMap.h>
>-#include <wtf/MediaTime.h>
>-#include <wtf/RefPtr.h>
>-#include <wtf/RetainPtr.h>
>-#include <wtf/Vector.h>
>-
>-OBJC_CLASS AVAsset;
>-OBJC_CLASS AVStreamDataParser;
>-OBJC_CLASS NSError;
>-OBJC_CLASS NSObject;
>-typedef struct opaqueCMSampleBuffer *CMSampleBufferRef;
>-
>-namespace WebCore {
>-
>-class CDMSession;
>-class MediaPlayerPrivateMediaStreamAVFObjC;
>-class MediaStreamPrivate;
>-class MediaStreamPrivateClient;
>-class PlatformTimeRanges;
>-class TimeRanges;
>-
>-class MediaStreamPrivateAVFObjC final : public MediaStreamPrivate {
>-public:
>-    static RefPtr<MediaStreamPrivateAVFObjC> create(MediaPlayerPrivateMediaStreamAVFObjC&, MediaStreamPrivate&);
>-    virtual ~MediaStreamPrivateAVFObjC();
>-
>-    MediaPlayerPrivateMediaStreamAVFObjC* player() const { return m_player; }
>-
>-    virtual MediaStreamPrivateClient* client() const { return m_client.get(); }
>-    virtual void setClient(MediaStreamPrivateClient* client) { m_client = client; }
>-
>-    MediaTime duration();
>-    std::unique_ptr<PlatformTimeRanges> buffered();
>-
>-    bool hasAudio() const;
>-    bool hasVideo() const;
>-
>-    void setReadyState(MediaPlayer::ReadyState);
>-    FloatSize naturalSize() const;
>-
>-    MediaPlayer::ReadyState readyState() const;
>-
>-private:
>-    MediaStreamPrivateAVFObjC(MediaPlayerPrivateMediaStreamAVFObjC&, MediaStreamPrivate&);
>-
>-    MediaPlayerPrivateMediaStreamAVFObjC* m_player;
>-    RefPtr<MediaStreamPrivateClient> m_client;
>-};
>-    
>-}
>-
>-#endif // ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
>-
>-#endif
>Index: Source/WebCore/platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.mm
>===================================================================
>--- Source/WebCore/platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.mm	(revision 191696)
>+++ Source/WebCore/platform/graphics/avfoundation/objc/MediaStreamPrivateAVFObjC.mm	(working copy)
>@@ -1,98 +0,0 @@
>-/*
>- * Copyright (C) 2015 Apple Inc. All rights reserved.
>- *
>- * Redistribution and use in source and binary forms, with or without
>- * modification, are permitted provided that the following conditions
>- * are met:
>- * 1. Redistributions of source code must retain the above copyright
>- *    notice, this list of conditions and the following disclaimer.
>- * 2. Redistributions in binary form must reproduce the above copyright
>- *    notice, this list of conditions and the following disclaimer in the
>- *    documentation and/or other materials provided with the distribution.
>- *
>- * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
>- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
>- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>- * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
>- * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
>- * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
>- * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
>- * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
>- * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>- * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>- */
>-
>-#include "config.h"
>-#include "MediaStreamPrivateAVFObjC.h"
>-
>-#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
>-
>-#import "ContentType.h"
>-#import "ExceptionCodePlaceholder.h"
>-#import "MediaPlayerPrivateMediaStreamAVFObjC.h"
>-#import "MediaStreamPrivate.h"
>-#import "SoftLinking.h"
>-#import "SourceBufferPrivateAVFObjC.h"
>-#import <objc/runtime.h>
>-#import <wtf/text/AtomicString.h>
>-#import <wtf/text/StringBuilder.h>
>-
>-namespace WebCore {
>-
>-#pragma mark -
>-#pragma mark MediaStreamPrivateAVFObjC
>-
>-RefPtr<MediaStreamPrivateAVFObjC> MediaStreamPrivateAVFObjC::create(MediaPlayerPrivateMediaStreamAVFObjC& parent, MediaStreamPrivate& stream)
>-{
>-    return adoptRef(new MediaStreamPrivateAVFObjC(parent, stream));
>-}
>-
>-MediaStreamPrivateAVFObjC::MediaStreamPrivateAVFObjC(MediaPlayerPrivateMediaStreamAVFObjC& parent, MediaStreamPrivate& stream)
>-    : MediaStreamPrivate(*stream.client())
>-    , m_player(&parent)
>-{
>-}
>-
>-MediaStreamPrivateAVFObjC::~MediaStreamPrivateAVFObjC()
>-{
>-}
>-
>-MediaTime MediaStreamPrivateAVFObjC::duration()
>-{
>-    return MediaTime::positiveInfiniteTime();
>-}
>-
>-std::unique_ptr<PlatformTimeRanges> MediaStreamPrivateAVFObjC::buffered()
>-{
>-    return std::unique_ptr<PlatformTimeRanges>();
>-}
>-
>-MediaPlayer::ReadyState MediaStreamPrivateAVFObjC::readyState() const
>-{
>-    return m_player->readyState();
>-}
>-
>-void MediaStreamPrivateAVFObjC::setReadyState(MediaPlayer::ReadyState readyState)
>-{
>-    m_player->setReadyState(readyState);
>-}
>-
>-bool MediaStreamPrivateAVFObjC::hasAudio() const
>-{
>-    return m_player->hasAudio();
>-}
>-
>-bool MediaStreamPrivateAVFObjC::hasVideo() const
>-{
>-    return m_player->hasVideo();
>-}
>-
>-FloatSize MediaStreamPrivateAVFObjC::naturalSize() const
>-{
>-    return m_player->naturalSize();
>-}
>-    
>-}
>-
>-#endif // ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
>Index: Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp
>===================================================================
>--- Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp	(revision 191696)
>+++ Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp	(working copy)
>@@ -2,6 +2,7 @@
>  * Copyright (C) 2011, 2015 Ericsson AB. All rights reserved.
>  * Copyright (C) 2013 Google Inc. All rights reserved.
>  * Copyright (C) 2013 Nokia Corporation and/or its subsidiary(-ies).
>+ * Copyright (C) 2015 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions are
>@@ -31,12 +32,14 @@
>  */
> 
> #include "config.h"
>+#include "MediaStreamPrivate.h"
> 
> #if ENABLE(MEDIA_STREAM)
> 
>-#include "MediaStreamPrivate.h"
>-
>+#include "GraphicsContext.h"
>+#include "IntRect.h"
> #include "UUID.h"
>+#include <wtf/MainThread.h>
> #include <wtf/RefCounted.h>
> #include <wtf/Vector.h>
> 
>@@ -62,25 +65,35 @@ RefPtr<MediaStreamPrivate> MediaStreamPr
> }
> 
> MediaStreamPrivate::MediaStreamPrivate(const String& id, const MediaStreamTrackPrivateVector& tracks)
>-    : m_id(id)
>+    : m_weakPtrFactory(this)
>+    , m_id(id)
> {
>     ASSERT(!m_id.isEmpty());
> 
>-    for (auto& track : tracks)
>+    for (auto& track : tracks) {
>+        track->addObserver(*this);
>         m_trackSet.add(track->id(), track);
>+    }
> 
>     updateActiveState(NotifyClientOption::DontNotify);
> }
> 
>-MediaStreamPrivate::MediaStreamPrivate(MediaStreamPrivateClient& client)
>-    : m_client(&client)
>+MediaStreamPrivate::~MediaStreamPrivate()
>+{
>+    for (auto& track : m_trackSet.values())
>+        track->removeObserver(*this);
>+}
>+
>+void MediaStreamPrivate::addObserver(MediaStreamPrivate::Observer& observer)
> {
>+    m_observers.append(&observer);
> }
> 
>-MediaStreamPrivate::~MediaStreamPrivate()
>+void MediaStreamPrivate::removeObserver(MediaStreamPrivate::Observer& observer)
> {
>-    m_client = nullptr;
>-    m_isActive = false;
>+    size_t pos = m_observers.find(&observer);
>+    if (pos != notFound)
>+        m_observers.remove(pos);
> }
> 
> MediaStreamTrackPrivateVector MediaStreamPrivate::tracks() const
>@@ -94,21 +107,29 @@ MediaStreamTrackPrivateVector MediaStrea
> 
> void MediaStreamPrivate::updateActiveState(NotifyClientOption notifyClientOption)
> {
>-    // A stream is active if it has at least one un-ended track.
>     bool newActiveState = false;
>+    m_activeVideoTrack = nullptr;
>     for (auto& track : m_trackSet.values()) {
>+
>         if (!track->ended()) {
>             newActiveState = true;
>-            break;
>+            if (track->source()->type() == RealtimeMediaSource::Type::Video) {
>+                m_activeVideoTrack = track.get();
>+                break;
>+            }
>         }
>     }
> 
>+    // A stream is active if it has at least one un-ended track.
>     if (newActiveState == m_isActive)
>         return;
>+
>     m_isActive = newActiveState;
> 
>-    if (m_client && notifyClientOption == NotifyClientOption::Notify)
>-        m_client->activeStatusChanged();
>+    if (notifyClientOption == NotifyClientOption::Notify) {
>+        for (auto& observer : m_observers)
>+            observer->activeStatusChanged();
>+    }
> }
> 
> void MediaStreamPrivate::addTrack(RefPtr<MediaStreamTrackPrivate>&& track, NotifyClientOption notifyClientOption)
>@@ -119,8 +140,10 @@ void MediaStreamPrivate::addTrack(RefPtr
>     track->addObserver(*this);
>     m_trackSet.add(track->id(), track);
> 
>-    if (m_client && notifyClientOption == NotifyClientOption::Notify)
>-        m_client->didAddTrack(*track.get());
>+    if (notifyClientOption == NotifyClientOption::Notify) {
>+        for (auto& observer : m_observers)
>+            observer->didAddTrack(*track.get());
>+    }
> 
>     updateActiveState(notifyClientOption);
> }
>@@ -132,12 +155,142 @@ void MediaStreamPrivate::removeTrack(Med
> 
>     track.removeObserver(*this);
> 
>-    if (m_client && notifyClientOption == NotifyClientOption::Notify)
>-        m_client->didRemoveTrack(track);
>+    if (notifyClientOption == NotifyClientOption::Notify) {
>+        for (auto& observer : m_observers)
>+            observer->didRemoveTrack(track);
>+    }
> 
>     updateActiveState(NotifyClientOption::Notify);
> }
> 
>+void MediaStreamPrivate::startProducingData()
>+{
>+    for (auto& track : m_trackSet.values())
>+        track->startProducingData();
>+}
>+
>+void MediaStreamPrivate::stopProducingData()
>+{
>+    for (auto& track : m_trackSet.values())
>+        track->stopProducingData();
>+}
>+
>+bool MediaStreamPrivate::isProducingData() const
>+{
>+    for (auto& track : m_trackSet.values()) {
>+        if (track->isProducingData())
>+            return true;
>+    }
>+    return false;
>+}
>+
>+bool MediaStreamPrivate::hasVideo()
>+{
>+    for (auto& track : m_trackSet.values()) {
>+        if (track->type() == RealtimeMediaSource::Type::Video && track->enabled())
>+            return true;
>+    }
>+    return false;
>+}
>+
>+bool MediaStreamPrivate::hasAudio()
>+{
>+    for (auto& track : m_trackSet.values()) {
>+        if (track->type() == RealtimeMediaSource::Type::Audio && track->enabled())
>+            return true;
>+    }
>+    return false;
>+}
>+
>+FloatSize MediaStreamPrivate::intrinsicSize() const
>+{
>+    FloatSize size;
>+
>+    if (m_activeVideoTrack) {
>+        const RealtimeMediaSourceStates& states = m_activeVideoTrack->source()->states();
>+        size.setWidth(states.width());
>+        size.setHeight(states.height());
>+    }
>+
>+    return size;
>+}
>+
>+PlatformLayer* MediaStreamPrivate::platformLayer() const
>+{
>+    if (!m_activeVideoTrack)
>+        return nullptr;
>+
>+    return m_activeVideoTrack->source()->platformLayer();
>+}
>+
>+void MediaStreamPrivate::paintCurrentFrameInContext(GraphicsContext& context, const FloatRect& rect)
>+{
>+    if (context.paintingDisabled())
>+        return;
>+
>+    if (active() && m_activeVideoTrack)
>+        m_activeVideoTrack->source()->paintCurrentFrameInContext(context, rect);
>+    else {
>+        GraphicsContextStateSaver stateSaver(context);
>+        context.translate(rect.x(), rect.y() + rect.height());
>+        context.scale(FloatSize(1, -1));
>+        IntRect paintRect(IntPoint(0, 0), IntSize(rect.width(), rect.height()));
>+        context.fillRect(paintRect, Color::black, ColorSpaceDeviceRGB);
>+    }
>+}
>+
>+RefPtr<Image> MediaStreamPrivate::currentFrameImage()
>+{
>+    if (!active() && !m_activeVideoTrack)
>+        return nullptr;
>+
>+    return m_activeVideoTrack->source()->currentFrameImage();
>+}
>+
>+void MediaStreamPrivate::characteristicsChanged()
>+{
>+    for (auto& observer : m_observers)
>+        observer->characteristicsChanged();
>+}
>+
>+void MediaStreamPrivate::trackMutedChanged(MediaStreamTrackPrivate&)
>+{
>+    scheduleDeferredTask([this] {
>+        characteristicsChanged();
>+    });
>+}
>+
>+void MediaStreamPrivate::trackStatesChanged(MediaStreamTrackPrivate&)
>+{
>+    characteristicsChanged();
>+}
>+
>+void MediaStreamPrivate::trackEnabledChanged(MediaStreamTrackPrivate&)
>+{
>+    scheduleDeferredTask([this] {
>+        characteristicsChanged();
>+    });
>+}
>+
>+void MediaStreamPrivate::trackProducingDataChanged(MediaStreamTrackPrivate&)
>+{
>+    scheduleDeferredTask([this] {
>+        characteristicsChanged();
>+    });
>+}
>+
>+void MediaStreamPrivate::scheduleDeferredTask(std::function<void()> function)
>+{
>+    ASSERT(function);
>+    auto weakThis = createWeakPtr();
>+    callOnMainThread([weakThis, function] {
>+        if (!weakThis)
>+            return;
>+
>+        function();
>+    });
>+}
>+
> } // namespace WebCore
> 
> #endif // ENABLE(MEDIA_STREAM)
>Index: Source/WebCore/platform/mediastream/MediaStreamPrivate.h
>===================================================================
>--- Source/WebCore/platform/mediastream/MediaStreamPrivate.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/MediaStreamPrivate.h	(working copy)
>@@ -2,6 +2,7 @@
>  * Copyright (C) 2011, 2015 Ericsson AB. All rights reserved.
>  * Copyright (C) 2012 Google Inc. All rights reserved.
>  * Copyright (C) 2013 Nokia Corporation and/or its subsidiary(-ies).
>+ * Copyright (C) 2015 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -35,28 +36,32 @@
> 
> #if ENABLE(MEDIA_STREAM)
> 
>+#include "FloatSize.h"
> #include "MediaStreamTrack.h"
> #include "MediaStreamTrackPrivate.h"
> #include <wtf/HashMap.h>
>+#include <wtf/MediaTime.h>
> #include <wtf/RefCounted.h>
> #include <wtf/RefPtr.h>
> #include <wtf/Vector.h>
>+#include <wtf/WeakPtr.h>
> 
> namespace WebCore {
> 
>-class MediaStreamTrackPrivate;
>-
>-class MediaStreamPrivateClient : public RefCounted<MediaStreamPrivateClient> {
>-public:
>-    virtual ~MediaStreamPrivateClient() { }
>-
>-    virtual void activeStatusChanged() = 0;
>-    virtual void didAddTrack(MediaStreamTrackPrivate&) = 0;
>-    virtual void didRemoveTrack(MediaStreamTrackPrivate&) = 0;
>-};
>+class MediaStream;
> 
> class MediaStreamPrivate : public MediaStreamTrackPrivate::Observer, public RefCounted<MediaStreamPrivate> {
> public:
>+    class Observer {
>+    public:
>+        virtual ~Observer() { }
>+
>+        virtual void characteristicsChanged() { }
>+        virtual void activeStatusChanged() { }
>+        virtual void didAddTrack(MediaStreamTrackPrivate&) { }
>+        virtual void didRemoveTrack(MediaStreamTrackPrivate&) { }
>+    };
>+
>     static RefPtr<MediaStreamPrivate> create(const Vector<RefPtr<RealtimeMediaSource>>& audioSources, const Vector<RefPtr<RealtimeMediaSource>>& videoSources);
>     static RefPtr<MediaStreamPrivate> create(const MediaStreamTrackPrivateVector&);
> 
>@@ -64,8 +69,8 @@ public:
> 
>     enum class NotifyClientOption { Notify, DontNotify };
> 
>-    MediaStreamPrivateClient* client() const { return m_client; }
>-    void setClient(MediaStreamPrivateClient* client) { m_client = client; }
>+    void addObserver(Observer&);
>+    void removeObserver(Observer&);
> 
>     String id() const { return m_id; }
> 
>@@ -77,21 +82,40 @@ public:
>     void addTrack(RefPtr<MediaStreamTrackPrivate>&&, NotifyClientOption = NotifyClientOption::Notify);
>     void removeTrack(MediaStreamTrackPrivate&, NotifyClientOption = NotifyClientOption::Notify);
> 
>-protected:
>-    explicit MediaStreamPrivate(MediaStreamPrivateClient&);
>+    void startProducingData();
>+    void stopProducingData();
>+    bool isProducingData() const;
>+
>+    PlatformLayer* platformLayer() const;
>+    RefPtr<Image> currentFrameImage();
>+    void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&);
>+
>+    bool hasVideo();
>+    bool hasAudio();
>+
>+    FloatSize intrinsicSize() const;
>+
>+    WeakPtr<MediaStreamPrivate> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(); }
> 
> private:
>     MediaStreamPrivate(const String&, const MediaStreamTrackPrivateVector&);
> 
>     void trackEnded(MediaStreamTrackPrivate&) override { }
>-    void trackMutedChanged(MediaStreamTrackPrivate&) override { }
>-    void trackStatesChanged(MediaStreamTrackPrivate&) override { }
>+    void trackMutedChanged(MediaStreamTrackPrivate&) override;
>+    void trackStatesChanged(MediaStreamTrackPrivate&) override;
>+    void trackProducingDataChanged(MediaStreamTrackPrivate&) override;
>+    void trackEnabledChanged(MediaStreamTrackPrivate&) override;
> 
>-    MediaStreamPrivateClient* m_client { nullptr };
>-    String m_id;
>-    bool m_isActive { false };
>+    void characteristicsChanged();
> 
>+    void scheduleDeferredTask(std::function<void()>);
>+
>+    WeakPtrFactory<MediaStreamPrivate> m_weakPtrFactory;
>+    Vector<Observer*> m_observers;
>+    String m_id;
>+    MediaStreamTrackPrivate* m_activeVideoTrack { nullptr };
>     HashMap<String, RefPtr<MediaStreamTrackPrivate>> m_trackSet;
>+    bool m_isActive { false };
> };
> 
> typedef Vector<RefPtr<MediaStreamPrivate>> MediaStreamPrivateVector;
>Index: Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp
>===================================================================
>--- Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp	(revision 191696)
>+++ Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.cpp	(working copy)
>@@ -178,6 +178,18 @@ void MediaStreamTrackPrivate::sourceStat
>         observer->trackStatesChanged(*this);
> }
> 
>+void MediaStreamTrackPrivate::sourceProducingDataChanged()
>+{
>+    for (auto& observer : m_observers)
>+        observer->trackProducingDataChanged(*this);
>+}
>+
>+void MediaStreamTrackPrivate::sourceEnabledChanged()
>+{
>+    for (auto& observer : m_observers)
>+        observer->trackEnabledChanged(*this);
>+}
>+
> bool MediaStreamTrackPrivate::preventSourceFromStopping()
> {
>     return !m_isEnded;
>Index: Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h
>===================================================================
>--- Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h	(working copy)
>@@ -48,6 +48,8 @@ public:
>         virtual void trackEnded(MediaStreamTrackPrivate&) = 0;
>         virtual void trackMutedChanged(MediaStreamTrackPrivate&) = 0;
>         virtual void trackStatesChanged(MediaStreamTrackPrivate&) = 0;
>+        virtual void trackProducingDataChanged(MediaStreamTrackPrivate&) = 0;
>+        virtual void trackEnabledChanged(MediaStreamTrackPrivate&) = 0;
>     };
>     
>     static RefPtr<MediaStreamTrackPrivate> create(RefPtr<RealtimeMediaSource>&&);
>@@ -62,6 +64,7 @@ public:
> 
>     void startProducingData() { m_source->startProducingData(); }
>     void stopProducingData() { m_source->stopProducingData(); }
>+    bool isProducingData() { return m_source->isProducingData(); }
> 
>     bool muted() const;
>     void setMuted(bool muted) const { m_source->setMuted(muted); }
>@@ -98,7 +101,9 @@ private:
>     void sourceStopped() override final;
>     void sourceMutedChanged() override final;
>     void sourceStatesChanged() override final;
>+    void sourceProducingDataChanged() override final;
>     bool preventSourceFromStopping() override final;
>+    void sourceEnabledChanged() override final;
> 
>     Vector<Observer*> m_observers;
>     RefPtr<RealtimeMediaSource> m_source;
>Index: Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
>===================================================================
>--- Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	(revision 191696)
>+++ Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	(working copy)
>@@ -101,6 +101,22 @@ void RealtimeMediaSource::statesDidChang
>         observer->sourceStatesChanged();
> }
> 
>+void RealtimeMediaSource::isProducingDataDidChange()
>+{
>+    for (auto& observer : m_observers)
>+        observer->sourceProducingDataChanged();
>+}
>+
>+void RealtimeMediaSource::setEnabled(bool enabled)
>+{
>+    if (m_enabled == enabled)
>+        return;
>+
>+    m_enabled = enabled;
>+    for (auto& observer : m_observers)
>+        observer->sourceEnabledChanged();
>+}
>+
> bool RealtimeMediaSource::readonly() const
> {
>     return m_readonly;
>@@ -119,6 +135,7 @@ void RealtimeMediaSource::stop(Observer*
>     }
> 
>     stopProducingData();
>+    reset();
> }
> 
> void RealtimeMediaSource::requestStop(Observer* callingObserver)
>Index: Source/WebCore/platform/mediastream/RealtimeMediaSource.h
>===================================================================
>--- Source/WebCore/platform/mediastream/RealtimeMediaSource.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/RealtimeMediaSource.h	(working copy)
>@@ -37,7 +37,9 @@
> #if ENABLE(MEDIA_STREAM)
> 
> #include "AudioSourceProvider.h"
>+#include "Image.h"
> #include "MediaConstraints.h"
>+#include "PlatformLayer.h"
> #include "RealtimeMediaSourceCapabilities.h"
> #include <wtf/RefCounted.h>
> #include <wtf/Vector.h>
>@@ -45,6 +47,8 @@
> 
> namespace WebCore {
> 
>+class FloatRect;
>+class GraphicsContext;
> class MediaConstraints;
> class MediaStreamPrivate;
> class RealtimeMediaSourceStates;
>@@ -59,6 +63,8 @@ public:
>         virtual void sourceStopped() = 0;
>         virtual void sourceMutedChanged() = 0;
>         virtual void sourceStatesChanged() = 0;
>+        virtual void sourceProducingDataChanged() = 0;
>+        virtual void sourceEnabledChanged() = 0;
> 
>         // Observer state queries.
>         virtual bool preventSourceFromStopping() = 0;
>@@ -66,8 +72,6 @@ public:
> 
>     virtual ~RealtimeMediaSource() { }
> 
>-    bool isAudioStreamSource() const { return type() == Audio; }
>-
>     const String& id() const { return m_id; }
> 
>     const String& persistentID() const { return m_persistentID; }
>@@ -82,7 +86,7 @@ public:
>     virtual unsigned fitnessScore() const { return m_fitnessScore; }
>     virtual void setFitnessScore(const unsigned fitnessScore) { m_fitnessScore = fitnessScore; }
> 
>-    virtual RefPtr<RealtimeMediaSourceCapabilities> capabilities() const = 0;
>+    virtual RefPtr<RealtimeMediaSourceCapabilities> capabilities() = 0;
>     virtual const RealtimeMediaSourceStates& states() = 0;
>     void statesDidChanged();
>     
>@@ -97,19 +101,28 @@ public:
>     virtual bool remote() const { return m_remote; }
>     virtual void setRemote(bool remote) { m_remote = remote; }
> 
>+    virtual void setEnabled(bool);
>+    virtual bool enabled() const { return m_enabled; }
>+
>     void addObserver(Observer*);
>     void removeObserver(Observer*);
> 
>     virtual void startProducingData() { }
>     virtual void stopProducingData() { }
>+    virtual bool isProducingData() const { return false; }
>+    virtual void isProducingDataDidChange();
> 
>     void stop(Observer* callingObserver = nullptr);
>     void requestStop(Observer* callingObserver = nullptr);
> 
>-    void reset();
>+    virtual void reset();
> 
>     virtual AudioSourceProvider* audioSourceProvider() { return nullptr; }
>-    
>+
>+    virtual PlatformLayer* platformLayer() const { return nullptr; }
>+    virtual RefPtr<Image> currentFrameImage() { return nullptr; }
>+    virtual void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&) { }
>+
> protected:
>     RealtimeMediaSource(const String& id, Type, const String& name);
> 
>@@ -124,6 +137,7 @@ private:
>     bool m_muted;
>     bool m_readonly;
>     bool m_remote;
>+    bool m_enabled { true };
>     
>     unsigned m_fitnessScore;
> };
>Index: Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.h
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.h	(working copy)
>@@ -58,10 +58,12 @@ private:
>     AVAudioCaptureSource(AVCaptureDevice*, const AtomicString&, PassRefPtr<MediaConstraints>);
>     virtual ~AVAudioCaptureSource();
>     
>-    RefPtr<RealtimeMediaSourceCapabilities> capabilities() const override;
>+    void initializeCapabilities(RealtimeMediaSourceCapabilities&) override;
>+
>     void captureOutputDidOutputSampleBufferFromConnection(AVCaptureOutput*, CMSampleBufferRef, AVCaptureConnection*) override;
>     
>     void setupCaptureSession() override;
>+    void shutdownCaptureSession() override;
>     void updateStates() override;
>     AudioSourceProvider* audioSourceProvider() override;
> 
>Index: Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/AVAudioCaptureSource.mm	(working copy)
>@@ -40,14 +40,16 @@
> 
> #import "CoreMediaSoftLink.h"
> 
>+typedef AVCaptureAudioChannel AVCaptureAudioChannelType;
>+typedef AVCaptureAudioDataOutput AVCaptureAudioDataOutputType;
> typedef AVCaptureConnection AVCaptureConnectionType;
> typedef AVCaptureDevice AVCaptureDeviceType;
> typedef AVCaptureDeviceInput AVCaptureDeviceInputType;
> typedef AVCaptureOutput AVCaptureOutputType;
>-typedef AVCaptureAudioDataOutput AVCaptureAudioDataOutputType;
> 
> SOFT_LINK_FRAMEWORK_OPTIONAL(AVFoundation)
> 
>+SOFT_LINK_CLASS(AVFoundation, AVCaptureAudioChannel)
> SOFT_LINK_CLASS(AVFoundation, AVCaptureAudioDataOutput)
> SOFT_LINK_CLASS(AVFoundation, AVCaptureConnection)
> SOFT_LINK_CLASS(AVFoundation, AVCaptureDevice)
>@@ -77,10 +79,10 @@ AVAudioCaptureSource::~AVAudioCaptureSou
> {
> }
> 
>-RefPtr<RealtimeMediaSourceCapabilities> AVAudioCaptureSource::capabilities() const
>+void AVAudioCaptureSource::initializeCapabilities(RealtimeMediaSourceCapabilities&capabilities)
> {
>-    notImplemented();
>-    return nullptr;
>+    // FIXME: finish this implementation - https://webkit.org/b/122430
>+    capabilities.addSourceType(RealtimeMediaSourceStates::Microphone);
> }
> 
> void AVAudioCaptureSource::updateStates()
>@@ -90,11 +92,8 @@ void AVAudioCaptureSource::updateStates(
> 
> void AVAudioCaptureSource::addObserver(AVAudioCaptureSource::Observer* observer)
> {
>-    {
>-        LockHolder lock(m_lock);
>-        m_observers.append(observer);
>-    }
>-
>+    LockHolder lock(m_lock);
>+    m_observers.append(observer);
>     if (m_inputDescription->mSampleRate)
>         observer->prepare(m_inputDescription.get());
> }
>@@ -110,16 +109,40 @@ void AVAudioCaptureSource::removeObserve
> void AVAudioCaptureSource::setupCaptureSession()
> {
>     RetainPtr<AVCaptureDeviceInputType> audioIn = adoptNS([allocAVCaptureDeviceInputInstance() initWithDevice:device() error:nil]);
>-    ASSERT([session() canAddInput:audioIn.get()]);
>-    if ([session() canAddInput:audioIn.get()])
>-        [session() addInput:audioIn.get()];
>-    
>+
>+    if (![session() canAddInput:audioIn.get()]) {
>+        LOG(Media, "AVVideoCaptureSource::setupCaptureSession(%p), unable to add audio input device", this);
>+        return;
>+    }
>+    [session() addInput:audioIn.get()];
>+
>     RetainPtr<AVCaptureAudioDataOutputType> audioOutput = adoptNS([allocAVCaptureAudioDataOutputInstance() init]);
>     setAudioSampleBufferDelegate(audioOutput.get());
>-    ASSERT([session() canAddOutput:audioOutput.get()]);
>-    if ([session() canAddOutput:audioOutput.get()])
>-        [session() addOutput:audioOutput.get()];
>-    m_audioConnection = adoptNS([audioOutput.get() connectionWithMediaType:AVMediaTypeAudio]);
>+
>+    if (![session() canAddOutput:audioOutput.get()]) {
>+        LOG(Media, "AVVideoCaptureSource::setupCaptureSession(%p), unable to add audio sample buffer output delegate", this);
>+        return;
>+    }
>+    [session() addOutput:audioOutput.get()];
>+    m_audioConnection = [audioOutput.get() connectionWithMediaType:AVMediaTypeAudio];
>+}
>+
>+void AVAudioCaptureSource::shutdownCaptureSession()
>+{
>+    {
>+        LockHolder lock(m_lock);
>+
>+        m_audioConnection = nullptr;
>+        m_inputDescription = std::make_unique<AudioStreamBasicDescription>();
>+
>+        for (auto& observer : m_observers)
>+            observer->unprepare();
>+        m_observers.shrink(0);
>+    }
>+
>+    // Don't hold the lock when destroying the audio provider, it will call back into this object
>+    // to remove itself as an observer.
>+    m_audioSourceProvider = nullptr;
> }
> 
> static bool operator==(const AudioStreamBasicDescription& a, const AudioStreamBasicDescription& b)
>@@ -141,27 +164,30 @@ static bool operator!=(const AudioStream
> 
> void AVAudioCaptureSource::captureOutputDidOutputSampleBufferFromConnection(AVCaptureOutputType*, CMSampleBufferRef sampleBuffer, AVCaptureConnectionType*)
> {
>-    Vector<Observer*> observers;
>-    {
>-        LockHolder lock(m_lock);
>-        if (m_observers.isEmpty())
>-            return;
>-
>-        copyToVector(m_observers, observers);
>-    }
>+    if (!enabled())
>+        return;
> 
>     CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer);
>     if (!formatDescription)
>         return;
> 
>+    std::unique_lock<Lock> lock(m_lock, std::try_to_lock);
>+    if (!lock.owns_lock()) {
>+        // Failed to acquire the lock, just return instead of blocking.
>+        return;
>+    }
>+
>+    if (m_observers.isEmpty())
>+        return;
>+
>     const AudioStreamBasicDescription* streamDescription = CMAudioFormatDescriptionGetStreamBasicDescription(formatDescription);
>     if (*m_inputDescription != *streamDescription) {
>         m_inputDescription = std::make_unique<AudioStreamBasicDescription>(*streamDescription);
>-        for (auto& observer : observers)
>+        for (auto& observer : m_observers)
>             observer->prepare(m_inputDescription.get());
>     }
> 
>-    for (auto& observer : observers)
>+    for (auto& observer : m_observers)
>         observer->process(formatDescription, sampleBuffer);
> }
> 
>Index: Source/WebCore/platform/mediastream/mac/AVCaptureDeviceManager.mm
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/AVCaptureDeviceManager.mm	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/AVCaptureDeviceManager.mm	(working copy)
>@@ -1,5 +1,5 @@
> /*
>- * Copyright (C) 2013 Apple Inc. All rights reserved.
>+ * Copyright (C) 2013-2015 Apple Inc. All rights reserved.
>  *
>  * Redistribution and use in source and binary forms, with or without
>  * modification, are permitted provided that the following conditions
>@@ -103,10 +103,7 @@ public:
>     String m_groupID;
> 
>     String m_audioSourceId;
>-    RefPtr<AVMediaCaptureSource> m_audioAVMediaCaptureSource;
>-
>     String m_videoSourceId;
>-    RefPtr<AVMediaCaptureSource> m_videoAVMediaCaptureSource;
> 
>     bool m_enabled;
> };
>@@ -146,7 +143,15 @@ static void refreshCaptureDeviceList()
>     
>     for (AVCaptureDeviceType *device in [AVCaptureDevice devices]) {
>         CaptureDevice source;
>-        
>+
>+        if (![device isConnected])
>+            continue;
>+
>+#if !PLATFORM(IOS)
>+        if ([device isSuspended] || [device isInUseByAnotherApplication])
>+            continue;
>+#endif
>+
>         if (!captureDeviceFromDeviceID(device.uniqueID, source)) {
>             // An AVCaptureDevice has a unique ID, but we can't use it for the source ID because:
>             // 1. if it provides both audio and video we will need to create two sources for it
>@@ -388,24 +393,20 @@ Vector<RefPtr<RealtimeMediaSource>> AVCa
>         if (!captureDevice.m_enabled)
>             continue;
> 
>-        // FIXME: consider the constraints when choosing among multiple devices. For now just select the first available
>-        // device of the appropriate type.
>         if (type == RealtimeMediaSource::Audio && !captureDevice.m_audioSourceId.isEmpty()) {
>-            if (!captureDevice.m_audioAVMediaCaptureSource) {
>-                AVCaptureDeviceType *device = [AVCaptureDevice deviceWithUniqueID:captureDevice.m_captureDeviceID];
>-                ASSERT(device);
>-                captureDevice.m_audioAVMediaCaptureSource = AVAudioCaptureSource::create(device, captureDevice.m_audioSourceId, constraints);
>-            }
>-            bestSourcesList.append(captureDevice.m_audioAVMediaCaptureSource);
>+            RefPtr<RealtimeMediaSource> captureSource = AVCaptureDeviceManager::sourceWithUID(captureDevice.m_captureDeviceID, RealtimeMediaSource::Audio, constraints.get());
>+            if (!captureSource)
>+                continue;
>+
>+            bestSourcesList.append(captureSource.leakRef());
>         }
> 
>         if (type == RealtimeMediaSource::Video && !captureDevice.m_videoSourceId.isEmpty()) {
>-            if (!captureDevice.m_videoAVMediaCaptureSource) {
>-                AVCaptureDeviceType *device = [AVCaptureDevice deviceWithUniqueID:captureDevice.m_captureDeviceID];
>-                ASSERT(device);
>-                captureDevice.m_videoAVMediaCaptureSource = AVVideoCaptureSource::create(device, captureDevice.m_videoSourceId, constraints);
>-            }
>-            bestSourcesList.append(captureDevice.m_videoAVMediaCaptureSource);
>+            RefPtr<RealtimeMediaSource> captureSource = AVCaptureDeviceManager::sourceWithUID(captureDevice.m_captureDeviceID, RealtimeMediaSource::Video, constraints.get());
>+            if (!captureSource)
>+                continue;
>+
>+            bestSourcesList.append(captureSource.leakRef());
>         }
>     }
>     std::sort(bestSourcesList.begin(), bestSourcesList.end(), sortBasedOffFitnessScore);
>@@ -416,41 +417,41 @@ RefPtr<RealtimeMediaSource> AVCaptureDev
> {
>     if (!isAvailable())
>         return 0;
>-    
>+
>     Vector<CaptureDevice>& devices = captureDeviceList();
>-    for (auto captureDevice : devices) {
>-        if (!captureDevice.m_enabled)
>+    for (auto& captureDevice : devices) {
>+        if (captureDevice.m_captureDeviceID != deviceUID || !captureDevice.m_enabled)
>             continue;
> 
>-        if (captureDevice.m_captureDeviceID != deviceUID)
>+        if (type == RealtimeMediaSource::Audio && captureDevice.m_audioSourceId.isEmpty())
>+            continue;
>+        if (type == RealtimeMediaSource::Video && captureDevice.m_videoSourceId.isEmpty())
>             continue;
> 
>+        AVCaptureDeviceType *device = [AVCaptureDevice deviceWithUniqueID:captureDevice.m_captureDeviceID];
>+        ASSERT(device);
>+
>+        RefPtr<AVMediaCaptureSource> captureSource;
>+        if (type == RealtimeMediaSource::Audio)
>+            captureSource = AVAudioCaptureSource::create(device, captureDevice.m_audioSourceId, constraints);
>+        else
>+            captureSource = AVVideoCaptureSource::create(device, captureDevice.m_videoSourceId, constraints);
>+
>         if (constraints) {
>             String invalidConstraints;
>             AVCaptureSessionType *session = nil;
> 
>-            if (type == RealtimeMediaSource::Video && captureDevice.m_videoAVMediaCaptureSource)
>-                captureDevice.m_videoAVMediaCaptureSource->session();
>-            else if (type == RealtimeMediaSource::Audio && captureDevice.m_audioAVMediaCaptureSource)
>-                captureDevice.m_audioAVMediaCaptureSource->session();
>+            if (type == RealtimeMediaSource::Video)
>+                session = captureSource->session();
>+            else if (type == RealtimeMediaSource::Audio)
>+                session = captureSource->session();
>             AVCaptureDeviceManager::singleton().verifyConstraintsForMediaType(session, type, constraints, invalidConstraints);
> 
>             if (!invalidConstraints.isEmpty())
>                 continue;
>         }
>         
>-        AVCaptureDeviceType *device = [AVCaptureDevice deviceWithUniqueID:captureDevice.m_captureDeviceID];
>-        ASSERT(device);
>-        if (type == RealtimeMediaSource::Audio && !captureDevice.m_audioSourceId.isEmpty()) {
>-            if (!captureDevice.m_audioAVMediaCaptureSource)
>-                captureDevice.m_audioAVMediaCaptureSource = AVAudioCaptureSource::create(device, captureDevice.m_audioSourceId, constraints);
>-            return captureDevice.m_audioAVMediaCaptureSource;
>-        }
>-        if (type == RealtimeMediaSource::Video && !captureDevice.m_videoSourceId.isEmpty()) {
>-            if (!captureDevice.m_videoAVMediaCaptureSource)
>-                captureDevice.m_videoAVMediaCaptureSource = AVVideoCaptureSource::create(device, captureDevice.m_videoSourceId, constraints);
>-            return captureDevice.m_videoAVMediaCaptureSource;
>-        }
>+        return captureSource.leakRef();
>     }
>     
>     return nullptr;
>Index: Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.h
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.h	(working copy)
>@@ -52,45 +52,52 @@ public:
> 
>     virtual void captureOutputDidOutputSampleBufferFromConnection(AVCaptureOutput*, CMSampleBufferRef, AVCaptureConnection*) = 0;
> 
>-    virtual void captureSessionStoppedRunning();
>+    virtual void captureSessionIsRunningDidChange(bool);
>     
>     AVCaptureSession *session() const { return m_session.get(); }
> 
>+    const RealtimeMediaSourceStates& states() override;
>+
>     void startProducingData() override;
>     void stopProducingData() override;
>+    bool isProducingData() const override { return m_isRunning; }
>+
>+    WeakPtr<AVMediaCaptureSource> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(); }
> 
> protected:
>     AVMediaCaptureSource(AVCaptureDevice*, const AtomicString&, RealtimeMediaSource::Type, PassRefPtr<MediaConstraints>);
> 
>-    const RealtimeMediaSourceStates& states() override;
>     AudioSourceProvider* audioSourceProvider() override;
> 
>     virtual void setupCaptureSession() = 0;
>+    virtual void shutdownCaptureSession() = 0;
>     virtual void updateStates() = 0;
>+    virtual void initializeCapabilities(RealtimeMediaSourceCapabilities&) = 0;
> 
>     AVCaptureDevice *device() const { return m_device.get(); }
>+
>     RealtimeMediaSourceStates* currentStates() { return &m_currentStates; }
>     MediaConstraints* constraints() { return m_constraints.get(); }
> 
>+    RefPtr<RealtimeMediaSourceCapabilities> capabilities() override;
>+
>     void setVideoSampleBufferDelegate(AVCaptureVideoDataOutput*);
>     void setAudioSampleBufferDelegate(AVCaptureAudioDataOutput*);
> 
>     void scheduleDeferredTask(std::function<void ()>);
> 
>-    void statesDidChanged() { }
>-    
> private:
>     void setupSession();
>-    WeakPtr<AVMediaCaptureSource> createWeakPtr() { return m_weakPtrFactory.createWeakPtr(); }
>+    void reset() override;
> 
>     WeakPtrFactory<AVMediaCaptureSource> m_weakPtrFactory;
>     RetainPtr<WebCoreAVMediaCaptureSourceObserver> m_objcObserver;
>     RefPtr<MediaConstraints> m_constraints;
>+    RefPtr<RealtimeMediaSourceCapabilities> m_capabilities;
>     RealtimeMediaSourceStates m_currentStates;
>     RetainPtr<AVCaptureSession> m_session;
>     RetainPtr<AVCaptureDevice> m_device;
>-    
>-    bool m_isRunning;
>+    bool m_isRunning { false};
> };
> 
> } // namespace WebCore
>Index: Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.mm
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.mm	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/AVMediaCaptureSource.mm	(working copy)
>@@ -94,12 +94,14 @@ @interface WebCoreAVMediaCaptureSourceOb
> 
> -(id)initWithCallback:(AVMediaCaptureSource*)callback;
> -(void)disconnect;
>--(void)captureSessionStoppedRunning:(NSNotification *)notification;
> -(void)captureOutput:(AVCaptureOutputType *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnectionType *)connection;
>+-(void)observeValueForKeyPath:keyPath ofObject:(id)object change:(NSDictionary *)change context:(void*)context;
> @end
> 
> namespace WebCore {
> 
>+static NSArray* sessionKVOProperties();
>+
> static dispatch_queue_t globaAudioCaptureSerialQueue()
> {
>     static dispatch_queue_t globalQueue;
>@@ -127,7 +129,6 @@ AVMediaCaptureSource::AVMediaCaptureSour
>     , m_objcObserver(adoptNS([[WebCoreAVMediaCaptureSourceObserver alloc] initWithCallback:this]))
>     , m_constraints(constraints)
>     , m_device(device)
>-    , m_isRunning(false)
> {
>     setName(device.localizedName);
>     setPersistentID(device.uniqueID);
>@@ -136,9 +137,13 @@ AVMediaCaptureSource::AVMediaCaptureSour
> 
> AVMediaCaptureSource::~AVMediaCaptureSource()
> {
>-    [[NSNotificationCenter defaultCenter] removeObserver:m_objcObserver.get()];
>     [m_objcObserver disconnect];
>-    [m_session.get() stopRunning];
>+
>+    if (m_session) {
>+        for (NSString *keyName in sessionKVOProperties())
>+            [m_session.get() removeObserver:m_objcObserver.get() forKeyPath:keyName];
>+        [m_session.get() stopRunning];
>+    }
> }
> 
> void AVMediaCaptureSource::startProducingData()
>@@ -150,8 +155,6 @@ void AVMediaCaptureSource::startProducin
>         return;
>     
>     [m_session.get() startRunning];
>-
>-    m_isRunning = true;
> }
> 
> void AVMediaCaptureSource::stopProducingData()
>@@ -160,8 +163,6 @@ void AVMediaCaptureSource::stopProducing
>         return;
> 
>     [m_session.get() stopRunning];
>-
>-    m_isRunning = true;
> }
> 
> const RealtimeMediaSourceStates& AVMediaCaptureSource::states()
>@@ -170,21 +171,47 @@ const RealtimeMediaSourceStates& AVMedia
>     return m_currentStates;
> }
> 
>+RefPtr<RealtimeMediaSourceCapabilities> AVMediaCaptureSource::capabilities()
>+{
>+    if (!m_capabilities) {
>+        m_capabilities = RealtimeMediaSourceCapabilities::create();
>+        m_capabilities->setSourceId(m_currentStates.sourceId());
>+        initializeCapabilities(*m_capabilities.get());
>+    }
>+    return m_capabilities;
>+}
>+
> void AVMediaCaptureSource::setupSession()
> {
>     if (m_session)
>         return;
> 
>     m_session = adoptNS([allocAVCaptureSessionInstance() init]);
>-    
>-    [[NSNotificationCenter defaultCenter] addObserver:m_objcObserver.get() selector:@selector(captureSessionStoppedRunning:) name:AVCaptureSessionDidStopRunningNotification object:nil];
>-    
>+    for (NSString *keyName in sessionKVOProperties())
>+        [m_session.get() addObserver:m_objcObserver.get() forKeyPath:keyName options:NSKeyValueObservingOptionNew context:(void *)nil];
>+
>     setupCaptureSession();
> }
>- 
>-void AVMediaCaptureSource::captureSessionStoppedRunning()
>+
>+void AVMediaCaptureSource::reset()
>+{
>+    RealtimeMediaSource::reset();
>+    m_isRunning = false;
>+    for (NSString *keyName in sessionKVOProperties())
>+        [m_session.get() removeObserver:m_objcObserver.get() forKeyPath:keyName];
>+    shutdownCaptureSession();
>+    m_session = nullptr;
>+}
>+
>+void AVMediaCaptureSource::captureSessionIsRunningDidChange(bool state)
> {
>-    stopProducingData();
>+    scheduleDeferredTask([this, state] {
>+        if (state == m_isRunning)
>+            return;
>+
>+        m_isRunning = state;
>+        isProducingDataDidChange();
>+    });
> }
> 
> void AVMediaCaptureSource::setVideoSampleBufferDelegate(AVCaptureVideoDataOutputType* videoOutput)
>@@ -216,6 +243,12 @@ AudioSourceProvider* AVMediaCaptureSourc
>     return nullptr;
> }
> 
>+NSArray* sessionKVOProperties()
>+{
>+    static NSArray* keys = [@[@"running"] retain];
>+    return keys;
>+}
>+
> } // namespace WebCore
> 
> @implementation WebCoreAVMediaCaptureSourceObserver
>@@ -225,6 +258,7 @@ - (id)initWithCallback:(AVMediaCaptureSo
>     self = [super init];
>     if (!self)
>         return nil;
>+
>     m_callback = callback;
>     return self;
> }
>@@ -235,25 +269,37 @@ - (void)disconnect
>     m_callback = 0;
> }
> 
>-- (void)captureSessionStoppedRunning:(NSNotification *)notification
>+- (void)captureOutput:(AVCaptureOutputType *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnectionType *)connection
> {
>-    UNUSED_PARAM(notification);
>-
>     if (!m_callback)
>         return;
> 
>-    dispatch_async(dispatch_get_main_queue(), ^{
>-        if (m_callback)
>-            m_callback->captureSessionStoppedRunning();
>-    });
>+    m_callback->captureOutputDidOutputSampleBufferFromConnection(captureOutput, sampleBuffer, connection);
> }
> 
>-- (void)captureOutput:(AVCaptureOutputType *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnectionType *)connection
>+-(void)observeValueForKeyPath:keyPath ofObject:(id)object change:(NSDictionary *)change context:(void*)context
> {
>+    UNUSED_PARAM(object);
>+    UNUSED_PARAM(context);
>+
>     if (!m_callback)
>         return;
> 
>-    m_callback->captureOutputDidOutputSampleBufferFromConnection(captureOutput, sampleBuffer, connection);
>+    id newValue = [change valueForKey:NSKeyValueChangeNewKey];
>+
>+#if !LOG_DISABLED
>+    bool willChange = [[change valueForKey:NSKeyValueChangeNotificationIsPriorKey] boolValue];
>+
>+    if (willChange)
>+        LOG(Media, "WebCoreAVMediaCaptureSourceObserver::observeValueForKeyPath(%p) - will change, keyPath = %s", self, [keyPath UTF8String]);
>+    else {
>+        RetainPtr<NSString> valueString = adoptNS([[NSString alloc] initWithFormat:@"%@", newValue]);
>+        LOG(Media, "WebCoreAVMediaCaptureSourceObserver::observeValueForKeyPath(%p) - did change, keyPath = %s, value = %s", self, [keyPath UTF8String], [valueString.get() UTF8String]);
>+    }
>+#endif
>+
>+    if ([keyPath isEqualToString:@"running"])
>+        m_callback->captureSessionIsRunningDidChange([newValue boolValue]);
> }
> 
> @end
>Index: Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h	(working copy)
>@@ -30,7 +30,7 @@
> 
> #include "AVMediaCaptureSource.h"
> 
>-OBJC_CLASS AVCaptureVideoPreviewLayer;
>+OBJC_CLASS CALayer;
> 
> typedef struct CGImage *CGImageRef;
> typedef const struct opaqueCMFormatDescription *CMFormatDescriptionRef;
>@@ -48,36 +48,38 @@ public:
>     int32_t width() const { return m_width; }
>     int32_t height() const { return m_height; }
> 
>-    AVCaptureVideoPreviewLayer* previewLayer() { return m_videoPreviewLayer.get(); }
>-    CMSampleBufferRef currentFrameSampleBuffer() const { return m_buffer.get(); }
>-    void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&);
>-    RetainPtr<CGImageRef> currentFrameImage();
>-
> private:
>     AVVideoCaptureSource(AVCaptureDevice*, const AtomicString&, PassRefPtr<MediaConstraints>);
>     virtual ~AVVideoCaptureSource();
> 
>     void setupCaptureSession() override;
>+    void shutdownCaptureSession() override;
>+
>     void updateStates() override;
> 
>-    RefPtr<RealtimeMediaSourceCapabilities> capabilities() const override;
>+    void initializeCapabilities(RealtimeMediaSourceCapabilities&) override;
> 
>     bool applyConstraints(MediaConstraints*);
>     bool setFrameRateConstraint(float minFrameRate, float maxFrameRate);
> 
>-    bool calculateFramerate(CMSampleBufferRef);
>+    bool updateFramerate(CMSampleBufferRef);
> 
>     void captureOutputDidOutputSampleBufferFromConnection(AVCaptureOutput*, CMSampleBufferRef, AVCaptureConnection*) override;
>     void processNewFrame(RetainPtr<CMSampleBufferRef>);
> 
>-    RetainPtr<AVCaptureConnection> m_videoConnection;
>-    RetainPtr<AVCaptureVideoPreviewLayer> m_videoPreviewLayer;
>+    void paintCurrentFrameInContext(GraphicsContext&, const FloatRect&) override;
>+    PlatformLayer* platformLayer() const override;
>+
>+    RetainPtr<CGImageRef> currentFrameCGImage();
>+    RefPtr<Image> currentFrameImage() override;
>+
>     RetainPtr<CMSampleBufferRef> m_buffer;
>     RetainPtr<CGImageRef> m_lastImage;
>     Vector<Float64> m_videoFrameTimeStamps;
>-    Float64 m_frameRate;
>-    int32_t m_width;
>-    int32_t m_height;
>+    mutable RetainPtr<PlatformLayer> m_videoPreviewLayer;
>+    Float64 m_frameRate { 0 };
>+    int32_t m_width { 0 };
>+    int32_t m_height { 0 };
> };
> 
> } // namespace WebCore
>Index: Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm	(working copy)
>@@ -31,10 +31,12 @@
> #import "AVCaptureDeviceManager.h"
> #import "BlockExceptions.h"
> #import "GraphicsContextCG.h"
>+#import "ImageBuffer.h"
> #import "IntRect.h"
> #import "Logging.h"
> #import "MediaConstraints.h"
> #import "NotImplemented.h"
>+#import "PlatformLayer.h"
> #import "RealtimeMediaSourceStates.h"
> #import <AVFoundation/AVFoundation.h>
> #import <objc/runtime.h>
>@@ -98,9 +100,6 @@ RefPtr<AVMediaCaptureSource> AVVideoCapt
> 
> AVVideoCaptureSource::AVVideoCaptureSource(AVCaptureDeviceType* device, const AtomicString& id, PassRefPtr<MediaConstraints> constraint)
>     : AVMediaCaptureSource(device, id, RealtimeMediaSource::Video, constraint)
>-    , m_frameRate(0)
>-    , m_width(0)
>-    , m_height(0)
> {
>     currentStates()->setSourceId(id);
>     currentStates()->setSourceType(RealtimeMediaSourceStates::Camera);
>@@ -110,10 +109,11 @@ AVVideoCaptureSource::~AVVideoCaptureSou
> {
> }
> 
>-RefPtr<RealtimeMediaSourceCapabilities> AVVideoCaptureSource::capabilities() const
>+void AVVideoCaptureSource::initializeCapabilities(RealtimeMediaSourceCapabilities& capabilities)
> {
>-    notImplemented();
>-    return 0;
>+    // FIXME: finish this implementation
>+    capabilities.setSourceId(currentStates()->sourceId());
>+    capabilities.addSourceType(RealtimeMediaSourceStates::Camera);
> }
> 
> void AVVideoCaptureSource::updateStates()
>@@ -218,9 +218,13 @@ bool AVVideoCaptureSource::applyConstrai
> void AVVideoCaptureSource::setupCaptureSession()
> {
>     RetainPtr<AVCaptureDeviceInputType> videoIn = adoptNS([allocAVCaptureDeviceInputInstance() initWithDevice:device() error:nil]);
>-    ASSERT([session() canAddInput:videoIn.get()]);
>-    if ([session() canAddInput:videoIn.get()])
>-        [session() addInput:videoIn.get()];
>+
>+    if (![session() canAddInput:videoIn.get()]) {
>+        LOG(Media, "AVVideoCaptureSource::setupCaptureSession(%p), unable to add video input device", this);
>+        return;
>+    }
>+
>+    [session() addInput:videoIn.get()];
> 
>     if (constraints())
>         applyConstraints(constraints());
>@@ -231,15 +235,26 @@ void AVVideoCaptureSource::setupCaptureS
>                                                          , nil]);
>     [videoOutput setVideoSettings:settingsDictionary.get()];
>     setVideoSampleBufferDelegate(videoOutput.get());
>-    ASSERT([session() canAddOutput:videoOutput.get()]);
>-    if ([session() canAddOutput:videoOutput.get()])
>-        [session() addOutput:videoOutput.get()];
> 
>-    m_videoConnection = adoptNS([videoOutput.get() connectionWithMediaType:AVMediaTypeVideo]);
>-    m_videoPreviewLayer = adoptNS([[AVCaptureVideoPreviewLayer alloc] initWithSession:session()]);
>+    if (![session() canAddOutput:videoOutput.get()]) {
>+        LOG(Media, "AVVideoCaptureSource::setupCaptureSession(%p), unable to add video sample buffer output delegate", this);
>+        return;
>+    }
>+    [session() addOutput:videoOutput.get()];
>+}
>+
>+void AVVideoCaptureSource::shutdownCaptureSession()
>+{
>+    m_videoPreviewLayer = nullptr;
>+    m_buffer = nullptr;
>+    m_lastImage = nullptr;
>+    m_videoFrameTimeStamps.clear();
>+    m_frameRate = 0;
>+    m_width = 0;
>+    m_height = 0;
> }
> 
>-bool AVVideoCaptureSource::calculateFramerate(CMSampleBufferRef sampleBuffer)
>+bool AVVideoCaptureSource::updateFramerate(CMSampleBufferRef sampleBuffer)
> {
>     CMTime sampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
>     if (!CMTIME_IS_NUMERIC(sampleTime))
>@@ -261,13 +276,19 @@ bool AVVideoCaptureSource::calculateFram
> 
> void AVVideoCaptureSource::processNewFrame(RetainPtr<CMSampleBufferRef> sampleBuffer)
> {
>+    // Ignore frames delivered when the session is not running, we want to hang onto the last image
>+    // delivered before it stopped.
>+    if (m_lastImage && (!isProducingData() || !enabled()))
>+        return;
>+
>     CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer.get());
>     if (!formatDescription)
>         return;
> 
>+    updateFramerate(sampleBuffer.get());
>+
>     bool statesChanged = false;
> 
>-    statesChanged = calculateFramerate(sampleBuffer.get());
>     m_buffer = sampleBuffer;
>     m_lastImage = nullptr;
> 
>@@ -291,7 +312,23 @@ void AVVideoCaptureSource::captureOutput
>     });
> }
> 
>-RetainPtr<CGImageRef> AVVideoCaptureSource::currentFrameImage()
>+RefPtr<Image> AVVideoCaptureSource::currentFrameImage()
>+{
>+    if (!currentFrameCGImage())
>+        return nullptr;
>+
>+    FloatRect imageRect(0, 0, m_width, m_height);
>+    std::unique_ptr<ImageBuffer> imageBuffer = ImageBuffer::create(imageRect.size(), Unaccelerated);
>+
>+    if (!imageBuffer)
>+        return nullptr;
>+
>+    paintCurrentFrameInContext(imageBuffer->context(), imageRect);
>+
>+    return imageBuffer->copyImage();
>+}
>+
>+RetainPtr<CGImageRef> AVVideoCaptureSource::currentFrameCGImage()
> {
>     if (m_lastImage)
>         return m_lastImage;
>@@ -318,7 +355,7 @@ RetainPtr<CGImageRef> AVVideoCaptureSour
> 
> void AVVideoCaptureSource::paintCurrentFrameInContext(GraphicsContext& context, const FloatRect& rect)
> {
>-    if (context.paintingDisabled() || !currentFrameImage())
>+    if (context.paintingDisabled() || !currentFrameCGImage())
>         return;
> 
>     GraphicsContextStateSaver stateSaver(context);
>@@ -329,6 +366,19 @@ void AVVideoCaptureSource::paintCurrentF
>     CGContextDrawImage(context.platformContext(), CGRectMake(0, 0, paintRect.width(), paintRect.height()), m_lastImage.get());
> }
> 
>+PlatformLayer* AVVideoCaptureSource::platformLayer() const
>+{
>+    if (m_videoPreviewLayer)
>+        return m_videoPreviewLayer.get();
>+
>+    m_videoPreviewLayer = adoptNS([allocAVCaptureVideoPreviewLayerInstance() initWithSession:session()]);
>+#ifndef NDEBUG
>+    m_videoPreviewLayer.get().name = @"AVVideoCaptureSource preview layer";
>+#endif
>+
>+    return m_videoPreviewLayer.get();
>+}
>+
> } // namespace WebCore
> 
> #endif // ENABLE(MEDIA_STREAM)
>Index: Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderAVFObjC.h
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderAVFObjC.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderAVFObjC.h	(working copy)
>@@ -68,7 +68,7 @@ private:
> 
>     size_t m_listBufferSize { 0 };
>     std::unique_ptr<AudioBufferList> m_list;
>-    RetainPtr<AudioConverterRef> m_converter;
>+    AudioConverterRef m_converter;
>     std::unique_ptr<AudioStreamBasicDescription> m_inputDescription;
>     std::unique_ptr<AudioStreamBasicDescription> m_outputDescription;
>     std::unique_ptr<CARingBuffer> m_ringBuffer;
>Index: Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderAVFObjC.mm
>===================================================================
>--- Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderAVFObjC.mm	(revision 191696)
>+++ Source/WebCore/platform/mediastream/mac/WebAudioSourceProviderAVFObjC.mm	(working copy)
>@@ -73,6 +73,11 @@ WebAudioSourceProviderAVFObjC::WebAudioS
> 
> WebAudioSourceProviderAVFObjC::~WebAudioSourceProviderAVFObjC()
> {
>+    if (m_converter) {
>+        // FIXME: make and use a smart pointer for AudioConverter
>+        AudioConverterDispose(m_converter);
>+        m_converter = nullptr;
>+    }
>     if (m_connected)
>         m_captureSource->removeObserver(this);
> }
>@@ -123,7 +128,7 @@ void WebAudioSourceProviderAVFObjC::prov
>     m_readCount += framesToProcess;
> 
>     if (m_converter)
>-        AudioConverterConvertComplexBuffer(m_converter.get(), framesToProcess, m_list.get(), m_list.get());
>+        AudioConverterConvertComplexBuffer(m_converter, framesToProcess, m_list.get(), m_list.get());
> }
> 
> void WebAudioSourceProviderAVFObjC::setClient(AudioSourceProviderClient* client)
>@@ -180,9 +185,19 @@ void WebAudioSourceProviderAVFObjC::prep
>     m_outputDescription->mBytesPerFrame = sizeof(Float32);
>     m_outputDescription->mFormatFlags |= kAudioFormatFlagIsNonInterleaved;
> 
>+    if (m_converter) {
>+        // FIXME: make and use a smart pointer for AudioConverter
>+        AudioConverterDispose(m_converter);
>+        m_converter = nullptr;
>+    }
>+
>     if (*m_inputDescription != *m_outputDescription) {
>         AudioConverterRef outConverter = nullptr;
>-        AudioConverterNew(m_inputDescription.get(), m_outputDescription.get(), &outConverter);
>+        OSStatus err = AudioConverterNew(m_inputDescription.get(), m_outputDescription.get(), &outConverter);
>+        if (err) {
>+            LOG(Media, "WebAudioSourceProviderAVFObjC::prepare(%p) - AudioConverterNew returned error %i", this, err);
>+            return;
>+        }
>         m_converter = outConverter;
>     }
> 
>@@ -221,6 +236,11 @@ void WebAudioSourceProviderAVFObjC::unpr
>     m_ringBuffer = nullptr;
>     m_list = nullptr;
>     m_listBufferSize = 0;
>+    if (m_converter) {
>+        // FIXME: make and use a smart pointer for AudioConverter
>+        AudioConverterDispose(m_converter);
>+        m_converter = nullptr;
>+    }
> }
> 
> void WebAudioSourceProviderAVFObjC::process(CMFormatDescriptionRef, CMSampleBufferRef sampleBuffer)
>@@ -234,7 +254,7 @@ void WebAudioSourceProviderAVFObjC::proc
>     OSStatus err = CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, nullptr, m_list.get(), m_listBufferSize, kCFAllocatorSystemDefault, kCFAllocatorSystemDefault, kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment, &buffer);
> 
>     if (err) {
>-        LOG(Media, "WebAudioSourceProviderAVFObjC::proess(%p) - CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer returned error %i", this, err);
>+        LOG(Media, "WebAudioSourceProviderAVFObjC::process(%p) - CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer returned error %i", this, err);
>         return;
>     }
> 
>Index: Source/WebCore/platform/mediastream/openwebrtc/RealtimeMediaSourceOwr.h
>===================================================================
>--- Source/WebCore/platform/mediastream/openwebrtc/RealtimeMediaSourceOwr.h	(revision 191696)
>+++ Source/WebCore/platform/mediastream/openwebrtc/RealtimeMediaSourceOwr.h	(working copy)
>@@ -58,7 +58,7 @@ RealtimeMediaSourceOwr(OwrMediaSource* m
> 
>     virtual ~RealtimeMediaSourceOwr() { }
> 
>-    virtual RefPtr<RealtimeMediaSourceCapabilities> capabilities() const { return m_capabilities; }
>+    virtual RefPtr<RealtimeMediaSourceCapabilities> capabilities() { return m_capabilities; }
>     virtual const RealtimeMediaSourceStates& states() { return m_currentStates; }
> 
>     OwrMediaSource* mediaSource() const { return m_mediaSource; }
>Index: Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp
>===================================================================
>--- Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp	(revision 191696)
>+++ Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp	(working copy)
>@@ -51,7 +51,7 @@ public:
> 
>     virtual ~MockSource() { }
> 
>-    virtual RefPtr<RealtimeMediaSourceCapabilities> capabilities() const { return m_capabilities; }
>+    virtual RefPtr<RealtimeMediaSourceCapabilities> capabilities() { return m_capabilities; }
>     virtual const RealtimeMediaSourceStates& states() { return m_currentStates; }
> 
>     RefPtr<RealtimeMediaSourceCapabilities> m_capabilities;

Actions: View | Formatted Diff | Diff

Attachments on bug 150449: 264192 | 264194 | 264214 | 264215 | 264217 | 264219 | 264223 | 264248 | 264268 | 264297