Attachment 451520 Details for Bug 236356 – Patch

[patch] Patch

bug-236356-20220210144442.patch (text/plain), 95.86 KB, created by youenn fablet on 2022-02-10 05:44:43 PST

(hide)

Description:

Filename:

MIME Type:

Creator: youenn fablet

Created: 2022-02-10 05:44:43 PST

Size: 95.86 KB

patch

obsolete

>Subversion Revision: 289396
>diff --git a/Source/ThirdParty/libwebrtc/ChangeLog b/Source/ThirdParty/libwebrtc/ChangeLog
>index 4d5f3a30f2b5bd18e1ad580fa27150293e291964..1a48de774d912d9927ccb8163baf0930fea1adde 100644
>--- a/Source/ThirdParty/libwebrtc/ChangeLog
>+++ b/Source/ThirdParty/libwebrtc/ChangeLog
>@@ -1,3 +1,24 @@
>+2022-02-10  Youenn Fablet  <youenn@apple.com>
>+
>+        Support remote video frames in WebRTC video pipeline
>+        https://bugs.webkit.org/show_bug.cgi?id=236356
>+
>+        Reviewed by NOBODY (OOPS!).
>+
>+        Add support for passing MediaSample as some kind of VideoFrameBuffer.
>+        If libwebrtc needs the actual data, it will call a function that will call WebKit to get the buffers synchronously.
>+        This is used for instance to provide the data to SW encoders.
>+
>+        * Configurations/libwebrtc.iOS.exp:
>+        * Configurations/libwebrtc.iOSsim.exp:
>+        * Configurations/libwebrtc.mac.exp:
>+        * Source/webrtc/sdk/WebKit/WebKitDecoder.h:
>+        * Source/webrtc/sdk/WebKit/WebKitDecoder.mm:
>+        * Source/webrtc/sdk/WebKit/WebKitUtilities.h:
>+        * Source/webrtc/sdk/WebKit/WebKitUtilities.mm:
>+        * Source/webrtc/sdk/objc/native/src/objc_frame_buffer.h:
>+        * Source/webrtc/sdk/objc/native/src/objc_frame_buffer.mm:
>+
> 2022-02-08  Youenn Fablet  <youenn@apple.com>
> 
>         Allow to pass webrtc video frame buffers through RealtimeMediaSource based pipelines
>diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
>index 28b01210e19fb06a3fab130036be5a62a8a20e89..e80eb592c2d13f900ef9da9c0f19982508c29f0b 100644
>--- a/Source/WebCore/ChangeLog
>+++ b/Source/WebCore/ChangeLog
>@@ -1,3 +1,19 @@
>+2022-02-10  Youenn Fablet  <youenn@apple.com>
>+
>+        Support remote video frames in WebRTC video pipeline
>+        https://bugs.webkit.org/show_bug.cgi?id=236356
>+
>+        Reviewed by NOBODY (OOPS!).
>+
>+        Make use of new capacity to directly pass MediaSample within webrtc::VideoFrameBuffer.
>+        Covered by existing tests.
>+
>+        * Configurations/WebCore.xcconfig:
>+        * platform/MediaSample.h:
>+        * platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h:
>+        * platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm:
>+        * platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp:
>+
> 2022-02-09  Youenn Fablet  <youenn@apple.com>
> 
>         Support remote video frames in WebRTC video pipeline
>diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog
>index ebab44733ccc43d4ec26e77663f776d84317b297..b2cb1c590b184adb368e0d5007ba48028f41e24f 100644
>--- a/Source/WebKit/ChangeLog
>+++ b/Source/WebKit/ChangeLog
>@@ -1,3 +1,50 @@
>+2022-02-10  Youenn Fablet  <youenn@apple.com>
>+
>+        Support remote video frames in WebRTC video pipeline
>+        https://bugs.webkit.org/show_bug.cgi?id=236356
>+
>+        Reviewed by NOBODY (OOPS!).
>+
>+        Introduce a way to asynchronously get raw data for remote video frames through shared memory.
>+        Make use of this to synchronoulsy get that data by calling this API and blocking using a semaphore.
>+        Update webrtc decoder and capture code to allow this remote access.
>+        Update MediaRecorder and webrtc encoders to deal with remote video frames.
>+
>+        * DerivedSources-input.xcfilelist:
>+        * DerivedSources-output.xcfilelist:
>+        * DerivedSources.make:
>+        * GPUProcess/GPUConnectionToWebProcess.cpp:
>+        * GPUProcess/GPUConnectionToWebProcess.h:
>+        * GPUProcess/media/RemoteVideoFrameObjectHeap.cpp:
>+        * GPUProcess/media/RemoteVideoFrameObjectHeap.h:
>+        * GPUProcess/media/RemoteVideoFrameObjectHeap.messages.in:
>+        * GPUProcess/webrtc/LibWebRTCCodecsProxy.h:
>+        * GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in:
>+        * GPUProcess/webrtc/LibWebRTCCodecsProxy.mm:
>+        * GPUProcess/webrtc/RemoteMediaRecorder.cpp:
>+        * GPUProcess/webrtc/RemoteMediaRecorder.h:
>+        * GPUProcess/webrtc/RemoteMediaRecorder.messages.in:
>+        * GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp:
>+        * Sources.txt:
>+        * WebKit.xcodeproj/project.pbxproj:
>+        * WebProcess/GPU/GPUProcessConnection.cpp:
>+        * WebProcess/GPU/GPUProcessConnection.h:
>+        * WebProcess/GPU/media/RemoteVideoFrameProxy.cpp:
>+        * WebProcess/GPU/media/RemoteVideoFrameProxy.h:
>+        * WebProcess/GPU/media/RemoteVideoFrameProxyIdentifier.h: Added.
>+        * WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp:
>+        * WebProcess/GPU/webrtc/LibWebRTCCodecs.h:
>+        * WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in:
>+        * WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp:
>+        * WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.cpp: Added.
>+        * WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.h: Added.
>+        * WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.messages.in: Added.
>+        * WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp:
>+        * WebProcess/cocoa/RemoteCaptureSampleManager.cpp:
>+        * WebProcess/cocoa/RemoteCaptureSampleManager.h:
>+        * WebProcess/cocoa/UserMediaCaptureManager.cpp:
>+        * WebProcess/cocoa/UserMediaCaptureManager.h:
>+
> 2022-02-09  Youenn Fablet  <youenn@apple.com>
> 
>         Support remote video frames in WebRTC video pipeline
>diff --git a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp
>index 512bbbf4e17b8cf3b3c0b0594fea7462f16c4cd2..19e5669bba9d3e70f39554fee12cb80f34a2c0fb 100644
>--- a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp
>+++ b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp
>@@ -223,8 +223,11 @@ __ZN6webrtc22createLocalH265DecoderEU13block_pointerFvP10__CVBufferjjE
> __ZN6webrtc21createLocalVP9DecoderEU13block_pointerFvP10__CVBufferjjE
> __ZN6webrtc19releaseLocalDecoderEPv
> __ZN6webrtc11decodeFrameEPvjPKhm
>-__ZN6webrtc24videoDecoderTaskCompleteEPvjP10__CVBufferj
>+__ZN6webrtc24videoDecoderTaskCompleteEPvjjP10__CVBuffer
>+__ZN6webrtc24videoDecoderTaskCompleteEPvjjS0_PFP10__CVBufferS0_EPFvS0_Eii
> __ZN6webrtc24setVideoDecoderCallbacksEPFPvRKNS_14SdpVideoFormatEEPFiS0_EPFiS0_jPKhmttEPFiS0_S0_E
>+__ZN6webrtc24toWebRTCVideoFrameBufferEPvPFP10__CVBufferS0_EPFvS0_Eii
>+__ZN6webrtc24videoFrameBufferProviderERKNS_10VideoFrameE
> __ZN6webrtc19setDecoderFrameSizeEPvtt
> __ZN6webrtc18RtpCodecCapabilityC1Ev
> __ZN6webrtc24encoderVideoTaskCompleteEPvNS_14VideoCodecTypeEPKhmRKNS_22WebKitEncodedFrameInfoE
>diff --git a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp
>index 512bbbf4e17b8cf3b3c0b0594fea7462f16c4cd2..19e5669bba9d3e70f39554fee12cb80f34a2c0fb 100644
>--- a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp
>+++ b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp
>@@ -223,8 +223,11 @@ __ZN6webrtc22createLocalH265DecoderEU13block_pointerFvP10__CVBufferjjE
> __ZN6webrtc21createLocalVP9DecoderEU13block_pointerFvP10__CVBufferjjE
> __ZN6webrtc19releaseLocalDecoderEPv
> __ZN6webrtc11decodeFrameEPvjPKhm
>-__ZN6webrtc24videoDecoderTaskCompleteEPvjP10__CVBufferj
>+__ZN6webrtc24videoDecoderTaskCompleteEPvjjP10__CVBuffer
>+__ZN6webrtc24videoDecoderTaskCompleteEPvjjS0_PFP10__CVBufferS0_EPFvS0_Eii
> __ZN6webrtc24setVideoDecoderCallbacksEPFPvRKNS_14SdpVideoFormatEEPFiS0_EPFiS0_jPKhmttEPFiS0_S0_E
>+__ZN6webrtc24toWebRTCVideoFrameBufferEPvPFP10__CVBufferS0_EPFvS0_Eii
>+__ZN6webrtc24videoFrameBufferProviderERKNS_10VideoFrameE
> __ZN6webrtc19setDecoderFrameSizeEPvtt
> __ZN6webrtc18RtpCodecCapabilityC1Ev
> __ZN6webrtc24encoderVideoTaskCompleteEPvNS_14VideoCodecTypeEPKhmRKNS_22WebKitEncodedFrameInfoE
>diff --git a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp
>index 512bbbf4e17b8cf3b3c0b0594fea7462f16c4cd2..fa280a4a4fd86851ccb31f3c9de76b8ea88958dc 100644
>--- a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp
>+++ b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp
>@@ -223,7 +223,10 @@ __ZN6webrtc22createLocalH265DecoderEU13block_pointerFvP10__CVBufferjjE
> __ZN6webrtc21createLocalVP9DecoderEU13block_pointerFvP10__CVBufferjjE
> __ZN6webrtc19releaseLocalDecoderEPv
> __ZN6webrtc11decodeFrameEPvjPKhm
>-__ZN6webrtc24videoDecoderTaskCompleteEPvjP10__CVBufferj
>+__ZN6webrtc24videoDecoderTaskCompleteEPvjjP10__CVBuffer
>+__ZN6webrtc24videoDecoderTaskCompleteEPvjjS0_PFP10__CVBufferS0_EPFvS0_Eii
>+__ZN6webrtc24toWebRTCVideoFrameBufferEPvPFP10__CVBufferS0_EPFvS0_Eii
>+__ZN6webrtc24videoFrameBufferProviderERKNS_10VideoFrameE
> __ZN6webrtc24setVideoDecoderCallbacksEPFPvRKNS_14SdpVideoFormatEEPFiS0_EPFiS0_jPKhmttEPFiS0_S0_E
> __ZN6webrtc19setDecoderFrameSizeEPvtt
> __ZN6webrtc18RtpCodecCapabilityC1Ev
>diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.h b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.h
>index b823ed32cb33fb3a674b4298408c2ab8f5cb06ad..ec732f5ccbb4d5a8f896567385b51c7186b579d4 100644
>--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.h
>+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.h
>@@ -54,7 +54,8 @@ using VideoDecoderRegisterDecodeCompleteCallback = int32_t(*)(WebKitVideoDecoder
> void setVideoDecoderCallbacks(VideoDecoderCreateCallback, VideoDecoderReleaseCallback, VideoDecoderDecodeCallback, VideoDecoderRegisterDecodeCompleteCallback);
> 
> std::unique_ptr<webrtc::VideoDecoderFactory> createWebKitDecoderFactory(WebKitH265, WebKitVP9, WebKitVP9VTB);
>-void videoDecoderTaskComplete(void* callback, uint32_t timeStamp, CVPixelBufferRef, uint32_t timeStampRTP);
>+void videoDecoderTaskComplete(void* callback, uint32_t timeStamp, uint32_t timeStampRTP, CVPixelBufferRef);
>+void videoDecoderTaskComplete(void* callback, uint32_t timeStamp, uint32_t timeStampRTP, void*, GetBufferCallback, ReleaseBufferCallback, int width, int height);
> 
> using LocalDecoder = void*;
> using LocalDecoderCallback = void (^)(CVPixelBufferRef, uint32_t timeStamp, uint32_t timeStampNs);
>diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.mm b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.mm
>index aa4188486d1abc7019645a8b58d9f73a5046ec51..b7e067f0c9f45c3a63e346999d9f10c309c7d566 100644
>--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.mm
>+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitDecoder.mm
>@@ -37,6 +37,7 @@
> #import "sdk/objc/components/video_codec/RTCVideoDecoderH264.h"
> #import "sdk/objc/components/video_codec/RTCVideoDecoderH265.h"
> #import "sdk/objc/components/video_codec/RTCVideoDecoderVTBVP9.h"
>+#import "sdk/objc/native/src/objc_frame_buffer.h"
> 
> @interface WK_RTCLocalVideoH264H265VP9Decoder : NSObject
> - (instancetype)initH264DecoderWithCallback:(webrtc::LocalDecoderCallback)callback;
>@@ -173,7 +174,7 @@ RemoteVideoDecoder::~RemoteVideoDecoder()
>     videoDecoderCallbacks().releaseCallback(m_internalDecoder);
> }
> 
>-void videoDecoderTaskComplete(void* callback, uint32_t timeStamp, CVPixelBufferRef pixelBuffer, uint32_t timeStampRTP)
>+void videoDecoderTaskComplete(void* callback, uint32_t timeStamp, uint32_t timeStampRTP, CVPixelBufferRef pixelBuffer)
> {
>     auto videoFrame = VideoFrame::Builder().set_video_frame_buffer(pixelBufferToFrame(pixelBuffer))
>         .set_timestamp_rtp(timeStampRTP)
>@@ -185,6 +186,18 @@ void videoDecoderTaskComplete(void* callback, uint32_t timeStamp, CVPixelBufferR
>     static_cast<DecodedImageCallback*>(callback)->Decoded(videoFrame);
> }
> 
>+void videoDecoderTaskComplete(void* callback, uint32_t timeStamp, uint32_t timeStampRTP, void* pointer, GetBufferCallback getBufferCallback, ReleaseBufferCallback releaseBufferCallback, int width, int height)
>+{
>+    auto videoFrame = VideoFrame::Builder().set_video_frame_buffer(toWebRTCVideoFrameBuffer(pointer, getBufferCallback, releaseBufferCallback, width, height))
>+        .set_timestamp_rtp(timeStampRTP)
>+        .set_timestamp_ms(0)
>+        .set_rotation((VideoRotation)RTCVideoRotation_0)
>+        .build();
>+    videoFrame.set_timestamp(timeStamp);
>+
>+    static_cast<DecodedImageCallback*>(callback)->Decoded(videoFrame);
>+}
>+
> bool RemoteVideoDecoder::Configure(const Settings&)
> {
>     return true;
>diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h
>index de10e83b6de12e12685ef009e8b4fb522b1995f4..a779cad1bd36002d799316a14805d3a33d4c18ba 100644
>--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h
>+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h
>@@ -53,4 +53,8 @@ CVPixelBufferRef pixelBufferFromFrame(const VideoFrame&) CF_RETURNS_RETAINED;
> rtc::scoped_refptr<webrtc::VideoFrameBuffer> pixelBufferToFrame(CVPixelBufferRef);
> bool copyVideoFrameBuffer(VideoFrameBuffer&, uint8_t*);
> 
>+typedef CVPixelBufferRef (*GetBufferCallback)(void*);
>+typedef void (*ReleaseBufferCallback)(void*);
>+rtc::scoped_refptr<webrtc::VideoFrameBuffer> toWebRTCVideoFrameBuffer(void*, GetBufferCallback, ReleaseBufferCallback, int width, int height);
>+void* videoFrameBufferProvider(const VideoFrame&);
> }
>diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm
>index d5695846eacff466cd09587d37a9a9db521487c1..c769450b87c31a44f94c70035411b1e5b7aba307 100644
>--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm
>+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm
>@@ -52,6 +52,21 @@ rtc::scoped_refptr<webrtc::VideoFrameBuffer> pixelBufferToFrame(CVPixelBufferRef
>     return new rtc::RefCountedObject<ObjCFrameBuffer>(frameBuffer);
> }
> 
>+rtc::scoped_refptr<webrtc::VideoFrameBuffer> toWebRTCVideoFrameBuffer(void* pointer, GetBufferCallback getBufferCallback, ReleaseBufferCallback releaseBufferCallback, int width, int height)
>+{
>+    return new rtc::RefCountedObject<ObjCFrameBuffer>(ObjCFrameBuffer::BufferProvider { pointer, getBufferCallback, releaseBufferCallback }, width, height);
>+}
>+
>+void* videoFrameBufferProvider(const VideoFrame& frame)
>+{
>+    auto buffer = frame.video_frame_buffer();
>+    if (buffer->type() != VideoFrameBuffer::Type::kNative)
>+        return nullptr;
>+
>+    return static_cast<ObjCFrameBuffer*>(buffer.get())->frame_buffer_provider();
>+
>+}
>+
> static bool CopyVideoFrameToPixelBuffer(const webrtc::I420BufferInterface* frame, CVPixelBufferRef pixel_buffer) {
>     RTC_DCHECK(pixel_buffer);
>     RTC_DCHECK(CVPixelBufferGetPixelFormatType(pixel_buffer) == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange || CVPixelBufferGetPixelFormatType(pixel_buffer) == kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange);
>diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.h b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.h
>index f941dad6e4fe56900eca9050c0a834edb855f681..3d2ebd77963abca44ccefe073f111bd5b51ed249 100644
>--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.h
>+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.h
>@@ -14,16 +14,27 @@
> #import <CoreVideo/CoreVideo.h>
> 
> #include "common_video/include/video_frame_buffer.h"
>+#include "rtc_base/synchronization/mutex.h"
> 
> @protocol RTCVideoFrameBuffer;
> 
> namespace webrtc {
> 
>+typedef CVPixelBufferRef (*GetBufferCallback)(void*);
>+typedef void (*ReleaseBufferCallback)(void*);
>+
> class ObjCFrameBuffer : public VideoFrameBuffer {
>  public:
>   explicit ObjCFrameBuffer(id<RTCVideoFrameBuffer>);
>   ~ObjCFrameBuffer() override;
> 
>+  struct BufferProvider {
>+    void *pointer { nullptr };
>+    GetBufferCallback getBuffer { nullptr };
>+    ReleaseBufferCallback releaseBuffer { nullptr };
>+  };
>+  ObjCFrameBuffer(BufferProvider, int width, int height);
>+
>   Type type() const override;
> 
>   int width() const override;
>@@ -32,11 +43,14 @@ class ObjCFrameBuffer : public VideoFrameBuffer {
>   rtc::scoped_refptr<I420BufferInterface> ToI420() override;
> 
>   id<RTCVideoFrameBuffer> wrapped_frame_buffer() const;
>+  void* frame_buffer_provider() { return frame_buffer_provider_.pointer; }
> 
>  private:
>   id<RTCVideoFrameBuffer> frame_buffer_;
>+  BufferProvider frame_buffer_provider_;
>   int width_;
>   int height_;
>+  mutable webrtc::Mutex mutex_;
> };
> 
> id<RTCVideoFrameBuffer> ToObjCVideoFrameBuffer(
>diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.mm b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.mm
>index a3d9ba812da59f8572a533ad62c52ab0ac0a18c2..2f56acb2943bd4c4d7dfb5db021a009182807a87 100644
>--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.mm
>+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/native/src/objc_frame_buffer.mm
>@@ -12,6 +12,7 @@
> 
> #import "base/RTCVideoFrameBuffer.h"
> #import "sdk/objc/api/video_frame_buffer/RTCNativeI420Buffer+Private.h"
>+#import "sdk/objc/components/video_frame_buffer/RTCCVPixelBuffer.h"
> 
> namespace webrtc {
> 
>@@ -55,10 +56,15 @@ class ObjCI420FrameBuffer : public I420BufferInterface {
> ObjCFrameBuffer::ObjCFrameBuffer(id<RTCVideoFrameBuffer> frame_buffer)
>     : frame_buffer_(frame_buffer), width_(frame_buffer.width), height_(frame_buffer.height) {}
> 
>+ObjCFrameBuffer::ObjCFrameBuffer(BufferProvider provider, int width, int height)
>+    : frame_buffer_provider_(provider), width_(width), height_(height) { }
>+
> ObjCFrameBuffer::~ObjCFrameBuffer() {
> #if defined(WEBRTC_WEBKIT_BUILD)
>   [frame_buffer_ close];
> #endif
>+  if (frame_buffer_provider_.releaseBuffer)
>+    frame_buffer_provider_.releaseBuffer(frame_buffer_provider_.pointer);
> }
> 
> VideoFrameBuffer::Type ObjCFrameBuffer::type() const {
>@@ -75,12 +81,21 @@ int ObjCFrameBuffer::height() const {
> 
> rtc::scoped_refptr<I420BufferInterface> ObjCFrameBuffer::ToI420() {
>   rtc::scoped_refptr<I420BufferInterface> buffer =
>-      new rtc::RefCountedObject<ObjCI420FrameBuffer>([frame_buffer_ toI420]);
>+      new rtc::RefCountedObject<ObjCI420FrameBuffer>([wrapped_frame_buffer() toI420]);
> 
>   return buffer;
> }
> 
> id<RTCVideoFrameBuffer> ObjCFrameBuffer::wrapped_frame_buffer() const {
>+  if (frame_buffer_)
>+    return frame_buffer_;
>+
>+  {
>+    webrtc::MutexLock lock(&mutex_);
>+    if (!frame_buffer_ && frame_buffer_provider_.getBuffer)
>+      const_cast<ObjCFrameBuffer*>(this)->frame_buffer_ = [[RTCCVPixelBuffer alloc] initWithPixelBuffer:frame_buffer_provider_.getBuffer(frame_buffer_provider_.pointer)];
>+  }
>+
>   return frame_buffer_;
> }
> 
>diff --git a/Source/WebCore/Configurations/WebCore.xcconfig b/Source/WebCore/Configurations/WebCore.xcconfig
>index 449cf6f9462d25a90c16204cc560f9d77be78604..a8cd5b5b27cd97a970c0558c12ed41cfff647f16 100644
>--- a/Source/WebCore/Configurations/WebCore.xcconfig
>+++ b/Source/WebCore/Configurations/WebCore.xcconfig
>@@ -117,7 +117,7 @@ WK_IOSURFACE_ACCELERATOR_LDFLAGS_watchos = $(WK_IOSURFACE_ACCELERATOR_LDFLAGS_ip
> WK_IOSURFACE_ACCELERATOR_LDFLAGS_appletvos = $(WK_IOSURFACE_ACCELERATOR_LDFLAGS_iphoneos);
> 
> WK_LIBWEBRTC_LDFLAGS = $(WK_LIBWEBRTC_LDFLAGS_$(ENABLE_WEB_RTC));
>-WK_LIBWEBRTC_LDFLAGS_ENABLE_WEB_RTC = -framework VideoToolbox -framework CoreMedia -weak-lwebrtc;
>+WK_LIBWEBRTC_LDFLAGS_ENABLE_WEB_RTC = -framework VideoToolbox -framework CoreMedia -weak-lwebrtc  -Xlinker --no-demangle;
> 
> WK_MOBILE_CORE_SERVICES_LDFLAGS = $(WK_MOBILE_CORE_SERVICES_LDFLAGS_$(WK_COCOA_TOUCH));
> WK_MOBILE_CORE_SERVICES_LDFLAGS_cocoatouch = -framework MobileCoreServices;
>diff --git a/Source/WebCore/platform/MediaSample.h b/Source/WebCore/platform/MediaSample.h
>index 130173c3d5777f4b47d98bbfdb4aa0d4e35c3696..6b213358a9f73c896c298d1a1f307eaf7c333779 100644
>--- a/Source/WebCore/platform/MediaSample.h
>+++ b/Source/WebCore/platform/MediaSample.h
>@@ -34,6 +34,7 @@
> #include <wtf/text/AtomString.h>
> 
> typedef struct opaqueCMSampleBuffer *CMSampleBufferRef;
>+typedef struct __CVBuffer *CVPixelBufferRef;
> typedef struct _GstSample GstSample;
> typedef struct OpaqueMTPluginByteSource *MTPluginByteSourceRef;
> typedef const struct opaqueCMFormatDescription *CMFormatDescriptionRef;
>@@ -113,6 +114,9 @@ public:
> #if defined(ENABLE_VIDEO) && ENABLE_VIDEO
>     virtual std::optional<MediaSampleVideoFrame> videoFrame() const { return std::nullopt; };
> #endif
>+#if PLATFORM(COCOA)
>+    virtual CVPixelBufferRef pixelBuffer() const { return nullptr; };
>+#endif
> 
>     bool isSync() const { return flags() & IsSync; }
>     bool isNonDisplaying() const { return flags() & IsNonDisplaying; }
>diff --git a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h
>index 9247fec97e53b90137feb7337115becd1a9a0e2e..7800c46d14863535cd279c2922169977c6b2253a 100644
>--- a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h
>+++ b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h
>@@ -50,7 +50,7 @@ public:
> private:
>     RealtimeIncomingVideoSourceCocoa(rtc::scoped_refptr<webrtc::VideoTrackInterface>&&, String&&);
>     CVPixelBufferPoolRef pixelBufferPool(size_t width, size_t height, webrtc::BufferType);
>-    std::unique_ptr<VideoFrameWebRTC> toVideoFrame(const webrtc::VideoFrame&);
>+    RefPtr<MediaSample> toVideoFrame(const webrtc::VideoFrame&);
> 
>     // rtc::VideoSinkInterface
>     void OnFrame(const webrtc::VideoFrame&) final;
>diff --git a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm
>index d974c1a4313a864068b216083a03933110b313ae..5d37fae6b6a2c033b0388b48e739b007b4eaeeae 100644
>--- a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm
>+++ b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm
>@@ -119,7 +119,7 @@ CVPixelBufferPoolRef RealtimeIncomingVideoSourceCocoa::pixelBufferPool(size_t wi
>     return m_pixelBufferPool.get();
> }
> 
>-std::unique_ptr<VideoFrameWebRTC> RealtimeIncomingVideoSourceCocoa::toVideoFrame(const webrtc::VideoFrame& frame)
>+RefPtr<MediaSample> RealtimeIncomingVideoSourceCocoa::toVideoFrame(const webrtc::VideoFrame& frame)
> {
>     MediaSample::VideoRotation rotation;
>     switch (frame.rotation()) {
>@@ -143,16 +143,20 @@ std::unique_ptr<VideoFrameWebRTC> RealtimeIncomingVideoSourceCocoa::toVideoFrame
>             m_blackFrameHeight = frame.height();
>             m_blackFrame = createBlackPixelBuffer(m_blackFrameWidth, m_blackFrameHeight);
>         }
>-        return makeUnique<VideoFrameWebRTC>(m_blackFrame.get(), rotation, false);
>+        return MediaSampleWebRTC::create(makeUniqueRef<VideoFrameWebRTC>(m_blackFrame.get(), rotation, false));
>     }
>+
>+    if (auto provider = webrtc::videoFrameBufferProvider(frame))
>+        return static_cast<MediaSample*>(provider);
>+
>     if (auto pixelBufffer = webrtc::pixelBufferFromFrame(frame))
>-        return makeUnique<VideoFrameWebRTC>(WTFMove(pixelBufffer), rotation, false);
>+        return MediaSampleWebRTC::create(makeUniqueRef<VideoFrameWebRTC>(WTFMove(pixelBufffer), rotation, false));
> 
>     auto bufferType = frame.video_frame_buffer()->type();
>     if (bufferType != webrtc::VideoFrameBuffer::Type::kI420 && bufferType != webrtc::VideoFrameBuffer::Type::kI010)
>         return nullptr;
> 
>-    return makeUnique<VideoFrameWebRTC>(frame.video_frame_buffer(), [protectedThis = Ref { *this }, this](auto& buffer) {
>+    return MediaSampleWebRTC::create(makeUniqueRef<VideoFrameWebRTC>(frame.video_frame_buffer(), [protectedThis = Ref { *this }, this](auto& buffer) {
>         return adoptCF(webrtc::createPixelBufferFromFrameBuffer(buffer, [this](size_t width, size_t height, webrtc::BufferType bufferType) -> CVPixelBufferRef {
>             Locker lock(m_pixelBufferPoolLock);
>             auto pixelBufferPool = this->pixelBufferPool(width, height, bufferType);
>@@ -168,7 +172,7 @@ std::unique_ptr<VideoFrameWebRTC> RealtimeIncomingVideoSourceCocoa::toVideoFrame
>             }
>             return pixelBuffer;
>         }));
>-    }, rotation, false);
>+    }, rotation, false));
> }
> 
> void RealtimeIncomingVideoSourceCocoa::OnFrame(const webrtc::VideoFrame& webrtcVideoFrame)
>@@ -191,7 +195,7 @@ void RealtimeIncomingVideoSourceCocoa::OnFrame(const webrtc::VideoFrame& webrtcV
>         std::swap(width, height);
> 
>     setIntrinsicSize(IntSize(width, height));
>-    videoSampleAvailable(MediaSampleWebRTC::create(makeUniqueRefFromNonNullUniquePtr(WTFMove(videoFrame))), metadataFromVideoFrame(webrtcVideoFrame));
>+    videoSampleAvailable(*videoFrame, metadataFromVideoFrame(webrtcVideoFrame));
> }
> 
> } // namespace WebCore
>diff --git a/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp b/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp
>index 1d14af284ae5272c50c3cc1b89415c9b1e9e20ec..e97716ca2384bb7696280878e18a8792cc4f6002 100644
>--- a/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp
>+++ b/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp
>@@ -85,8 +85,17 @@ void RealtimeOutgoingVideoSourceCocoa::videoSampleAvailable(MediaSample& sample,
>         break;
>     }
> 
>-    if (sample.platformSample().type == PlatformSample::CMSampleBufferType) {
>-        auto pixelBuffer = static_cast<CVPixelBufferRef>(PAL::CMSampleBufferGetImageBuffer(sample.platformSample().sample.cmSampleBuffer));
>+    auto platformSample = sample.platformSample();
>+    if (platformSample.type == PlatformSample::RemoteVideoFrameProxyType) {
>+        sample.ref();
>+        sendFrame(webrtc::toWebRTCVideoFrameBuffer(&sample,
>+            [](auto* sample) { return static_cast<MediaSample*>(sample)->pixelBuffer(); },
>+            [](auto* sample) { static_cast<MediaSample*>(sample)->deref(); },
>+            static_cast<int>(sample.presentationSize().width()), static_cast<int>(sample.presentationSize().height())));
>+        return;
>+    }
>+    if (platformSample.type == PlatformSample::CMSampleBufferType) {
>+        auto pixelBuffer = static_cast<CVPixelBufferRef>(PAL::CMSampleBufferGetImageBuffer(platformSample.sample.cmSampleBuffer));
>         auto pixelFormatType = CVPixelBufferGetPixelFormatType(pixelBuffer);
> 
>         RetainPtr<CVPixelBufferRef> convertedBuffer = pixelBuffer;
>@@ -99,8 +108,8 @@ void RealtimeOutgoingVideoSourceCocoa::videoSampleAvailable(MediaSample& sample,
>         sendFrame(webrtc::pixelBufferToFrame(convertedBuffer.get()));
>         return;
>     }
>-    ASSERT(sample.platformSample().type == PlatformSample::VideoFrameSampleType);
>-    sendFrame(sample.platformSample().sample.videoFrame->toWebRTCVideoFrameBuffer());
>+    ASSERT(platformSample.type == PlatformSample::VideoFrameSampleType);
>+    sendFrame(platformSample.sample.videoFrame->toWebRTCVideoFrameBuffer());
> }
> 
> rtc::scoped_refptr<webrtc::VideoFrameBuffer> RealtimeOutgoingVideoSourceCocoa::createBlackFrame(size_t  width, size_t  height)
>diff --git a/Source/WebKit/DerivedSources-input.xcfilelist b/Source/WebKit/DerivedSources-input.xcfilelist
>index 5fe634712c0ecbe61d0af3750c98b62e9921a7d0..a7bc01ae7bdcf7f4c358b36a01944c33fb6b4393 100644
>--- a/Source/WebKit/DerivedSources-input.xcfilelist
>+++ b/Source/WebKit/DerivedSources-input.xcfilelist
>@@ -197,6 +197,7 @@ $(PROJECT_DIR)/WebProcess/GPU/media/SourceBufferPrivateRemote.messages.in
> $(PROJECT_DIR)/WebProcess/GPU/media/ios/RemoteMediaSessionHelper.messages.in
> $(PROJECT_DIR)/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in
> $(PROJECT_DIR)/WebProcess/GPU/webrtc/LibWebRTCRemoteCodecs.messages.in
>+$(PROJECT_DIR)/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.messages.in
> $(PROJECT_DIR)/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.messages.in
> $(PROJECT_DIR)/WebProcess/Geolocation/WebGeolocationManager.messages.in
> $(PROJECT_DIR)/WebProcess/Inspector/RemoteWebInspectorUI.messages.in
>diff --git a/Source/WebKit/DerivedSources-output.xcfilelist b/Source/WebKit/DerivedSources-output.xcfilelist
>index 93e9cf2bd936948924645a9204fbadaff6ab7219..03ff987a73c9dfed5833d74bc8431394bd307cdf 100644
>--- a/Source/WebKit/DerivedSources-output.xcfilelist
>+++ b/Source/WebKit/DerivedSources-output.xcfilelist
>@@ -384,6 +384,9 @@ $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteTextureViewMessagesReplies.h
> $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameObjectHeapMessageReceiver.cpp
> $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameObjectHeapMessages.h
> $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameObjectHeapMessagesReplies.h
>+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameObjectHeapProxyMessageReceiver.cpp
>+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameObjectHeapProxyMessages.h
>+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameObjectHeapProxyMessagesReplies.h
> $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameProxyMessageReceiver.cpp
> $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameProxyMessages.h
> $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit/RemoteVideoFrameProxyMessagesReplies.h
>diff --git a/Source/WebKit/DerivedSources.make b/Source/WebKit/DerivedSources.make
>index c3c0d5ac6e1ba840adce66c8c4a5967789008286..dd3a511018f57aa25e25bac55cb168c6e24bb152 100644
>--- a/Source/WebKit/DerivedSources.make
>+++ b/Source/WebKit/DerivedSources.make
>@@ -205,6 +205,7 @@ MESSAGE_RECEIVERS = \
> 	WebProcess/GPU/media/RemoteRemoteCommandListener \
> 	WebProcess/GPU/media/SourceBufferPrivateRemote \
> 	WebProcess/GPU/media/ios/RemoteMediaSessionHelper \
>+	WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy \
> 	WebProcess/WebStorage/StorageAreaMap \
> 	WebProcess/UserContent/WebUserContentController \
> 	WebProcess/Inspector/WebInspectorInterruptDispatcher \
>diff --git a/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp b/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
>index 7bc32d00933b796927297bb9f1412a739d17519b..790a0f2f4e62bfc576b25f4b75d670156a580092 100644
>--- a/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
>+++ b/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
>@@ -238,15 +238,18 @@ GPUConnectionToWebProcess::GPUConnectionToWebProcess(GPUProcess& gpuProcess, Web
>     , m_webProcessIdentity(WTFMove(parameters.webProcessIdentity))
>     , m_remoteMediaPlayerManagerProxy(makeUniqueRef<RemoteMediaPlayerManagerProxy>(*this))
>     , m_sessionID(sessionID)
>-#if PLATFORM(COCOA) && USE(LIBWEBRTC)
>-    , m_libWebRTCCodecsProxy(LibWebRTCCodecsProxy::create(*this))
>-#endif
> #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
>     , m_sampleBufferDisplayLayerManager(RemoteSampleBufferDisplayLayerManager::create(*this))
> #endif
> #if ENABLE(MEDIA_STREAM)
>     , m_captureOrigin(SecurityOrigin::createUnique())
> #endif
>+#if ENABLE(MEDIA_STREAM)
>+    , m_videoFrameObjectHeap(RemoteVideoFrameObjectHeap::create(*this))
>+#endif
>+#if PLATFORM(COCOA) && USE(LIBWEBRTC)
>+    , m_libWebRTCCodecsProxy(LibWebRTCCodecsProxy::create(*this))
>+#endif
> #if HAVE(AUDIT_TOKEN)
>     , m_presentingApplicationAuditToken(WTFMove(parameters.presentingApplicationAuditToken))
> #endif
>@@ -259,9 +262,6 @@ GPUConnectionToWebProcess::GPUConnectionToWebProcess(GPUProcess& gpuProcess, Web
> #if ENABLE(ROUTING_ARBITRATION) && HAVE(AVAUDIO_ROUTING_ARBITER)
>     gpuProcess.audioSessionManager().session().setRoutingArbitrationClient(m_routingArbitrator.get());
> #endif
>-#if ENABLE(MEDIA_STREAM)
>-    m_videoFrameObjectHeap = RemoteVideoFrameObjectHeap::create(*this);
>-#endif
> 
>     if (!parameters.overrideLanguages.isEmpty())
>         overrideUserPreferredLanguages(parameters.overrideLanguages);
>diff --git a/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h b/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
>index 11acc00c74eb5d12f8282d4466397d87d7e9759d..fb2026d1013c7985f0faaad8dd84b49d116344a5 100644
>--- a/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
>+++ b/Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
>@@ -294,9 +294,6 @@ private:
>     std::unique_ptr<RemoteMediaResourceManager> m_remoteMediaResourceManager;
>     UniqueRef<RemoteMediaPlayerManagerProxy> m_remoteMediaPlayerManagerProxy;
>     PAL::SessionID m_sessionID;
>-#if PLATFORM(COCOA) && USE(LIBWEBRTC)
>-    Ref<LibWebRTCCodecsProxy> m_libWebRTCCodecsProxy;
>-#endif
> #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
>     std::unique_ptr<UserMediaCaptureManagerProxy> m_userMediaCaptureManagerProxy;
>     std::unique_ptr<RemoteAudioMediaStreamTrackRendererInternalUnitManager> m_audioMediaStreamTrackRendererInternalUnitManager;
>@@ -313,6 +310,9 @@ private:
>     bool m_allowsDisplayCapture { false };
>     IPC::ScopedActiveMessageReceiveQueue<RemoteVideoFrameObjectHeap> m_videoFrameObjectHeap;
> #endif
>+#if PLATFORM(COCOA) && USE(LIBWEBRTC)
>+    Ref<LibWebRTCCodecsProxy> m_libWebRTCCodecsProxy;
>+#endif
> #if HAVE(AUDIT_TOKEN)
>     std::optional<audit_token_t> m_presentingApplicationAuditToken;
> #endif
>diff --git a/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp b/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp
>index 3cc50e64bc80aa123c978fa1a9d8d3482f64f79f..8c42f3a0db34303e96f5fc8c755b0bcce0ea46ef 100644
>--- a/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp
>+++ b/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp
>@@ -29,9 +29,12 @@
> #if ENABLE(GPU_PROCESS) && ENABLE(MEDIA_STREAM)
> #include "GPUConnectionToWebProcess.h"
> #include "RemoteVideoFrameObjectHeapMessages.h"
>+#include "RemoteVideoFrameObjectHeapProxyMessages.h"
> #include "RemoteVideoFrameProxy.h"
> #include "RemoteVideoFrameProxyMessages.h"
> 
>+#include <pal/cf/CoreMediaSoftLink.h>
>+
> namespace WebKit {
> 
> Ref<RemoteVideoFrameObjectHeap> RemoteVideoFrameObjectHeap::create(GPUConnectionToWebProcess& connectionToWebProcess)
>@@ -85,6 +88,23 @@ void RemoteVideoFrameObjectHeap::releaseVideoFrame(RemoteVideoFrameWriteReferenc
>     retireRemove(WTFMove(write));
> }
> 
>+void RemoteVideoFrameObjectHeap::getVideoFrameBuffer(RemoteVideoFrameReadReference&& read)
>+{
>+    auto identifier = read.identifier();
>+    auto mediaSample = retire(WTFMove(read), 0_s);
>+    if (!mediaSample)
>+        m_connection->send(Messages::RemoteVideoFrameObjectHeapProxy::VideoFrameBufferNotFound { identifier }, 0);
>+
>+    auto platformSample = mediaSample->platformSample();
>+    ASSERT(platformSample.type == PlatformSample::CMSampleBufferType);
>+
>+    m_sharedVideoFrameWriter.write(static_cast<CVPixelBufferRef>(PAL::CMSampleBufferGetImageBuffer(platformSample.sample.cmSampleBuffer)),
>+        [&](auto& semaphore) { m_connection->send(Messages::RemoteVideoFrameObjectHeapProxy::SetSharedVideoFrameSemaphore { semaphore }, 0); },
>+        [&](auto& handle) { m_connection->send(Messages::RemoteVideoFrameObjectHeapProxy::SetSharedVideoFrameMemory { handle }, 0); }
>+    );
>+    m_connection->send(Messages::RemoteVideoFrameObjectHeapProxy::NewVideoFrameBuffer { identifier }, 0);
>+}
>+
> }
> 
> #endif
>diff --git a/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.h b/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.h
>index fa78f3ace5127711f0675f9e5ada4a53c1d2055a..334e6c86868abe75aed5c52092a975df3c170f0b 100644
>--- a/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.h
>+++ b/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.h
>@@ -27,6 +27,7 @@
> 
> #if ENABLE(GPU_PROCESS) && ENABLE(MEDIA_STREAM)
> #include "RemoteVideoFrameIdentifier.h"
>+#include "SharedVideoFrame.h"
> #include "ThreadSafeObjectHeap.h"
> #include <WebCore/MediaSample.h>
> #include <wtf/ThreadAssertions.h>
>@@ -56,10 +57,12 @@ private:
> 
>     // Messages.
>     void releaseVideoFrame(RemoteVideoFrameWriteReference&&);
>+    void getVideoFrameBuffer(RemoteVideoFrameReadReference&&);
> 
>     GPUConnectionToWebProcess* m_gpuConnectionToWebProcess WTF_GUARDED_BY_LOCK(m_consumeThread);
>     const Ref<IPC::Connection> m_connection;
>     ThreadAssertion m_consumeThread NO_UNIQUE_ADDRESS;
>+    SharedVideoFrameWriter m_sharedVideoFrameWriter;
> };
> 
> 
>diff --git a/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.messages.in b/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.messages.in
>index a278ce3d039dbe1731d5af6c8e49910a3952a712..1d3d243e15175eff9c8d7efcbbbd0b152de5ca62 100644
>--- a/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.messages.in
>+++ b/Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.messages.in
>@@ -23,6 +23,7 @@
> 
> #if ENABLE(GPU_PROCESS) && ENABLE(MEDIA_STREAM)
> messages -> RemoteVideoFrameObjectHeap NotRefCounted {
>+    GetVideoFrameBuffer(WebKit::RemoteVideoFrameReadReference identifier)
>     ReleaseVideoFrame(WebKit::RemoteVideoFrameWriteReference identifier)
> }
>-#endif
>\ No newline at end of file
>+#endif
>diff --git a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h
>index c772cb7760f2d5829afcd2b85f5913a30c007444..8ef29cae24091defcd89c4140604c6fec9822026 100644
>--- a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h
>+++ b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h
>@@ -31,6 +31,7 @@
> #include "DataReference.h"
> #include "RTCDecoderIdentifier.h"
> #include "RTCEncoderIdentifier.h"
>+#include "RemoteVideoFrameIdentifier.h"
> #include "SharedMemory.h"
> #include "SharedVideoFrame.h"
> #include <wtf/Lock.h>
>@@ -53,6 +54,7 @@ using LocalEncoder = void*;
> namespace WebKit {
> 
> class GPUConnectionToWebProcess;
>+class RemoteVideoFrameObjectHeap;
> class SharedVideoFrameReader;
> 
> class LibWebRTCCodecsProxy : public IPC::Connection::ThreadMessageReceiverRefCounted {
>@@ -83,7 +85,7 @@ private:
>     void createEncoder(RTCEncoderIdentifier, const String&, const Vector<std::pair<String, String>>&, bool useLowLatency);
>     void releaseEncoder(RTCEncoderIdentifier);
>     void initializeEncoder(RTCEncoderIdentifier, uint16_t width, uint16_t height, unsigned startBitrate, unsigned maxBitrate, unsigned minBitrate, uint32_t maxFramerate);
>-    void encodeFrame(RTCEncoderIdentifier, WebCore::RemoteVideoSample&&, uint32_t timeStamp, bool shouldEncodeAsKeyFrame);
>+    void encodeFrame(RTCEncoderIdentifier, WebCore::RemoteVideoSample&&, uint32_t timeStamp, bool shouldEncodeAsKeyFrame, std::optional<RemoteVideoFrameReadReference>);
>     void setEncodeRates(RTCEncoderIdentifier, uint32_t bitRate, uint32_t frameRate);
>     void setSharedVideoFrameSemaphore(RTCEncoderIdentifier, IPC::Semaphore&&);
>     void setSharedVideoFrameMemory(RTCEncoderIdentifier, const SharedMemory::IPCHandle&);
>@@ -101,6 +103,7 @@ private:
>     HashMap<RTCDecoderIdentifier, webrtc::LocalDecoder> m_decoders WTF_GUARDED_BY_LOCK(m_lock); // Only modified on the libWebRTCCodecsQueue but may get accessed from the main thread.
>     HashMap<RTCEncoderIdentifier, Encoder> m_encoders WTF_GUARDED_BY_LOCK(m_lock); // Only modified on the libWebRTCCodecsQueue but may get accessed from the main thread.
>     Ref<WorkQueue> m_queue;
>+    Ref<RemoteVideoFrameObjectHeap> m_videoFrameObjectHeap;
> };
> 
> }
>diff --git a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
>index 6bb208638dc22a89efb89c9af906a03ea69e744e..73d0b58d47bd160775d48e669a7e3d1ee4b51aa1 100644
>--- a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
>+++ b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
>@@ -34,7 +34,7 @@ messages -> LibWebRTCCodecsProxy NotRefCounted {
>     CreateEncoder(WebKit::RTCEncoderIdentifier id, String formatName, Vector<std::pair<String, String>> parameters, bool useLowLatency);
>     ReleaseEncoder(WebKit::RTCEncoderIdentifier id)
>     InitializeEncoder(WebKit::RTCEncoderIdentifier id, uint16_t width, uint16_t height, unsigned startBitrate, unsigned maxBitrate, unsigned minBitrate, uint32_t maxFramerate)
>-    EncodeFrame(WebKit::RTCEncoderIdentifier id, WebCore::RemoteVideoSample sample, uint32_t timeStamp, bool shouldEncodeAsKeyFrame)
>+    EncodeFrame(WebKit::RTCEncoderIdentifier id, WebCore::RemoteVideoSample sample, uint32_t timeStamp, bool shouldEncodeAsKeyFrame, std::optional<WebKit::RemoteVideoFrameReadReference> sampleReference)
>     SetEncodeRates(WebKit::RTCEncoderIdentifier id, uint32_t bitRate, uint32_t frameRate)
>     SetSharedVideoFrameSemaphore(WebKit::RTCEncoderIdentifier id, IPC::Semaphore semaphore)
>     SetSharedVideoFrameMemory(WebKit::RTCEncoderIdentifier id, WebKit::SharedMemory::IPCHandle storageHandle)
>diff --git a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm
>index 22e206ab9a932a59bced53393bdc8b301ac6834b..5fe9d7b92971daf1f61d27bcc04cb465740a0e39 100644
>--- a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm
>+++ b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm
>@@ -32,21 +32,27 @@
> #import "GPUProcess.h"
> #import "LibWebRTCCodecsMessages.h"
> #import "LibWebRTCCodecsProxyMessages.h"
>+#import "RemoteVideoFrameIdentifier.h"
>+#import "RemoteVideoFrameObjectHeap.h"
> #import "SharedVideoFrame.h"
> #import "WebCoreArgumentCoders.h"
> #import <WebCore/CVUtilities.h>
> #import <WebCore/LibWebRTCProvider.h>
>+#import <WebCore/MediaSampleAVFObjC.h>
> #import <WebCore/RemoteVideoSample.h>
> #import <webrtc/sdk/WebKit/WebKitDecoder.h>
> #import <webrtc/sdk/WebKit/WebKitEncoder.h>
> #import <wtf/BlockPtr.h>
> #import <wtf/MediaTime.h>
> 
>+#import <pal/cf/CoreMediaSoftLink.h>
>+
> namespace WebKit {
> 
> LibWebRTCCodecsProxy::LibWebRTCCodecsProxy(GPUConnectionToWebProcess& connection)
>     : m_gpuConnectionToWebProcess(connection)
>     , m_queue(connection.gpuProcess().libWebRTCCodecsQueue())
>+    , m_videoFrameObjectHeap(connection.videoFrameObjectHeap())
> {
>     m_gpuConnectionToWebProcess.connection().addThreadMessageReceiver(Messages::LibWebRTCCodecsProxy::messageReceiverName(), this);
> }
>@@ -77,12 +83,22 @@ void LibWebRTCCodecsProxy::close()
> 
> static Function<void(CVPixelBufferRef pixelBuffer, uint32_t timeStampNs, uint32_t timeStamp)> createDecoderCallback(RTCDecoderIdentifier identifier, GPUConnectionToWebProcess& gpuConnectionToWebProcess)
> {
>-    return [connection = Ref { gpuConnectionToWebProcess.connection() }, resourceOwner = gpuConnectionToWebProcess.webProcessIdentity(), identifier] (CVPixelBufferRef pixelBuffer, uint32_t timeStampNs, uint32_t timeStamp) {
>-        if (auto sample = WebCore::RemoteVideoSample::create(pixelBuffer, MediaTime(timeStampNs, 1))) {
>-            if (resourceOwner)
>-                sample->setOwnershipIdentity(resourceOwner);
>-            connection->send(Messages::LibWebRTCCodecs::CompletedDecoding { identifier, timeStamp, *sample }, 0);
>-        }
>+    return [connection = Ref { gpuConnectionToWebProcess.connection() }, resourceOwner = gpuConnectionToWebProcess.webProcessIdentity(), videoFrameObjectHeap = Ref { gpuConnectionToWebProcess.videoFrameObjectHeap() }, identifier] (CVPixelBufferRef pixelBuffer, uint32_t timeStampNs, uint32_t timeStamp)  mutable {
>+        auto sample = WebCore::MediaSampleAVFObjC::createImageSample(pixelBuffer, WebCore::MediaSample::VideoRotation::None, false, MediaTime(timeStampNs, 1), { });
>+        if (!sample)
>+            return;
>+
>+        auto remoteSample = WebCore::RemoteVideoSample::create(*sample);
>+        if (!remoteSample)
>+            return;
>+
>+        if (resourceOwner)
>+            remoteSample->setOwnershipIdentity(resourceOwner);
>+
>+        auto remoteIdentifier = videoFrameObjectHeap->createVideoFrame(sample.releaseNonNull());
>+        remoteSample->clearIOSurface();
>+
>+        connection->send(Messages::LibWebRTCCodecs::CompletedDecoding { identifier, timeStamp, *remoteSample, remoteIdentifier }, 0);
>     };
> }
> 
>@@ -211,15 +227,26 @@ static inline webrtc::VideoRotation toWebRTCVideoRotation(WebCore::MediaSample::
> 
> // For performance reasons, this function accesses m_encoders without locking. This is safe because this function runs on the libWebRTCCodecsQueue
> // and m_encoders only get modified on this queue.
>-void LibWebRTCCodecsProxy::encodeFrame(RTCEncoderIdentifier identifier, WebCore::RemoteVideoSample&& sample, uint32_t timeStamp, bool shouldEncodeAsKeyFrame)
>+void LibWebRTCCodecsProxy::encodeFrame(RTCEncoderIdentifier identifier, WebCore::RemoteVideoSample&& sample, uint32_t timeStamp, bool shouldEncodeAsKeyFrame, std::optional<RemoteVideoFrameReadReference> sampleReference)
> {
>+    RetainPtr<CVPixelBufferRef> pixelBuffer;
>+    if (sampleReference) {
>+        auto sample = m_videoFrameObjectHeap->retire(WTFMove(*sampleReference), 30_s);
>+        if (!sample) {
>+            ASSERT_IS_TESTING_IPC();
>+            return;
>+        }
>+        auto platformSample = sample->platformSample();
>+        ASSERT(platformSample.type == WebCore::PlatformSample::CMSampleBufferType);
>+        pixelBuffer = static_cast<CVPixelBufferRef>(PAL::CMSampleBufferGetImageBuffer(platformSample.sample.cmSampleBuffer));
>+    }
>+
>     ASSERT(findEncoderWithoutLock(identifier));
>     auto* encoder = findEncoderWithoutLock(identifier);
>     if (!encoder)
>         return;
> 
> #if !PLATFORM(MACCATALYST)
>-    RetainPtr<CVPixelBufferRef> pixelBuffer;
>     if (sample.surface()) {
>         if (auto buffer = WebCore::createCVPixelBuffer(sample.surface()))
>             pixelBuffer = WTFMove(*buffer);
>diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp
>index 51e8a37d7ac2b3247ba70d9404f22813d6ecb1fe..ffd15304375f72880bb19b393a35d9dc8affbc2c 100644
>--- a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp
>+++ b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp
>@@ -30,6 +30,7 @@
> 
> #include "Connection.h"
> #include "GPUConnectionToWebProcess.h"
>+#include "RemoteVideoFrameObjectHeap.h"
> #include "SharedRingBufferStorage.h"
> #include <WebCore/CARingBuffer.h>
> #include <WebCore/ImageTransferSessionVT.h>
>@@ -42,6 +43,7 @@
> 
> namespace WebKit {
> using namespace WebCore;
>+static constexpr Seconds mediaRecorderDefaultTimeout { 30_s };
> 
> std::unique_ptr<RemoteMediaRecorder> RemoteMediaRecorder::create(GPUConnectionToWebProcess& gpuConnectionToWebProcess, MediaRecorderIdentifier identifier, bool recordAudio, bool recordVideo, const MediaRecorderPrivateOptions& options)
> {
>@@ -55,6 +57,7 @@ RemoteMediaRecorder::RemoteMediaRecorder(GPUConnectionToWebProcess& gpuConnectio
>     : m_gpuConnectionToWebProcess(gpuConnectionToWebProcess)
>     , m_identifier(identifier)
>     , m_writer(WTFMove(writer))
>+    , m_videoFrameObjectHeap(gpuConnectionToWebProcess.videoFrameObjectHeap())
> {
>     if (recordAudio)
>         m_ringBuffer = makeUnique<CARingBuffer>();
>@@ -86,10 +89,16 @@ void RemoteMediaRecorder::audioSamplesAvailable(MediaTime time, uint64_t numberO
>     m_writer->appendAudioSampleBuffer(*m_audioBufferList, m_description, time, numberOfFrames);
> }
> 
>-void RemoteMediaRecorder::videoSampleAvailable(WebCore::RemoteVideoSample&& remoteSample)
>+void RemoteMediaRecorder::videoSampleAvailable(WebCore::RemoteVideoSample&& remoteSample, std::optional<RemoteVideoFrameReadReference> sampleReference)
> {
>     RefPtr<MediaSample> sample;
>-    if (!remoteSample.surface()) {
>+    if (sampleReference) {
>+        sample = m_videoFrameObjectHeap->retire(WTFMove(*sampleReference), mediaRecorderDefaultTimeout);
>+        if (!sample) {
>+            ASSERT_IS_TESTING_IPC();
>+            return;
>+        }
>+    } else if (!remoteSample.surface()) {
>         auto pixelBuffer = m_sharedVideoFrameReader.read();
>         if (!pixelBuffer)
>             return;
>diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h
>index c6629dbc6cabb36f61b7e4627d1f505093765a06..60b8d0214e77a53cc045a0f9feb81a39049d3dd2 100644
>--- a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h
>+++ b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h
>@@ -30,6 +30,7 @@
> #include "DataReference.h"
> #include "MediaRecorderIdentifier.h"
> #include "MessageReceiver.h"
>+#include "RemoteVideoFrameIdentifier.h"
> #include "SharedMemory.h"
> #include "SharedVideoFrame.h"
> #include <WebCore/CAAudioStreamDescription.h>
>@@ -52,6 +53,7 @@ struct MediaRecorderPrivateOptions;
> namespace WebKit {
> 
> class GPUConnectionToWebProcess;
>+class RemoteVideoFrameObjectHeap;
> class SharedRingBufferStorage;
> 
> class RemoteMediaRecorder : private IPC::MessageReceiver {
>@@ -72,7 +74,7 @@ private:
>     // IPC::MessageReceiver
>     void audioSamplesStorageChanged(const SharedMemory::IPCHandle&, const WebCore::CAAudioStreamDescription&, uint64_t numberOfFrames);
>     void audioSamplesAvailable(MediaTime, uint64_t numberOfFrames);
>-    void videoSampleAvailable(WebCore::RemoteVideoSample&&);
>+    void videoSampleAvailable(WebCore::RemoteVideoSample&&, std::optional<RemoteVideoFrameReadReference>);
>     void fetchData(CompletionHandler<void(IPC::DataReference&&, double)>&&);
>     void stopRecording(CompletionHandler<void()>&&);
>     void pause(CompletionHandler<void()>&&);
>@@ -90,6 +92,7 @@ private:
>     std::unique_ptr<WebCore::ImageTransferSessionVT> m_imageTransferSession;
> 
>     SharedVideoFrameReader m_sharedVideoFrameReader;
>+    Ref<RemoteVideoFrameObjectHeap> m_videoFrameObjectHeap;
> };
> 
> }
>diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
>index acaef8b8c40dbd5d7fb9b6c8848f9da468871493..498f281af801878e054d31ee6ed2b79708b90e84 100644
>--- a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
>+++ b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
>@@ -26,7 +26,7 @@
> messages -> RemoteMediaRecorder NotRefCounted {
>     AudioSamplesStorageChanged(WebKit::SharedMemory::IPCHandle storageHandle, WebCore::CAAudioStreamDescription description, uint64_t numberOfFrames)
>     AudioSamplesAvailable(MediaTime time, uint64_t numberOfFrames)
>-    VideoSampleAvailable(WebCore::RemoteVideoSample sample)
>+    VideoSampleAvailable(WebCore::RemoteVideoSample sample, std::optional<WebKit::RemoteVideoFrameReadReference> remoteVideoFrameReadReference)
>     FetchData() -> (IPC::DataReference buffer, double timeCode) Async
>     StopRecording() -> () Async
>     Pause() -> () Async
>diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp
>index 8a0349f233932c1df675e96c8d3d25261437f62a..6e0c0f32a8bc441391ed283c6a2884a293d19e83 100644
>--- a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp
>+++ b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp
>@@ -140,7 +140,6 @@ void RemoteSampleBufferDisplayLayer::enqueueSampleCV(WebCore::RemoteVideoSample&
> {
>     RefPtr<MediaSample> sample;
>     if (!remoteSample.surface()) {
>-        
>         auto pixelBuffer = m_sharedVideoFrameReader.read();
>         if (!pixelBuffer)
>             return;
>diff --git a/Source/WebKit/Sources.txt b/Source/WebKit/Sources.txt
>index 168ecb1eca79025943c0d82f87ad7d59eae9d007..09d5f1928f8700662cd2e6be26dd33af7d0a4cd5 100644
>--- a/Source/WebKit/Sources.txt
>+++ b/Source/WebKit/Sources.txt
>@@ -751,6 +751,7 @@ WebProcess/GPU/media/WebMediaStrategy.cpp
> WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
> WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
> WebProcess/GPU/webrtc/MediaRecorderProvider.cpp
>+WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.cpp
> WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp
> WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.cpp
> 
>@@ -881,6 +882,7 @@ RemoteExternalTextureMessageReceiver.cpp
> RemoteGPUMessageReceiver.cpp
> RemoteGPUProxyMessageReceiver.cpp
> RemoteVideoFrameObjectHeapMessageReceiver.cpp
>+RemoteVideoFrameObjectHeapProxyMessageReceiver.cpp
> RemoteVideoFrameProxyMessageReceiver.cpp
> RemotePipelineLayoutMessageReceiver.cpp
> RemoteQuerySetMessageReceiver.cpp
>diff --git a/Source/WebKit/WebKit.xcodeproj/project.pbxproj b/Source/WebKit/WebKit.xcodeproj/project.pbxproj
>index 1a3f090ab9bb0197b82f978acfe097033807e988..4f5014088f8e8c05419938fecf711c62eb24811d 100644
>--- a/Source/WebKit/WebKit.xcodeproj/project.pbxproj
>+++ b/Source/WebKit/WebKit.xcodeproj/project.pbxproj
>@@ -838,6 +838,7 @@
> 		410F0D4C2701EFF900F96DFC /* GPUProcessConnectionInitializationParameters.h in Headers */ = {isa = PBXBuildFile; fileRef = 410F0D4B2701EFEA00F96DFC /* GPUProcessConnectionInitializationParameters.h */; };
> 		411A8DDB20DDD1AC0060D34F /* WKMockMediaDevice.h in Headers */ = {isa = PBXBuildFile; fileRef = 411A8DD920DDB6050060D34F /* WKMockMediaDevice.h */; settings = {ATTRIBUTES = (Private, ); }; };
> 		411B22641E371BA6004F7363 /* LibWebRTCNetwork.h in Headers */ = {isa = PBXBuildFile; fileRef = 411B22621E371244004F7363 /* LibWebRTCNetwork.h */; };
>+		411B89DE27B5068600F9EBD3 /* RemoteVideoFrameObjectHeapProxy.h in Headers */ = {isa = PBXBuildFile; fileRef = 411B89DD27B4266300F9EBD3 /* RemoteVideoFrameObjectHeapProxy.h */; };
> 		41287D4E225D1ECB009A3E26 /* WebSocketTaskCocoa.mm in Sources */ = {isa = PBXBuildFile; fileRef = 41287D4B225C05C4009A3E26 /* WebSocketTaskCocoa.mm */; };
> 		413075AC1DE85F370039EC69 /* NetworkRTCMonitor.h in Headers */ = {isa = PBXBuildFile; fileRef = 4130759B1DE84FB00039EC69 /* NetworkRTCMonitor.h */; };
> 		413075AD1DE85F580039EC69 /* LibWebRTCSocket.h in Headers */ = {isa = PBXBuildFile; fileRef = 413075A01DE85EE70039EC69 /* LibWebRTCSocket.h */; };
>@@ -4063,6 +4064,10 @@
> 		411A8DD920DDB6050060D34F /* WKMockMediaDevice.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKMockMediaDevice.h; sourceTree = "<group>"; };
> 		411A8DDA20DDB6050060D34F /* WKMockMediaDevice.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WKMockMediaDevice.cpp; sourceTree = "<group>"; };
> 		411B22621E371244004F7363 /* LibWebRTCNetwork.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = LibWebRTCNetwork.h; path = Network/webrtc/LibWebRTCNetwork.h; sourceTree = "<group>"; };
>+		411B89DA27B423CB00F9EBD3 /* RemoteVideoFrameObjectHeap.messages.in */ = {isa = PBXFileReference; lastKnownFileType = text; path = RemoteVideoFrameObjectHeap.messages.in; sourceTree = "<group>"; };
>+		411B89DB27B4266300F9EBD3 /* RemoteVideoFrameObjectHeapProxy.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteVideoFrameObjectHeapProxy.cpp; sourceTree = "<group>"; };
>+		411B89DC27B4266300F9EBD3 /* RemoteVideoFrameObjectHeapProxy.messages.in */ = {isa = PBXFileReference; lastKnownFileType = text; path = RemoteVideoFrameObjectHeapProxy.messages.in; sourceTree = "<group>"; };
>+		411B89DD27B4266300F9EBD3 /* RemoteVideoFrameObjectHeapProxy.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = RemoteVideoFrameObjectHeapProxy.h; sourceTree = "<group>"; };
> 		41287D4B225C05C4009A3E26 /* WebSocketTaskCocoa.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebSocketTaskCocoa.mm; sourceTree = "<group>"; };
> 		41287D4C225C05C5009A3E26 /* WebSocketTaskCocoa.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebSocketTaskCocoa.h; sourceTree = "<group>"; };
> 		41287D4D225C161F009A3E26 /* WebSocketTask.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebSocketTask.h; sourceTree = "<group>"; };
>@@ -6885,6 +6890,7 @@
> 				07E19F0623D4DC880094FFB4 /* RemoteTextTrackProxy.h */,
> 				7B5A3DA827A7DCBC006C6F97 /* RemoteVideoFrameObjectHeap.cpp */,
> 				7B5A3DA727A7DCBC006C6F97 /* RemoteVideoFrameObjectHeap.h */,
>+				411B89DA27B423CB00F9EBD3 /* RemoteVideoFrameObjectHeap.messages.in */,
> 				071BC58423CC2C0900680D7C /* RemoteVideoTrackProxy.cpp */,
> 				071BC58523CC2C0900680D7C /* RemoteVideoTrackProxy.h */,
> 				07E19F0523D4DC880094FFB4 /* TextTrackPrivateRemoteConfiguration.h */,
>@@ -9264,6 +9270,9 @@
> 				4176E89223C34D6E003E83FE /* MediaRecorderProvider.h */,
> 				4172198C23B612E800AE5686 /* RTCDecoderIdentifier.h */,
> 				4158649A23BE092400A0A61E /* RTCEncoderIdentifier.h */,
>+				411B89DB27B4266300F9EBD3 /* RemoteVideoFrameObjectHeapProxy.cpp */,
>+				411B89DD27B4266300F9EBD3 /* RemoteVideoFrameObjectHeapProxy.h */,
>+				411B89DC27B4266300F9EBD3 /* RemoteVideoFrameObjectHeapProxy.messages.in */,
> 				41FCD6B723CDE53D00C62567 /* SampleBufferDisplayLayer.cpp */,
> 				41FCD6B323CDC60400C62567 /* SampleBufferDisplayLayer.h */,
> 				41FCD6B823CDE53E00C62567 /* SampleBufferDisplayLayer.messages.in */,
>@@ -13133,6 +13142,7 @@
> 				9B47908F253151CC00EC11AB /* JSIPCBinding.h in Headers */,
> 				C1663E5B24AEAA2F00C6A3B2 /* LaunchServicesDatabaseXPCConstants.h in Headers */,
> 				BCE0937814FB128C001138D9 /* LayerHostingContext.h in Headers */,
>+				411B89DE27B5068600F9EBD3 /* RemoteVideoFrameObjectHeapProxy.h in Headers */,
> 				1A92DC1112F8BA460017AF65 /* LayerTreeContext.h in Headers */,
> 				44E936FD2447C2D8009FA3E3 /* LegacyCustomProtocolID.h in Headers */,
> 				5C1427181C23F8B700D41183 /* LegacyCustomProtocolManager.h in Headers */,
>diff --git a/Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp b/Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp
>index 5c5de1821267bd856dee577165f454cbd42427ea..28f37876393bc3ac57e44fd73ea12477d2492de2 100644
>--- a/Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp
>+++ b/Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp
>@@ -45,6 +45,7 @@
> #include "RemoteMediaEngineConfigurationFactory.h"
> #include "RemoteMediaPlayerManager.h"
> #include "RemoteRemoteCommandListenerMessages.h"
>+#include "RemoteVideoFrameObjectHeapProxy.h"
> #include "SampleBufferDisplayLayerManager.h"
> #include "SampleBufferDisplayLayerMessages.h"
> #include "SourceBufferPrivateRemoteMessages.h"
>@@ -164,6 +165,13 @@ void GPUProcessConnection::resetAudioMediaStreamTrackRendererInternalUnit(AudioM
> {
>     WebProcess::singleton().audioMediaStreamTrackRendererInternalUnitManager().reset(identifier);
> }
>+
>+RemoteVideoFrameObjectHeapProxy& GPUProcessConnection::remoteVideoFrameObjectHeapProxy()
>+{
>+    if (!m_remoteVideoFrameObjectHeapProxy)
>+        m_remoteVideoFrameObjectHeapProxy = RemoteVideoFrameObjectHeapProxy::create(*this);
>+    return *m_remoteVideoFrameObjectHeapProxy;
>+}
> #endif
> 
> RemoteMediaPlayerManager& GPUProcessConnection::mediaPlayerManager()
>diff --git a/Source/WebKit/WebProcess/GPU/GPUProcessConnection.h b/Source/WebKit/WebProcess/GPU/GPUProcessConnection.h
>index 3f98cb1097763466ab205daa00ea6c516f9123f5..813a0546972cc32ad62ac9fa28d11e54ca5fef68 100644
>--- a/Source/WebKit/WebProcess/GPU/GPUProcessConnection.h
>+++ b/Source/WebKit/WebProcess/GPU/GPUProcessConnection.h
>@@ -51,6 +51,7 @@ namespace WebKit {
> 
> class RemoteAudioSourceProviderManager;
> class RemoteMediaPlayerManager;
>+class RemoteVideoFrameObjectHeapProxy;
> class SampleBufferDisplayLayerManager;
> class WebPage;
> struct GPUProcessConnectionInitializationParameters;
>@@ -77,6 +78,7 @@ public:
> #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
>     SampleBufferDisplayLayerManager& sampleBufferDisplayLayerManager();
>     void resetAudioMediaStreamTrackRendererInternalUnit(AudioMediaStreamTrackRendererInternalUnitIdentifier);
>+    RemoteVideoFrameObjectHeapProxy& remoteVideoFrameObjectHeapProxy();
> #endif
> 
>     RemoteMediaPlayerManager& mediaPlayerManager();
>@@ -140,6 +142,7 @@ private:
> #endif
> #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
>     std::unique_ptr<SampleBufferDisplayLayerManager> m_sampleBufferDisplayLayerManager;
>+    RefPtr<RemoteVideoFrameObjectHeapProxy> m_remoteVideoFrameObjectHeapProxy;
> #endif
> #if PLATFORM(COCOA) && ENABLE(WEB_AUDIO)
>     RefPtr<RemoteAudioSourceProviderManager> m_audioSourceProviderManager;
>diff --git a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp
>index b299e2e608b227d03c0fc2ec46e547ddfc04d557..46175e2ea8274dd7dbe7d55cf48554d9ebf39417 100644
>--- a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp
>+++ b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp
>@@ -33,14 +33,14 @@
> 
> namespace WebKit {
> 
>-Ref<RemoteVideoFrameProxy> RemoteVideoFrameProxy::create(GPUProcessConnection& gpuProcessConnection)
>+Ref<RemoteVideoFrameProxy> RemoteVideoFrameProxy::create(GPUProcessConnection& gpuProcessConnection, PixelBufferCallback&& callback)
> {
>-    return adoptRef(*new RemoteVideoFrameProxy(gpuProcessConnection, RemoteVideoFrameIdentifier::generate()));
>+    return adoptRef(*new RemoteVideoFrameProxy(gpuProcessConnection, RemoteVideoFrameIdentifier::generate(), WTFMove(callback)));
> }
> 
>-Ref<RemoteVideoFrameProxy> RemoteVideoFrameProxy::createAlreadyInitialized(IPC::Connection::UniqueID connectionID, RemoteVideoFrameIdentifier identifier, Properties properties)
>+Ref<RemoteVideoFrameProxy> RemoteVideoFrameProxy::createAlreadyInitialized(IPC::Connection::UniqueID connectionID, RemoteVideoFrameIdentifier identifier, Properties properties, PixelBufferCallback&& callback)
> {
>-    auto frame = adoptRef(*new RemoteVideoFrameProxy(connectionID, identifier));
>+    auto frame = adoptRef(*new RemoteVideoFrameProxy(connectionID, identifier, WTFMove(callback)));
>     frame->wasInitialized(properties);
>     return frame;
> }
>@@ -56,19 +56,21 @@ RemoteVideoFrameProxy::Properties RemoteVideoFrameProxy::properties(const WebCor
>     };
> }
> 
>-RemoteVideoFrameProxy::RemoteVideoFrameProxy(GPUProcessConnection& gpuProcessConnection, RemoteVideoFrameIdentifier identifier)
>+RemoteVideoFrameProxy::RemoteVideoFrameProxy(GPUProcessConnection& gpuProcessConnection, RemoteVideoFrameIdentifier identifier, PixelBufferCallback&& callback)
>     : m_gpuProcessConnection(&gpuProcessConnection)
>     , m_connectionID(gpuProcessConnection.connection().uniqueID())
>     , m_referenceTracker(identifier)
>+    , m_pixelBufferCallback(WTFMove(callback))
> {
>     m_gpuProcessConnection->addClient(*this);
>     if (identifier.processIdentifier() == Process::identifier())
>         m_gpuProcessConnection->messageReceiverMap().addMessageReceiver(Messages::RemoteVideoFrameProxy::messageReceiverName(), identifier.object().toUInt64(), *this);
> }
> 
>-RemoteVideoFrameProxy::RemoteVideoFrameProxy(IPC::Connection::UniqueID connectionID, RemoteVideoFrameIdentifier identifier)
>+RemoteVideoFrameProxy::RemoteVideoFrameProxy(IPC::Connection::UniqueID connectionID, RemoteVideoFrameIdentifier identifier, PixelBufferCallback&& callback)
>     : m_connectionID(connectionID)
>     , m_referenceTracker(identifier)
>+    , m_pixelBufferCallback(WTFMove(callback))
> {
> }
> 
>@@ -98,6 +100,24 @@ RemoteVideoFrameReadReference RemoteVideoFrameProxy::read() const
>     return m_referenceTracker.read();
> }
> 
>+void RemoteVideoFrameProxy::getPixelBuffer()
>+{
>+    ASSERT(m_pixelBufferLock.isHeld());
>+    std::exchange(m_pixelBufferCallback, { })(*this, [this](auto pixelBuffer) {
>+        m_pixelBuffer = WTFMove(pixelBuffer);
>+        m_semaphore.signal();
>+    });
>+    m_semaphore.wait();
>+}
>+
>+CVPixelBufferRef RemoteVideoFrameProxy::pixelBuffer() const
>+{
>+    Locker lock(m_pixelBufferLock);
>+    if (!m_pixelBuffer && m_pixelBufferCallback)
>+        const_cast<RemoteVideoFrameProxy*>(this)->getPixelBuffer();
>+    return m_pixelBuffer.get();
>+}
>+
> MediaTime RemoteVideoFrameProxy::presentationTime() const
> {
>     (void) waitForWasInitialized();
>diff --git a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h
>index d42413304c18fd8c9eb3d069a50e22cd6c94be69..976cc6ce52abc9134055070de4bda87a431bfa7b 100644
>--- a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h
>+++ b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h
>@@ -31,6 +31,8 @@
> #include "MessageReceiver.h"
> #include "RemoteVideoFrameIdentifier.h"
> #include <WebCore/VideoFrame.h>
>+#include <wtf/Function.h>
>+#include <wtf/threads/BinarySemaphore.h>
> 
> namespace IPC {
> class Connection;
>@@ -45,8 +47,12 @@ class RemoteVideoFrameProxy final : public WebCore::VideoFrame, private IPC::Mes
>     WTF_MAKE_FAST_ALLOCATED;
>     WTF_MAKE_NONCOPYABLE(RemoteVideoFrameProxy);
> public:
>+    using PixelBufferResultCallback = Function<void(RetainPtr<CVPixelBufferRef>&&)>;
>+    // CompletionHandler but it might not be called on the same thread.
>+    using PixelBufferCallback = Function<void(const RemoteVideoFrameProxy&, PixelBufferResultCallback&&)>;
>+
>     // Creates a promise of a MediaSample that exists in GPU process. The object can be used as out parameter of the creation message.
>-    static Ref<RemoteVideoFrameProxy> create(GPUProcessConnection&);
>+    static Ref<RemoteVideoFrameProxy> create(GPUProcessConnection&, PixelBufferCallback&& = { });
> 
>     struct Properties {
>         MediaTime presentationTime;
>@@ -59,7 +65,7 @@ public:
>         template<typename Decoder> static std::optional<Properties> decode(Decoder&);
>     };
> 
>-    static Ref<RemoteVideoFrameProxy> createAlreadyInitialized(IPC::Connection::UniqueID, RemoteVideoFrameIdentifier, Properties);
>+    static Ref<RemoteVideoFrameProxy> createAlreadyInitialized(IPC::Connection::UniqueID, RemoteVideoFrameIdentifier, Properties, PixelBufferCallback&&);
> 
>     static Properties properties(const WebCore::MediaSample&);
> 
>@@ -69,10 +75,12 @@ public:
>     RemoteVideoFrameWriteReference write() const;
>     RemoteVideoFrameReadReference read() const;
> 
>+    CVPixelBufferRef pixelBuffer() const final;
>+
>     static bool handleMessageToRemovedDestination(IPC::Connection&, IPC::Decoder&);
> private:
>-    RemoteVideoFrameProxy(GPUProcessConnection&, RemoteVideoFrameIdentifier);
>-    RemoteVideoFrameProxy(IPC::Connection::UniqueID, RemoteVideoFrameIdentifier);
>+    RemoteVideoFrameProxy(GPUProcessConnection&, RemoteVideoFrameIdentifier, PixelBufferCallback&&);
>+    RemoteVideoFrameProxy(IPC::Connection::UniqueID, RemoteVideoFrameIdentifier, PixelBufferCallback&&);
>     RemoteVideoFrameProxy();
>     void disconnectGPUProcessIfNeeded() const;
>     bool waitForWasInitialized() const WARN_UNUSED_RETURN;
>@@ -97,6 +105,8 @@ private:
>     void wasInitialized(std::optional<Properties>&&);
>     static inline Seconds defaultSendTimeout = 30_s;
> 
>+    void getPixelBuffer();
>+
>     mutable GPUProcessConnection* m_gpuProcessConnection { nullptr };
>     std::optional<IPC::Connection::UniqueID> m_connectionID;
>     RemoteVideoFrameReferenceTracker m_referenceTracker;
>@@ -106,6 +116,10 @@ private:
>     VideoRotation m_videoRotation { VideoRotation::None };
>     WebCore::FloatSize m_size;
>     uint32_t m_pixelFormat { 0 };
>+    mutable Lock m_pixelBufferLock;
>+    RetainPtr<CVPixelBufferRef> m_pixelBuffer;
>+    BinarySemaphore m_semaphore;
>+    PixelBufferCallback m_pixelBufferCallback;
> };
> 
> template<typename Encoder> void RemoteVideoFrameProxy::Properties::encode(Encoder& encoder) const
>diff --git a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxyIdentifier.h b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxyIdentifier.h
>new file mode 100644
>index 0000000000000000000000000000000000000000..4792c048650d0ca264600072346703282e72a4d0
>--- /dev/null
>+++ b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxyIdentifier.h
>@@ -0,0 +1,42 @@
>+/*
>+ * Copyright (C) 2022 Apple Inc. All rights reserved.
>+ *
>+ * Redistribution and use in source and binary forms, with or without
>+ * modification, are permitted provided that the following conditions
>+ * are met:
>+ * 1. Redistributions of source code must retain the above copyright
>+ *    notice, this list of conditions and the following disclaimer.
>+ * 2. Redistributions in binary form must reproduce the above copyright
>+ *    notice, this list of conditions and the following disclaimer in the
>+ *    documentation and/or other materials provided with the distribution.
>+ *
>+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
>+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
>+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
>+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>+ * THE POSSIBILITY OF SUCH DAMAGE.
>+ */
>+
>+#pragma once
>+
>+#if ENABLE(MEDIA_STREAM)
>+
>+#include "ObjectIdentifierReferenceTracker.h"
>+#include <WebCore/RemoteVideoFrameIdentifier.h>
>+
>+namespace WebKit {
>+
>+using RemoteVideoFrameReadReference = ObjectIdentifierReadReference<WebCore::RemoteVideoFrameIdentifier>;
>+using RemoteVideoFrameWriteReference = ObjectIdentifierWriteReference<WebCore::RemoteVideoFrameIdentifier>;
>+using RemoteVideoFrameReference = ObjectIdentifierReference<WebCore::RemoteVideoFrameIdentifier>;
>+using RemoteVideoFrameReferenceTracker = ObjectIdentifierReferenceTracker<WebCore::RemoteVideoFrameIdentifier>;
>+
>+} // namespace WebKit
>+
>+#endif // ENABLE(MEDIA_STREAM)
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
>index 24ccf557b2f65940bf4bdf099f0b9d3f07cbf3f4..6bc3722d1959dda3d938c86245f55b43b1b765ae 100644
>--- a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
>@@ -32,6 +32,8 @@
> #include "LibWebRTCCodecsMessages.h"
> #include "LibWebRTCCodecsProxyMessages.h"
> #include "Logging.h"
>+#include "RemoteVideoFrameObjectHeapProxy.h"
>+#include "RemoteVideoFrameProxy.h"
> #include "WebCoreArgumentCoders.h"
> #include "WebProcess.h"
> #include <WebCore/CVUtilities.h>
>@@ -184,6 +186,7 @@ void LibWebRTCCodecs::ensureGPUProcessConnectionOnMainThreadWithLock()
>     auto& gpuConnection = WebProcess::singleton().ensureGPUProcessConnection();
>     gpuConnection.addClient(*this);
>     m_connection = &gpuConnection.connection();
>+    m_remoteVideoFrameObjectHeapProxy = &gpuConnection.remoteVideoFrameObjectHeapProxy();
>     m_connection->addThreadMessageReceiver(Messages::LibWebRTCCodecs::messageReceiverName(), this);
> 
>     if (m_loggingLevel)
>@@ -315,10 +318,23 @@ void LibWebRTCCodecs::failedDecoding(RTCDecoderIdentifier decoderIdentifier)
>         decoder->hasError = true;
> }
> 
>-void LibWebRTCCodecs::completedDecoding(RTCDecoderIdentifier decoderIdentifier, uint32_t timeStamp, WebCore::RemoteVideoSample&& remoteSample)
>+void LibWebRTCCodecs::completedDecoding(RTCDecoderIdentifier decoderIdentifier, uint32_t timeStamp, WebCore::RemoteVideoSample&& remoteSample, std::optional<RemoteVideoFrameIdentifier> remoteFrameIdentifier)
> {
>     ASSERT(!isMainRunLoop());
> 
>+    RefPtr<RemoteVideoFrameProxy> remoteVideoFrame;
>+    // We always create RemoteVideoFrameProxy so that we can release the corresponding GPUProcess IOSurface right away if there is no video source.
>+    if (remoteFrameIdentifier) {
>+        Locker locker { m_connectionLock };
>+        RemoteVideoFrameProxy::Properties properties { remoteSample.time(), remoteSample.mirrored(), remoteSample.rotation(), { static_cast<float>(remoteSample.size().width()), static_cast<float>(remoteSample.size().height()) }, remoteSample.videoFormat() };
>+        remoteVideoFrame = RemoteVideoFrameProxy::createAlreadyInitialized(m_connection->uniqueID(), *remoteFrameIdentifier, properties, [proxy = m_remoteVideoFrameObjectHeapProxy](auto& frame, auto&& callback) {
>+            if (!proxy) {
>+                callback({ });
>+                return;
>+            }
>+            proxy->getVideoFrameBuffer(frame, WTFMove(callback));
>+        });
>+    }
>     // FIXME: Do error logging.
>     auto* decoder = m_decoders.get(decoderIdentifier);
>     if (!decoder)
>@@ -332,6 +348,14 @@ void LibWebRTCCodecs::completedDecoding(RTCDecoderIdentifier decoderIdentifier,
>     if (!decoder->decodedImageCallback)
>         return;
> 
>+    if (remoteVideoFrame) {
>+        webrtc::videoDecoderTaskComplete(decoder->decodedImageCallback, timeStamp, remoteSample.time().toDouble(), remoteVideoFrame.leakRef(),
>+            [](auto* pointer) { return static_cast<RemoteVideoFrameProxy*>(pointer)->pixelBuffer(); },
>+            [](auto* pointer) { static_cast<RemoteVideoFrameProxy*>(pointer)->deref(); },
>+            remoteSample.size().width(), remoteSample.size().height());
>+        return;
>+    }
>+
>     if (!remoteSample.surface())
>         return;
>     auto pixelBuffer = createCVPixelBuffer(remoteSample.surface()).value_or(nullptr);
>@@ -340,7 +364,7 @@ void LibWebRTCCodecs::completedDecoding(RTCDecoderIdentifier decoderIdentifier,
>         return;
>     }
> 
>-    webrtc::videoDecoderTaskComplete(decoder->decodedImageCallback, timeStamp, pixelBuffer.get(), remoteSample.time().toDouble());
>+    webrtc::videoDecoderTaskComplete(decoder->decodedImageCallback, timeStamp, remoteSample.time().toDouble(), pixelBuffer.get());
> }
> 
> static inline String formatNameFromCodecType(LibWebRTCCodecs::Type type)
>@@ -438,11 +462,21 @@ int32_t LibWebRTCCodecs::encodeFrame(Encoder& encoder, const webrtc::VideoFrame&
>     if (!encoder.connection)
>         return WEBRTC_VIDEO_CODEC_ERROR;
> 
>-    auto buffer = adoptCF(webrtc::pixelBufferFromFrame(frame));
>-    if (!buffer) {
>-        // buffer is not native, we need to copy to shared video frame.
>-        if (!copySharedVideoFrame(encoder, *frame.video_frame_buffer()))
>-            return WEBRTC_VIDEO_CODEC_ERROR;
>+    std::optional<RemoteVideoFrameReadReference> remoteVideoFrameReadReference;
>+    if (auto provider = webrtc::videoFrameBufferProvider(frame)) {
>+        auto& mediaSample = *static_cast<MediaSample*>(provider);
>+        if (is<RemoteVideoFrameProxy>(mediaSample))
>+            remoteVideoFrameReadReference = downcast<RemoteVideoFrameProxy>(mediaSample).read();
>+    }
>+
>+    RetainPtr<CVPixelBufferRef> buffer;
>+    if (!remoteVideoFrameReadReference) {
>+        buffer = adoptCF(webrtc::pixelBufferFromFrame(frame));
>+        if (!buffer) {
>+            // buffer is not native, we need to copy to shared video frame.
>+            if (!copySharedVideoFrame(encoder, *frame.video_frame_buffer()))
>+                return WEBRTC_VIDEO_CODEC_ERROR;
>+        }
>     }
>     auto sample = RemoteVideoSample::create(buffer.get(), MediaTime(frame.timestamp_us() * 1000, 1000000), toMediaSampleVideoRotation(frame.rotation()), RemoteVideoSample::ShouldCheckForIOSurface::No);
>     if (buffer && !sample->surface()) {
>@@ -451,7 +485,7 @@ int32_t LibWebRTCCodecs::encodeFrame(Encoder& encoder, const webrtc::VideoFrame&
>             return WEBRTC_VIDEO_CODEC_ERROR;
>     }
> 
>-    encoder.connection->send(Messages::LibWebRTCCodecsProxy::EncodeFrame { encoder.identifier, *sample, frame.timestamp(), shouldEncodeAsKeyFrame }, 0);
>+    encoder.connection->send(Messages::LibWebRTCCodecsProxy::EncodeFrame { encoder.identifier, *sample, frame.timestamp(), shouldEncodeAsKeyFrame, remoteVideoFrameReadReference }, 0);
>     return WEBRTC_VIDEO_CODEC_OK;
> }
> 
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h
>index 2952e0bbf46e5208d05c3f9c92b669b2961800d3..48d73387d573a59daee94ec9c86dce70f69bd4b5 100644
>--- a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.h
>@@ -34,6 +34,7 @@
> #include "MessageReceiver.h"
> #include "RTCDecoderIdentifier.h"
> #include "RTCEncoderIdentifier.h"
>+#include "RemoteVideoFrameIdentifier.h"
> #include "SharedVideoFrame.h"
> #include <WebCore/PixelBufferConformerCV.h>
> #include <map>
>@@ -61,6 +62,8 @@ struct WebKitEncodedFrameInfo;
> 
> namespace WebKit {
> 
>+class RemoteVideoFrameObjectHeapProxy;
>+
> class LibWebRTCCodecs : public IPC::Connection::ThreadMessageReceiverRefCounted, public GPUProcessConnection::Client {
>     WTF_MAKE_FAST_ALLOCATED;
> public:
>@@ -129,7 +132,7 @@ private:
>     void gpuProcessConnectionMayNoLongerBeNeeded();
> 
>     void failedDecoding(RTCDecoderIdentifier);
>-    void completedDecoding(RTCDecoderIdentifier, uint32_t timeStamp, WebCore::RemoteVideoSample&&);
>+    void completedDecoding(RTCDecoderIdentifier, uint32_t timeStamp, WebCore::RemoteVideoSample&&, std::optional<RemoteVideoFrameIdentifier>);
>     void completedEncoding(RTCEncoderIdentifier, IPC::DataReference&&, const webrtc::WebKitEncodedFrameInfo&);
>     RetainPtr<CVPixelBufferRef> convertToBGRA(CVPixelBufferRef);
> 
>@@ -150,6 +153,7 @@ private:
> 
>     Lock m_connectionLock;
>     RefPtr<IPC::Connection> m_connection WTF_GUARDED_BY_LOCK(m_connectionLock);
>+    RefPtr<RemoteVideoFrameObjectHeapProxy> m_remoteVideoFrameObjectHeapProxy WTF_GUARDED_BY_LOCK(m_connectionLock);
>     Vector<Function<void()>> m_tasksToDispatchAfterEstablishingConnection;
> 
>     Ref<WorkQueue> m_queue;
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in
>index 25f27fea83de7b56fcafd8d1f8f38a347ab06cbd..7db795081ea5f0602f6f22627b445553a8b70740 100644
>--- a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in
>@@ -24,7 +24,7 @@
> 
> messages -> LibWebRTCCodecs NotRefCounted {
>     FailedDecoding(WebKit::RTCDecoderIdentifier identifier)
>-    CompletedDecoding(WebKit::RTCDecoderIdentifier identifier, uint32_t timeStamp, WebCore::RemoteVideoSample sample)
>+    CompletedDecoding(WebKit::RTCDecoderIdentifier identifier, uint32_t timeStamp, WebCore::RemoteVideoSample sample, std::optional<WebKit::RemoteVideoFrameIdentifier> remoteFrameIdentifier)
> 
>     CompletedEncoding(WebKit::RTCEncoderIdentifier identifier, IPC::DataReference data, struct webrtc::WebKitEncodedFrameInfo info);
> }
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp b/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
>index d129cff7e332b96b1585914b9b1cb48321828ece..bbf14645bf2a6407406a46c77c1d7a38267a73d8 100644
>--- a/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
>@@ -32,6 +32,7 @@
> #include "GPUProcessConnection.h"
> #include "RemoteMediaRecorderManagerMessages.h"
> #include "RemoteMediaRecorderMessages.h"
>+#include "RemoteVideoFrameProxy.h"
> #include "WebProcess.h"
> #include <WebCore/CARingBuffer.h>
> #include <WebCore/MediaStreamPrivate.h>
>@@ -102,10 +103,14 @@ void MediaRecorderPrivate::videoSampleAvailable(MediaSample& sample, VideoSample
>         remoteSample = RemoteVideoSample::create(m_blackFrame.get(), sample.presentationTime(), sample.videoRotation(), RemoteVideoSample::ShouldCheckForIOSurface::No);
>     } else {
>         m_blackFrame = nullptr;
>+
>         remoteSample = RemoteVideoSample::create(sample, RemoteVideoSample::ShouldCheckForIOSurface::No);
>     }
> 
>-    if (!remoteSample->surface()) {
>+    std::optional<RemoteVideoFrameReadReference> remoteVideoFrameReadReference;
>+    if (is<RemoteVideoFrameProxy>(sample))
>+        remoteVideoFrameReadReference = downcast<RemoteVideoFrameProxy>(sample).read();
>+    else if (!remoteSample->surface()) {
>         // buffer is not IOSurface, we need to copy to shared video frame.
>         if (auto buffer = remoteSample->imageBuffer()) {
>             if (!copySharedVideoFrame(buffer))
>@@ -118,7 +123,7 @@ void MediaRecorderPrivate::videoSampleAvailable(MediaSample& sample, VideoSample
>         }
>     }
> 
>-    m_connection->send(Messages::RemoteMediaRecorder::VideoSampleAvailable { WTFMove(*remoteSample) }, m_identifier);
>+    m_connection->send(Messages::RemoteMediaRecorder::VideoSampleAvailable { WTFMove(*remoteSample), remoteVideoFrameReadReference }, m_identifier);
> }
> 
> template<typename Buffer>
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.cpp b/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.cpp
>new file mode 100644
>index 0000000000000000000000000000000000000000..dfb99879e4cfcc5f85959763c553b1ad5b06401b
>--- /dev/null
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.cpp
>@@ -0,0 +1,121 @@
>+/*
>+ * Copyright (C) 2022 Apple Inc. All rights reserved.
>+ *
>+ * Redistribution and use in source and binary forms, with or without
>+ * modification, are permitted provided that the following conditions
>+ * are met:
>+ * 1. Redistributions of source code must retain the above copyright
>+ *    notice, this list of conditions and the following disclaimer.
>+ * 2. Redistributions in binary form must reproduce the above copyright
>+ *    notice, this list of conditions and the following disclaimer in the
>+ *    documentation and/or other materials provided with the distribution.
>+ *
>+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
>+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
>+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
>+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>+ * THE POSSIBILITY OF SUCH DAMAGE.
>+ */
>+
>+#include "config.h"
>+#include "RemoteVideoFrameObjectHeapProxy.h"
>+
>+#if ENABLE(MEDIA_STREAM) && PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
>+
>+#include "RemoteVideoFrameObjectHeapMessages.h"
>+#include "RemoteVideoFrameObjectHeapProxyMessages.h"
>+
>+namespace WebKit {
>+
>+Ref<RemoteVideoFrameObjectHeapProxy> RemoteVideoFrameObjectHeapProxy::create(GPUProcessConnection& connection)
>+{
>+    return adoptRef(*new RemoteVideoFrameObjectHeapProxy(connection));
>+}
>+
>+RemoteVideoFrameObjectHeapProxy::RemoteVideoFrameObjectHeapProxy(GPUProcessConnection& connection)
>+    : m_connectionID(connection.connection().uniqueID())
>+    , m_queue(WorkQueue::create("RemoteVideoFrameObjectHeapProxy", WorkQueue::QOS::UserInteractive))
>+{
>+    connection.addClient(*this);
>+    connection.connection().addWorkQueueMessageReceiver(Messages::RemoteVideoFrameObjectHeapProxy::messageReceiverName(), m_queue, this);
>+}
>+
>+RemoteVideoFrameObjectHeapProxy::~RemoteVideoFrameObjectHeapProxy()
>+{
>+    clearCallbacks();
>+}
>+
>+void RemoteVideoFrameObjectHeapProxy::gpuProcessConnectionDidClose(GPUProcessConnection& connection)
>+{
>+    {
>+        Locker lock(m_connectionLock);
>+        m_connectionID = { };
>+    }
>+    connection.connection().removeWorkQueueMessageReceiver(Messages::RemoteVideoFrameObjectHeapProxy::messageReceiverName());
>+    clearCallbacks();
>+}
>+
>+void RemoteVideoFrameObjectHeapProxy::clearCallbacks()
>+{
>+    HashMap<RemoteVideoFrameIdentifier, Callback> callbacks;
>+    {
>+        Locker lock(m_callbacksLock);
>+        callbacks = std::exchange(m_callbacks, { });
>+    }
>+    for (auto& callback : callbacks.values())
>+        callback({ });
>+}
>+
>+void RemoteVideoFrameObjectHeapProxy::setSharedVideoFrameSemaphore(IPC::Semaphore&& semaphore)
>+{
>+    m_sharedVideoFrameReader.setSemaphore(WTFMove(semaphore));
>+}
>+
>+void RemoteVideoFrameObjectHeapProxy::setSharedVideoFrameMemory(const SharedMemory::IPCHandle& ipcHandle)
>+{
>+    m_sharedVideoFrameReader.setSharedMemory(ipcHandle);
>+}
>+
>+RemoteVideoFrameObjectHeapProxy::Callback RemoteVideoFrameObjectHeapProxy::takeCallback(RemoteVideoFrameIdentifier identifier)
>+{
>+    Locker lock(m_callbacksLock);
>+    return m_callbacks.take(identifier);
>+}
>+
>+void RemoteVideoFrameObjectHeapProxy::videoFrameBufferNotFound(RemoteVideoFrameIdentifier identifier)
>+{
>+    if (auto callback = takeCallback(identifier))
>+        callback(nullptr);
>+}
>+
>+void RemoteVideoFrameObjectHeapProxy::newVideoFrameBuffer(RemoteVideoFrameIdentifier identifier)
>+{
>+    auto result = m_sharedVideoFrameReader.read();
>+    if (auto callback = takeCallback(identifier))
>+        callback(WTFMove(result));
>+}
>+
>+void RemoteVideoFrameObjectHeapProxy::getVideoFrameBuffer(const RemoteVideoFrameProxy& frame, Callback&& callback)
>+{
>+    {
>+        Locker lock(m_callbacksLock);
>+        ASSERT(!m_callbacks.contains(frame.identifier()));
>+        m_callbacks.add(frame.identifier(), WTFMove(callback));
>+    }
>+    Locker lock(m_connectionLock);
>+    if (!m_connectionID) {
>+        videoFrameBufferNotFound(frame.identifier());
>+        return;
>+    }
>+    IPC::Connection::send(m_connectionID, Messages::RemoteVideoFrameObjectHeap::GetVideoFrameBuffer(frame.read()), 0, IPC::SendOption::DispatchMessageEvenWhenWaitingForSyncReply);
>+}
>+
>+}
>+
>+#endif
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.h b/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.h
>new file mode 100644
>index 0000000000000000000000000000000000000000..63e8934fd9962c666d3965e579712097f0fcd3bd
>--- /dev/null
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.h
>@@ -0,0 +1,88 @@
>+/*
>+ * Copyright (C) 2020 Apple Inc. All rights reserved.
>+ *
>+ * Redistribution and use in source and binary forms, with or without
>+ * modification, are permitted provided that the following conditions
>+ * are met:
>+ * 1. Redistributions of source code must retain the above copyright
>+ *    notice, this list of conditions and the following disclaimer.
>+ * 2. Redistributions in binary form must reproduce the above copyright
>+ *    notice, this list of conditions and the following disclaimer in the
>+ *    documentation and/or other materials provided with the distribution.
>+ *
>+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
>+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
>+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
>+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>+ * THE POSSIBILITY OF SUCH DAMAGE.
>+ */
>+
>+#pragma once
>+
>+#if ENABLE(MEDIA_STREAM) && PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
>+
>+#include "Connection.h"
>+#include "GPUProcessConnection.h"
>+#include "MessageReceiver.h"
>+#include "RemoteVideoFrameIdentifier.h"
>+#include "SharedVideoFrame.h"
>+#include <wtf/Function.h>
>+#include <wtf/HashMap.h>
>+#include <wtf/Lock.h>
>+#include <wtf/WorkQueue.h>
>+
>+using CVPixelBufferPoolRef = struct __CVPixelBufferPool*;
>+
>+namespace IPC {
>+class Connection;
>+class Decoder;
>+}
>+
>+namespace WebKit {
>+
>+class RemoteVideoFrameProxy;
>+
>+class RemoteVideoFrameObjectHeapProxy : public IPC::Connection::WorkQueueMessageReceiver, public GPUProcessConnection::Client {
>+    WTF_MAKE_FAST_ALLOCATED;
>+public:
>+    static Ref<RemoteVideoFrameObjectHeapProxy> create(GPUProcessConnection&);
>+    ~RemoteVideoFrameObjectHeapProxy();
>+
>+    using Callback = Function<void(RetainPtr<CVPixelBufferRef>&&)>;
>+    void getVideoFrameBuffer(const RemoteVideoFrameProxy&, Callback&&);
>+
>+private:
>+    explicit RemoteVideoFrameObjectHeapProxy(GPUProcessConnection&);
>+
>+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
>+
>+    // Messages
>+    void setSharedVideoFrameSemaphore(IPC::Semaphore&&);
>+    void setSharedVideoFrameMemory(const SharedMemory::IPCHandle&);
>+    void videoFrameBufferNotFound(RemoteVideoFrameIdentifier);
>+    void newVideoFrameBuffer(RemoteVideoFrameIdentifier);
>+
>+    // GPUProcessConnection::Client
>+    void gpuProcessConnectionDidClose(GPUProcessConnection&);
>+
>+    void clearCallbacks();
>+    Callback takeCallback(RemoteVideoFrameIdentifier);
>+
>+private:
>+    Lock m_connectionLock;
>+    IPC::Connection::UniqueID m_connectionID WTF_GUARDED_BY_LOCK(m_connectionLock);
>+    Lock m_callbacksLock;
>+    HashMap<RemoteVideoFrameIdentifier, Callback> m_callbacks WTF_GUARDED_BY_LOCK(m_callbacksLock);
>+    Ref<WorkQueue> m_queue;
>+    SharedVideoFrameReader m_sharedVideoFrameReader;
>+};
>+
>+} // namespace WebKit
>+
>+#endif
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.messages.in b/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.messages.in
>new file mode 100644
>index 0000000000000000000000000000000000000000..47e0d68387bf39d470c68069722026681f0989a9
>--- /dev/null
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxy.messages.in
>@@ -0,0 +1,32 @@
>+# Copyright (C) 2022 Apple Inc. All rights reserved.
>+#
>+# Redistribution and use in source and binary forms, with or without
>+# modification, are permitted provided that the following conditions
>+# are met:
>+# 1.  Redistributions of source code must retain the above copyright
>+#     notice, this list of conditions and the following disclaimer.
>+# 2.  Redistributions in binary form must reproduce the above copyright
>+#     notice, this list of conditions and the following disclaimer in the
>+#     documentation and/or other materials provided with the distribution.
>+#
>+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND
>+# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
>+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR
>+# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
>+# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
>+# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
>+# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
>+# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
>+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>+
>+#if ENABLE(MEDIA_STREAM) && PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
>+
>+messages -> RemoteVideoFrameObjectHeapProxy NotRefCounted {
>+    SetSharedVideoFrameSemaphore(IPC::Semaphore semaphore)
>+    SetSharedVideoFrameMemory(WebKit::SharedMemory::IPCHandle storageHandle)
>+    VideoFrameBufferNotFound(WebKit::RemoteVideoFrameIdentifier identifier)
>+    NewVideoFrameBuffer(WebKit::RemoteVideoFrameIdentifier identifier)
>+}
>+
>+#endif // ENABLE(MEDIA_STREAM) && PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
>diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp
>index c29783939656fa0b001fc001807dd94193d3b9bf..4a82ee116e714ea9b7500ee25288d926152eb9c3 100644
>--- a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp
>+++ b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp
>@@ -144,8 +144,8 @@ void SampleBufferDisplayLayer::enqueueSample(MediaSample& sample)
>         return;
>     if (!m_gpuProcessConnection)
>         return;
>-    if (is<RemoteVideoFrameProxy>(sample)) {
> 
>+    if (is<RemoteVideoFrameProxy>(sample)) {
>         auto& remoteSample = downcast<RemoteVideoFrameProxy>(sample);
>         m_connection->send(Messages::RemoteSampleBufferDisplayLayer::EnqueueSample { remoteSample.read() }, m_identifier);
>         return;
>diff --git a/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp b/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp
>index c73f1cbb8f4525d671ec4e69d3caaf775ea6fb5b..156a9eacff98d7059cae2ee57c5b618828e1c0af 100644
>--- a/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp
>+++ b/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.cpp
>@@ -28,6 +28,7 @@
> 
> #include "Logging.h"
> #include "RemoteCaptureSampleManagerMessages.h"
>+#include "RemoteVideoFrameObjectHeapProxy.h"
> #include "RemoteVideoFrameProxy.h"
> #include "SharedRingBufferStorage.h"
> #include "WebProcess.h"
>@@ -134,6 +135,12 @@ void RemoteCaptureSampleManager::didUpdateSourceConnection(IPC::Connection* conn
>     setConnection(connection);
> }
> 
>+void RemoteCaptureSampleManager::setRemoteVideoFrameObjectHeapProxy(RemoteVideoFrameObjectHeapProxy* proxy)
>+{
>+    Locker lock(m_remoteVideoFrameObjectHeapProxyLock);
>+    m_remoteVideoFrameObjectHeapProxy = proxy;
>+}
>+
> void RemoteCaptureSampleManager::dispatchToThread(Function<void()>&& callback)
> {
>     m_queue->dispatch(WTFMove(callback));
>@@ -157,8 +164,19 @@ void RemoteCaptureSampleManager::videoSampleAvailable(RealtimeMediaSourceIdentif
> 
>     RefPtr<MediaSample> videoFrame;
>     // We always create RemoteVideoFrameProxy so that we can release the corresponding GPUProcess IOSurface right away if there is no video source.
>-    if (remoteIdentifier)
>-        videoFrame = RemoteVideoFrameProxy::createAlreadyInitialized(m_connection->uniqueID(), *remoteIdentifier, { sample.time(), sample.mirrored(), sample.rotation(), { static_cast<float>(sample.size().width()), static_cast<float>(sample.size().height()) }, sample.videoFormat() });
>+    if (remoteIdentifier) {
>+        RemoteVideoFrameProxy::Properties properties { sample.time(), sample.mirrored(), sample.rotation(), { static_cast<float>(sample.size().width()), static_cast<float>(sample.size().height()) }, sample.videoFormat() };
>+        
>+        // FIXME: We need to either get GPUProcess or UIProcess object heap proxy. For now we always go to GPUProcess.
>+        Locker lock(m_remoteVideoFrameObjectHeapProxyLock);
>+        videoFrame = RemoteVideoFrameProxy::createAlreadyInitialized(m_connection->uniqueID(), *remoteIdentifier, properties, [proxy = m_remoteVideoFrameObjectHeapProxy](auto& frame, auto&& callback) {
>+            if (!proxy) {
>+                callback({ });
>+                return;
>+            }
>+            proxy->getVideoFrameBuffer(frame, WTFMove(callback));
>+        });
>+    }
> 
>     auto iterator = m_videoSources.find(identifier);
>     if (iterator == m_videoSources.end()) {
>diff --git a/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.h b/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.h
>index 7cf20d7c5094fdba3767927036aa86429c9f0f7a..bf1da9ddce8014098b5a25abf547458405172a3c 100644
>--- a/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.h
>+++ b/Source/WebKit/WebProcess/cocoa/RemoteCaptureSampleManager.h
>@@ -39,6 +39,7 @@
> #include <WebCore/CARingBuffer.h>
> #include <WebCore/WebAudioBufferList.h>
> #include <wtf/HashMap.h>
>+#include <wtf/Lock.h>
> #include <wtf/WorkQueue.h>
> 
> namespace WebCore {
>@@ -47,6 +48,7 @@ class RemoteVideoSample;
> }
> 
> namespace WebKit {
>+class RemoteVideoFrameObjectHeapProxy;
> 
> class RemoteCaptureSampleManager : public IPC::Connection::ThreadMessageReceiverRefCounted {
>     WTF_MAKE_FAST_ALLOCATED;
>@@ -61,6 +63,7 @@ public:
>     void removeSource(WebCore::RealtimeMediaSourceIdentifier);
> 
>     void didUpdateSourceConnection(IPC::Connection*);
>+    void setRemoteVideoFrameObjectHeapProxy(RemoteVideoFrameObjectHeapProxy*);
> 
>     void didReceiveMessage(IPC::Connection&, IPC::Decoder&);
> 
>@@ -120,6 +123,9 @@ private:
>     // background thread member
>     HashMap<WebCore::RealtimeMediaSourceIdentifier, std::unique_ptr<RemoteAudio>> m_audioSources;
>     HashMap<WebCore::RealtimeMediaSourceIdentifier, std::unique_ptr<RemoteVideo>> m_videoSources;
>+
>+    Lock m_remoteVideoFrameObjectHeapProxyLock;
>+    RefPtr<RemoteVideoFrameObjectHeapProxy> m_remoteVideoFrameObjectHeapProxy WTF_GUARDED_BY_LOCK(m_remoteVideoFrameObjectHeapProxyLock);
> };
> 
> } // namespace WebKit
>diff --git a/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp b/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp
>index 1ee923069cdc58c9d7a21027ec946630b2c0eb1e..711b5585a5d5966b35b9409a3c9429829f1e376c 100644
>--- a/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp
>+++ b/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp
>@@ -230,12 +230,19 @@ void UserMediaCaptureManager::AudioFactory::setShouldCaptureInGPUProcess(bool va
>     m_shouldCaptureInGPUProcess = value;
> }
> 
>+void UserMediaCaptureManager::VideoFactory::setShouldCaptureInGPUProcess(bool value)
>+{
>+    m_shouldCaptureInGPUProcess = value;
>+}
>+
> CaptureSourceOrError UserMediaCaptureManager::VideoFactory::createVideoCaptureSource(const CaptureDevice& device, String&& hashSalt, const MediaConstraints* constraints)
> {
> #if !ENABLE(GPU_PROCESS)
>     if (m_shouldCaptureInGPUProcess)
>         return CaptureSourceOrError { "Video capture in GPUProcess is not implemented"_s };
> #endif
>+    if (m_shouldCaptureInGPUProcess)
>+        m_manager.m_remoteCaptureSampleManager.setRemoteVideoFrameObjectHeapProxy(&WebProcess::singleton().ensureGPUProcessConnection().remoteVideoFrameObjectHeapProxy());
> 
>     bool shouldUseIOSurface = false;
>     return CaptureSourceOrError(RealtimeVideoSource::create(RemoteRealtimeVideoSource::create(device, constraints, { }, WTFMove(hashSalt), m_manager, m_shouldCaptureInGPUProcess), shouldUseIOSurface));
>diff --git a/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.h b/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.h
>index 7c54a2a30cd5ea7fdbdb98633ba840aaeea0dae5..f4ab9439da8612481ff4ab5c30a3ff62ce9113bc 100644
>--- a/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.h
>+++ b/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.h
>@@ -85,7 +85,7 @@ private:
>     class VideoFactory : public WebCore::VideoCaptureFactory {
>     public:
>         explicit VideoFactory(UserMediaCaptureManager& manager) : m_manager(manager) { }
>-        void setShouldCaptureInGPUProcess(bool value) { m_shouldCaptureInGPUProcess = value; }
>+        void setShouldCaptureInGPUProcess(bool);
> 
>     private:
>         WebCore::CaptureSourceOrError createVideoCaptureSource(const WebCore::CaptureDevice&, String&& hashSalt, const WebCore::MediaConstraints*) final;

Actions: View | Formatted Diff | Diff

Attachments on bug 236356: 451352 | 451520 | 451610 | 451611 | 451652 | 451654 | 451673 | 451692 | 451694