WebKit Bugzilla
New
Browse
Search+
Log In
×
Sign in with GitHub
or
Remember my login
Create Account
·
Forgot Password
Forgotten password account recovery
[patch]
Patch
bug-236131-20220223170738.patch (text/plain), 56.90 KB, created by
youenn fablet
on 2022-02-23 08:07:42 PST
(
hide
)
Description:
Patch
Filename:
MIME Type:
Creator:
youenn fablet
Created:
2022-02-23 08:07:42 PST
Size:
56.90 KB
patch
obsolete
>Subversion Revision: 290366 >diff --git a/Source/ThirdParty/libwebrtc/ChangeLog b/Source/ThirdParty/libwebrtc/ChangeLog >index da3e6b98e3de6666a56f31ea3b8773dd8a47b56c..c7bc5e061fc1bae81fd6fd8438596dabcbf3202a 100644 >--- a/Source/ThirdParty/libwebrtc/ChangeLog >+++ b/Source/ThirdParty/libwebrtc/ChangeLog >@@ -1,3 +1,20 @@ >+2022-02-23 Youenn Fablet <youenn@apple.com> >+ >+ Allow to pass webrtc video frame buffers through RealtimeMediaSource based pipelines >+ https://bugs.webkit.org/show_bug.cgi?id=236131 >+ <rdar://problem/88805580> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Update routines to take VideoFrameBuffer instead of VideoFrame. >+ Fix bugs in handling of YUV conversion libwebrtc routines. >+ >+ * Configurations/libwebrtc.iOS.exp: >+ * Configurations/libwebrtc.iOSsim.exp: >+ * Configurations/libwebrtc.mac.exp: >+ * Source/webrtc/sdk/WebKit/WebKitUtilities.h: >+ * Source/webrtc/sdk/WebKit/WebKitUtilities.mm: >+ > 2022-02-18 Robert Jenner <Jenner@apple.com> > > Unreviewed, reverting r290149. >diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog >index ed1206a13d7a54e075a747c4c9e602640ede9e17..2a7d2ef2d3d9ad3785e5e3e5e0bf17971d20676d 100644 >--- a/Source/WebCore/ChangeLog >+++ b/Source/WebCore/ChangeLog >@@ -1,3 +1,34 @@ >+2022-02-23 Youenn Fablet <youenn@apple.com> >+ >+ Allow to pass webrtc video frame buffers through RealtimeMediaSource based pipelines >+ https://bugs.webkit.org/show_bug.cgi?id=236131 >+ <rdar://problem/88805580> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Introduce VideoFrameLibWebRTC as a wrapper to libwebrtc video frame buffer. >+ In case we send VideoFrameLibWebRTC through RealtimeOutgoingVideoSourceCocoa, directly send the video frame buffer. >+ When receiving libwebrtc video frame buffers in RealtimeIncomingVideoSourceCocoa, make use of VideoFrameLibWebRTC to delay conversion of the video frame buffer in a CVPixelBufferRef. >+ In most cases, the conversion is unneeded as the video frame buffer will be used for rendering and will be copied to shared memory through SharedVideoFrameWriter. >+ Fix bugs in handling of YUV conversion libwebrtc routines. >+ Minor refactoring to have a default asVideoFrameCV method implementatin in VideoFrame. >+ >+ Covered by existing tests. >+ >+ * Sources.txt: >+ * WebCore.xcodeproj/project.pbxproj: >+ * platform/VideoFrame.cpp: >+ * platform/VideoFrame.h: >+ * platform/cocoa/SharedVideoFrameInfo.h: >+ * platform/cocoa/SharedVideoFrameInfo.mm: >+ * platform/graphics/RemoteVideoSample.cpp: >+ * platform/graphics/cv/VideoFrameCV.mm: >+ * platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp: Added. >+ * platform/mediastream/libwebrtc/VideoFrameLibWebRTC.h: Added. >+ * platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h: >+ * platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm: >+ * platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp: >+ > 2022-02-23 Zan Dobersek <zdobersek@igalia.com> > > [GStreamer] Add WebKitDMABufVideoSink >diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog >index de8fe2e690115497f561999576207e7c1dad4b4d..9d249ba0b228ec6129d507a7fa4a217dc936c063 100644 >--- a/Source/WebKit/ChangeLog >+++ b/Source/WebKit/ChangeLog >@@ -1,3 +1,22 @@ >+2022-02-23 Youenn Fablet <youenn@apple.com> >+ >+ Allow to pass webrtc video frame buffers through RealtimeMediaSource based pipelines >+ https://bugs.webkit.org/show_bug.cgi?id=236131 >+ <rdar://problem/88805580> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Add optimization support for VideoFrameLibWebRTC writing into shared memory. >+ This removes the need to convert the webrtc video frame into a CVPixelBuffer, just to send it to GPUProcess for rendering. >+ >+ * GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp: >+ This assertion might be buggy in case of GPUProcess crash (old GPU process frames submitted to rendering in new GPU Process). >+ * WebProcess/GPU/media/RemoteVideoFrameProxy.cpp: >+ * WebProcess/GPU/media/RemoteVideoFrameProxy.h: >+ * WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp: >+ * WebProcess/GPU/webrtc/SharedVideoFrame.cpp: >+ * WebProcess/GPU/webrtc/SharedVideoFrame.h: >+ > 2022-02-23 Youenn Fablet <youenn@apple.com> > > Optimise black frame sending in MediaRecorderPrivate >diff --git a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp >index bbbc05827cf74a93a002e5074782110fcf5cddd3..19e5669bba9d3e70f39554fee12cb80f34a2c0fb 100644 >--- a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp >+++ b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOS.exp >@@ -334,4 +334,5 @@ __ZN6webrtc32webrtc_sequence_checker_internal19SequenceCheckerImplC2Ev > __ZNK3rtc14RTCCertificate17GetSSLCertificateEv > __ZTVN6webrtc30WrappingAsyncDnsResolverResultE > __ZN6webrtc20pixelBufferFromFrameERKNS_10VideoFrameE >-__ZN6webrtc14copyVideoFrameERKNS_10VideoFrameEPh >+__ZN6webrtc20copyVideoFrameBufferERNS_16VideoFrameBufferEPh >+__ZN6webrtc32createPixelBufferFromFrameBufferERNS_16VideoFrameBufferERKNSt3__18functionIFP10__CVBuffermmNS_10BufferTypeEEEE >diff --git a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp >index bbbc05827cf74a93a002e5074782110fcf5cddd3..19e5669bba9d3e70f39554fee12cb80f34a2c0fb 100644 >--- a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp >+++ b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.iOSsim.exp >@@ -334,4 +334,5 @@ __ZN6webrtc32webrtc_sequence_checker_internal19SequenceCheckerImplC2Ev > __ZNK3rtc14RTCCertificate17GetSSLCertificateEv > __ZTVN6webrtc30WrappingAsyncDnsResolverResultE > __ZN6webrtc20pixelBufferFromFrameERKNS_10VideoFrameE >-__ZN6webrtc14copyVideoFrameERKNS_10VideoFrameEPh >+__ZN6webrtc20copyVideoFrameBufferERNS_16VideoFrameBufferEPh >+__ZN6webrtc32createPixelBufferFromFrameBufferERNS_16VideoFrameBufferERKNSt3__18functionIFP10__CVBuffermmNS_10BufferTypeEEEE >diff --git a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp >index 2a57dca5af38edf3e3a0e9a4efef64c8a63e9e6c..fa280a4a4fd86851ccb31f3c9de76b8ea88958dc 100644 >--- a/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp >+++ b/Source/ThirdParty/libwebrtc/Configurations/libwebrtc.mac.exp >@@ -334,4 +334,5 @@ __ZN6webrtc32webrtc_sequence_checker_internal19SequenceCheckerImplC2Ev > __ZNK3rtc14RTCCertificate17GetSSLCertificateEv > __ZTVN6webrtc30WrappingAsyncDnsResolverResultE > __ZN6webrtc20pixelBufferFromFrameERKNS_10VideoFrameE >-__ZN6webrtc14copyVideoFrameERKNS_10VideoFrameEPh >+__ZN6webrtc20copyVideoFrameBufferERNS_16VideoFrameBufferEPh >+__ZN6webrtc32createPixelBufferFromFrameBufferERNS_16VideoFrameBufferERKNSt3__18functionIFP10__CVBuffermmNS_10BufferTypeEEEE >diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h >index 2ef8a1e91538611efe5af734f80cd7a9ba4f53c6..a779cad1bd36002d799316a14805d3a33d4c18ba 100644 >--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h >+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.h >@@ -48,9 +48,10 @@ bool isH264HardwareEncoderAllowed(); > > enum class BufferType { I420, I010 }; > CVPixelBufferRef createPixelBufferFromFrame(const VideoFrame&, const std::function<CVPixelBufferRef(size_t, size_t, BufferType)>& createPixelBuffer) CF_RETURNS_RETAINED; >+CVPixelBufferRef createPixelBufferFromFrameBuffer(VideoFrameBuffer&, const std::function<CVPixelBufferRef(size_t, size_t, BufferType)>& createPixelBuffer) CF_RETURNS_RETAINED; > CVPixelBufferRef pixelBufferFromFrame(const VideoFrame&) CF_RETURNS_RETAINED; > rtc::scoped_refptr<webrtc::VideoFrameBuffer> pixelBufferToFrame(CVPixelBufferRef); >-bool copyVideoFrame(const VideoFrame&, uint8_t*); >+bool copyVideoFrameBuffer(VideoFrameBuffer&, uint8_t*); > > typedef CVPixelBufferRef (*GetBufferCallback)(void*); > typedef void (*ReleaseBufferCallback)(void*); >diff --git a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm >index 62f624c4976fa618d80cd140a3fd91d6ba9e909f..66f8019ea596cacc472a15b16f0a3b35dacae9de 100644 >--- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm >+++ b/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/WebKit/WebKitUtilities.mm >@@ -160,28 +160,32 @@ static bool CopyVideoFrameToPixelBuffer(const webrtc::I010BufferInterface* frame > > CVPixelBufferRef createPixelBufferFromFrame(const VideoFrame& frame, const std::function<CVPixelBufferRef(size_t, size_t, BufferType)>& createPixelBuffer) > { >- auto buffer = frame.video_frame_buffer(); >- if (buffer->type() != VideoFrameBuffer::Type::kNative) { >- auto type = buffer->type(); >+ return createPixelBufferFromFrameBuffer(*frame.video_frame_buffer(), createPixelBuffer); >+} >+ >+CVPixelBufferRef createPixelBufferFromFrameBuffer(VideoFrameBuffer& buffer, const std::function<CVPixelBufferRef(size_t, size_t, BufferType)>& createPixelBuffer) >+{ >+ if (buffer.type() != VideoFrameBuffer::Type::kNative) { >+ auto type = buffer.type(); > if (type != VideoFrameBuffer::Type::kI420 && type != VideoFrameBuffer::Type::kI010) { > RTC_LOG(WARNING) << "Video frame buffer type is not expected."; > return nullptr; > } > >- auto pixelBuffer = createPixelBuffer(buffer->width(), buffer->height(), type == VideoFrameBuffer::Type::kI420 ? BufferType::I420 : BufferType::I010); >+ auto pixelBuffer = createPixelBuffer(buffer.width(), buffer.height(), type == VideoFrameBuffer::Type::kI420 ? BufferType::I420 : BufferType::I010); > if (!pixelBuffer) { > RTC_LOG(WARNING) << "Pixel buffer creation failed."; > return nullptr; > } > > if (type == VideoFrameBuffer::Type::kI420) >- CopyVideoFrameToPixelBuffer(buffer->GetI420(), pixelBuffer); >+ CopyVideoFrameToPixelBuffer(buffer.GetI420(), pixelBuffer); > else >- CopyVideoFrameToPixelBuffer(buffer->GetI010(), pixelBuffer); >+ CopyVideoFrameToPixelBuffer(buffer.GetI010(), pixelBuffer); > return pixelBuffer; > } > >- auto *frameBuffer = static_cast<ObjCFrameBuffer*>(buffer.get())->wrapped_frame_buffer(); >+ auto *frameBuffer = static_cast<ObjCFrameBuffer*>(&buffer)->wrapped_frame_buffer(); > if (![frameBuffer isKindOfClass:[RTCCVPixelBuffer class]]) > return nullptr; > >@@ -203,18 +207,17 @@ CVPixelBufferRef pixelBufferFromFrame(const VideoFrame& frame) > return CVPixelBufferRetain(rtcPixelBuffer.pixelBuffer); > } > >-bool copyVideoFrame(const VideoFrame& frame, uint8_t* data) >+bool copyVideoFrameBuffer(VideoFrameBuffer& buffer, uint8_t* data) > { >- auto buffer = frame.video_frame_buffer(); >- if (buffer->type() == VideoFrameBuffer::Type::kNative) >+ if (buffer.type() == VideoFrameBuffer::Type::kNative) > return false; > >- auto type = buffer->type(); >+ auto type = buffer.type(); > if (type == VideoFrameBuffer::Type::kI420) { >- auto* i420Frame = buffer->GetI420(); >+ auto* i420Frame = buffer.GetI420(); > auto* dataY = data; > auto strideY = i420Frame->width(); >- auto strideUV = i420Frame->width() / 2; >+ auto strideUV = i420Frame->width(); > auto* dataUV = data + (i420Frame->width() * i420Frame->height()); > return !libyuv::I420ToNV12(i420Frame->DataY(), i420Frame->StrideY(), > i420Frame->DataU(), i420Frame->StrideU(), >@@ -223,10 +226,10 @@ bool copyVideoFrame(const VideoFrame& frame, uint8_t* data) > i420Frame->width(), i420Frame->height()); > } > if (type == VideoFrameBuffer::Type::kI010) { >- auto* i010Frame = buffer->GetI010(); >+ auto* i010Frame = buffer.GetI010(); > auto* dataY = reinterpret_cast<uint16_t*>(data); > auto strideY = i010Frame->width(); >- auto strideUV = i010Frame->width() / 2; >+ auto strideUV = i010Frame->width(); > auto* dataUV = dataY + (i010Frame->width() * i010Frame->height()); > return !libyuv::I010ToP010(i010Frame->DataY(), i010Frame->StrideY(), > i010Frame->DataU(), i010Frame->StrideU(), >diff --git a/Source/WebCore/Sources.txt b/Source/WebCore/Sources.txt >index ef63b542ccbf190cbb51e1c45f5e9a823635d274..c20037e88a35c0b953304bcc08673ece834e5fed 100644 >--- a/Source/WebCore/Sources.txt >+++ b/Source/WebCore/Sources.txt >@@ -2200,6 +2200,7 @@ platform/mediastream/RealtimeVideoCaptureSource.cpp > platform/mediastream/RealtimeVideoSource.cpp > platform/mediastream/libwebrtc/LibWebRTCDTMFSenderBackend.cpp > platform/mediastream/libwebrtc/LibWebRTCProvider.cpp >+platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp > platform/mock/DeviceOrientationClientMock.cpp > platform/mock/GeolocationClientMock.cpp > platform/mock/MediaEngineConfigurationFactoryMock.cpp >diff --git a/Source/WebCore/WebCore.xcodeproj/project.pbxproj b/Source/WebCore/WebCore.xcodeproj/project.pbxproj >index bc36fcfa0110d1e812a45614dc1de6d53114cc32..701d3473ef50e2a18363252e62646b4fc9705530 100644 >--- a/Source/WebCore/WebCore.xcodeproj/project.pbxproj >+++ b/Source/WebCore/WebCore.xcodeproj/project.pbxproj >@@ -1226,6 +1226,7 @@ > 41E6D71B26EA1A4600B765D6 /* RTCError.h in Headers */ = {isa = PBXBuildFile; fileRef = 41B7C04B26E6917100D36C07 /* RTCError.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 41E9DCE7231974BF00F35949 /* BlobLoader.h in Headers */ = {isa = PBXBuildFile; fileRef = 41E9DCE4231973FE00F35949 /* BlobLoader.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 41E9DCE92319CA7600F35949 /* NetworkSendQueue.h in Headers */ = {isa = PBXBuildFile; fileRef = 41E9DCE82319CA7500F35949 /* NetworkSendQueue.h */; settings = {ATTRIBUTES = (Private, ); }; }; >+ 41EB797E27C3E45E001C3E3E /* VideoFrameLibWebRTC.h in Headers */ = {isa = PBXBuildFile; fileRef = 41EB797C27C3E45D001C3E3E /* VideoFrameLibWebRTC.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 41F062140F5F192600A07EAC /* InspectorDatabaseResource.h in Headers */ = {isa = PBXBuildFile; fileRef = 41F062120F5F192600A07EAC /* InspectorDatabaseResource.h */; }; > 41F1D21F0EF35C2A00DA8753 /* ScriptCachedFrameData.h in Headers */ = {isa = PBXBuildFile; fileRef = 41F1D21D0EF35C2A00DA8753 /* ScriptCachedFrameData.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 41FABD2D1F4DFE4A006A6C97 /* DOMCacheEngine.h in Headers */ = {isa = PBXBuildFile; fileRef = 41FABD2B1F4DFE42006A6C97 /* DOMCacheEngine.h */; settings = {ATTRIBUTES = (Private, ); }; }; >@@ -8992,6 +8993,8 @@ > 41E9DCE62319742300F35949 /* EndingType.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = EndingType.idl; sourceTree = "<group>"; }; > 41E9DCE82319CA7500F35949 /* NetworkSendQueue.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkSendQueue.h; sourceTree = "<group>"; }; > 41E9DCEA2319CAE800F35949 /* NetworkSendQueue.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkSendQueue.cpp; sourceTree = "<group>"; }; >+ 41EB797B27C3E45D001C3E3E /* VideoFrameLibWebRTC.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = VideoFrameLibWebRTC.cpp; path = libwebrtc/VideoFrameLibWebRTC.cpp; sourceTree = "<group>"; }; >+ 41EB797C27C3E45D001C3E3E /* VideoFrameLibWebRTC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = VideoFrameLibWebRTC.h; path = libwebrtc/VideoFrameLibWebRTC.h; sourceTree = "<group>"; }; > 41F062120F5F192600A07EAC /* InspectorDatabaseResource.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = InspectorDatabaseResource.h; sourceTree = "<group>"; }; > 41F062130F5F192600A07EAC /* InspectorDatabaseResource.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = InspectorDatabaseResource.cpp; sourceTree = "<group>"; }; > 41F0963424F90DEA0009FB93 /* TextDecoderStreamDecoder.idl */ = {isa = PBXFileReference; lastKnownFileType = text; path = TextDecoderStreamDecoder.idl; sourceTree = "<group>"; }; >@@ -21066,6 +21069,8 @@ > 41B28B381F860BD100FB52AC /* LibWebRTCProviderCocoa.cpp */, > 41B28B361F860BD000FB52AC /* LibWebRTCProviderCocoa.h */, > 410E445F234373AD000173D4 /* LibWebRTCSocketIdentifier.h */, >+ 41EB797B27C3E45D001C3E3E /* VideoFrameLibWebRTC.cpp */, >+ 41EB797C27C3E45D001C3E3E /* VideoFrameLibWebRTC.h */, > ); > name = libwebrtc; > sourceTree = "<group>"; >@@ -37908,6 +37913,7 @@ > CD3EEF3B25799FA1006563BB /* VideoConfiguration.h in Headers */, > 7B5A3DAF27ABF8C5006C6F97 /* VideoFrame.h in Headers */, > 7BE37F8727BBB708007A6CD3 /* VideoFrameCV.h in Headers */, >+ 41EB797E27C3E45E001C3E3E /* VideoFrameLibWebRTC.h in Headers */, > 41DEEFB62719BA1A00CB8D74 /* VideoFrameMetadata.h in Headers */, > 41DEEFB02719B29000CB8D74 /* VideoFrameRequestCallback.h in Headers */, > 52B0D4C01C57FD660077CE53 /* VideoFullscreenChangeObserver.h in Headers */, >diff --git a/Source/WebCore/platform/VideoFrame.cpp b/Source/WebCore/platform/VideoFrame.cpp >index a549c1a5c48eb23d64ba3fd3538408bd155234f0..54f8a3d8c0c4138f5dfc64c999a11fef15710d0e 100644 >--- a/Source/WebCore/platform/VideoFrame.cpp >+++ b/Source/WebCore/platform/VideoFrame.cpp >@@ -26,6 +26,8 @@ > #include "config.h" > #include "VideoFrame.h" > >+#include "VideoFrameCV.h" >+ > #if ENABLE(VIDEO) > > namespace WebCore { >@@ -131,6 +133,17 @@ void VideoFrame::dump(PrintStream&) const > { > } > >+#if PLATFORM(COCOA) >+RefPtr<VideoFrameCV> VideoFrame::asVideoFrameCV() >+{ >+ auto buffer = pixelBuffer(); >+ if (!buffer) >+ return nullptr; >+ >+ return VideoFrameCV::create(presentationTime(), videoMirrored(), videoRotation(), buffer); >+} >+#endif >+ > } > > #endif >diff --git a/Source/WebCore/platform/VideoFrame.h b/Source/WebCore/platform/VideoFrame.h >index 6411df7d640ccded1d1e3e11fd1a428a291b31ed..5909c348f6beeface6e5f69a031504a04a2d9659 100644 >--- a/Source/WebCore/platform/VideoFrame.h >+++ b/Source/WebCore/platform/VideoFrame.h >@@ -52,9 +52,10 @@ public: > WEBCORE_EXPORT PlatformSample::Type platformSampleType() const final; > > virtual bool isRemoteProxy() const { return false; } >+ virtual bool isLibWebRTC() const { return false; } > #if USE(AVFOUNDATION) > virtual bool isCV() const { return false; } >- virtual RefPtr<WebCore::VideoFrameCV> asVideoFrameCV() = 0; >+ WEBCORE_EXPORT virtual RefPtr<VideoFrameCV> asVideoFrameCV(); > #endif > > protected: >diff --git a/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.h b/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.h >index be3fdb797803a140eb02c469d33624b48b627503..a4f5cf450e0aa1846bccaa265bffb516d904afaf 100644 >--- a/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.h >+++ b/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.h >@@ -34,7 +34,7 @@ typedef struct __CVBuffer* CVPixelBufferRef; > typedef struct __CVPixelBufferPool* CVPixelBufferPoolRef; > > namespace webrtc { >-class VideoFrame; >+class VideoFrameBuffer; > } > > namespace WebCore { >@@ -51,8 +51,8 @@ public: > WEBCORE_EXPORT bool writePixelBuffer(CVPixelBufferRef, uint8_t* data); > > #if USE(LIBWEBRTC) >- WEBCORE_EXPORT static SharedVideoFrameInfo fromVideoFrame(const webrtc::VideoFrame&); >- WEBCORE_EXPORT bool writeVideoFrame(const webrtc::VideoFrame&, uint8_t* data); >+ WEBCORE_EXPORT static SharedVideoFrameInfo fromVideoFrame(const webrtc::VideoFrameBuffer&); >+ WEBCORE_EXPORT bool writeVideoFrameBuffer(webrtc::VideoFrameBuffer&, uint8_t* data); > #endif > > WEBCORE_EXPORT size_t storageSize() const; >diff --git a/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.mm b/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.mm >index 08c6ccd885371faf1d118e39da23ae32d8459bde..ec5650ea37667a902f328f532c49ceafa7147f38 100644 >--- a/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.mm >+++ b/Source/WebCore/platform/cocoa/SharedVideoFrameInfo.mm >@@ -78,6 +78,7 @@ size_t SharedVideoFrameInfo::storageSize() const > void SharedVideoFrameInfo::encode(uint8_t* destination) > { > WTF::Persistence::Encoder encoder; >+ > encoder << (uint32_t)m_bufferType; > encoder << m_width; > encoder << m_height; >@@ -207,7 +208,6 @@ RetainPtr<CVPixelBufferRef> SharedVideoFrameInfo::createPixelBufferFromMemory(co > data += m_bytesPerRowPlaneB; > } > >- CVPixelBufferUnlockBaseAddress(rawPixelBuffer, 0); > return pixelBuffer; > } > >@@ -266,31 +266,31 @@ RetainPtr<CVPixelBufferPoolRef> SharedVideoFrameInfo::createCompatibleBufferPool > } > > #if USE(LIBWEBRTC) >-SharedVideoFrameInfo SharedVideoFrameInfo::fromVideoFrame(const webrtc::VideoFrame& frame) >+SharedVideoFrameInfo SharedVideoFrameInfo::fromVideoFrame(const webrtc::VideoFrameBuffer& frame) > { >- auto buffer = frame.video_frame_buffer(); >- if (buffer->type() == webrtc::VideoFrameBuffer::Type::kNative) >+ if (frame.type() == webrtc::VideoFrameBuffer::Type::kNative) > return SharedVideoFrameInfo { }; > >- auto type = buffer->type(); >- >+ auto type = frame.type(); > if (type == webrtc::VideoFrameBuffer::Type::kI420) > return SharedVideoFrameInfo { kCVPixelFormatType_420YpCbCr8BiPlanarFullRange, > static_cast<uint32_t>(frame.width()), static_cast<uint32_t>(frame.height()), static_cast<uint32_t>(frame.width()), >- static_cast<uint32_t>(frame.width()), static_cast<uint32_t>(frame.height()), static_cast<uint32_t>(frame.width()) / 2 }; >+ static_cast<uint32_t>(frame.width()) / 2, static_cast<uint32_t>(frame.height()) / 2, static_cast<uint32_t>(frame.width()) }; > > if (type == webrtc::VideoFrameBuffer::Type::kI010) > return SharedVideoFrameInfo { kCVPixelFormatType_420YpCbCr10BiPlanarFullRange, > static_cast<uint32_t>(frame.width()), static_cast<uint32_t>(frame.height()), static_cast<uint32_t>(frame.width() * 2), >- static_cast<uint32_t>(frame.width()), static_cast<uint32_t>(frame.height()), static_cast<uint32_t>(frame.width()) }; >+ static_cast<uint32_t>(frame.width()) / 2, static_cast<uint32_t>(frame.height()) / 2, static_cast<uint32_t>(frame.width()) * 2 }; > > return SharedVideoFrameInfo { }; > } > >-bool SharedVideoFrameInfo::writeVideoFrame(const webrtc::VideoFrame& frame, uint8_t* data) >+bool SharedVideoFrameInfo::writeVideoFrameBuffer(webrtc::VideoFrameBuffer& frameBuffer, uint8_t* data) > { > ASSERT(m_bufferType == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange || m_bufferType == kCVPixelFormatType_420YpCbCr10BiPlanarFullRange); >- return webrtc::copyVideoFrame(frame, data); >+ encode(data); >+ data += sizeof(SharedVideoFrameInfo); >+ return webrtc::copyVideoFrameBuffer(frameBuffer, data); > } > #endif > >diff --git a/Source/WebCore/platform/graphics/RemoteVideoSample.cpp b/Source/WebCore/platform/graphics/RemoteVideoSample.cpp >index dab6f2a41773a5b27815ab20d4bd4469d62b92ea..875daa40e28e8df9ae22b667460ec6e29d58b260 100644 >--- a/Source/WebCore/platform/graphics/RemoteVideoSample.cpp >+++ b/Source/WebCore/platform/graphics/RemoteVideoSample.cpp >@@ -33,6 +33,7 @@ > #include "Logging.h" > #include "MediaSampleAVFObjC.h" > #include "VideoFrameCV.h" >+#include "VideoFrameLibWebRTC.h" > > #if USE(ACCELERATE) > #include <Accelerate/Accelerate.h> >@@ -93,11 +94,11 @@ std::unique_ptr<RemoteVideoSample> RemoteVideoSample::create(MediaSample& sample > imageBuffer = downcast<MediaSampleAVFObjC>(sample).pixelBuffer(); > else if (is<VideoFrameCV>(sample)) > imageBuffer = downcast<VideoFrameCV>(sample).pixelBuffer(); >- else { >+ else if (shouldCheckForIOSurface == ShouldCheckForIOSurface::Yes) { > RELEASE_LOG_ERROR(Media, "RemoteVideoSample::create: cannot obtain CVPixelBuffer from sample. Unknown sample type."); > return nullptr; > } >- if (!imageBuffer) { >+ if (!imageBuffer && shouldCheckForIOSurface == ShouldCheckForIOSurface::Yes) { > RELEASE_LOG_ERROR(Media, "RemoteVideoSample::create: cannot obtain CVPixelBuffer from sample. No buffer."); > return nullptr; > } >diff --git a/Source/WebCore/platform/graphics/cv/VideoFrameCV.mm b/Source/WebCore/platform/graphics/cv/VideoFrameCV.mm >index d39c90e0c790a5751137b05a6fd601112d6a981a..89f1dc6f15332b2f0a1e6c2e45be5941637a29cb 100644 >--- a/Source/WebCore/platform/graphics/cv/VideoFrameCV.mm >+++ b/Source/WebCore/platform/graphics/cv/VideoFrameCV.mm >@@ -54,6 +54,7 @@ uint32_t VideoFrameCV::videoPixelFormat() const > { > return CVPixelBufferGetPixelFormatType(m_pixelBuffer.get()); > } >+ > RefPtr<WebCore::VideoFrameCV> VideoFrameCV::asVideoFrameCV() > { > return this; >diff --git a/Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp b/Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..bbaa662b62d025cdc6f59146522975e474a72f53 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp >@@ -0,0 +1,68 @@ >+/* >+ * Copyright (C) 2022 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+#include "VideoFrameLibWebRTC.h" >+ >+#if USE(LIBWEBRTC) >+ >+#include <pal/cf/CoreMediaSoftLink.h> >+#include "CoreVideoSoftLink.h" >+ >+namespace WebCore { >+ >+Ref<VideoFrameLibWebRTC> VideoFrameLibWebRTC::create(MediaTime presentationTime, bool isMirrored, VideoRotation rotation, rtc::scoped_refptr<webrtc::VideoFrameBuffer>&& buffer, ConversionCallback&& conversionCallback) >+{ >+ return adoptRef(*new VideoFrameLibWebRTC(presentationTime, isMirrored, rotation, WTFMove(buffer), WTFMove(conversionCallback))); >+} >+ >+VideoFrameLibWebRTC::VideoFrameLibWebRTC(MediaTime presentationTime, bool isMirrored, VideoRotation rotation, rtc::scoped_refptr<webrtc::VideoFrameBuffer>&& buffer, ConversionCallback&& conversionCallback) >+ : VideoFrame(presentationTime, isMirrored, rotation) >+ , m_buffer(WTFMove(buffer)) >+ , m_size(m_buffer->width(), m_buffer->height()) >+ , m_conversionCallback(WTFMove(conversionCallback)) >+{ >+ switch (m_buffer->type()) { >+ case webrtc::VideoFrameBuffer::Type::kI420: >+ m_videoPixelFormat = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange; >+ break; >+ case webrtc::VideoFrameBuffer::Type::kI010: >+ m_videoPixelFormat = kCVPixelFormatType_420YpCbCr10BiPlanarFullRange; >+ break; >+ default: >+ ASSERT_NOT_REACHED(); >+ } >+} >+ >+CVPixelBufferRef VideoFrameLibWebRTC::pixelBuffer() const >+{ >+ Locker locker { m_pixelBufferLock }; >+ if (!m_pixelBuffer && m_conversionCallback) >+ m_pixelBuffer = std::exchange(m_conversionCallback, { })(*m_buffer); >+ return m_pixelBuffer.get(); >+} >+ >+} >+#endif >diff --git a/Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.h b/Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.h >new file mode 100644 >index 0000000000000000000000000000000000000000..073483f8a19a332937ddb03bda861fe9292929c9 >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.h >@@ -0,0 +1,73 @@ >+/* >+ * Copyright (C) 2022 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if USE(LIBWEBRTC) >+ >+#include "VideoFrame.h" >+ >+ALLOW_UNUSED_PARAMETERS_BEGIN >+#include <webrtc/api/video/video_frame.h> >+#include <webrtc/sdk/WebKit/WebKitUtilities.h> >+ALLOW_UNUSED_PARAMETERS_END >+ >+using CVPixelBufferRef = struct __CVBuffer*; >+ >+ >+namespace WebCore { >+ >+class VideoFrameLibWebRTC final : public VideoFrame { >+public: >+ using ConversionCallback = Function<RetainPtr<CVPixelBufferRef>(webrtc::VideoFrameBuffer&)>; >+ static Ref<VideoFrameLibWebRTC> create(MediaTime, bool isMirrored, VideoRotation, rtc::scoped_refptr<webrtc::VideoFrameBuffer>&&, ConversionCallback&&); >+ >+ rtc::scoped_refptr<webrtc::VideoFrameBuffer> buffer() { return m_buffer; } >+ >+private: >+ VideoFrameLibWebRTC(MediaTime, bool isMirrored, VideoRotation, rtc::scoped_refptr<webrtc::VideoFrameBuffer>&&, ConversionCallback&&); >+ >+ // VideoFrame >+ FloatSize presentationSize() const final { return m_size; } >+ uint32_t videoPixelFormat() const final { return m_videoPixelFormat; } >+ CVPixelBufferRef pixelBuffer() const final; >+ >+ const rtc::scoped_refptr<webrtc::VideoFrameBuffer> m_buffer; >+ FloatSize m_size; >+ uint32_t m_videoPixelFormat { 0 }; >+ >+ mutable ConversionCallback m_conversionCallback; >+ mutable RetainPtr<CVPixelBufferRef> m_pixelBuffer WTF_GUARDED_BY_LOCK(m_pixelBufferLock); >+ mutable Lock m_pixelBufferLock; >+}; >+ >+} >+ >+SPECIALIZE_TYPE_TRAITS_BEGIN(WebCore::VideoFrameLibWebRTC) >+ static bool isType(const WebCore::VideoFrame& videoFrame) { return videoFrame.isLibWebRTC(); } >+ static bool isType(const WebCore::MediaSample& mediaSample) { return is<WebCore::VideoFrame>(mediaSample) && is<WebCore::VideoFrameLibWebRTC>(downcast<WebCore::VideoFrame>(mediaSample)); } >+SPECIALIZE_TYPE_TRAITS_END() >+ >+#endif >diff --git a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h >index 60338ea0737db6168fc610b1c4205c3aaa69abe9..8a0252962136a223ed342f7807725af848d1b8b5 100644 >--- a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h >+++ b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.h >@@ -62,7 +62,8 @@ private: > #if !RELEASE_LOG_DISABLED > size_t m_numberOfFrames { 0 }; > #endif >- RetainPtr<CVPixelBufferPoolRef> m_pixelBufferPool; >+ Lock m_pixelBufferPoolLock; >+ RetainPtr<CVPixelBufferPoolRef> m_pixelBufferPool WTF_GUARDED_BY_LOCK(m_pixelBufferPoolLock); > size_t m_pixelBufferPoolWidth { 0 }; > size_t m_pixelBufferPoolHeight { 0 }; > webrtc::BufferType m_pixelBufferPoolBufferType; >diff --git a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm >index f10cfe1f029bf664cc8232d617afb1604bc974b0..b4498e3a58dfef9ead8466b1484013593840e187 100644 >--- a/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm >+++ b/Source/WebCore/platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.mm >@@ -33,7 +33,7 @@ > #import "CVUtilities.h" > #import "Logging.h" > #import "MediaSampleAVFObjC.h" >-#import "VideoFrame.h" >+#import "VideoFrameLibWebRTC.h" > #import <wtf/cf/TypeCastsCF.h> > > ALLOW_UNUSED_PARAMETERS_BEGIN >@@ -62,7 +62,7 @@ RealtimeIncomingVideoSourceCocoa::RealtimeIncomingVideoSourceCocoa(rtc::scoped_r > { > } > >-CVPixelBufferPoolRef RealtimeIncomingVideoSourceCocoa::pixelBufferPool(size_t width, size_t height, webrtc::BufferType bufferType) >+CVPixelBufferPoolRef RealtimeIncomingVideoSourceCocoa::pixelBufferPool(size_t width, size_t height, webrtc::BufferType bufferType) WTF_IGNORES_THREAD_SAFETY_ANALYSIS > { > if (!m_pixelBufferPool || m_pixelBufferPoolWidth != width || m_pixelBufferPoolHeight != height || m_pixelBufferPoolBufferType != bufferType) { > OSType poolBufferType; >@@ -133,23 +133,26 @@ RefPtr<MediaSample> RealtimeIncomingVideoSourceCocoa::toVideoFrame(const webrtc: > return static_cast<VideoFrame*>(provider); > } > >- // In case of in memory samples, we have non interleaved YUV data while CVPixelBuffers prefer interleaved YUV data. >- // Maybe we should introduce a MediaSample that would represent non interleaved YUV data as an optimization. >- auto pixelBuffer = adoptCF(webrtc::createPixelBufferFromFrame(frame, [this](size_t width, size_t height, webrtc::BufferType bufferType) -> CVPixelBufferRef { >- auto pixelBufferPool = this->pixelBufferPool(width, height, bufferType); >- if (!pixelBufferPool) >- return nullptr; >- >- CVPixelBufferRef pixelBuffer = nullptr; >- auto status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, m_pixelBufferPool.get(), &pixelBuffer); >- >- if (status != kCVReturnSuccess) { >- ERROR_LOG_IF(loggerPtr(), LOGIDENTIFIER, "Failed creating a pixel buffer with error ", status); >- return nullptr; >- } >- return pixelBuffer; >- })); >- return createMediaSampleFromCVPixelBuffer(pixelBuffer.get(), rotation, frame.timestamp_us()); >+ // If we already have a CVPixelBufferRef, use it directly. >+ if (auto pixelBuffer = webrtc::pixelBufferFromFrame(frame)) >+ return createMediaSampleFromCVPixelBuffer(pixelBuffer, rotation, frame.timestamp_us()); >+ >+ // In case of in memory libwebrtc samples, we have non interleaved YUV data, let's lazily create CVPixelBuffers if needed. >+ return VideoFrameLibWebRTC::create(MediaTime(frame.timestamp_us(), 1000000), false, rotation, frame.video_frame_buffer(), [protectedThis = Ref { *this }, this](auto& buffer) { >+ return adoptCF(webrtc::createPixelBufferFromFrameBuffer(buffer, [this](size_t width, size_t height, webrtc::BufferType bufferType) -> CVPixelBufferRef { >+ Locker lock(m_pixelBufferPoolLock); >+ auto pixelBufferPool = this->pixelBufferPool(width, height, bufferType); >+ if (!pixelBufferPool) >+ return nullptr; >+ CVPixelBufferRef pixelBuffer = nullptr; >+ auto status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, m_pixelBufferPool.get(), &pixelBuffer); >+ if (status != kCVReturnSuccess) { >+ ERROR_LOG_IF(loggerPtr(), LOGIDENTIFIER, "Failed creating a pixel buffer with error ", status); >+ return nullptr; >+ } >+ return pixelBuffer; >+ })); >+ }); > } > > void RealtimeIncomingVideoSourceCocoa::OnFrame(const webrtc::VideoFrame& webrtcVideoFrame) >diff --git a/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp b/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp >index a6eb19d32a244414d951255412d589cb046c7098..f70b41554cbd510831301556dcaddfdb7fcaa21b 100644 >--- a/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp >+++ b/Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp >@@ -33,7 +33,7 @@ > #include "Logging.h" > #include "RealtimeIncomingVideoSourceCocoa.h" > #include "RealtimeVideoUtilities.h" >-#include "VideoFrame.h" >+#include "VideoFrameLibWebRTC.h" > > ALLOW_UNUSED_PARAMETERS_BEGIN > >@@ -86,14 +86,20 @@ void RealtimeOutgoingVideoSourceCocoa::videoSampleAvailable(MediaSample& sample, > } > > bool shouldApplyRotation = m_shouldApplyRotation && m_currentRotation != webrtc::kVideoRotation_0; >- if (!shouldApplyRotation && is<VideoFrame>(sample) && downcast<VideoFrame>(sample).isRemoteProxy()) { >- Ref videoFrame { downcast<VideoFrame>(sample) }; >- auto size = sample.presentationSize(); >- sendFrame(webrtc::toWebRTCVideoFrameBuffer(&videoFrame.leakRef(), >- [](auto* pointer) { return static_cast<VideoFrame*>(pointer)->pixelBuffer(); }, >- [](auto* pointer) { static_cast<VideoFrame*>(pointer)->deref(); }, >- static_cast<int>(size.width()), static_cast<int>(size.height()))); >- return; >+ if (!shouldApplyRotation && is<VideoFrame>(sample)) { >+ if (downcast<VideoFrame>(sample).isRemoteProxy()) { >+ Ref videoFrame { downcast<VideoFrame>(sample) }; >+ auto size = sample.presentationSize(); >+ sendFrame(webrtc::toWebRTCVideoFrameBuffer(&videoFrame.leakRef(), >+ [](auto* pointer) { return static_cast<VideoFrame*>(pointer)->pixelBuffer(); }, >+ [](auto* pointer) { static_cast<VideoFrame*>(pointer)->deref(); }, >+ static_cast<int>(size.width()), static_cast<int>(size.height()))); >+ return; >+ } >+ if (downcast<VideoFrame>(sample).isLibWebRTC()) { >+ sendFrame(downcast<VideoFrameLibWebRTC>(sample).buffer()); >+ return; >+ } > } > > auto pixelBuffer = sample.pixelBuffer(); >diff --git a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h >index b75850b9329b9718cf555dd69be2e7184004e32f..321942be69fae9d3fe4d1a3c2c0cb7f31406e0a5 100644 >--- a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h >+++ b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.h >@@ -55,6 +55,7 @@ namespace WebKit { > > class GPUConnectionToWebProcess; > class RemoteVideoFrameObjectHeap; >+struct SharedVideoFrame; > class SharedVideoFrameReader; > > class LibWebRTCCodecsProxy : public IPC::Connection::ThreadMessageReceiverRefCounted { >@@ -85,7 +86,7 @@ private: > void createEncoder(RTCEncoderIdentifier, const String&, const Vector<std::pair<String, String>>&, bool useLowLatency); > void releaseEncoder(RTCEncoderIdentifier); > void initializeEncoder(RTCEncoderIdentifier, uint16_t width, uint16_t height, unsigned startBitrate, unsigned maxBitrate, unsigned minBitrate, uint32_t maxFramerate); >- void encodeFrame(RTCEncoderIdentifier, WebCore::RemoteVideoSample&&, uint32_t timeStamp, bool shouldEncodeAsKeyFrame, std::optional<RemoteVideoFrameReadReference>); >+ void encodeFrame(RTCEncoderIdentifier, SharedVideoFrame&&, uint32_t timeStamp, bool shouldEncodeAsKeyFrame); > void setEncodeRates(RTCEncoderIdentifier, uint32_t bitRate, uint32_t frameRate); > void setSharedVideoFrameSemaphore(RTCEncoderIdentifier, IPC::Semaphore&&); > void setSharedVideoFrameMemory(RTCEncoderIdentifier, const SharedMemory::IPCHandle&); >diff --git a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in >index dfaf4a26bd1e31219034abb0652de23d9ce2d1aa..ca15ab364c5607d174fa7c31c778e757cbd88ce8 100644 >--- a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in >+++ b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in >@@ -34,7 +34,7 @@ messages -> LibWebRTCCodecsProxy NotRefCounted { > CreateEncoder(WebKit::RTCEncoderIdentifier id, String formatName, Vector<std::pair<String, String>> parameters, bool useLowLatency); > ReleaseEncoder(WebKit::RTCEncoderIdentifier id) > InitializeEncoder(WebKit::RTCEncoderIdentifier id, uint16_t width, uint16_t height, unsigned startBitrate, unsigned maxBitrate, unsigned minBitrate, uint32_t maxFramerate) >- EncodeFrame(WebKit::RTCEncoderIdentifier id, WebCore::RemoteVideoSample sample, uint32_t timeStamp, bool shouldEncodeAsKeyFrame, std::optional<WebKit::RemoteVideoFrameReadReference> sampleReference) >+ EncodeFrame(WebKit::RTCEncoderIdentifier id, struct WebKit::SharedVideoFrame frame, uint32_t timeStamp, bool shouldEncodeAsKeyFrame) > SetEncodeRates(WebKit::RTCEncoderIdentifier id, uint32_t bitRate, uint32_t frameRate) > SetSharedVideoFrameSemaphore(WebKit::RTCEncoderIdentifier id, IPC::Semaphore semaphore) > SetSharedVideoFrameMemory(WebKit::RTCEncoderIdentifier id, WebKit::SharedMemory::IPCHandle storageHandle) >diff --git a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm >index f00681b79f5fce0386a4ee63b72d6f7ee42880ba..758cb419dff2bd58f43cdffb5c891ba41c95cbc3 100644 >--- a/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm >+++ b/Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.mm >@@ -179,7 +179,7 @@ void LibWebRTCCodecsProxy::createEncoder(RTCEncoderIdentifier identifier, const > connection->send(Messages::LibWebRTCCodecs::CompletedEncoding { identifier, IPC::DataReference { buffer, size }, info }, 0); > }).get()); > webrtc::setLocalEncoderLowLatency(encoder, useLowLatency); >- m_encoders.add(identifier, Encoder { encoder, nullptr }); >+ m_encoders.add(identifier, Encoder { encoder, makeUnique<SharedVideoFrameReader>(m_videoFrameObjectHeap.ptr()) }); > } > > void LibWebRTCCodecsProxy::releaseEncoder(RTCEncoderIdentifier identifier) >@@ -230,37 +230,23 @@ static inline webrtc::VideoRotation toWebRTCVideoRotation(WebCore::MediaSample:: > > // For performance reasons, this function accesses m_encoders without locking. This is safe because this function runs on the libWebRTCCodecsQueue > // and m_encoders only get modified on this queue. >-void LibWebRTCCodecsProxy::encodeFrame(RTCEncoderIdentifier identifier, WebCore::RemoteVideoSample&& sample, uint32_t timeStamp, bool shouldEncodeAsKeyFrame, std::optional<RemoteVideoFrameReadReference> sampleReference) >+void LibWebRTCCodecsProxy::encodeFrame(RTCEncoderIdentifier identifier, SharedVideoFrame&& frame, uint32_t timeStamp, bool shouldEncodeAsKeyFrame) > { >- RetainPtr<CVPixelBufferRef> pixelBuffer; >- if (sampleReference) { >- auto sample = m_videoFrameObjectHeap->retire(WTFMove(*sampleReference), 0_s); >- if (!sample) >- return; >- >- auto platformSample = sample->platformSample(); >- ASSERT(platformSample.type == WebCore::PlatformSample::CMSampleBufferType); >- pixelBuffer = static_cast<CVPixelBufferRef>(PAL::CMSampleBufferGetImageBuffer(platformSample.sample.cmSampleBuffer)); >- } >- > ASSERT(findEncoderWithoutLock(identifier)); > auto* encoder = findEncoderWithoutLock(identifier); >- if (!encoder) >+ if (!encoder) { >+ // Let's prevent memory leaks if encoder is gone. >+ if (std::holds_alternative<RemoteVideoFrameReadReference>(frame.buffer)) >+ m_videoFrameObjectHeap->retire(WTFMove(std::get<RemoteVideoFrameReadReference>(frame.buffer)), 0_s); > return; >+ } > > #if !PLATFORM(MACCATALYST) >- if (!pixelBuffer) { >- if (sample.surface()) { >- if (auto buffer = WebCore::createCVPixelBuffer(sample.surface())) >- pixelBuffer = WTFMove(*buffer); >- } else if (encoder->frameReader) >- pixelBuffer = encoder->frameReader->read(); >- >- if (!pixelBuffer) >- return; >- } >+ auto mediaSample = encoder->frameReader->read(WTFMove(frame)); >+ if (!mediaSample) >+ return; > >- webrtc::encodeLocalEncoderFrame(encoder->webrtcEncoder, pixelBuffer.get(), sample.time().toTimeScale(1000000).timeValue(), timeStamp, toWebRTCVideoRotation(sample.rotation()), shouldEncodeAsKeyFrame); >+ webrtc::encodeLocalEncoderFrame(encoder->webrtcEncoder, mediaSample->pixelBuffer(), frame.time.toTimeScale(1000000).timeValue(), timeStamp, toWebRTCVideoRotation(frame.rotation), shouldEncodeAsKeyFrame); > #endif > } > >@@ -284,8 +270,6 @@ void LibWebRTCCodecsProxy::setSharedVideoFrameSemaphore(RTCEncoderIdentifier ide > if (!encoder) > return; > >- if (!encoder->frameReader) >- encoder->frameReader = makeUnique<SharedVideoFrameReader>(Ref { m_videoFrameObjectHeap }); > encoder->frameReader->setSemaphore(WTFMove(semaphore)); > } > >@@ -298,8 +282,6 @@ void LibWebRTCCodecsProxy::setSharedVideoFrameMemory(RTCEncoderIdentifier identi > if (!encoder) > return; > >- if (!encoder->frameReader) >- encoder->frameReader = makeUnique<SharedVideoFrameReader>(Ref { m_videoFrameObjectHeap }); > encoder->frameReader->setSharedMemory(ipcHandle); > } > >diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp >index d1e47ae589ef6de256fdedcef2938f0ca780d689..ed7625a96db2a48dcd460ac73491ff9641720d2c 100644 >--- a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp >+++ b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp >@@ -122,7 +122,6 @@ void RemoteSampleBufferDisplayLayer::pause() > void RemoteSampleBufferDisplayLayer::enqueue(SharedVideoFrame&& frame) > { > auto sample = m_sharedVideoFrameReader.read(WTFMove(frame)); >- ASSERT(sample); > if (!sample) > return; > >diff --git a/Source/WebKit/WebKit.xcodeproj/project.pbxproj b/Source/WebKit/WebKit.xcodeproj/project.pbxproj >index 5956855ac59bf7a5d17b291fa4e1f12e1ea7d5ba..37511e8ab8010f55cccfe43a254e0a2ef1400c05 100644 >--- a/Source/WebKit/WebKit.xcodeproj/project.pbxproj >+++ b/Source/WebKit/WebKit.xcodeproj/project.pbxproj >@@ -846,6 +846,7 @@ > 413075B01DE85F580039EC69 /* WebRTCMonitor.h in Headers */ = {isa = PBXBuildFile; fileRef = 413075A31DE85EE70039EC69 /* WebRTCMonitor.h */; }; > 413075B21DE85F580039EC69 /* LibWebRTCSocketFactory.h in Headers */ = {isa = PBXBuildFile; fileRef = 413075A61DE85EE70039EC69 /* LibWebRTCSocketFactory.h */; }; > 413075B41DE85F580039EC69 /* LibWebRTCProvider.h in Headers */ = {isa = PBXBuildFile; fileRef = 413075A81DE85EE70039EC69 /* LibWebRTCProvider.h */; }; >+ 413C541A27C67C39002FCDA8 /* LibWebRTCCodecsProxy.mm in Sources */ = {isa = PBXBuildFile; fileRef = 41E0A7C823B6397900561060 /* LibWebRTCCodecsProxy.mm */; }; > 4176901422FDD41B00B1576D /* NetworkRTCProvider.mm in Sources */ = {isa = PBXBuildFile; fileRef = 4176901322FDD41B00B1576D /* NetworkRTCProvider.mm */; }; > 417915AF2256BB7500D6F97E /* WebSocketChannel.h in Headers */ = {isa = PBXBuildFile; fileRef = 417915AD2256BB7400D6F97E /* WebSocketChannel.h */; }; > 417915B12256C0D600D6F97E /* WebSocketChannelManager.h in Headers */ = {isa = PBXBuildFile; fileRef = 417915B02256C0D600D6F97E /* WebSocketChannelManager.h */; }; >@@ -16056,6 +16057,7 @@ > buildActionMask = 2147483647; > files = ( > 5CE912102293C277005BEC78 /* AuxiliaryProcessMain.cpp in Sources */, >+ 413C541A27C67C39002FCDA8 /* LibWebRTCCodecsProxy.mm in Sources */, > 5C9A9111228F69CC005C5B17 /* XPCServiceMain.mm in Sources */, > ); > runOnlyForDeploymentPostprocessing = 0; >diff --git a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp >index d18034b46311ad389a6ab6a895f1ca2b18179918..adcaaa6872f8fca239e2e00b54eb3beed69003b6 100644 >--- a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp >+++ b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.cpp >@@ -121,15 +121,6 @@ CVPixelBufferRef RemoteVideoFrameProxy::pixelBuffer() const > } > return m_pixelBuffer.get(); > } >- >-RefPtr<WebCore::VideoFrameCV> RemoteVideoFrameProxy::asVideoFrameCV() >-{ >- auto buffer = pixelBuffer(); >- if (!buffer) >- return nullptr; >- return VideoFrameCV::create(m_presentationTime, m_isMirrored, m_rotation, RetainPtr { buffer }); >-} >- > #endif > > TextStream& operator<<(TextStream& ts, const RemoteVideoFrameProxy::Properties& properties) >diff --git a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h >index dacd88648b12ba617d7038f86d4942d48bd8b3ad..1d82ed9c18ef1cf190770dd8b36c42d9c35f7cc4 100644 >--- a/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h >+++ b/Source/WebKit/WebProcess/GPU/media/RemoteVideoFrameProxy.h >@@ -89,7 +89,6 @@ public: > bool isRemoteProxy() const final { return true; } > #if PLATFORM(COCOA) > CVPixelBufferRef pixelBuffer() const final; >- RefPtr<WebCore::VideoFrameCV> asVideoFrameCV() final; > #endif > > private: >diff --git a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp >index 8afc1ac03225e643d875256b11cb17933812d9a9..7189994d024e28c0e0cb9c3bccc12a9d4518df76 100644 >--- a/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp >+++ b/Source/WebKit/WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp >@@ -457,31 +457,15 @@ int32_t LibWebRTCCodecs::encodeFrame(Encoder& encoder, const webrtc::VideoFrame& > if (!encoder.connection) > return WEBRTC_VIDEO_CODEC_ERROR; > >- std::optional<RemoteVideoFrameReadReference> remoteVideoFrameReadReference; >- if (auto* provider = webrtc::videoFrameBufferProvider(frame)) { >- auto* videoFrame = static_cast<VideoFrame*>(provider); >- if (is<RemoteVideoFrameProxy>(videoFrame)) >- remoteVideoFrameReadReference = downcast<RemoteVideoFrameProxy>(videoFrame)->read(); >- } >- >- RetainPtr<CVPixelBufferRef> buffer; >- if (!remoteVideoFrameReadReference) { >- buffer = adoptCF(webrtc::pixelBufferFromFrame(frame)); >- if (!buffer) { >- // buffer is not native, we need to copy to shared video frame. >- if (!copySharedVideoFrame(encoder, frame)) >- return WEBRTC_VIDEO_CODEC_ERROR; >- } >- } >- >- auto sample = RemoteVideoSample::create(buffer.get(), MediaTime(frame.timestamp_us() * 1000, 1000000), toMediaSampleVideoRotation(frame.rotation()), RemoteVideoSample::ShouldCheckForIOSurface::No); >- if (buffer && !sample->surface()) { >- // buffer is not IOSurface, we need to copy to shared video frame. >- if (!copySharedVideoFrame(encoder, buffer.get())) >- return WEBRTC_VIDEO_CODEC_ERROR; >- } >+ auto sharedVideoFrame = encoder.sharedVideoFrameWriter.write(frame, >+ [&](auto& semaphore) { encoder.connection->send(Messages::LibWebRTCCodecsProxy::SetSharedVideoFrameSemaphore { encoder.identifier, semaphore }, 0); }, >+ [&](auto& handle) { encoder.connection->send(Messages::LibWebRTCCodecsProxy::SetSharedVideoFrameMemory { encoder.identifier, handle }, 0); } >+ ); >+ ASSERT(sharedVideoFrame); >+ if (!sharedVideoFrame) >+ return WEBRTC_VIDEO_CODEC_ERROR; > >- encoder.connection->send(Messages::LibWebRTCCodecsProxy::EncodeFrame { encoder.identifier, *sample, frame.timestamp(), shouldEncodeAsKeyFrame, remoteVideoFrameReadReference }, 0); >+ encoder.connection->send(Messages::LibWebRTCCodecsProxy::EncodeFrame { encoder.identifier, *sharedVideoFrame, frame.timestamp(), shouldEncodeAsKeyFrame }, 0); > return WEBRTC_VIDEO_CODEC_OK; > } > >diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.cpp b/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.cpp >index 8a8ba8966ff29cf4295816a0c718f139949dfc6b..ced566088aae0f5e389c4c754840a36821b51336 100644 >--- a/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.cpp >+++ b/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.cpp >@@ -35,6 +35,7 @@ > #include <WebCore/IOSurface.h> > #include <WebCore/MediaSampleAVFObjC.h> > #include <WebCore/SharedVideoFrameInfo.h> >+#include <WebCore/VideoFrameLibWebRTC.h> > #include <wtf/Scope.h> > > #include <pal/cf/CoreMediaSoftLink.h> >@@ -95,6 +96,13 @@ std::optional<SharedVideoFrame> SharedVideoFrameWriter::write(MediaSample& frame > sharedVideoFrame.buffer = downcast<RemoteVideoFrameProxy>(frame).read(); > return sharedVideoFrame; > } >+#if USE(LIBWEBRTC) >+ if (is<VideoFrameLibWebRTC>(frame)) { >+ if (!write(*downcast<VideoFrameLibWebRTC>(frame).buffer(), newSemaphoreCallback, newMemoryCallback)) >+ return { }; >+ return sharedVideoFrame; >+ } >+#endif > if (is<MediaSampleAVFObjC>(frame)) { > if (auto pixelBuffer = downcast<MediaSampleAVFObjC>(frame).pixelBuffer()) { > if (auto surface = CVPixelBufferGetIOSurface(pixelBuffer)) { >@@ -103,6 +111,7 @@ std::optional<SharedVideoFrame> SharedVideoFrameWriter::write(MediaSample& frame > } > } > } >+ > if (!write(frame.pixelBuffer(), newSemaphoreCallback, newMemoryCallback)) > return { }; > return sharedVideoFrame; >@@ -118,13 +127,30 @@ bool SharedVideoFrameWriter::write(CVPixelBufferRef pixelBuffer, const Function< > } > > #if USE(LIBWEBRTC) >-bool SharedVideoFrameWriter::write(const webrtc::VideoFrame& frame, const Function<void(IPC::Semaphore&)>& newSemaphoreCallback, const Function<void(const SharedMemory::IPCHandle&)>& newMemoryCallback) >+std::optional<SharedVideoFrame> SharedVideoFrameWriter::write(const webrtc::VideoFrame& frame, const Function<void(IPC::Semaphore&)>& newSemaphoreCallback, const Function<void(const SharedMemory::IPCHandle&)>& newMemoryCallback) >+{ >+ if (auto* provider = webrtc::videoFrameBufferProvider(frame)) >+ return write(*static_cast<VideoFrame*>(provider), newSemaphoreCallback, newMemoryCallback); >+ >+ SharedVideoFrame sharedVideoFrame { MediaTime(frame.timestamp_us() * 1000, 1000000), false, MediaSample::VideoRotation::None, nullptr }; >+ if (auto pixelBuffer = adoptCF(webrtc::pixelBufferFromFrame(frame))) { >+ if (!write(pixelBuffer.get(), newSemaphoreCallback, newMemoryCallback)) >+ return { }; >+ return sharedVideoFrame; >+ } >+ >+ if (!write(*frame.video_frame_buffer(), newSemaphoreCallback, newMemoryCallback)) >+ return { }; >+ return sharedVideoFrame; >+} >+ >+bool SharedVideoFrameWriter::write(webrtc::VideoFrameBuffer& frameBuffer, const Function<void(IPC::Semaphore&)>& newSemaphoreCallback, const Function<void(const SharedMemory::IPCHandle&)>& newMemoryCallback) > { >- auto info = SharedVideoFrameInfo::fromVideoFrame(frame); >+ auto info = SharedVideoFrameInfo::fromVideoFrame(frameBuffer); > if (!prepareWriting(info, newSemaphoreCallback, newMemoryCallback)) > return { }; > >- return info.writeVideoFrame(frame, static_cast<uint8_t*>(m_storage->data())); >+ return info.writeVideoFrameBuffer(frameBuffer, static_cast<uint8_t*>(m_storage->data())); > } > #endif > >@@ -176,9 +202,9 @@ RefPtr<MediaSample> SharedVideoFrameReader::read(SharedVideoFrame&& sharedVideoF > return nullptr; > > auto sample = m_objectHeap->retire(WTFMove(reference), 0_s); >- ASSERT(sample && sample->pixelBuffer()); > if (!sample) > return nullptr; >+ ASSERT(sample->pixelBuffer()); > return sample->pixelBuffer(); > } , [](MachSendRight&& sendRight) -> RetainPtr<CVPixelBufferRef> { > auto surface = WebCore::IOSurface::createFromSendRight(WTFMove(sendRight), DestinationColorSpace::SRGB()); >diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.h b/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.h >index c46b7e27ff1a7c6cd58d470d425365b1e2331a6b..efefa7901616d1e4a6fe2f6538e60025e0e263aa 100644 >--- a/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.h >+++ b/Source/WebKit/WebProcess/GPU/webrtc/SharedVideoFrame.h >@@ -44,6 +44,7 @@ class SharedVideoFrameInfo; > > namespace webrtc { > class VideoFrame; >+class VideoFrameBuffer; > } > > namespace WebKit { >@@ -66,10 +67,10 @@ public: > SharedVideoFrameWriter(); > > std::optional<SharedVideoFrame> write(WebCore::MediaSample&, const Function<void(IPC::Semaphore&)>&, const Function<void(const SharedMemory::IPCHandle&)>&); >- bool write(CVPixelBufferRef, const Function<void(IPC::Semaphore&)>&, const Function<void(const SharedMemory::IPCHandle&)>&); > #if USE(LIBWEBRTC) >- bool write(const webrtc::VideoFrame&, const Function<void(IPC::Semaphore&)>&, const Function<void(const SharedMemory::IPCHandle&)>&); >+ std::optional<SharedVideoFrame> write(const webrtc::VideoFrame&, const Function<void(IPC::Semaphore&)>&, const Function<void(const SharedMemory::IPCHandle&)>&); > #endif >+ bool write(CVPixelBufferRef, const Function<void(IPC::Semaphore&)>&, const Function<void(const SharedMemory::IPCHandle&)>&); > void disable(); > > private: >@@ -77,6 +78,10 @@ private: > bool allocateStorage(size_t, const Function<void(const SharedMemory::IPCHandle&)>&); > bool prepareWriting(const WebCore::SharedVideoFrameInfo&, const Function<void(IPC::Semaphore&)>&, const Function<void(const SharedMemory::IPCHandle&)>&); > >+#if USE(LIBWEBRTC) >+ bool write(webrtc::VideoFrameBuffer&, const Function<void(IPC::Semaphore&)>&, const Function<void(const SharedMemory::IPCHandle&)>&); >+#endif >+ > UniqueRef<IPC::Semaphore> m_semaphore; > RefPtr<SharedMemory> m_storage; > bool m_isSemaphoreInUse { false };
<b>You cannot view the attachment while viewing its details because your browser does not support IFRAMEs. <a href="attachment.cgi?id=452982">View the attachment on a separate page</a>.</b>
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 236131
:
450881
|
452864
|
452982
|
452998
|
453374
|
453720