WebKit Bugzilla
Attachment 349772 Details for
Bug 178109
: Support arbitrary video resolution in getUserMedia API
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Patch for landing
bug-178109-20180914102807.patch (text/plain), 131.52 KB, created by
Eric Carlson
on 2018-09-14 10:28:09 PDT
(
hide
)
Description:
Patch for landing
Filename:
MIME Type:
Creator:
Eric Carlson
Created:
2018-09-14 10:28:09 PDT
Size:
131.52 KB
patch
obsolete
>Subversion Revision: 236010 >diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog >index 99eb5ecebb3a84fe7979b3d9585d86f6b4589bd0..6fa42962979765f676db3ecc10c768960c7fa789 100644 >--- a/Source/WebCore/ChangeLog >+++ b/Source/WebCore/ChangeLog >@@ -1,3 +1,148 @@ >+2018-09-12 Eric Carlson <eric.carlson@apple.com> >+ >+ Support arbitrary video resolution in getUserMedia API >+ https://bugs.webkit.org/show_bug.cgi?id=178109 >+ <rdar://problem/35083128> >+ >+ Reviewed by Youenn Fablet. >+ >+ Support arbitrary video resolutions by configuring the camera to capture at the closest >+ larger size it supports and scaling/cropping frames as necessary. >+ >+ No new tests, existing tests updated. >+ >+ * Modules/mediastream/CanvasCaptureMediaStreamTrack.h: >+ * Modules/webaudio/MediaStreamAudioSource.cpp: >+ (WebCore::MediaStreamAudioSource::capabilities): Make non-const, it wasn't helpful. >+ (WebCore::MediaStreamAudioSource::settings): Ditto. >+ (WebCore::MediaStreamAudioSource::capabilities const): Deleted. >+ (WebCore::MediaStreamAudioSource::settings const): Deleted. >+ * Modules/webaudio/MediaStreamAudioSource.h: >+ >+ * SourcesCocoa.txt: Add PixelBufferResizer. >+ >+ * WebCore.xcodeproj/project.pbxproj: Ditto. >+ >+ * platform/cocoa/CoreVideoSoftLink.h: >+ * platform/cocoa/VideoToolboxSoftLink.cpp: >+ * platform/cocoa/VideoToolboxSoftLink.h: >+ >+ * platform/graphics/cv/PixelBufferResizer.h: Added. >+ (WebCore::PixelBufferResizer::canResizeTo): >+ * platform/graphics/cv/PixelBufferResizer.mm: Added. >+ (WebCore::PixelBufferResizer::PixelBufferResizer): >+ (WebCore::PixelBufferResizer::resize): >+ >+ * platform/mediastream/RealtimeIncomingAudioSource.cpp: >+ (WebCore::RealtimeIncomingAudioSource::capabilities): Make non-const, it wasn't helpful. >+ (WebCore::RealtimeIncomingAudioSource::settings): Ditto. >+ (WebCore::RealtimeIncomingAudioSource::capabilities const): Deleted. >+ (WebCore::RealtimeIncomingAudioSource::settings const): Deleted. >+ * platform/mediastream/RealtimeIncomingAudioSource.h: >+ >+ * platform/mediastream/RealtimeIncomingVideoSource.cpp: >+ (WebCore::RealtimeIncomingVideoSource::capabilities): Make non-const, it wasn't helpful. >+ (WebCore::RealtimeIncomingVideoSource::settings): Ditto. >+ (WebCore::RealtimeIncomingVideoSource::capabilities const): Deleted. >+ (WebCore::RealtimeIncomingVideoSource::settings const): Deleted. >+ * platform/mediastream/RealtimeIncomingVideoSource.h: >+ >+ * platform/mediastream/RealtimeMediaSource.cpp: >+ (WebCore::RealtimeMediaSource::supportsConstraint): >+ (WebCore::RealtimeMediaSource::supportsConstraint const): Deleted. >+ * platform/mediastream/RealtimeMediaSource.h: >+ >+ * platform/mediastream/RealtimeVideoSource.cpp: >+ (WebCore::RealtimeVideoSource::presets): >+ (WebCore::RealtimeVideoSource::setSupportedPresets): >+ (WebCore::standardVideoSizes): >+ (WebCore::RealtimeVideoSource::updateCapabilities): Make non-const, it wasn't helpful. >+ (WebCore::presetSupportsFrameRate): >+ (WebCore::RealtimeVideoSource::supportsCaptureSize): >+ (WebCore::RealtimeVideoSource::shouldUsePreset): >+ (WebCore::RealtimeVideoSource::bestSupportedSizeAndFrameRate): >+ (WebCore::RealtimeVideoSource::setSizeAndFrameRate): >+ (WebCore::RealtimeVideoSource::addSupportedCapabilities const): Deleted. >+ * platform/mediastream/RealtimeVideoSource.h: >+ (WebCore::VideoPresetData::encode const): >+ (WebCore::VideoPresetData::decode): >+ (WebCore::VideoPreset::create): >+ (WebCore::VideoPreset::VideoPreset): >+ (WebCore::RealtimeVideoSource::prefersPreset): >+ (WebCore::RealtimeVideoSource::canResizeVideoFrames const): >+ (WebCore::RealtimeVideoSource::setDefaultSize): >+ (WebCore::RealtimeVideoSource::observedFrameRate const): >+ (WebCore::VideoPreset::encode const): Deleted. >+ (WebCore::VideoPreset::decode): Deleted. >+ >+ * platform/mediastream/mac/AVVideoCaptureSource.h: >+ (WebCore::AVVideoCaptureSource::videoPresets): Deleted. >+ * platform/mediastream/mac/AVVideoCaptureSource.mm: >+ (WebCore::AVVideoPreset::create): >+ (WebCore::AVVideoPreset::AVVideoPreset): >+ (WebCore::AVVideoCaptureSource::AVVideoCaptureSource): >+ (WebCore::AVVideoCaptureSource::settings): >+ (WebCore::AVVideoCaptureSource::capabilities): Make non-const, it wasn't helpful. >+ (WebCore::AVVideoCaptureSource::setFrameRate): >+ (WebCore::AVVideoCaptureSource::prefersPreset): >+ (WebCore::AVVideoCaptureSource::setSizeAndFrameRateWithPreset): >+ (WebCore::AVVideoCaptureSource::frameDurationForFrameRate): >+ (WebCore::AVVideoCaptureSource::setupCaptureSession): >+ (WebCore::AVVideoCaptureSource::processNewFrame): >+ (WebCore::AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection): >+ (WebCore::AVVideoCaptureSource::isFrameRateSupported): >+ (WebCore::AVVideoCaptureSource::generatePresets): >+ (WebCore::updateSizeMinMax): Deleted. >+ (WebCore::updateAspectRatioMinMax): Deleted. >+ (WebCore::AVVideoCaptureSource::settings const): Deleted. >+ (WebCore::AVVideoCaptureSource::capabilities const): Deleted. >+ (WebCore::AVVideoCaptureSource::sizeForPreset): Deleted. >+ (WebCore::AVVideoCaptureSource::setPreset): Deleted. >+ (WebCore::AVVideoCaptureSource::setSizeAndFrameRate): Deleted. >+ (WebCore::AVVideoCaptureSource::bestSessionPresetForVideoDimensions): Deleted. >+ (WebCore::AVVideoCaptureSource::supportsSizeAndFrameRate): Deleted. >+ >+ * platform/mediastream/mac/CoreAudioCaptureSource.cpp: >+ (WebCore::CoreAudioCaptureSource::capabilities): Make non-const, it wasn't helpful. >+ (WebCore::CoreAudioCaptureSource::settings): Ditto. >+ (WebCore::CoreAudioCaptureSource::capabilities const): Deleted. >+ (WebCore::CoreAudioCaptureSource::settings const): Deleted. >+ * platform/mediastream/mac/CoreAudioCaptureSource.h: >+ >+ * platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp: >+ (WebCore::DisplayCaptureSourceCocoa::capabilities): Make non-const, it wasn't helpful. >+ (WebCore::DisplayCaptureSourceCocoa::settings): Ditto. >+ (WebCore::DisplayCaptureSourceCocoa::capabilities const): Deleted. >+ (WebCore::DisplayCaptureSourceCocoa::settings const): Deleted. >+ * platform/mediastream/mac/DisplayCaptureSourceCocoa.h: >+ >+ * platform/mediastream/mac/MockRealtimeVideoSourceMac.h: >+ * platform/mediastream/mac/MockRealtimeVideoSourceMac.mm: >+ (WebCore::MockRealtimeVideoSourceMac::CMSampleBufferFromPixelBuffer): >+ (WebCore::MockRealtimeVideoSourceMac::updateSampleBuffer): >+ (WebCore::MockRealtimeVideoSourceMac::setSizeAndFrameRateWithPreset): >+ * platform/mock/MockMediaDevice.h: >+ (WebCore::MockCameraProperties::decode): >+ >+ * platform/mock/MockRealtimeAudioSource.cpp: >+ (WebCore::MockRealtimeAudioSource::settings): Make non-const, it wasn't helpful. >+ (WebCore::MockRealtimeAudioSource::capabilities): Ditto. >+ (WebCore::MockRealtimeAudioSource::settings const): Deleted. >+ (WebCore::MockRealtimeAudioSource::capabilities const): Deleted. >+ * platform/mock/MockRealtimeAudioSource.h: >+ >+ * platform/mock/MockRealtimeMediaSourceCenter.cpp: >+ (WebCore::defaultDevices): Change video device presets to trigger resize code more often. >+ >+ * platform/mock/MockRealtimeVideoSource.cpp: >+ (WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource): >+ (WebCore::MockRealtimeVideoSource::generatePresets): >+ (WebCore::MockRealtimeVideoSource::capabilities): Make non-const, it wasn't helpful. >+ (WebCore::MockRealtimeVideoSource::settings): Ditto. >+ (WebCore::MockRealtimeVideoSource::capabilities const): Deleted. >+ (WebCore::MockRealtimeVideoSource::settings const): Deleted. >+ * platform/mock/MockRealtimeVideoSource.h: >+ > 2018-09-14 Jer Noble <jer.noble@apple.com> > > Turn SourceBufferChangeTypeEnabled on by default >diff --git a/Source/WebCore/PAL/ChangeLog b/Source/WebCore/PAL/ChangeLog >index d038276a5058b1e8425bb9a88c6b5b2a0a3d2c5b..67bc21816cdbb1b1d631c043a0bef87dff43ee2a 100644 >--- a/Source/WebCore/PAL/ChangeLog >+++ b/Source/WebCore/PAL/ChangeLog >@@ -1,3 +1,14 @@ >+2018-09-12 Eric Carlson <eric.carlson@apple.com> >+ >+ Support arbitrary video resolution in getUserMedia API >+ https://bugs.webkit.org/show_bug.cgi?id=178109 >+ <rdar://problem/35083128> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ * pal/cf/CoreMediaSoftLink.cpp: >+ * pal/cf/CoreMediaSoftLink.h: >+ > 2018-09-12 Guillaume Emont <guijemont@igalia.com> > > Add IGNORE_WARNING_.* macros >diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog >index 666db6954d693ab2dbe80d08208c81299af865b0..97fdc1ebaa612341ba52a50e2b7896008265d3c8 100644 >--- a/Source/WebKit/ChangeLog >+++ b/Source/WebKit/ChangeLog >@@ -1,3 +1,13 @@ >+2018-09-12 Eric Carlson <eric.carlson@apple.com> >+ >+ Support arbitrary video resolution in getUserMedia API >+ https://bugs.webkit.org/show_bug.cgi?id=178109 >+ <rdar://problem/35083128> >+ >+ Reviewed by Youenn Fablet. >+ >+ * WebProcess/cocoa/UserMediaCaptureManager.cpp: >+ > 2018-09-14 Jer Noble <jer.noble@apple.com> > > Turn SourceBufferChangeTypeEnabled on by default >diff --git a/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.h b/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.h >index b006af93bf4d11253ff7c16d2376c0ae1c34c4ae..b2290591d0b34fbea77ff5e5c7352bbfc01a0989 100644 >--- a/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.h >+++ b/Source/WebCore/Modules/mediastream/CanvasCaptureMediaStreamTrack.h >@@ -65,8 +65,8 @@ private: > // RealtimeMediaSource API > void startProducingData() final; > void stopProducingData() final; >- const RealtimeMediaSourceCapabilities& capabilities() const final { return RealtimeMediaSourceCapabilities::emptyCapabilities(); } >- const RealtimeMediaSourceSettings& settings() const final { return m_settings; } >+ const RealtimeMediaSourceCapabilities& capabilities() final { return RealtimeMediaSourceCapabilities::emptyCapabilities(); } >+ const RealtimeMediaSourceSettings& settings() final { return m_settings; } > > void captureCanvas(); > void requestFrameTimerFired(); >diff --git a/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp b/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp >index d797b67c05dfeaf61d94bcabff0a23850af36e37..56f37c1166b6bf9e863c429678709c48bc602624 100644 >--- a/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp >+++ b/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.cpp >@@ -39,7 +39,7 @@ MediaStreamAudioSource::MediaStreamAudioSource(float sampleRate) > m_currentSettings.setSampleRate(sampleRate); > } > >-const RealtimeMediaSourceCapabilities& MediaStreamAudioSource::capabilities() const >+const RealtimeMediaSourceCapabilities& MediaStreamAudioSource::capabilities() > { > // FIXME: implement this. > // https://bugs.webkit.org/show_bug.cgi?id=122430 >@@ -47,7 +47,7 @@ const RealtimeMediaSourceCapabilities& MediaStreamAudioSource::capabilities() co > return RealtimeMediaSourceCapabilities::emptyCapabilities(); > } > >-const RealtimeMediaSourceSettings& MediaStreamAudioSource::settings() const >+const RealtimeMediaSourceSettings& MediaStreamAudioSource::settings() > { > // FIXME: implement this. > // https://bugs.webkit.org/show_bug.cgi?id=122430 >diff --git a/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h b/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h >index 4b90c8af6e2079970534fb8121b74ca8c1adbaca..fc6ddb8e6d2b87036d262138ded4c6c77796be9e 100644 >--- a/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h >+++ b/Source/WebCore/Modules/webaudio/MediaStreamAudioSource.h >@@ -43,8 +43,8 @@ public: > > ~MediaStreamAudioSource() = default; > >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > > const String& deviceId() const { return m_deviceId; } > void setDeviceId(const String& deviceId) { m_deviceId = deviceId; } >diff --git a/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp b/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp >index b60a645fb1d763505c33b30113675ba9c8188e20..bda1bf825d941446a8cba7a0cca09f94ad634f24 100644 >--- a/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp >+++ b/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.cpp >@@ -97,6 +97,7 @@ SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreate, OSStatus, (C > SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateCopy, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef sbuf, CMSampleBufferRef *sbufCopyOut), (allocator, sbuf, sbufCopyOut)) > SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateCopyWithNewTiming, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef originalSBuf, CMItemCount numSampleTimingEntries, const CMSampleTimingInfo *sampleTimingArray, CMSampleBufferRef *sBufCopyOut), (allocator, originalSBuf, numSampleTimingEntries, sampleTimingArray, sBufCopyOut)) > SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateReadyWithImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo* sampleTiming, CMSampleBufferRef* sBufOut), (allocator, imageBuffer, formatDescription, sampleTiming, sBufOut)) >+SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferCreateForImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, Boolean dataReady, CMSampleBufferMakeDataReadyCallback makeDataReadyCallback, void* makeDataReadyRefcon, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo* sampleTiming, CMSampleBufferRef* sampleBufferOut), (allocator, imageBuffer, dataReady, makeDataReadyCallback, makeDataReadyRefcon, formatDescription, sampleTiming, sampleBufferOut)) > SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetDecodeTimeStamp, CMTime, (CMSampleBufferRef sbuf), (sbuf)) > SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetDuration, CMTime, (CMSampleBufferRef sbuf), (sbuf)) > SOFT_LINK_FUNCTION_FOR_SOURCE(PAL, CoreMedia, CMSampleBufferGetImageBuffer, CVImageBufferRef, (CMSampleBufferRef sbuf), (sbuf)) >diff --git a/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h b/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h >index e95fbd32b8bf6201da239c96e4fe50b6f0cdab0b..1a67a3ce8c1bf3fbb0b56a447f1dc686a8d47a9e 100644 >--- a/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h >+++ b/Source/WebCore/PAL/pal/cf/CoreMediaSoftLink.h >@@ -151,6 +151,8 @@ SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCreateCopyWithNewTim > #define CMSampleBufferCreateCopyWithNewTiming softLink_CoreMedia_CMSampleBufferCreateCopyWithNewTiming > SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCreateReadyWithImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo *sampleTiming, CMSampleBufferRef *sBufOut), (allocator, imageBuffer, formatDescription, sampleTiming, sBufOut)) > #define CMSampleBufferCreateReadyWithImageBuffer softLink_CoreMedia_CMSampleBufferCreateReadyWithImageBuffer >+SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferCreateForImageBuffer, OSStatus, (CFAllocatorRef allocator, CVImageBufferRef imageBuffer, Boolean dataReady, CMSampleBufferMakeDataReadyCallback makeDataReadyCallback, void* makeDataReadyRefcon, CMVideoFormatDescriptionRef formatDescription, const CMSampleTimingInfo* sampleTiming, CMSampleBufferRef* sampleBufferOut), (allocator, imageBuffer, dataReady, makeDataReadyCallback, makeDataReadyRefcon, formatDescription, sampleTiming, sampleBufferOut)) >+#define CMSampleBufferCreateForImageBuffer softLink_CoreMedia_CMSampleBufferCreateForImageBuffer > SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferGetDecodeTimeStamp, CMTime, (CMSampleBufferRef sbuf), (sbuf)) > #define CMSampleBufferGetDecodeTimeStamp softLink_CoreMedia_CMSampleBufferGetDecodeTimeStamp > SOFT_LINK_FUNCTION_FOR_HEADER(PAL, CoreMedia, CMSampleBufferGetDuration, CMTime, (CMSampleBufferRef sbuf), (sbuf)) >diff --git a/Source/WebCore/SourcesCocoa.txt b/Source/WebCore/SourcesCocoa.txt >index bc6565a152600b09d4ca0d92a81c998e36a84583..8033a36a1e645a4d4bc516ac8fe7a56f1e419f49 100644 >--- a/Source/WebCore/SourcesCocoa.txt >+++ b/Source/WebCore/SourcesCocoa.txt >@@ -311,6 +311,7 @@ platform/graphics/cocoa/WebGPULayer.mm > platform/graphics/cocoa/TextTrackRepresentationCocoa.mm > > platform/graphics/cv/PixelBufferConformerCV.cpp >+platform/graphics/cv/PixelBufferResizer.mm > platform/graphics/cv/TextureCacheCV.mm > platform/graphics/cv/VideoTextureCopierCV.cpp > >diff --git a/Source/WebCore/WebCore.xcodeproj/project.pbxproj b/Source/WebCore/WebCore.xcodeproj/project.pbxproj >index a0216bc5ef1deb829c9e1c4b2948fc8c8bc42aa5..a0398a551a7a2ba1b3506087a976772e0bc4959d 100644 >--- a/Source/WebCore/WebCore.xcodeproj/project.pbxproj >+++ b/Source/WebCore/WebCore.xcodeproj/project.pbxproj >@@ -120,6 +120,7 @@ > 074E82BB18A69F0E007EF54C /* PlatformTimeRanges.h in Headers */ = {isa = PBXBuildFile; fileRef = 074E82B918A69F0E007EF54C /* PlatformTimeRanges.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 0753860214489E9800B78452 /* CachedTextTrack.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 0753860014489E9800B78452 /* CachedTextTrack.cpp */; }; > 0753860314489E9800B78452 /* CachedTextTrack.h in Headers */ = {isa = PBXBuildFile; fileRef = 0753860114489E9800B78452 /* CachedTextTrack.h */; }; >+ 0757B13E214AE79900794B0D /* VideoPreset.h in Headers */ = {isa = PBXBuildFile; fileRef = 0757B13C214AE79700794B0D /* VideoPreset.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 075BA84920618AA500FCB4AD /* VideoFullscreenLayerManager.h in Headers */ = {isa = PBXBuildFile; fileRef = 075BA84720618AA500FCB4AD /* VideoFullscreenLayerManager.h */; }; > 07638A991884487200E15A1B /* MediaSessionManagerIOS.h in Headers */ = {isa = PBXBuildFile; fileRef = 07638A971884487200E15A1B /* MediaSessionManagerIOS.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 07638A9A1884487200E15A1B /* MediaSessionManagerIOS.mm in Sources */ = {isa = PBXBuildFile; fileRef = 07638A981884487200E15A1B /* MediaSessionManagerIOS.mm */; }; >@@ -5333,10 +5334,13 @@ > 073B87651E43859D0071C0EC /* AudioSampleDataSource.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioSampleDataSource.h; sourceTree = "<group>"; }; > 0744ECEB1E0C4AE5000D0944 /* MockRealtimeAudioSourceMac.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MockRealtimeAudioSourceMac.h; sourceTree = "<group>"; }; > 0744ECEC1E0C4AE5000D0944 /* MockRealtimeAudioSourceMac.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MockRealtimeAudioSourceMac.mm; sourceTree = "<group>"; }; >+ 0746D30A2146EA37003DDF84 /* PixelBufferResizer.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = PixelBufferResizer.mm; sourceTree = "<group>"; }; >+ 0746D30C2146EA38003DDF84 /* PixelBufferResizer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PixelBufferResizer.h; sourceTree = "<group>"; }; > 074E82B818A69F0E007EF54C /* PlatformTimeRanges.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PlatformTimeRanges.cpp; sourceTree = "<group>"; }; > 074E82B918A69F0E007EF54C /* PlatformTimeRanges.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PlatformTimeRanges.h; sourceTree = "<group>"; }; > 0753860014489E9800B78452 /* CachedTextTrack.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CachedTextTrack.cpp; sourceTree = "<group>"; }; > 0753860114489E9800B78452 /* CachedTextTrack.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CachedTextTrack.h; sourceTree = "<group>"; }; >+ 0757B13C214AE79700794B0D /* VideoPreset.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoPreset.h; sourceTree = "<group>"; }; > 075BA84720618AA500FCB4AD /* VideoFullscreenLayerManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = VideoFullscreenLayerManager.h; sourceTree = "<group>"; }; > 07638A971884487200E15A1B /* MediaSessionManagerIOS.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaSessionManagerIOS.h; sourceTree = "<group>"; }; > 07638A981884487200E15A1B /* MediaSessionManagerIOS.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = MediaSessionManagerIOS.mm; sourceTree = "<group>"; }; >@@ -15301,6 +15305,7 @@ > 07221BAD17CF0AD400848E51 /* RTCSessionDescriptionRequest.h */, > 313591051E7DDC6000F30630 /* RTCSignalingState.h */, > 07221BB017CF0AD400848E51 /* RTCVoidRequest.h */, >+ 0757B13C214AE79700794B0D /* VideoPreset.h */, > 070E81D01BF27656001FDA48 /* VideoTrackPrivateMediaStream.h */, > 41B2A6251EF1BF60002B9D7A /* WebAudioSourceProvider.h */, > ); >@@ -24781,6 +24786,8 @@ > children = ( > CD7D33411C7A123F00041293 /* PixelBufferConformerCV.cpp */, > CD7D33421C7A123F00041293 /* PixelBufferConformerCV.h */, >+ 0746D30C2146EA38003DDF84 /* PixelBufferResizer.h */, >+ 0746D30A2146EA37003DDF84 /* PixelBufferResizer.mm */, > CD9D82741C7AE535006FF066 /* TextureCacheCV.h */, > CD9D82731C7AE535006FF066 /* TextureCacheCV.mm */, > CD9D82771C7B8EE1006FF066 /* VideoTextureCopierCV.cpp */, >@@ -30861,6 +30868,7 @@ > 52D5A1A71C57489D00DE34A3 /* VideoFullscreenModel.h in Headers */, > 52D5A1A81C5748A300DE34A3 /* VideoFullscreenModelVideoElement.h in Headers */, > CDE83DB2183C44060031EAA3 /* VideoPlaybackQuality.h in Headers */, >+ 0757B13E214AE79900794B0D /* VideoPreset.h in Headers */, > CD9D827A1C7B8EE1006FF066 /* VideoTextureCopierCV.h in Headers */, > CDC939A81E9BDFB100BB768D /* VideoToolboxSoftLink.h in Headers */, > BE88E0DF1715D2A200658D98 /* VideoTrack.h in Headers */, >diff --git a/Source/WebCore/platform/cocoa/CoreVideoSoftLink.h b/Source/WebCore/platform/cocoa/CoreVideoSoftLink.h >index a213c7c927c5fca1a35e64268492629a3399a45f..a7d870e0d88e5479c48aa110c86e9e0de5ae4da1 100644 >--- a/Source/WebCore/platform/cocoa/CoreVideoSoftLink.h >+++ b/Source/WebCore/platform/cocoa/CoreVideoSoftLink.h >@@ -62,6 +62,8 @@ SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferPoolCreatePixelBu > #define CVPixelBufferPoolCreatePixelBuffer softLink_CoreVideo_CVPixelBufferPoolCreatePixelBuffer > SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferGetIOSurface, IOSurfaceRef, (CVPixelBufferRef pixelBuffer), (pixelBuffer)) > #define CVPixelBufferGetIOSurface softLink_CoreVideo_CVPixelBufferGetIOSurface >+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreVideo, CVPixelBufferPoolGetPixelBufferAttributes, CFDictionaryRef, (CVPixelBufferPoolRef pool), (pool)) >+#define CVPixelBufferPoolGetPixelBufferAttributes softLink_CoreVideo_CVPixelBufferPoolGetPixelBufferAttributes > > SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, CoreVideo, kCVPixelBufferPixelFormatTypeKey, CFStringRef) > #define kCVPixelBufferPixelFormatTypeKey get_CoreVideo_kCVPixelBufferPixelFormatTypeKey() >diff --git a/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.cpp b/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.cpp >index 23c4978ca7afb0f9fa6a5547772ad9fed65f1435..23287aae3f3303c2ac3c5e64a82d9649ddb13810 100644 >--- a/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.cpp >+++ b/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.cpp >@@ -29,6 +29,7 @@ > #include <wtf/SoftLinking.h> > > typedef struct OpaqueVTImageRotationSession* VTImageRotationSessionRef; >+typedef struct OpaqueVTPixelTransferSession* VTPixelTransferSessionRef; > > SOFT_LINK_FRAMEWORK_FOR_SOURCE(WebCore, VideoToolbox) > >@@ -50,3 +51,9 @@ SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTDecompressionPropertyKey > SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTImageRotationPropertyKey_EnableHighSpeedTransfer, CFStringRef) > SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTImageRotationPropertyKey_FlipHorizontalOrientation, CFStringRef) > SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTImageRotationPropertyKey_FlipVerticalOrientation, CFStringRef) >+ >+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, VideoToolbox, VTPixelTransferSessionCreate, OSStatus, (CFAllocatorRef allocator, VTPixelTransferSessionRef* pixelTransferSessionOut), (allocator, pixelTransferSessionOut)) >+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, VideoToolbox, VTPixelTransferSessionTransferImage, OSStatus, (VTPixelTransferSessionRef session, CVPixelBufferRef sourceBuffer, CVPixelBufferRef destinationBuffer), (session, sourceBuffer, destinationBuffer)) >+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, VideoToolbox, VTSessionSetProperty, OSStatus, (VTSessionRef session, CFStringRef propertyKey, CFTypeRef propertyValue), (session, propertyKey, propertyValue)) >+SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTPixelTransferPropertyKey_ScalingMode, CFStringRef) >+SOFT_LINK_CONSTANT_FOR_SOURCE(WebCore, VideoToolbox, kVTScalingMode_Trim, CFStringRef) >diff --git a/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.h b/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.h >index 8f46a33ee89d7ccce09e7f67a7e22f0312a83829..7215d3aea34e742638db2f8b299ad77c788291de 100644 >--- a/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.h >+++ b/Source/WebCore/platform/cocoa/VideoToolboxSoftLink.h >@@ -29,6 +29,7 @@ > #include <wtf/SoftLinking.h> > > typedef struct OpaqueVTImageRotationSession* VTImageRotationSessionRef; >+typedef struct OpaqueVTPixelTransferSession* VTPixelTransferSessionRef; > > SOFT_LINK_FRAMEWORK_FOR_HEADER(WebCore, VideoToolbox) > >@@ -68,3 +69,15 @@ SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTImageRotationPropertyKey > #define kVTImageRotationPropertyKey_FlipHorizontalOrientation get_VideoToolbox_kVTImageRotationPropertyKey_FlipHorizontalOrientation() > SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTImageRotationPropertyKey_FlipVerticalOrientation, CFStringRef) > #define kVTImageRotationPropertyKey_FlipVerticalOrientation get_VideoToolbox_kVTImageRotationPropertyKey_FlipVerticalOrientation() >+ >+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, VideoToolbox, VTPixelTransferSessionCreate, OSStatus, (CFAllocatorRef allocator, VTPixelTransferSessionRef* pixelTransferSessionOut), (allocator, pixelTransferSessionOut)) >+#define VTPixelTransferSessionCreate softLink_VideoToolbox_VTPixelTransferSessionCreate >+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, VideoToolbox, VTPixelTransferSessionTransferImage, OSStatus, (VTPixelTransferSessionRef session, CVPixelBufferRef sourceBuffer, CVPixelBufferRef destinationBuffer), (session, sourceBuffer, destinationBuffer)) >+#define VTPixelTransferSessionTransferImage softLink_VideoToolbox_VTPixelTransferSessionTransferImage >+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, VideoToolbox, VTSessionSetProperty, OSStatus, (VTSessionRef session, CFStringRef propertyKey, CFTypeRef propertyValue), (session, propertyKey, propertyValue)) >+#define VTSessionSetProperty softLink_VideoToolbox_VTSessionSetProperty >+ >+SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTPixelTransferPropertyKey_ScalingMode, CFStringRef) >+#define kVTPixelTransferPropertyKey_ScalingMode get_VideoToolbox_kVTPixelTransferPropertyKey_ScalingMode() >+SOFT_LINK_CONSTANT_FOR_HEADER(WebCore, VideoToolbox, kVTScalingMode_Trim, CFStringRef) >+#define kVTScalingMode_Trim get_VideoToolbox_kVTScalingMode_Trim() >diff --git a/Source/WebCore/platform/graphics/cv/PixelBufferResizer.h b/Source/WebCore/platform/graphics/cv/PixelBufferResizer.h >new file mode 100644 >index 0000000000000000000000000000000000000000..db19536338441dde26717ff1453c793e8e8c7597 >--- /dev/null >+++ b/Source/WebCore/platform/graphics/cv/PixelBufferResizer.h >@@ -0,0 +1,55 @@ >+/* >+ * Copyright (C) 2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if USE(VIDEOTOOLBOX) >+ >+#include <wtf/RetainPtr.h> >+ >+typedef struct OpaqueVTPixelTransferSession* VTPixelTransferSessionRef; >+typedef struct __CVBuffer *CVPixelBufferRef; >+typedef struct __CVPixelBufferPool *CVPixelBufferPoolRef; >+typedef struct opaqueCMSampleBuffer *CMSampleBufferRef; >+ >+namespace WebCore { >+ >+class PixelBufferResizer { >+public: >+ PixelBufferResizer(IntSize, OSType); >+ >+ RetainPtr<CVPixelBufferRef> resize(CVPixelBufferRef); >+ RetainPtr<CMSampleBufferRef> resize(CMSampleBufferRef); >+ bool canResizeTo(IntSize size) { return size == m_size; } >+ >+private: >+ RetainPtr<VTPixelTransferSessionRef> m_transferSession; >+ RetainPtr<CVPixelBufferPoolRef> m_bufferPool; >+ IntSize m_size; >+}; >+ >+} >+ >+#endif // USE(VIDEOTOOLBOX) >diff --git a/Source/WebCore/platform/graphics/cv/PixelBufferResizer.mm b/Source/WebCore/platform/graphics/cv/PixelBufferResizer.mm >new file mode 100644 >index 0000000000000000000000000000000000000000..a53ab1af3b24b316ccebf6f424ed3e1a060e39ab >--- /dev/null >+++ b/Source/WebCore/platform/graphics/cv/PixelBufferResizer.mm >@@ -0,0 +1,133 @@ >+/* >+ * Copyright (C) 2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+#include "PixelBufferResizer.h" >+ >+#if USE(VIDEOTOOLBOX) >+ >+#include <wtf/SoftLinking.h> >+ >+#include "CoreVideoSoftLink.h" >+#include "VideoToolboxSoftLink.h" >+ >+namespace WebCore { >+ >+PixelBufferResizer::PixelBufferResizer(IntSize size, OSType videoFormat) >+{ >+ VTPixelTransferSessionRef transferSession; >+ VTPixelTransferSessionCreate(NULL, &transferSession); >+ ASSERT(transferSession); >+ m_transferSession = adoptCF(transferSession); >+ >+ VTSessionSetProperty(transferSession, kVTPixelTransferPropertyKey_ScalingMode, kVTScalingMode_Trim); >+ >+ CFDictionaryRef sourcePixelBufferOptions = (__bridge CFDictionaryRef) @{ >+ (__bridge NSString *)kCVPixelBufferWidthKey :@(size.width()), >+ (__bridge NSString *)kCVPixelBufferHeightKey:@(size.height()), >+ (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey:@(videoFormat), >+#if PLATFORM(IOS) >+ (__bridge NSString *)kCVPixelFormatOpenGLESCompatibility : @(YES), >+#else >+ (__bridge NSString *)kCVPixelBufferOpenGLCompatibilityKey : @(YES), >+#endif >+ (__bridge NSString *)kCVPixelBufferIOSurfacePropertiesKey : @{ /*empty dictionary*/ }, >+ }; >+ CVPixelBufferPoolRef bufferPool; >+ auto status = CVPixelBufferPoolCreate(kCFAllocatorDefault, NULL, sourcePixelBufferOptions, &bufferPool); >+ ASSERT(!status); >+ if (status != kCVReturnSuccess) >+ return; >+ >+ m_bufferPool = adoptCF(bufferPool); >+ m_size = size; >+} >+ >+RetainPtr<CVPixelBufferRef> PixelBufferResizer::resize(CVPixelBufferRef inputBuffer) >+{ >+ RetainPtr<CVPixelBufferRef> result; >+ CVPixelBufferRef outputBuffer = nullptr; >+ >+ auto status = CVPixelBufferPoolCreatePixelBuffer(kCFAllocatorDefault, m_bufferPool.get(), &outputBuffer); >+ if (status) { >+ RELEASE_LOG(Media, "PixelBufferResizer::resize, CVPixelBufferPoolCreatePixelBuffer failed with error %d", static_cast<int>(status)); >+ return nullptr; >+ } >+ result = adoptCF(outputBuffer); >+ >+ auto err = VTPixelTransferSessionTransferImage(m_transferSession.get(), inputBuffer, outputBuffer); >+ if (err) { >+ RELEASE_LOG(Media, "PixelBufferResizer::resize, VTPixelTransferSessionTransferImage failed with error %d", static_cast<int>(err)); >+ return nullptr; >+ } >+ >+ return result; >+} >+ >+RetainPtr<CMSampleBufferRef> PixelBufferResizer::resize(CMSampleBufferRef sampleBuffer) >+{ >+ auto convertedPixelBuffer = resize(CMSampleBufferGetImageBuffer(sampleBuffer)); >+ if (!convertedPixelBuffer) >+ return nullptr; >+ >+ CMVideoFormatDescriptionRef formatDescription = nullptr; >+ auto status = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, convertedPixelBuffer.get(), &formatDescription); >+ if (status != noErr) { >+ RELEASE_LOG(Media, "PixelBufferResizer::resize: failed to create CMVideoFormatDescription with error code: %d", static_cast<int>(status)); >+ return nullptr; >+ } >+ >+ CMItemCount itemCount = 0; >+ status = CMSampleBufferGetSampleTimingInfoArray(sampleBuffer, 1, nullptr, &itemCount); >+ if (status != noErr) { >+ RELEASE_LOG(Media, "PixelBufferResizer::resize: CMSampleBufferGetSampleTimingInfoArray failed with error code: %d", static_cast<int>(status)); >+ return nullptr; >+ } >+ Vector<CMSampleTimingInfo> timingInfoArray; >+ CMSampleTimingInfo* timeingInfoPtr = nullptr; >+ if (itemCount) { >+ timingInfoArray.grow(itemCount); >+ status = CMSampleBufferGetSampleTimingInfoArray(sampleBuffer, itemCount, timingInfoArray.data(), nullptr); >+ if (status != noErr) { >+ RELEASE_LOG(Media, "PixelBufferResizer::resize: CMSampleBufferGetSampleTimingInfoArray failed with error code: %d", static_cast<int>(status)); >+ return nullptr; >+ } >+ timeingInfoPtr = timingInfoArray.data(); >+ } >+ >+ CMSampleBufferRef resizedSampleBuffer; >+ status = CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, convertedPixelBuffer.get(), formatDescription, timeingInfoPtr, &resizedSampleBuffer); >+ CFRelease(formatDescription); >+ if (status != noErr) { >+ RELEASE_LOG(Media, "PixelBufferResizer::resize: failed to create CMSampleBuffer with error code: %d", static_cast<int>(status)); >+ return nullptr; >+ } >+ >+ return adoptCF(resizedSampleBuffer); >+} >+ >+} // namespace WebCore >+ >+#endif // USE(VIDEOTOOLBOX) >diff --git a/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.cpp b/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.cpp >index 52331381fe5bbce3dc3cb47cef201eb03219d234..b3762947d68b6814f9f42fc4a9cc72b8146dce71 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.cpp >+++ b/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.cpp >@@ -73,12 +73,12 @@ void RealtimeIncomingAudioSource::setSourceTrack(rtc::scoped_refptr<webrtc::Audi > m_audioTrack->AddSink(this); > } > >-const RealtimeMediaSourceCapabilities& RealtimeIncomingAudioSource::capabilities() const >+const RealtimeMediaSourceCapabilities& RealtimeIncomingAudioSource::capabilities() > { > return RealtimeMediaSourceCapabilities::emptyCapabilities(); > } > >-const RealtimeMediaSourceSettings& RealtimeIncomingAudioSource::settings() const >+const RealtimeMediaSourceSettings& RealtimeIncomingAudioSource::settings() > { > return m_currentSettings; > } >diff --git a/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.h b/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.h >index 044bfa85f0cac460c75cb33ce7f6b47c4e524f33..9add53a8c10ca2852f95fdb736817fab97af2584 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.h >+++ b/Source/WebCore/platform/mediastream/RealtimeIncomingAudioSource.h >@@ -63,8 +63,8 @@ private: > void startProducingData() final; > void stopProducingData() final; > >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > > RealtimeMediaSourceSettings m_currentSettings; > rtc::scoped_refptr<webrtc::AudioTrackInterface> m_audioTrack; >diff --git a/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp b/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp >index f760d40f75f60aadc8af21d2d0f57d67cc0c5508..bda76d3191abbd3a40aef0e35c7e4819933b1f6f 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp >+++ b/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.cpp >@@ -70,12 +70,12 @@ void RealtimeIncomingVideoSource::stopProducingData() > m_videoTrack->RemoveSink(this); > } > >-const RealtimeMediaSourceCapabilities& RealtimeIncomingVideoSource::capabilities() const >+const RealtimeMediaSourceCapabilities& RealtimeIncomingVideoSource::capabilities() > { > return RealtimeMediaSourceCapabilities::emptyCapabilities(); > } > >-const RealtimeMediaSourceSettings& RealtimeIncomingVideoSource::settings() const >+const RealtimeMediaSourceSettings& RealtimeIncomingVideoSource::settings() > { > return m_currentSettings; > } >diff --git a/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h b/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h >index 06680ab3d7f0c9c5a0f7af071f160c892f7d10d9..4bbc07d92bef8a7b147806938750552f6a9b0e27 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h >+++ b/Source/WebCore/platform/mediastream/RealtimeIncomingVideoSource.h >@@ -67,8 +67,8 @@ private: > void startProducingData() final; > void stopProducingData() final; > >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > > rtc::scoped_refptr<webrtc::VideoTrackInterface> m_videoTrack; > }; >diff --git a/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp b/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp >index f2793f38c05a9a31c708e1d422303e85d06a5fde..b3ab3e0dae812fcdfb232d60878a32b5e04329a9 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp >+++ b/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp >@@ -693,7 +693,7 @@ bool RealtimeMediaSource::selectSettings(const MediaConstraints& constraints, Fl > return true; > } > >-bool RealtimeMediaSource::supportsConstraint(const MediaConstraint& constraint) const >+bool RealtimeMediaSource::supportsConstraint(const MediaConstraint& constraint) > { > auto& capabilities = this->capabilities(); > >diff --git a/Source/WebCore/platform/mediastream/RealtimeMediaSource.h b/Source/WebCore/platform/mediastream/RealtimeMediaSource.h >index 057240ffc4404d0675fbf53eba6d4b97ba9fcf9a..6d8b96298db3e5e088895f133ce88d016a390138 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeMediaSource.h >+++ b/Source/WebCore/platform/mediastream/RealtimeMediaSource.h >@@ -191,8 +191,8 @@ public: > bool echoCancellation() const { return m_echoCancellation; } > void setEchoCancellation(bool); > >- virtual const RealtimeMediaSourceCapabilities& capabilities() const = 0; >- virtual const RealtimeMediaSourceSettings& settings() const = 0; >+ virtual const RealtimeMediaSourceCapabilities& capabilities() = 0; >+ virtual const RealtimeMediaSourceSettings& settings() = 0; > > using SuccessHandler = WTF::Function<void()>; > using FailureHandler = WTF::Function<void(const String& badConstraint, const String& errorString)>; >@@ -200,7 +200,7 @@ public: > std::optional<std::pair<String, String>> applyConstraints(const MediaConstraints&); > > bool supportsConstraints(const MediaConstraints&, String&); >- bool supportsConstraint(const MediaConstraint&) const; >+ bool supportsConstraint(const MediaConstraint&); > > virtual void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>); > >diff --git a/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp b/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp >index b3ad6cb9e4a9a80194933b15c7f4b45f222f8ae8..f5db9b097e76439ab1198e7ee62be05a22d3d7ab 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp >+++ b/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp >@@ -58,22 +58,85 @@ void RealtimeVideoSource::prepareToProduceData() > setSize(m_defaultSize); > } > >-void RealtimeVideoSource::setSupportedPresets(RealtimeVideoSource::VideoPresets&& presets) >+const Vector<Ref<VideoPreset>>& RealtimeVideoSource::presets() > { >- m_presets = WTFMove(presets); >- std::sort(m_presets.begin(), m_presets.end(), >- [&] (const auto& a, const auto& b) -> bool { >- return (a.size.width() * a.size.height()) < (b.size.width() * b.size.height()); >+ if (m_presets.isEmpty()) >+ generatePresets(); >+ >+ ASSERT(!m_presets.isEmpty()); >+ return m_presets; >+} >+ >+void RealtimeVideoSource::setSupportedPresets(Vector<VideoPresetData>&& presetData) >+{ >+ Vector<Ref<VideoPreset>> presets; >+ >+ for (auto& data : presetData) >+ presets.append(VideoPreset::create(WTFMove(data))); >+ >+ setSupportedPresets(WTFMove(presets)); >+} >+ >+void RealtimeVideoSource::setSupportedPresets(const Vector<Ref<VideoPreset>>& presets) >+{ >+ m_presets = WTF::map(presets, [](auto& preset) { >+ return preset.copyRef(); > }); >- >+ > for (auto& preset : m_presets) { >- std::sort(preset.frameRateRanges.begin(), preset.frameRateRanges.end(), >+ std::sort(preset->frameRateRanges.begin(), preset->frameRateRanges.end(), > [&] (const auto& a, const auto& b) -> bool { > return a.minimum < b.minimum; > }); > } > } > >+static const Vector<IntSize>& standardVideoSizes() >+{ >+ static const auto sizes = makeNeverDestroyed([] { >+ static IntSize videoSizes[] = { >+ { 112, 112 }, >+ { 160, 160 }, >+ { 160, 120 }, // 4:3, QQVGA >+ { 192, 192 }, >+ { 192, 112 }, // 16:9 >+ { 192, 144 }, // 3:4 >+ { 240, 240 }, >+ { 240, 160 }, // 3:2, HQVGA >+ { 320, 320 }, >+ { 320, 176 }, // 16:9 >+ { 320, 240 }, // 4:3, QVGA >+ { 352, 288 }, // CIF >+ { 480, 272 }, // 16:9 >+ { 480, 360 }, // 4:3 >+ { 480, 480 }, >+ { 640, 640 }, >+ { 640, 368 }, // 16:9 >+ { 640, 480 }, // 4:3 >+ { 720, 720 }, >+ { 800, 600 }, // 4:3, SVGA >+ { 960, 540 }, // 16:9, qHD >+ { 1024, 600 }, // 16:9, WSVGA >+ { 1024, 768 }, // 4:3, XGA >+ { 1280, 960 }, // 4:3 >+ { 1280, 1024 }, // 5:4, SXGA >+ { 1280, 720 }, // 16:9, WXGA >+ { 1366, 768 }, // 16:9, HD >+ { 1920, 1080 }, // 16:9, FHD >+ { 2560, 1440 }, // 16:9, QHD >+ { 2592, 1936 }, >+ { 3264, 2448 }, // 3:4 >+ { 3840, 2160 }, // 16:9, 4K UHD >+ }; >+ Vector<IntSize> sizes; >+ for (auto& size : videoSizes) >+ sizes.append(size); >+ >+ return sizes; >+ }()); >+ >+ return sizes.get(); >+} > template <typename ValueType> > static void updateMinMax(ValueType& min, ValueType& max, ValueType value) > { >@@ -81,9 +144,9 @@ static void updateMinMax(ValueType& min, ValueType& max, ValueType value) > max = std::max<ValueType>(max, value); > } > >-void RealtimeVideoSource::addSupportedCapabilities(RealtimeMediaSourceCapabilities& capabilities) const >+void RealtimeVideoSource::updateCapabilities(RealtimeMediaSourceCapabilities& capabilities) > { >- ASSERT(!m_presets.isEmpty()); >+ ASSERT(!presets().isEmpty()); > > int minimumWidth = std::numeric_limits<int>::max(); > int maximumWidth = 0; >@@ -93,18 +156,28 @@ void RealtimeVideoSource::addSupportedCapabilities(RealtimeMediaSourceCapabiliti > double maximumAspectRatio = 0; > double minimumFrameRate = std::numeric_limits<double>::max(); > double maximumFrameRate = 0; >- for (const auto& preset : m_presets) { >- const auto& size = preset.size; >+ for (const auto& preset : presets()) { >+ const auto& size = preset->size; > updateMinMax(minimumWidth, maximumWidth, size.width()); > updateMinMax(minimumHeight, maximumHeight, size.height()); >- > updateMinMax(minimumAspectRatio, maximumAspectRatio, static_cast<double>(size.width()) / size.height()); > >- for (const auto& rate : preset.frameRateRanges) { >+ for (const auto& rate : preset->frameRateRanges) { > updateMinMax(minimumFrameRate, maximumFrameRate, rate.minimum); > updateMinMax(minimumFrameRate, maximumFrameRate, rate.maximum); > } > } >+ >+ if (canResizeVideoFrames()) { >+ for (auto& size : standardVideoSizes()) { >+ if (size.width() < minimumWidth || size.height() < minimumHeight) { >+ minimumWidth = std::min(minimumWidth, size.width()); >+ minimumHeight = std::min(minimumHeight, size.height()); >+ minimumAspectRatio = std::min(minimumAspectRatio, static_cast<double>(size.width()) / size.height()); >+ } >+ } >+ } >+ > capabilities.setWidth({ minimumWidth, maximumWidth }); > capabilities.setHeight({ minimumHeight, maximumHeight }); > capabilities.setAspectRatio({ minimumAspectRatio, maximumAspectRatio }); >@@ -119,55 +192,174 @@ bool RealtimeVideoSource::supportsSizeAndFrameRate(std::optional<int> width, std > return !!bestSupportedSizeAndFrameRate(width, height, frameRate); > } > >-std::optional<RealtimeVideoSource::CaptureSizeAndFrameRate> RealtimeVideoSource::bestSupportedSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate) >+bool RealtimeVideoSource::frameRateRangeIncludesRate(const FrameRateRange& range, double frameRate) > { >- if (!width && !height && !frameRate) >+ const double epsilon = 0.001; >+ return frameRate + epsilon >= range.minimum && frameRate - epsilon <= range.maximum; >+} >+ >+bool RealtimeVideoSource::presetSupportsFrameRate(RefPtr<VideoPreset> preset, double frameRate) >+{ >+ for (const auto& range : preset->frameRateRanges) { >+ if (frameRateRangeIncludesRate(range, frameRate)) >+ return true; >+ } >+ >+ return false; >+} >+ >+bool RealtimeVideoSource::supportsCaptureSize(std::optional<int> width, std::optional<int> height, const Function<bool(const IntSize&)>&& function) >+{ >+ if (width && height) >+ return function({ width.value(), height.value() }); >+ >+ if (width) { >+ for (auto& size : standardVideoSizes()) { >+ if (width.value() == size.width() && function({ size.width(), size.height() })) >+ return true; >+ } >+ >+ return false; >+ } >+ >+ for (auto& size : standardVideoSizes()) { >+ if (height.value() == size.height() && function({ size.width(), size.height() })) >+ return true; >+ } >+ >+ return false; >+} >+ >+bool RealtimeVideoSource::shouldUsePreset(VideoPreset& current, VideoPreset& candidate) >+{ >+ return candidate.size.width() <= current.size.width() && candidate.size.height() <= current.size.height() && prefersPreset(candidate); >+} >+ >+std::optional<RealtimeVideoSource::CaptureSizeAndFrameRate> RealtimeVideoSource::bestSupportedSizeAndFrameRate(std::optional<int> requestedWidth, std::optional<int> requestedHeight, std::optional<double> requestedFrameRate) >+{ >+ if (!requestedWidth && !requestedHeight && !requestedFrameRate) > return { }; > >- CaptureSizeAndFrameRate match; >- for (const auto& preset : m_presets) { >- const auto& size = preset.size; >- if ((width && width.value() != size.width()) || (height && height.value() != size.height())) >+ if (!requestedWidth && !requestedHeight && !size().isEmpty()) { >+ requestedWidth = size().width(); >+ requestedHeight = size().height(); >+ } >+ if (!requestedFrameRate) >+ requestedFrameRate = frameRate(); >+ >+ CaptureSizeAndFrameRate result; >+ RefPtr<VideoPreset> exactSizePreset; >+ RefPtr<VideoPreset> aspectRatioPreset; >+ IntSize aspectRatioMatchSize; >+ RefPtr<VideoPreset> resizePreset; >+ IntSize resizeSize; >+ >+ for (const auto& preset : presets()) { >+ const auto& presetSize = preset->size; >+ >+ if (!presetSupportsFrameRate(&preset.get(), requestedFrameRate.value())) >+ continue; >+ >+ if (!requestedWidth && !requestedHeight) { >+ result.requestedFrameRate = requestedFrameRate.value(); >+ return result; >+ } >+ >+ // Don't look at presets smaller than the requested resolution because we never want to resize larger. >+ if ((requestedWidth && presetSize.width() < requestedWidth.value()) || (requestedHeight && presetSize.height() < requestedHeight.value())) >+ continue; >+ >+ auto lookForExactSizeMatch = [&] (const IntSize& size) -> bool { >+ return preset->size == size; >+ }; >+ if (supportsCaptureSize(requestedWidth, requestedHeight, WTFMove(lookForExactSizeMatch))) { >+ if (!exactSizePreset || prefersPreset(preset)) >+ exactSizePreset = &preset.get(); > continue; >+ } > >- match.size = size; >- if (!frameRate) { >- match.frameRate = preset.frameRateRanges[preset.frameRateRanges.size() - 1].maximum; >- return match; >+ IntSize encodingSize; >+ auto lookForAspectRatioMatch = [this, &preset, &encodingSize] (const IntSize& size) -> bool { >+ auto aspectRatio = [] (const IntSize size) -> double { >+ return size.width() / static_cast<double>(size.height()); >+ }; >+ if (std::abs(aspectRatio(preset->size) - aspectRatio(size)) > 10e-7 || !canResizeVideoFrames()) >+ return false; >+ >+ encodingSize = size; >+ return true; >+ }; >+ if (supportsCaptureSize(requestedWidth, requestedHeight, WTFMove(lookForAspectRatioMatch))) { >+ if (!aspectRatioPreset || shouldUsePreset(*aspectRatioPreset, preset)) { >+ aspectRatioPreset = &preset.get(); >+ aspectRatioMatchSize = encodingSize; >+ } > } > >- const double epsilon = 0.001; >- for (const auto& rate : preset.frameRateRanges) { >- if (frameRate.value() + epsilon >= rate.minimum && frameRate.value() - epsilon <= rate.maximum) { >- match.frameRate = frameRate.value(); >- return match; >+ if (requestedWidth && requestedHeight) { >+ const auto& minStandardSize = standardVideoSizes()[0]; >+ if (requestedWidth.value() >= minStandardSize.width() && requestedHeight.value() >= minStandardSize.height()) { >+ if (!resizePreset || shouldUsePreset(*resizePreset, preset)) { >+ resizePreset = &preset.get(); >+ resizeSize = { requestedWidth.value(), requestedHeight.value() }; >+ } >+ } >+ } else { >+ for (auto& standardSize : standardVideoSizes()) { >+ if (standardSize.width() > preset->size.width() || standardSize.height() > preset->size.height()) >+ break; >+ if ((requestedWidth && requestedWidth.value() != standardSize.width()) || (requestedHeight && requestedHeight.value() != standardSize.height())) >+ continue; >+ >+ if (!resizePreset || shouldUsePreset(*resizePreset, preset)) { >+ resizePreset = &preset.get(); >+ resizeSize = standardSize; >+ } > } > } >+ } > >- break; >+ if (!exactSizePreset && !aspectRatioPreset && !resizePreset) >+ return { }; >+ >+ result.requestedFrameRate = requestedFrameRate.value(); >+ if (exactSizePreset) { >+ result.encodingPreset = exactSizePreset; >+ result.requestedSize = exactSizePreset->size; >+ return result; >+ } >+ >+ if (aspectRatioPreset) { >+ result.encodingPreset = aspectRatioPreset; >+ result.requestedSize = aspectRatioMatchSize; >+ return result; > } > >- return { }; >+ result.encodingPreset = resizePreset; >+ result.requestedSize = resizeSize; >+ return result; > } > > void RealtimeVideoSource::setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate) > { > std::optional<RealtimeVideoSource::CaptureSizeAndFrameRate> match; > >- // If only the framerate is changing, first see if it is supported with the current width and height. > auto size = this->size(); >- if (!width && !height && !size.isEmpty()) >- match = bestSupportedSizeAndFrameRate(size.width(), size.height(), frameRate); >- >- if (!match) >- match = bestSupportedSizeAndFrameRate(width, height, frameRate); >+ if (!width && !height && !size.isEmpty()) { >+ width = size.width(); >+ height = size.height(); >+ } > >+ match = bestSupportedSizeAndFrameRate(width, height, frameRate); > ASSERT(match); > if (!match) > return; > >- setSize(match->size); >- setFrameRate(match->frameRate); >+ setSizeAndFrameRateWithPreset(match->requestedSize, match->requestedFrameRate, match->encodingPreset); >+ >+ if (!match->requestedSize.isEmpty()) >+ setSize(match->requestedSize); >+ setFrameRate(match->requestedFrameRate); > } > > void RealtimeVideoSource::dispatchMediaSampleToObservers(MediaSample& sample) >diff --git a/Source/WebCore/platform/mediastream/RealtimeVideoSource.h b/Source/WebCore/platform/mediastream/RealtimeVideoSource.h >index 1fb0334c71c5bacd7e6d6e83f703ea578da27736..3ba373a31db00fd7734804216873c53f4aa17269 100644 >--- a/Source/WebCore/platform/mediastream/RealtimeVideoSource.h >+++ b/Source/WebCore/platform/mediastream/RealtimeVideoSource.h >@@ -31,14 +31,12 @@ > #include "ImageBuffer.h" > #include "MediaSample.h" > #include "RealtimeMediaSource.h" >+#include "VideoPreset.h" > #include <wtf/Lock.h> > #include <wtf/RunLoop.h> > > namespace WebCore { > >-struct FrameRateRange; >-struct VideoPreset; >- > class RealtimeVideoSource : public RealtimeMediaSource { > public: > virtual ~RealtimeVideoSource(); >@@ -50,16 +48,19 @@ protected: > bool supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final; > void setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final; > >- using VideoPresets = Vector<VideoPreset>; >- void setSupportedPresets(VideoPresets&&); >+ virtual void generatePresets() = 0; >+ virtual bool prefersPreset(VideoPreset&) { return true; } >+ virtual void setSizeAndFrameRateWithPreset(IntSize, double, RefPtr<VideoPreset>) { }; >+ virtual bool canResizeVideoFrames() const { return false; } >+ bool shouldUsePreset(VideoPreset& current, VideoPreset& candidate); > >- struct CaptureSizeAndFrameRate { >- IntSize size; >- double frameRate; >- }; >- std::optional<CaptureSizeAndFrameRate> bestSupportedSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>); >+ void setSupportedPresets(const Vector<Ref<VideoPreset>>&); >+ void setSupportedPresets(Vector<VideoPresetData>&&); >+ const Vector<Ref<VideoPreset>>& presets(); > >- void addSupportedCapabilities(RealtimeMediaSourceCapabilities&) const; >+ bool frameRateRangeIncludesRate(const FrameRateRange&, double); >+ >+ void updateCapabilities(RealtimeMediaSourceCapabilities&); > > void setDefaultSize(const IntSize& size) { m_defaultSize = size; } > >@@ -68,75 +69,21 @@ protected: > void dispatchMediaSampleToObservers(MediaSample&); > > private: >+ struct CaptureSizeAndFrameRate { >+ RefPtr<VideoPreset> encodingPreset; >+ IntSize requestedSize; >+ double requestedFrameRate { 0 }; >+ }; >+ bool supportsCaptureSize(std::optional<int>, std::optional<int>, const Function<bool(const IntSize&)>&&); >+ std::optional<CaptureSizeAndFrameRate> bestSupportedSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>); >+ bool presetSupportsFrameRate(RefPtr<VideoPreset>, double); > >- VideoPresets m_presets; >+ Vector<Ref<VideoPreset>> m_presets; > Deque<double> m_observedFrameTimeStamps; > double m_observedFrameRate { 0 }; > IntSize m_defaultSize; > }; > >-struct FrameRateRange { >- double minimum; >- double maximum; >- >- template<class Encoder> void encode(Encoder&) const; >- template<class Decoder> static std::optional<FrameRateRange> decode(Decoder&); >-}; >- >-template<class Encoder> >-void FrameRateRange::encode(Encoder& encoder) const >-{ >- encoder << minimum; >- encoder << maximum; >-} >- >-template <class Decoder> >-std::optional<FrameRateRange> FrameRateRange::decode(Decoder& decoder) >-{ >- std::optional<double> minimum; >- decoder >> minimum; >- if (!minimum) >- return std::nullopt; >- >- std::optional<double> maximum; >- decoder >> maximum; >- if (!maximum) >- return std::nullopt; >- >- return FrameRateRange { *minimum, *maximum }; >-} >- >-struct VideoPreset { >- IntSize size; >- Vector<FrameRateRange> frameRateRanges; >- >- template<class Encoder> void encode(Encoder&) const; >- template<class Decoder> static std::optional<VideoPreset> decode(Decoder&); >-}; >- >-template<class Encoder> >-void VideoPreset::encode(Encoder& encoder) const >-{ >- encoder << size; >- encoder << frameRateRanges; >-} >- >-template <class Decoder> >-std::optional<VideoPreset> VideoPreset::decode(Decoder& decoder) >-{ >- std::optional<IntSize> size; >- decoder >> size; >- if (!size) >- return std::nullopt; >- >- std::optional<Vector<FrameRateRange>> frameRateRanges; >- decoder >> frameRateRanges; >- if (!frameRateRanges) >- return std::nullopt; >- >- return VideoPreset { *size, *frameRateRanges }; >-} >- > } // namespace WebCore > > #endif // ENABLE(MEDIA_STREAM) >diff --git a/Source/WebCore/platform/mediastream/VideoPreset.h b/Source/WebCore/platform/mediastream/VideoPreset.h >new file mode 100644 >index 0000000000000000000000000000000000000000..9f04098fcda06b40f50182a009557696c71e594b >--- /dev/null >+++ b/Source/WebCore/platform/mediastream/VideoPreset.h >@@ -0,0 +1,133 @@ >+/* >+ * Copyright (C) 2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#if ENABLE(MEDIA_STREAM) >+ >+#include "FontCascade.h" >+#include "ImageBuffer.h" >+#include "MediaSample.h" >+#include "RealtimeMediaSource.h" >+#include <wtf/Lock.h> >+#include <wtf/RunLoop.h> >+ >+namespace WebCore { >+ >+struct FrameRateRange { >+ double minimum; >+ double maximum; >+ >+ template<class Encoder> void encode(Encoder&) const; >+ template<class Decoder> static std::optional<FrameRateRange> decode(Decoder&); >+}; >+ >+template<class Encoder> >+void FrameRateRange::encode(Encoder& encoder) const >+{ >+ encoder << minimum; >+ encoder << maximum; >+} >+ >+template <class Decoder> >+std::optional<FrameRateRange> FrameRateRange::decode(Decoder& decoder) >+{ >+ std::optional<double> minimum; >+ decoder >> minimum; >+ if (!minimum) >+ return std::nullopt; >+ >+ std::optional<double> maximum; >+ decoder >> maximum; >+ if (!maximum) >+ return std::nullopt; >+ >+ return FrameRateRange { *minimum, *maximum }; >+} >+ >+struct VideoPresetData { >+ IntSize size; >+ Vector<FrameRateRange> frameRateRanges; >+ >+ template<class Encoder> void encode(Encoder&) const; >+ template<class Decoder> static std::optional<VideoPresetData> decode(Decoder&); >+}; >+ >+template<class Encoder> >+void VideoPresetData::encode(Encoder& encoder) const >+{ >+ encoder << size; >+ encoder << frameRateRanges; >+} >+ >+template <class Decoder> >+std::optional<VideoPresetData> VideoPresetData::decode(Decoder& decoder) >+{ >+ std::optional<IntSize> size; >+ decoder >> size; >+ if (!size) >+ return std::nullopt; >+ >+ std::optional<Vector<FrameRateRange>> frameRateRanges; >+ decoder >> frameRateRanges; >+ if (!frameRateRanges) >+ return std::nullopt; >+ >+ return VideoPresetData { *size, *frameRateRanges }; >+} >+ >+class VideoPreset : public RefCounted<VideoPreset> { >+public: >+ static Ref<VideoPreset> create(VideoPresetData&& data) >+ { >+ return adoptRef(*new VideoPreset(data.size, WTFMove(data.frameRateRanges), Base)); >+ } >+ >+ enum VideoPresetType { >+ Base, >+ AVCapture >+ }; >+ >+ IntSize size; >+ Vector<FrameRateRange> frameRateRanges; >+ VideoPresetType type; >+ >+protected: >+ VideoPreset(IntSize size, Vector<FrameRateRange>&& frameRateRanges, VideoPresetType type) >+ : size(size) >+ , frameRateRanges(WTFMove(frameRateRanges)) >+ , type(type) >+ { >+ } >+}; >+ >+} // namespace WebCore >+ >+SPECIALIZE_TYPE_TRAITS_BEGIN(WebCore::VideoPreset) >+ static bool isType(const WebCore::VideoPreset& preset) { return preset.type == WebCore::VideoPreset::VideoPresetType::Base; } >+SPECIALIZE_TYPE_TRAITS_END() >+ >+#endif // ENABLE(MEDIA_STREAM) >+ >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >index 7e216735d8c137c9c65c0b333a41501199259973..584f87be2aa56f539675f75f4798f166c571ad84 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.cpp >@@ -136,7 +136,7 @@ void GStreamerAudioCaptureSource::stopProducingData() > m_capturer->stop(); > } > >-const RealtimeMediaSourceCapabilities& GStreamerAudioCaptureSource::capabilities() const >+const RealtimeMediaSourceCapabilities& GStreamerAudioCaptureSource::capabilities() > { > if (m_capabilities) > return m_capabilities.value(); >@@ -180,7 +180,7 @@ void GStreamerAudioCaptureSource::settingsDidChange(OptionSet<RealtimeMediaSourc > RealtimeMediaSource::settingsDidChange(settings); > } > >-const RealtimeMediaSourceSettings& GStreamerAudioCaptureSource::settings() const >+const RealtimeMediaSourceSettings& GStreamerAudioCaptureSource::settings() > { > if (!m_currentSettings) { > RealtimeMediaSourceSettings settings; >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h >index 985a2e42d0f60a80690dc71e2d58f3f74f00e150..759e814711527aefd5f0de0ce6dab060504400f1 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCaptureSource.h >@@ -33,8 +33,8 @@ public: > static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*); > WEBCORE_EXPORT static AudioCaptureFactory& factory(); > >- const RealtimeMediaSourceCapabilities& capabilities() const override; >- const RealtimeMediaSourceSettings& settings() const override; >+ const RealtimeMediaSourceCapabilities& capabilities() override; >+ const RealtimeMediaSourceSettings& settings() override; > > GstElement* pipeline() { return m_capturer->pipeline(); } > GStreamerCapturer* capturer() { return m_capturer.get(); } >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >index 0758e540b441ecf4703f35640795cd70c1d4a633..30d157bfe9f0fbc0ac84714555a6f1ec6d39e24e 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp >@@ -148,7 +148,7 @@ void GStreamerVideoCaptureSource::stopProducingData() > setSize({ 0, 0 }); > } > >-const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities() const >+const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities() > { > if (m_capabilities) > return m_capabilities.value(); >@@ -234,7 +234,7 @@ const RealtimeMediaSourceCapabilities& GStreamerVideoCaptureSource::capabilities > return m_capabilities.value(); > } > >-const RealtimeMediaSourceSettings& GStreamerVideoCaptureSource::settings() const >+const RealtimeMediaSourceSettings& GStreamerVideoCaptureSource::settings() > { > if (!m_currentSettings) { > RealtimeMediaSourceSettings settings; >diff --git a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h >index 1609b00c49b6bc81c83b7a8c43f84fecf478cb6c..751296a63ca75edd97c782fb5aed1c08d68d29e9 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h >+++ b/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h >@@ -32,8 +32,8 @@ public: > static CaptureSourceOrError create(const String& deviceID, const MediaConstraints*); > WEBCORE_EXPORT static VideoCaptureFactory& factory(); > >- const RealtimeMediaSourceCapabilities& capabilities() const override; >- const RealtimeMediaSourceSettings& settings() const override; >+ const RealtimeMediaSourceCapabilities& capabilities() override; >+ const RealtimeMediaSourceSettings& settings() override; > GstElement* pipeline() { return m_capturer->pipeline(); } > GStreamerCapturer* capturer() { return m_capturer.get(); } > >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp >index 995f05720eb90ad4350a1ab7b4545b49d285fcec..9f01e6712020b1afc039ffefc0f076ebdf5574d8 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.cpp >@@ -83,12 +83,12 @@ void MockGStreamerAudioCaptureSource::startProducingData() > m_wrappedSource->start(); > } > >-const RealtimeMediaSourceSettings& MockGStreamerAudioCaptureSource::settings() const >+const RealtimeMediaSourceSettings& MockGStreamerAudioCaptureSource::settings() > { > return m_wrappedSource->settings(); > } > >-const RealtimeMediaSourceCapabilities& MockGStreamerAudioCaptureSource::capabilities() const >+const RealtimeMediaSourceCapabilities& MockGStreamerAudioCaptureSource::capabilities() > { > m_capabilities = m_wrappedSource->capabilities(); > m_currentSettings = m_wrappedSource->settings(); >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h >index bc11f179b986b63f86b04a01f0b4bd5857b8a9b6..b57181d9a1c9c9ad70f1f56679d3755e454b1083 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerAudioCaptureSource.h >@@ -37,8 +37,8 @@ public: > private: > void stopProducingData() final; > void startProducingData() final; >- const RealtimeMediaSourceSettings& settings() const final; >- const RealtimeMediaSourceCapabilities& capabilities() const final; >+ const RealtimeMediaSourceSettings& settings() final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; > > void captureFailed(); > std::unique_ptr<RealtimeMediaSource> m_wrappedSource; >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp >index ead7272bf82a419cb848aa49b0def04a66e81c7a..44d94cc27e32672bb4c442063531bc82f2bcf4cb 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.cpp >@@ -118,12 +118,12 @@ void MockGStreamerVideoCaptureSource::applyConstraints(const MediaConstraints& c > m_wrappedSource->applyConstraints(constraints, WTFMove(successHandler), WTFMove(failureHandler)); > } > >-const RealtimeMediaSourceSettings& MockGStreamerVideoCaptureSource::settings() const >+const RealtimeMediaSourceSettings& MockGStreamerVideoCaptureSource::settings() > { > return m_wrappedSource->settings(); > } > >-const RealtimeMediaSourceCapabilities& MockGStreamerVideoCaptureSource::capabilities() const >+const RealtimeMediaSourceCapabilities& MockGStreamerVideoCaptureSource::capabilities() > { > m_capabilities = m_wrappedSource->capabilities(); > m_currentSettings = m_wrappedSource->settings(); >diff --git a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h >index ed3227f9ec566ad21d4a20ec48326fdbff7823fe..d08aa4899585e9a335d2937ec713fcd8979c5f5d 100644 >--- a/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h >+++ b/Source/WebCore/platform/mediastream/gstreamer/MockGStreamerVideoCaptureSource.h >@@ -42,9 +42,9 @@ public: > private: > void stopProducingData() final; > void startProducingData() final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceSettings& settings() final; > std::unique_ptr<RealtimeMediaSource> m_wrappedSource; >- const RealtimeMediaSourceCapabilities& capabilities() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; > void captureFailed() override; > > void videoSampleAvailable(MediaSample&) override; >diff --git a/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h b/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h >index 9f2fc3456c4b2a5bdd4b44423e45a3633edfdae9..d38e1ae62650a6e36473b11cb1ec9f1e01422e29 100644 >--- a/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h >+++ b/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h >@@ -29,23 +29,29 @@ > > #include "IntSizeHash.h" > #include "OrientationNotifier.h" >-#include "RealtimeMediaSource.h" >+#include "RealtimeVideoSource.h" >+#include <wtf/Lock.h> > #include <wtf/text/StringHash.h> > > typedef struct opaqueCMSampleBuffer* CMSampleBufferRef; > > OBJC_CLASS AVCaptureConnection; > OBJC_CLASS AVCaptureDevice; >+OBJC_CLASS AVCaptureDeviceFormat; > OBJC_CLASS AVCaptureOutput; > OBJC_CLASS AVCaptureSession; > OBJC_CLASS AVCaptureVideoDataOutput; >+OBJC_CLASS AVFrameRateRange; > OBJC_CLASS NSError; > OBJC_CLASS NSNotification; > OBJC_CLASS WebCoreAVVideoCaptureSourceObserver; > > namespace WebCore { > >-class AVVideoCaptureSource : public RealtimeMediaSource, private OrientationNotifier::Observer { >+class AVVideoPreset; >+class PixelBufferResizer; >+ >+class AVVideoCaptureSource : public RealtimeVideoSource, private OrientationNotifier::Observer { > public: > static CaptureSourceOrError create(const AtomicString&, const MediaConstraints*); > >@@ -72,13 +78,10 @@ private: > bool setupCaptureSession(); > void shutdownCaptureSession(); > >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- void setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final; >- void setFrameRate(double); >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > void startProducingData() final; > void stopProducingData() final; >- bool supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final; > void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) final; > void monitorOrientation(OrientationNotifier&) final; > void beginConfiguration() final; >@@ -86,12 +89,14 @@ private: > bool isCaptureSource() const final { return true; } > bool interrupted() const final; > >+ void setSizeAndFrameRateWithPreset(IntSize, double, RefPtr<VideoPreset>) final; >+ bool prefersPreset(VideoPreset&) final; >+ void generatePresets() final; >+ bool canResizeVideoFrames() const final { return true; } >+ > bool setPreset(NSString*); > void computeSampleRotation(); >- >- bool isFrameRateSupported(double frameRate); >- >- NSString* bestSessionPresetForVideoDimensions(std::optional<int> width, std::optional<int> height); >+ AVFrameRateRange* frameDurationForFrameRate(double); > > // OrientationNotifier::Observer API > void orientationChanged(int orientation) final; >@@ -101,34 +106,38 @@ private: > void processNewFrame(RetainPtr<CMSampleBufferRef>, RetainPtr<AVCaptureConnection>); > IntSize sizeForPreset(NSString*); > >- using VideoPresetMap = HashMap<String, IntSize>; >- VideoPresetMap& videoPresets() { return m_supportedPresets; } >- > AVCaptureDevice* device() const { return m_device.get(); } > >- RetainPtr<NSString> m_pendingPreset; > RetainPtr<CMSampleBufferRef> m_buffer; > RetainPtr<AVCaptureVideoDataOutput> m_videoOutput; > >- IntSize m_presetSize; >+ IntSize m_requestedSize; > int32_t m_width { 0 }; > int32_t m_height { 0 }; > int m_sensorOrientation { 0 }; > int m_deviceOrientation { 0 }; > MediaSample::VideoRotation m_sampleRotation { MediaSample::VideoRotation::None }; > >- VideoPresetMap m_supportedPresets; >- >- mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >- mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ std::unique_ptr<PixelBufferResizer> m_pixelBufferResizer; >+ std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ std::optional<RealtimeMediaSourceCapabilities> m_capabilities; > RetainPtr<WebCoreAVVideoCaptureSourceObserver> m_objcObserver; > RetainPtr<AVCaptureSession> m_session; > RetainPtr<AVCaptureDevice> m_device; >+ RefPtr<VideoPreset> m_pendingPreset; >+ >+ Lock m_presetMutex; >+ RefPtr<AVVideoPreset> m_currentPreset; >+ IntSize m_pendingSize; >+ double m_pendingFrameRate; > InterruptionReason m_interruption { InterruptionReason::None }; > bool m_isRunning { false }; >- > }; > > } // namespace WebCore > >+SPECIALIZE_TYPE_TRAITS_BEGIN(WebCore::AVVideoPreset) >+ static bool isType(const WebCore::VideoPreset& preset) { return preset.type == WebCore::VideoPreset::VideoPresetType::AVCapture; } >+SPECIALIZE_TYPE_TRAITS_END() >+ > #endif // ENABLE(MEDIA_STREAM) >diff --git a/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm b/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm >index f3bd4100246d55e90783f2b4f1d2f7e3e852a06e..adf1ff346e73b3581250b24d5347d98db98233fa 100644 >--- a/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm >+++ b/Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm >@@ -33,6 +33,7 @@ > #import "Logging.h" > #import "MediaConstraints.h" > #import "MediaSampleAVFObjC.h" >+#import "PixelBufferResizer.h" > #import "PlatformLayer.h" > #import "RealtimeMediaSourceCenterMac.h" > #import "RealtimeMediaSourceSettings.h" >@@ -75,29 +76,14 @@ SOFT_LINK_CLASS(AVFoundation, AVCaptureSession) > #define AVFrameRateRange getAVFrameRateRangeClass() > > SOFT_LINK_CONSTANT(AVFoundation, AVMediaTypeVideo, NSString *) >-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset1280x720, NSString *) >-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset960x540, NSString *) >-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset640x480, NSString *) >-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset352x288, NSString *) >-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset320x240, NSString *) >- >-#define AVCaptureSessionPreset1280x720 getAVCaptureSessionPreset1280x720() >-#define AVCaptureSessionPreset960x540 getAVCaptureSessionPreset960x540() >-#define AVCaptureSessionPreset640x480 getAVCaptureSessionPreset640x480() >-#define AVCaptureSessionPreset352x288 getAVCaptureSessionPreset352x288() >-#define AVCaptureSessionPreset320x240 getAVCaptureSessionPreset320x240() > > #if PLATFORM(IOS) >-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset3840x2160, NSString *) >-SOFT_LINK_CONSTANT_MAY_FAIL(AVFoundation, AVCaptureSessionPreset1920x1080, NSString *) > SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionRuntimeErrorNotification, NSString *) > SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionWasInterruptedNotification, NSString *) > SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionInterruptionEndedNotification, NSString *) > SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionInterruptionReasonKey, NSString *) > SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionErrorKey, NSString *) > >-#define AVCaptureSessionPreset3840x2160 getAVCaptureSessionPreset3840x2160() >-#define AVCaptureSessionPreset1920x1080 getAVCaptureSessionPreset1920x1080() > #define AVCaptureSessionRuntimeErrorNotification getAVCaptureSessionRuntimeErrorNotification() > #define AVCaptureSessionWasInterruptedNotification getAVCaptureSessionWasInterruptedNotification() > #define AVCaptureSessionInterruptionEndedNotification getAVCaptureSessionInterruptionEndedNotification() >@@ -106,6 +92,7 @@ SOFT_LINK_POINTER_OPTIONAL(AVFoundation, AVCaptureSessionErrorKey, NSString *) > #endif > > using namespace WebCore; >+using namespace PAL; > > @interface WebCoreAVVideoCaptureSourceObserver : NSObject<AVCaptureVideoDataOutputSampleBufferDelegate> { > AVVideoCaptureSource* m_callback; >@@ -142,6 +129,22 @@ static dispatch_queue_t globaVideoCaptureSerialQueue() > return globalQueue; > } > >+class AVVideoPreset : public VideoPreset { >+public: >+ static Ref<AVVideoPreset> create(IntSize size, Vector<FrameRateRange>&& frameRateRanges, AVCaptureDeviceFormatType* format) >+ { >+ return adoptRef(*new AVVideoPreset(size, WTFMove(frameRateRanges), format)); >+ } >+ >+ AVVideoPreset(IntSize size, Vector<FrameRateRange>&& frameRateRanges, AVCaptureDeviceFormatType* format) >+ : VideoPreset(size, WTFMove(frameRateRanges), AVCapture) >+ , format(format) >+ { >+ } >+ >+ RetainPtr<AVCaptureDeviceFormatType> format; >+}; >+ > CaptureSourceOrError AVVideoCaptureSource::create(const AtomicString& id, const MediaConstraints* constraints) > { > AVCaptureDeviceTypedef *device = [getAVCaptureDeviceClass() deviceWithUniqueID:id]; >@@ -159,46 +162,10 @@ CaptureSourceOrError AVVideoCaptureSource::create(const AtomicString& id, const > } > > AVVideoCaptureSource::AVVideoCaptureSource(AVCaptureDeviceTypedef* device, const AtomicString& id) >- : RealtimeMediaSource(id, Type::Video, device.localizedName) >+ : RealtimeVideoSource(id, device.localizedName) > , m_objcObserver(adoptNS([[WebCoreAVVideoCaptureSourceObserver alloc] initWithCallback:this])) > , m_device(device) > { >- struct VideoPreset { >- NSString* name; >- int width; >- int height; >- }; >- >- static NeverDestroyed<Vector<VideoPreset>> presets = [] { >- Vector<VideoPreset> sets; >-#if PLATFORM(IOS) >- if (canLoadAVCaptureSessionPreset3840x2160()) >- sets.append({ AVCaptureSessionPreset3840x2160, 3840, 2160 }); >- if (canLoadAVCaptureSessionPreset1920x1080()) >- sets.append({ AVCaptureSessionPreset1920x1080, 1920, 1080 }); >-#endif >- if (canLoadAVCaptureSessionPreset1280x720()) >- sets.append({ AVCaptureSessionPreset1280x720, 1280, 720 }); >- if (canLoadAVCaptureSessionPreset960x540()) >- sets.append({ AVCaptureSessionPreset960x540, 960, 540 }); >- if (canLoadAVCaptureSessionPreset640x480()) >- sets.append({ AVCaptureSessionPreset640x480, 640, 480 }); >- if (canLoadAVCaptureSessionPreset352x288()) >- sets.append({ AVCaptureSessionPreset352x288, 352, 288 }); >- if (canLoadAVCaptureSessionPreset320x240()) >- sets.append({ AVCaptureSessionPreset320x240, 320, 240 }); >- >- return sets; >- }(); >- >- auto* presetsMap = &videoPresets(); >- for (auto& preset : presets.get()) { >- if (![device supportsAVCaptureSessionPreset:preset.name]) >- continue; >- >- presetsMap->add(String(preset.name), IntSize(preset.width, preset.height)); >- } >- > #if PLATFORM(IOS) > static_assert(static_cast<int>(InterruptionReason::VideoNotAllowedInBackground) == static_cast<int>(AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableInBackground), "InterruptionReason::VideoNotAllowedInBackground is not AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableInBackground as expected"); > static_assert(static_cast<int>(InterruptionReason::VideoNotAllowedInSideBySide) == AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableWithMultipleForegroundApps, "InterruptionReason::VideoNotAllowedInSideBySide is not AVCaptureSessionInterruptionReasonVideoDeviceNotAvailableWithMultipleForegroundApps as expected"); >@@ -255,18 +222,6 @@ void AVVideoCaptureSource::stopProducingData() > #endif > } > >-static void updateSizeMinMax(int& min, int& max, int value) >-{ >- min = std::min<int>(min, value); >- max = std::max<int>(max, value); >-} >- >-static void updateAspectRatioMinMax(double& min, double& max, double value) >-{ >- min = std::min<double>(min, value); >- max = std::max<double>(max, value); >-} >- > void AVVideoCaptureSource::beginConfiguration() > { > if (m_session) >@@ -285,7 +240,7 @@ void AVVideoCaptureSource::settingsDidChange(OptionSet<RealtimeMediaSourceSettin > RealtimeMediaSource::settingsDidChange(settings); > } > >-const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings() const >+const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings() > { > if (m_currentSettings) > return *m_currentSettings; >@@ -298,10 +253,10 @@ const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings() const > else > settings.setFacingMode(RealtimeMediaSourceSettings::Unknown); > >- auto maxFrameDuration = [device() activeVideoMaxFrameDuration]; >- settings.setFrameRate(maxFrameDuration.timescale / maxFrameDuration.value); >- settings.setWidth(m_width); >- settings.setHeight(m_height); >+ settings.setFrameRate(frameRate()); >+ auto& size = this->size(); >+ settings.setWidth(size.width()); >+ settings.setHeight(size.height()); > settings.setDeviceId(id()); > > RealtimeMediaSourceSupportedConstraints supportedConstraints; >@@ -319,147 +274,92 @@ const RealtimeMediaSourceSettings& AVVideoCaptureSource::settings() const > return *m_currentSettings; > } > >-const RealtimeMediaSourceCapabilities& AVVideoCaptureSource::capabilities() const >+const RealtimeMediaSourceCapabilities& AVVideoCaptureSource::capabilities() > { > if (m_capabilities) > return *m_capabilities; > > RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); > capabilities.setDeviceId(id()); >- AVCaptureDeviceTypedef *videoDevice = device(); > >+ AVCaptureDeviceTypedef *videoDevice = device(); > if ([videoDevice position] == AVCaptureDevicePositionFront) > capabilities.addFacingMode(RealtimeMediaSourceSettings::User); > if ([videoDevice position] == AVCaptureDevicePositionBack) > capabilities.addFacingMode(RealtimeMediaSourceSettings::Environment); > >- Float64 lowestFrameRateRange = std::numeric_limits<double>::infinity(); >- Float64 highestFrameRateRange = 0; >- int minimumWidth = std::numeric_limits<int>::infinity(); >- int maximumWidth = 0; >- int minimumHeight = std::numeric_limits<int>::infinity(); >- int maximumHeight = 0; >- double minimumAspectRatio = std::numeric_limits<double>::infinity(); >- double maximumAspectRatio = 0; >- >- for (AVCaptureDeviceFormatType *format in [videoDevice formats]) { >- >- for (AVFrameRateRangeType *range in [format videoSupportedFrameRateRanges]) { >- lowestFrameRateRange = std::min<Float64>(lowestFrameRateRange, range.minFrameRate); >- highestFrameRateRange = std::max<Float64>(highestFrameRateRange, range.maxFrameRate); >- } >- } >- >- for (auto& preset : m_supportedPresets) { >- auto values = preset.value; >- updateSizeMinMax(minimumWidth, maximumWidth, values.width()); >- updateSizeMinMax(minimumHeight, maximumHeight, values.height()); >- updateAspectRatioMinMax(minimumAspectRatio, maximumAspectRatio, static_cast<double>(values.width()) / values.height()); >- } >- capabilities.setFrameRate(CapabilityValueOrRange(lowestFrameRateRange, highestFrameRateRange)); >- capabilities.setWidth(CapabilityValueOrRange(minimumWidth, maximumWidth)); >- capabilities.setHeight(CapabilityValueOrRange(minimumHeight, maximumHeight)); >- capabilities.setAspectRatio(CapabilityValueOrRange(minimumAspectRatio, maximumAspectRatio)); >+ updateCapabilities(capabilities); > > m_capabilities = WTFMove(capabilities); > > return *m_capabilities; > } > >-IntSize AVVideoCaptureSource::sizeForPreset(NSString* preset) >+bool AVVideoCaptureSource::prefersPreset(VideoPreset& preset) > { >- if (!preset) >- return { }; >- >- auto& presets = videoPresets(); >- auto it = presets.find(String(preset)); >- if (it != presets.end()) >- return { it->value.width(), it->value.height() }; >+#if PLATFORM(IOS) >+ return ![static_cast<AVVideoPreset*>(&preset)->format.get() isVideoBinned]; >+#else >+ UNUSED_PARAM(preset); >+#endif > >- return { }; >+ return true; > } > >-bool AVVideoCaptureSource::setPreset(NSString *preset) >+void AVVideoCaptureSource::setSizeAndFrameRateWithPreset(IntSize requestedSize, double requestedFrameRate, RefPtr<VideoPreset> preset) > { >- if (!session()) { >- m_pendingPreset = preset; >- return true; >- } >- >- auto size = sizeForPreset(preset); >- if (m_presetSize == size) >- return true; >- >- m_presetSize = size; >+ auto* avPreset = preset ? downcast<AVVideoPreset>(preset.get()) : nullptr; > >- @try { >- session().sessionPreset = preset; >-#if PLATFORM(MAC) >- auto settingsDictionary = @{ >- (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(videoCaptureFormat), >- (__bridge NSString *)kCVPixelBufferWidthKey: @(size.width()), >- (__bridge NSString *)kCVPixelBufferHeightKey: @(size.height()) >- }; >- [m_videoOutput setVideoSettings:settingsDictionary]; >-#endif >- } @catch(NSException *exception) { >- LOG(Media, "AVVideoCaptureSource::setPreset(%p), exception thrown configuring device: <%s> %s", this, [[exception name] UTF8String], [[exception reason] UTF8String]); >- return false; >+ if (!m_session) { >+ m_pendingPreset = avPreset; >+ m_pendingSize = requestedSize; >+ m_pendingFrameRate = requestedFrameRate; >+ return; > } > >- return true; >-} >+ m_pendingPreset = nullptr; >+ m_pendingFrameRate = 0; > >-void AVVideoCaptureSource::setFrameRate(double rate) >-{ >- using namespace PAL; >- double epsilon = 0.00001; >- AVFrameRateRangeType *bestFrameRateRange = nil; >- for (AVFrameRateRangeType *frameRateRange in [[device() activeFormat] videoSupportedFrameRateRanges]) { >- if (rate + epsilon >= [frameRateRange minFrameRate] && rate - epsilon <= [frameRateRange maxFrameRate]) { >- if (!bestFrameRateRange || CMTIME_COMPARE_INLINE([frameRateRange minFrameDuration], >, [bestFrameRateRange minFrameDuration])) >- bestFrameRateRange = frameRateRange; >- } >- } >+ auto* frameRateRange = frameDurationForFrameRate(requestedFrameRate); >+ ASSERT(frameRateRange); >+ if (!frameRateRange) >+ return; > >- if (!bestFrameRateRange || !isFrameRateSupported(rate)) { >- LOG(Media, "AVVideoCaptureSource::setFrameRate(%p), frame rate %f not supported by video device", this, rate); >+ if (!avPreset) > return; >- } >+ >+ ASSERT(avPreset->format); >+ >+ m_requestedSize = requestedSize; > > NSError *error = nil; >+ [m_session beginConfiguration]; > @try { > if ([device() lockForConfiguration:&error]) { >- if (bestFrameRateRange.minFrameRate == bestFrameRateRange.maxFrameRate) { >- [device() setActiveVideoMinFrameDuration:[bestFrameRateRange minFrameDuration]]; >- [device() setActiveVideoMaxFrameDuration:[bestFrameRateRange maxFrameDuration]]; >- } else { >- [device() setActiveVideoMinFrameDuration: CMTimeMake(1, rate)]; >- [device() setActiveVideoMaxFrameDuration: CMTimeMake(1, rate)]; >+ if (!m_currentPreset || ![m_currentPreset->format.get() isEqual:avPreset->format.get()]) { >+ [device() setActiveFormat:avPreset->format.get()]; >+#if PLATFORM(MAC) >+ auto settingsDictionary = @{ >+ (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(videoCaptureFormat), >+ (__bridge NSString *)kCVPixelBufferWidthKey: @(avPreset->size.width()), >+ (__bridge NSString *)kCVPixelBufferHeightKey: @(avPreset->size.height()) >+ }; >+ [m_videoOutput setVideoSettings:settingsDictionary]; >+#endif > } >+ [device() setActiveVideoMinFrameDuration:[frameRateRange minFrameDuration]]; >+ [device() setActiveVideoMaxFrameDuration:[frameRateRange maxFrameDuration]]; > [device() unlockForConfiguration]; > } > } @catch(NSException *exception) { >- LOG(Media, "AVVideoCaptureSource::setFrameRate(%p), exception thrown configuring device: <%s> %s", this, [[exception name] UTF8String], [[exception reason] UTF8String]); >- return; >+ RELEASE_LOG(Media, "AVVideoCaptureSource::setFrameRate - exception thrown configuring device: <%s> %s", [[exception name] UTF8String], [[exception reason] UTF8String]); > } >+ [m_session commitConfiguration]; > >- if (error) { >- LOG(Media, "AVVideoCaptureSource::setFrameRate(%p), failed to lock video device for configuration: %s", this, [[error localizedDescription] UTF8String]); >- return; >- } >- >- LOG(Media, "AVVideoCaptureSource::setFrameRate(%p) - set frame rate range to %f", this, rate); >- return; >-} >- >-void AVVideoCaptureSource::setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate) >-{ >- if (width || height) >- setPreset(bestSessionPresetForVideoDimensions(WTFMove(width), WTFMove(height))); >+ m_currentPreset = avPreset; > >- if (frameRate) >- setFrameRate(frameRate.value()); >+ if (error) >+ RELEASE_LOG(Media, "AVVideoCaptureSource::setFrameRate - failed to lock video device for configuration: %s", [[error localizedDescription] UTF8String]); > } > > static inline int sensorOrientation(AVCaptureVideoOrientation videoOrientation) >@@ -513,6 +413,22 @@ bool AVVideoCaptureSource::setupSession() > return success; > } > >+AVFrameRateRangeType* AVVideoCaptureSource::frameDurationForFrameRate(double rate) >+{ >+ AVFrameRateRangeType *bestFrameRateRange = nil; >+ for (AVFrameRateRangeType *frameRateRange in [[device() activeFormat] videoSupportedFrameRateRanges]) { >+ if (frameRateRangeIncludesRate({ [frameRateRange minFrameRate], [frameRateRange maxFrameRate] }, rate)) { >+ if (!bestFrameRateRange || CMTIME_COMPARE_INLINE([frameRateRange minFrameDuration], >, [bestFrameRateRange minFrameDuration])) >+ bestFrameRateRange = frameRateRange; >+ } >+ } >+ >+ if (!bestFrameRateRange) >+ RELEASE_LOG(Media, "AVVideoCaptureSource::frameDurationForFrameRate, no frame rate range for rate %g", rate); >+ >+ return bestFrameRateRange; >+} >+ > bool AVVideoCaptureSource::setupCaptureSession() > { > #if PLATFORM(IOS) >@@ -534,13 +450,6 @@ bool AVVideoCaptureSource::setupCaptureSession() > > m_videoOutput = adoptNS([allocAVCaptureVideoDataOutputInstance() init]); > auto settingsDictionary = adoptNS([[NSMutableDictionary alloc] initWithObjectsAndKeys: [NSNumber numberWithInt:videoCaptureFormat], kCVPixelBufferPixelFormatTypeKey, nil]); >- if (m_pendingPreset) { >-#if PLATFORM(MAC) >- auto size = sizeForPreset(m_pendingPreset.get()); >- [settingsDictionary setObject:@(size.width()) forKey:(__bridge NSString *)kCVPixelBufferWidthKey]; >- [settingsDictionary setObject:@(size.height()) forKey:(__bridge NSString *)kCVPixelBufferHeightKey]; >-#endif >- } > > [m_videoOutput setVideoSettings:settingsDictionary.get()]; > [m_videoOutput setAlwaysDiscardsLateVideoFrames:YES]; >@@ -552,9 +461,8 @@ bool AVVideoCaptureSource::setupCaptureSession() > } > [session() addOutput:m_videoOutput.get()]; > >-#if PLATFORM(IOS) >- setPreset(m_pendingPreset.get()); >-#endif >+ if (m_pendingPreset || m_pendingFrameRate) >+ setSizeAndFrameRateWithPreset(m_pendingSize, m_pendingFrameRate, m_pendingPreset); > > m_sensorOrientation = sensorOrientationFromVideoOutput(m_videoOutput.get()); > computeSampleRotation(); >@@ -615,25 +523,20 @@ void AVVideoCaptureSource::processNewFrame(RetainPtr<CMSampleBufferRef> sampleBu > if (!isProducingData() || muted()) > return; > >- CMFormatDescriptionRef formatDescription = PAL::CMSampleBufferGetFormatDescription(sampleBuffer.get()); >+ >+ CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer.get()); > if (!formatDescription) > return; > > m_buffer = sampleBuffer; >- CMVideoDimensions dimensions = PAL::CMVideoFormatDescriptionGetDimensions(formatDescription); >+ CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(formatDescription); > if (m_sampleRotation == MediaSample::VideoRotation::Left || m_sampleRotation == MediaSample::VideoRotation::Right) > std::swap(dimensions.width, dimensions.height); > > if (dimensions.width != m_width || dimensions.height != m_height) { >- OptionSet<RealtimeMediaSourceSettings::Flag> changed; >- if (m_width != dimensions.width) >- changed.add(RealtimeMediaSourceSettings::Flag::Width); >- if (m_height != dimensions.height) >- changed.add(RealtimeMediaSourceSettings::Flag::Height); >- > m_width = dimensions.width; > m_height = dimensions.height; >- settingsDidChange(changed); >+ setSize({ dimensions.width, dimensions.height }); > } > > videoSampleAvailable(MediaSampleAVFObjC::create(m_buffer.get(), m_sampleRotation, [connection isVideoMirrored])); >@@ -641,55 +544,30 @@ void AVVideoCaptureSource::processNewFrame(RetainPtr<CMSampleBufferRef> sampleBu > > void AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection(AVCaptureOutputType*, CMSampleBufferRef sampleBuffer, AVCaptureConnectionType* captureConnection) > { >- RetainPtr<CMSampleBufferRef> buffer = sampleBuffer; >- RetainPtr<AVCaptureConnectionType> connection = captureConnection; >- >- scheduleDeferredTask([this, buffer, connection] { >- this->processNewFrame(buffer, connection); >- }); >-} >- >-NSString* AVVideoCaptureSource::bestSessionPresetForVideoDimensions(std::optional<int> width, std::optional<int> height) >-{ >- if (!width && !height) >- return nil; >- >- int widthValue = width ? width.value() : 0; >- int heightValue = height ? height.value() : 0; >+ CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer); >+ if (!formatDescription) >+ return; > >- for (auto& preset : videoPresets()) { >- auto size = preset.value; >- NSString* name = preset.key; >+ CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(formatDescription); >+ if (dimensions.width != m_requestedSize.width() || dimensions.height != m_requestedSize.height()) { >+ if (m_pixelBufferResizer && !m_pixelBufferResizer->canResizeTo(m_requestedSize)) >+ m_pixelBufferResizer = nullptr; > >- if ((!widthValue || widthValue == size.width()) && (!heightValue || heightValue == size.height())) >- return name; >- } >+ if (!m_pixelBufferResizer) >+ m_pixelBufferResizer = std::make_unique<PixelBufferResizer>(m_requestedSize, videoCaptureFormat); >+ } else >+ m_pixelBufferResizer = nullptr; > >- return nil; >-} >- >-bool AVVideoCaptureSource::isFrameRateSupported(double frameRate) >-{ >- double epsilon = 0.001; >- for (AVFrameRateRangeType *range in [[device() activeFormat] videoSupportedFrameRateRanges]) { >- if (frameRate + epsilon >= range.minFrameRate && frameRate - epsilon <= range.maxFrameRate) >- return true; >+ auto buffer = retainPtr(sampleBuffer); >+ if (m_pixelBufferResizer) { >+ buffer = m_pixelBufferResizer->resize(sampleBuffer); >+ if (!buffer) >+ return; > } >- return false; >-} >- >-bool AVVideoCaptureSource::supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double> frameRate) >-{ >- if (!height && !width && !frameRate) >- return true; > >- if ((height || width) && !bestSessionPresetForVideoDimensions(WTFMove(width), WTFMove(height))) >- return false; >- >- if (!frameRate) >- return true; >- >- return isFrameRateSupported(frameRate.value()); >+ scheduleDeferredTask([this, buffer, connection = retainPtr(captureConnection)] { >+ this->processNewFrame(buffer, connection); >+ }); > } > > void AVVideoCaptureSource::captureSessionIsRunningDidChange(bool state) >@@ -711,6 +589,29 @@ bool AVVideoCaptureSource::interrupted() const > return RealtimeMediaSource::interrupted(); > } > >+void AVVideoCaptureSource::generatePresets() >+{ >+ Vector<Ref<VideoPreset>> presets; >+ for (AVCaptureDeviceFormatType* format in [device() formats]) { >+ >+ CMVideoDimensions dimensions = CMVideoFormatDescriptionGetDimensions(format.formatDescription); >+ IntSize size = {dimensions.width, dimensions.height}; >+ auto index = presets.findMatching([&size](auto& preset) { >+ return size == preset->size; >+ }); >+ if (index != notFound) >+ continue; >+ >+ Vector<FrameRateRange> frameRates; >+ for (AVFrameRateRangeType *range in [format videoSupportedFrameRateRanges]) >+ frameRates.append({ range.minFrameRate, range.maxFrameRate}); >+ >+ presets.append(AVVideoPreset::create(size, WTFMove(frameRates), format)); >+ } >+ >+ setSupportedPresets(WTFMove(presets)); >+} >+ > #if PLATFORM(IOS) > void AVVideoCaptureSource::captureSessionRuntimeError(RetainPtr<NSError> error) > { >diff --git a/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp b/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp >index 7d727cca63b5fa8169c7015392cf34b57c5f55c8..e6cc41f7a2808546b4d4877754f030113356b37d 100644 >--- a/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp >+++ b/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp >@@ -841,7 +841,7 @@ void CoreAudioCaptureSource::stopProducingData() > unit.stopProducingData(); > } > >-const RealtimeMediaSourceCapabilities& CoreAudioCaptureSource::capabilities() const >+const RealtimeMediaSourceCapabilities& CoreAudioCaptureSource::capabilities() > { > if (!m_capabilities) { > RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >@@ -854,7 +854,7 @@ const RealtimeMediaSourceCapabilities& CoreAudioCaptureSource::capabilities() co > return m_capabilities.value(); > } > >-const RealtimeMediaSourceSettings& CoreAudioCaptureSource::settings() const >+const RealtimeMediaSourceSettings& CoreAudioCaptureSource::settings() > { > if (!m_currentSettings) { > RealtimeMediaSourceSettings settings; >diff --git a/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h b/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h >index c40d3c6c34d60cf4ba9ccedbf021d3d593d2ddba..476d3d63fb719fe191b4dc01d1b32a55cd19640e 100644 >--- a/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h >+++ b/Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h >@@ -86,16 +86,16 @@ private: > > std::optional<Vector<int>> discreteSampleRates() const final { return { { 8000, 16000, 32000, 44100, 48000, 96000 } }; } > >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) final; > > bool interrupted() const final; > > uint32_t m_captureDeviceID { 0 }; > >- mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >- mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ std::optional<RealtimeMediaSourceSettings> m_currentSettings; > > enum class SuspensionType { None, WhilePaused, WhilePlaying }; > SuspensionType m_suspendType { SuspensionType::None }; >diff --git a/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp b/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp >index 3ecd5ca2cded4175f5d47fa07e8c8990916a2341..119185c49c62687fa7ce4772083e899aeb6508be 100644 >--- a/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp >+++ b/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp >@@ -60,7 +60,7 @@ DisplayCaptureSourceCocoa::~DisplayCaptureSourceCocoa() > #endif > } > >-const RealtimeMediaSourceCapabilities& DisplayCaptureSourceCocoa::capabilities() const >+const RealtimeMediaSourceCapabilities& DisplayCaptureSourceCocoa::capabilities() > { > if (!m_capabilities) { > RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >@@ -75,7 +75,7 @@ const RealtimeMediaSourceCapabilities& DisplayCaptureSourceCocoa::capabilities() > return m_capabilities.value(); > } > >-const RealtimeMediaSourceSettings& DisplayCaptureSourceCocoa::settings() const >+const RealtimeMediaSourceSettings& DisplayCaptureSourceCocoa::settings() > { > if (!m_currentSettings) { > RealtimeMediaSourceSettings settings; >diff --git a/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h b/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h >index d8263d5968b0bb1b77619cefa90c3986ec0b94c8..7e39eee35bab6b4a859fed43b51c2f684a48dd22 100644 >--- a/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h >+++ b/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h >@@ -69,13 +69,13 @@ private: > > bool isCaptureSource() const final { return true; } > >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > > void emitFrame(); > >- mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >- mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ std::optional<RealtimeMediaSourceSettings> m_currentSettings; > RealtimeMediaSourceSupportedConstraints m_supportedConstraints; > > MonotonicTime m_startTime { MonotonicTime::nan() }; >diff --git a/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h b/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h >index 73129e1be7cb6dcc166fd556e00fd195ec44374e..0b6e0ab47252495c5b550b68f4d27ce799f19314 100644 >--- a/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h >+++ b/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.h >@@ -43,6 +43,8 @@ typedef struct __CVPixelBufferPool *CVPixelBufferPoolRef; > > namespace WebCore { > >+class PixelBufferResizer; >+ > class MockRealtimeVideoSourceMac final : public MockRealtimeVideoSource, private OrientationNotifier::Observer { > public: > virtual ~MockRealtimeVideoSourceMac() = default; >@@ -57,15 +59,17 @@ private: > PlatformLayer* platformLayer() const; > void updateSampleBuffer() final; > void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) final; >+ bool canResizeVideoFrames() const final { return true; } >+ void setSizeAndFrameRateWithPreset(IntSize, double, RefPtr<VideoPreset>) final; > > void orientationChanged(int orientation) final; > void monitorOrientation(OrientationNotifier&) final; > >- mutable RetainPtr<CGImageRef> m_previewImage; >- mutable RetainPtr<PlatformLayer> m_previewLayer; > mutable RetainPtr<CVPixelBufferPoolRef> m_bufferPool; > MediaSample::VideoRotation m_deviceOrientation { MediaSample::VideoRotation::None }; > std::unique_ptr<PixelBufferConformerCV> m_pixelBufferConformer; >+ std::unique_ptr<PixelBufferResizer> m_pixelBufferResizer; >+ IntSize m_presetSize; > }; > > } // namespace WebCore >diff --git a/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm b/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm >index 17b35baa157d5f446bb90397cc336151092fe8dc..613f224dade6b881884d757e4dffee9aad48970b 100644 >--- a/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm >+++ b/Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm >@@ -37,6 +37,7 @@ > #import "MediaConstraints.h" > #import "MediaSampleAVFObjC.h" > #import "NotImplemented.h" >+#import "PixelBufferResizer.h" > #import "PlatformLayer.h" > #import "RealtimeMediaSourceSettings.h" > #import <QuartzCore/CALayer.h> >@@ -49,6 +50,11 @@ > namespace WebCore { > using namespace PAL; > >+#if PLATFORM(MAC) >+const OSType videoCaptureFormat = kCVPixelFormatType_420YpCbCr8Planar; >+#else >+const OSType videoCaptureFormat = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange; >+#endif > static const int videoSampleRate = 90000; > > CaptureSourceOrError MockRealtimeVideoSource::create(const String& deviceID, const String& name, const MediaConstraints* constraints) >@@ -71,23 +77,18 @@ RetainPtr<CMSampleBufferRef> MockRealtimeVideoSourceMac::CMSampleBufferFromPixel > if (!pixelBuffer) > return nullptr; > >- if (!m_pixelBufferConformer) >- m_pixelBufferConformer = std::make_unique<PixelBufferConformerCV>((__bridge CFDictionaryRef)@{ (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_420YpCbCr8Planar) }); >- >- auto convertedPixelBuffer = m_pixelBufferConformer->convert(pixelBuffer); >- > CMTime sampleTime = CMTimeMake(((elapsedTime() + 100_ms) * videoSampleRate).seconds(), videoSampleRate); > CMSampleTimingInfo timingInfo = { kCMTimeInvalid, sampleTime, sampleTime }; > > CMVideoFormatDescriptionRef formatDescription = nullptr; >- OSStatus status = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)convertedPixelBuffer, &formatDescription); >+ OSStatus status = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)pixelBuffer, &formatDescription); > if (status != noErr) { > LOG_ERROR("Failed to initialize CMVideoFormatDescription with error code: %d", status); > return nullptr; > } > > CMSampleBufferRef sampleBuffer; >- status = CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)convertedPixelBuffer, formatDescription, &timingInfo, &sampleBuffer); >+ status = CMSampleBufferCreateReadyWithImageBuffer(kCFAllocatorDefault, (CVImageBufferRef)pixelBuffer, formatDescription, &timingInfo, &sampleBuffer); > CFRelease(formatDescription); > if (status != noErr) { > LOG_ERROR("Failed to initialize CMSampleBuffer with error code: %d", status); >@@ -148,6 +149,17 @@ void MockRealtimeVideoSourceMac::updateSampleBuffer() > return; > > auto pixelBuffer = pixelBufferFromCGImage(imageBuffer->copyImage()->nativeImage().get()); >+ if (m_pixelBufferResizer) >+ pixelBuffer = m_pixelBufferResizer->resize(pixelBuffer.get()); >+ else { >+ if (!m_pixelBufferConformer) { >+ m_pixelBufferConformer = std::make_unique<PixelBufferConformerCV>((__bridge CFDictionaryRef)@{ >+ (__bridge NSString *)kCVPixelBufferPixelFormatTypeKey: @(videoCaptureFormat) >+ }); >+ } >+ >+ pixelBuffer = m_pixelBufferConformer->convert(pixelBuffer.get()); >+ } > auto sampleBuffer = CMSampleBufferFromPixelBuffer(pixelBuffer.get()); > > // We use m_deviceOrientation to emulate sensor orientation >@@ -188,6 +200,21 @@ void MockRealtimeVideoSourceMac::monitorOrientation(OrientationNotifier& notifie > orientationChanged(notifier.orientation()); > } > >+void MockRealtimeVideoSourceMac::setSizeAndFrameRateWithPreset(IntSize requestedSize, double, RefPtr<VideoPreset> preset) >+{ >+ if (!preset) >+ return; >+ >+ if (preset->size != requestedSize) { >+ if (m_pixelBufferResizer && !m_pixelBufferResizer->canResizeTo(requestedSize)) >+ m_pixelBufferResizer = nullptr; >+ >+ if (!m_pixelBufferResizer) >+ m_pixelBufferResizer = std::make_unique<PixelBufferResizer>(requestedSize, videoCaptureFormat); >+ } else >+ m_pixelBufferResizer = nullptr; >+} >+ > } // namespace WebCore > > #endif // ENABLE(MEDIA_STREAM) >diff --git a/Source/WebCore/platform/mock/MockMediaDevice.h b/Source/WebCore/platform/mock/MockMediaDevice.h >index 9136e632d9e21dfbda9f1314ff3271529c2b2176..9c96cbb406b0aa51b6cffb02ba0a21ecfef2edf4 100644 >--- a/Source/WebCore/platform/mock/MockMediaDevice.h >+++ b/Source/WebCore/platform/mock/MockMediaDevice.h >@@ -79,7 +79,7 @@ struct MockCameraProperties { > if (!facingMode) > return std::nullopt; > >- std::optional<Vector<VideoPreset>> presets; >+ std::optional<Vector<VideoPresetData>> presets; > decoder >> presets; > if (!presets) > return std::nullopt; >@@ -94,7 +94,7 @@ struct MockCameraProperties { > > double defaultFrameRate { 30 }; > RealtimeMediaSourceSettings::VideoFacingMode facingMode { RealtimeMediaSourceSettings::VideoFacingMode::User }; >- Vector<VideoPreset> presets { { { 640, 480 }, { { 30, 30}, { 15, 15 } } } }; >+ Vector<VideoPresetData> presets { { { 640, 480 }, { { 30, 30}, { 15, 15 } } } }; > Color fillColor { Color::black }; > }; > >diff --git a/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp b/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp >index 965727082cb323989596b81ac6a13a6d910f1095..2e8e55cf6460bc23d12fab8bbbd626442de05ad5 100644 >--- a/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp >+++ b/Source/WebCore/platform/mock/MockRealtimeAudioSource.cpp >@@ -98,7 +98,7 @@ MockRealtimeAudioSource::~MockRealtimeAudioSource() > #endif > } > >-const RealtimeMediaSourceSettings& MockRealtimeAudioSource::settings() const >+const RealtimeMediaSourceSettings& MockRealtimeAudioSource::settings() > { > if (!m_currentSettings) { > RealtimeMediaSourceSettings settings; >@@ -119,7 +119,7 @@ const RealtimeMediaSourceSettings& MockRealtimeAudioSource::settings() const > return m_currentSettings.value(); > } > >-const RealtimeMediaSourceCapabilities& MockRealtimeAudioSource::capabilities() const >+const RealtimeMediaSourceCapabilities& MockRealtimeAudioSource::capabilities() > { > if (!m_capabilities) { > RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >diff --git a/Source/WebCore/platform/mock/MockRealtimeAudioSource.h b/Source/WebCore/platform/mock/MockRealtimeAudioSource.h >index 79c6678ad0153116c7ab7177449ba6895332b2a0..ded280d7df5840fe903f340b607cbebd93928b66 100644 >--- a/Source/WebCore/platform/mock/MockRealtimeAudioSource.h >+++ b/Source/WebCore/platform/mock/MockRealtimeAudioSource.h >@@ -59,8 +59,8 @@ protected: > static Seconds renderInterval() { return 60_ms; } > > private: >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > > void tick(); > >@@ -68,8 +68,8 @@ private: > > void delaySamples(Seconds) final; > >- mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >- mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ std::optional<RealtimeMediaSourceSettings> m_currentSettings; > RealtimeMediaSourceSupportedConstraints m_supportedConstraints; > > RunLoop::Timer<MockRealtimeAudioSource> m_timer; >diff --git a/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp b/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp >index 642d3ca6c8cd33259558601596df389b4b7f5c00..ae00d0debdbca1f8a75ad9c2b792d6e42f06df39 100644 >--- a/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp >+++ b/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp >@@ -53,13 +53,8 @@ static inline Vector<MockMediaDevice> defaultDevices() > MockCameraProperties { > 30, > RealtimeMediaSourceSettings::VideoFacingMode::User, { >- { { 3840, 2160 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, >- { { 1920, 1080 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, > { { 1280, 720 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, >- { { 960, 540 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, >- { { 640, 480 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, >- { { 352, 288 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, >- { { 320, 240 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, >+ { { 640, 480 }, { { 30, 30}, { 27.5, 27.5}, { 25, 25}, { 22.5, 22.5}, { 20, 20}, { 17.5, 17.5}, { 15, 15}, { 12.5, 12.5}, { 10, 10}, { 7.5, 7.5}, { 5, 5} } }, > }, > Color::black, > } }, >@@ -68,12 +63,14 @@ static inline Vector<MockMediaDevice> defaultDevices() > MockCameraProperties { > 15, > RealtimeMediaSourceSettings::VideoFacingMode::Environment, { >- { { 1280, 720 }, { { 3, 120 } } }, >- { { 960, 540 }, { { 3, 60 } } }, >- { { 640, 480 }, { { 2, 30 } } }, >- { { 352, 288 }, { { 2, 30 } } }, >- { { 320, 240 }, { { 2, 30 } } }, >- { { 160, 120 }, { { 2, 30 } } }, >+ { { 3840, 2160 }, { { 2, 30 } } }, >+ { { 1920, 1080 }, { { 2, 30 } } }, >+ { { 1280, 720 }, { { 3, 120 } } }, >+ { { 960, 540 }, { { 3, 60 } } }, >+ { { 640, 480 }, { { 2, 30 } } }, >+ { { 352, 288 }, { { 2, 30 } } }, >+ { { 320, 240 }, { { 2, 30 } } }, >+ { { 160, 120 }, { { 2, 30 } } }, > }, > Color::darkGray, > } }, >diff --git a/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp b/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp >index 0e78d56c440d791767baad0de25c7b4f898d444f..27b53e459c06bd15a1475de1fe4fcf75a6e0d274 100644 >--- a/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp >+++ b/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp >@@ -131,11 +131,15 @@ MockRealtimeVideoSource::MockRealtimeVideoSource(const String& deviceID, const S > auto& properties = WTF::get<MockCameraProperties>(m_device.properties); > setFrameRate(properties.defaultFrameRate); > setFacingMode(properties.facingMode); >- setSupportedPresets(WTFMove(properties.presets)); > m_fillColor = properties.fillColor; > } > >-const RealtimeMediaSourceCapabilities& MockRealtimeVideoSource::capabilities() const >+void MockRealtimeVideoSource::generatePresets() >+{ >+ setSupportedPresets(WTFMove(WTF::get<MockCameraProperties>(m_device.properties).presets)); >+} >+ >+const RealtimeMediaSourceCapabilities& MockRealtimeVideoSource::capabilities() > { > if (!m_capabilities) { > RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints()); >@@ -149,13 +153,13 @@ const RealtimeMediaSourceCapabilities& MockRealtimeVideoSource::capabilities() c > capabilities.setFrameRate(CapabilityValueOrRange(.01, 60.0)); > } > >- addSupportedCapabilities(capabilities); >+ updateCapabilities(capabilities); > m_capabilities = WTFMove(capabilities); > } > return m_capabilities.value(); > } > >-const RealtimeMediaSourceSettings& MockRealtimeVideoSource::settings() const >+const RealtimeMediaSourceSettings& MockRealtimeVideoSource::settings() > { > if (m_currentSettings) > return m_currentSettings.value(); >@@ -168,7 +172,7 @@ const RealtimeMediaSourceSettings& MockRealtimeVideoSource::settings() const > settings.setLogicalSurface(true); > } > settings.setFrameRate(frameRate()); >- IntSize size = this->size(); >+ auto& size = this->size(); > settings.setWidth(size.width()); > settings.setHeight(size.height()); > if (aspectRatio()) >diff --git a/Source/WebCore/platform/mock/MockRealtimeVideoSource.h b/Source/WebCore/platform/mock/MockRealtimeVideoSource.h >index 91f656afc1771db2f6fe6c6883f2e124007a9160..eec40965a934bbe2456be0ab6ac07121f1557160 100644 >--- a/Source/WebCore/platform/mock/MockRealtimeVideoSource.h >+++ b/Source/WebCore/platform/mock/MockRealtimeVideoSource.h >@@ -63,18 +63,19 @@ protected: > void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) override; > > private: >- const RealtimeMediaSourceCapabilities& capabilities() const final; >- const RealtimeMediaSourceSettings& settings() const final; >+ const RealtimeMediaSourceCapabilities& capabilities() final; >+ const RealtimeMediaSourceSettings& settings() final; > > void startProducingData() final; > void stopProducingData() final; >+ bool isCaptureSource() const final { return true; } >+ >+ void generatePresets() final; > > void drawAnimation(GraphicsContext&); > void drawText(GraphicsContext&); > void drawBoxes(GraphicsContext&); > >- bool isCaptureSource() const final { return true; } >- > void generateFrame(); > void startCaptureTimer(); > >@@ -98,8 +99,8 @@ private: > > unsigned m_frameNumber { 0 }; > RunLoop::Timer<MockRealtimeVideoSource> m_emitFrameTimer; >- mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >- mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings; >+ std::optional<RealtimeMediaSourceCapabilities> m_capabilities; >+ std::optional<RealtimeMediaSourceSettings> m_currentSettings; > RealtimeMediaSourceSupportedConstraints m_supportedConstraints; > Color m_fillColor { Color::black }; > MockMediaDevice m_device; >diff --git a/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp b/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp >index bc39dccf58e145c4984ba58d4c51b6c6835886a8..480bad73ec271f91faec7a50316a724ee88c5e94 100644 >--- a/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp >+++ b/Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp >@@ -65,13 +65,15 @@ public: > } > > SharedRingBufferStorage& storage() { return static_cast<SharedRingBufferStorage&>(m_ringBuffer.storage()); } >- const RealtimeMediaSourceCapabilities& capabilities() const final { >+ >+ const RealtimeMediaSourceCapabilities& capabilities() final >+ { > if (!m_capabilities) > m_capabilities = m_manager.capabilities(m_id); > return m_capabilities.value(); > } > >- const RealtimeMediaSourceSettings& settings() const final { return m_settings; } >+ const RealtimeMediaSourceSettings& settings() final { return m_settings; } > void setSettings(RealtimeMediaSourceSettings&& settings) > { > auto changed = m_settings.difference(settings); >diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog >index 2babe41ea7bccaf66d782bbdae6e2769bf763159..aaf782c758800653b8c08bcc56efa49a8c2162c2 100644 >--- a/LayoutTests/ChangeLog >+++ b/LayoutTests/ChangeLog >@@ -1,3 +1,16 @@ >+2018-09-12 Eric Carlson <eric.carlson@apple.com> >+ >+ Support arbitrary video resolution in getUserMedia API >+ https://bugs.webkit.org/show_bug.cgi?id=178109 >+ <rdar://problem/35083128> >+ >+ Reviewed by Youenn Fablet. >+ >+ * fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt: >+ * fast/mediastream/apply-constraints-advanced-expected.txt: >+ * fast/mediastream/apply-constraints-advanced.html: >+ * fast/mediastream/apply-constraints-video.html: >+ > 2018-09-14 Devin Rousso <webkit@devinrousso.com> > > Web Inspector: Record actions performed on ImageBitmapRenderingContext >diff --git a/LayoutTests/fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt b/LayoutTests/fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt >index 431f1f01a5aa31ef4a6cd80f961c05a51e2d3984..43e9f4209feeb8a0b97078f76f33ec6b34a8640d 100644 >--- a/LayoutTests/fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt >+++ b/LayoutTests/fast/mediastream/MediaStreamTrack-getCapabilities-expected.txt >@@ -4,12 +4,12 @@ On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE > > > video track capabilities: >- capabilities.aspectRatio = { max: 1.778, min: 1.222 } >+ capabilities.aspectRatio = { max: 1.778, min: 1 } > capabilities.deviceId = <UUID> > capabilities.facingMode = [ user ] > capabilities.frameRate = { max: 30, min: 5 } >- capabilities.height = { max: 2160, min: 240 } >- capabilities.width = { max: 3840, min: 320 } >+ capabilities.height = { max: 720, min: 112 } >+ capabilities.width = { max: 1280, min: 112 } > > audio track capabilities: > capabilities.deviceId = <UUID> >diff --git a/LayoutTests/fast/mediastream/apply-constraints-advanced-expected.txt b/LayoutTests/fast/mediastream/apply-constraints-advanced-expected.txt >index 75d0bdc6634a323c3284e802b2459bc5d8222e16..4e22260fcbebf8cfbd1cdd51e018ec86f6bdb487 100644 >--- a/LayoutTests/fast/mediastream/apply-constraints-advanced-expected.txt >+++ b/LayoutTests/fast/mediastream/apply-constraints-advanced-expected.txt >@@ -16,9 +16,9 @@ PASS settings['height'] is 480 > PASS settings['width'] is 640 > PASS settings['height'] is 480 > >-** Constraint: {"width":{"min":640},"height":{"min":480},"advanced":[{"width":6000,"height":6000},{"width":1920,"height":1080}]} - first width and height in advanced are too big, second is used. >-PASS settings['width'] is 1920 >-PASS settings['height'] is 1080 >+** Constraint: {"width":{"min":640},"height":{"min":480},"advanced":[{"width":6000,"height":6000},{"width":1280,"height":720}]} - first width and height in advanced are too big, second is used. >+PASS settings['width'] is 1280 >+PASS settings['height'] is 720 > > ** Constraint: {"width":320,"height":240} - reset width and height. > PASS settings['width'] is 320 >diff --git a/LayoutTests/fast/mediastream/apply-constraints-advanced.html b/LayoutTests/fast/mediastream/apply-constraints-advanced.html >index 922e1350ccf9a3f9ab2b834c55af81e6cca0db6b..1eab37963c5b78e9be117456ab84f62b07959f2d 100644 >--- a/LayoutTests/fast/mediastream/apply-constraints-advanced.html >+++ b/LayoutTests/fast/mediastream/apply-constraints-advanced.html >@@ -29,10 +29,10 @@ > height: { min: 480 }, > advanced: [ > { width: 6000, height: 6000 }, >- { width: 1920, height: 1080 }, >+ { width: 1280, height: 720 }, > ] > }, >- expected: { width: 1920, height: 1080 }, >+ expected: { width: 1280, height: 720 }, > }, > { > message: "reset width and height.", >diff --git a/LayoutTests/fast/mediastream/apply-constraints-video-expected.txt b/LayoutTests/fast/mediastream/apply-constraints-video-expected.txt >index d890d367b6c784840d00c293b4eba5626bf6f3b4..37d3ee680269ab6ac9c67acb13fecc42679ab67d 100644 >--- a/LayoutTests/fast/mediastream/apply-constraints-video-expected.txt >+++ b/LayoutTests/fast/mediastream/apply-constraints-video-expected.txt >@@ -32,34 +32,34 @@ PASS settings['width'] is 320 > PASS settings['height'] is 240 > PASS settings['frameRate'] is 30 > >-** Constraint: {"frameRate":{"max":6}} - the 'max' constraint can't be satisfied, promise should reject and no settings should change. >+** Constraint: {"frameRate":{"max":8}} - the 'max' constraint can't be satisfied, promise should reject and no settings should change. > PASS Promise was rejected > PASS error.constraint is "frameRate" > PASS settings['frameRate'] is 30 > >-** Constraint: {"width":{"exact":960}} - the 'exact' constraint can be satisfied. >-PASS settings['width'] is 960 >-PASS settings['height'] is 540 >+** Constraint: {"width":{"exact":640}} - the 'exact' constraint can be satisfied. >+PASS settings['width'] is 640 >+PASS settings['height'] is 480 > > ** Constraint: {"width":{"min":300,"ideal":5000}} - the 'ideal' constraint can't be satisfied but the 'min' can, maximum value should be chosen. >-PASS settings['width'] is 3840 >-PASS settings['height'] is 2160 >- >-** Constraint: {"width":{"min":320,"ideal":1280},"height":{"min":480,"ideal":720}} - 'ideal' and 'min' constraints can be satisfied, 'ideal' should be chosen. > PASS settings['width'] is 1280 > PASS settings['height'] is 720 > >+** Constraint: {"width":{"min":320,"ideal":640},"height":{"min":480,"ideal":720}} - 'ideal' and 'min' constraints can be satisfied, 'ideal' should be chosen. >+PASS settings['width'] is 640 >+PASS settings['height'] is 720 >+ > ** Constraint: {"width":5000} - ideal width is greater than track capability, should be clamped to the maximum value. >-PASS settings['width'] is 3840 >+PASS settings['width'] is 1280 > >-** Constraint: {"width":160,"height":120,"frameRate":4} - all values are less than track capabilities, should be clamped to the minimum values. >-PASS settings['width'] is 320 >-PASS settings['height'] is 240 >+** Constraint: {"width":100,"height":100,"frameRate":4} - all values are less than track capabilities, should be clamped to the minimum values. >+PASS settings['width'] is 112 >+PASS settings['height'] is 112 > PASS settings['frameRate'] is 5 > > ** Constraint: {"frameRate":20} - set frame rate, width and height should remain unchanged >-PASS settings['width'] is 320 >-PASS settings['height'] is 240 >+PASS settings['width'] is 112 >+PASS settings['height'] is 112 > PASS settings['frameRate'] is 20 > > ** Constraint: {"facingMode":"xnvironment","height":720} - illegal facing mode value should be ignored, height should change. >diff --git a/LayoutTests/fast/mediastream/apply-constraints-video.html b/LayoutTests/fast/mediastream/apply-constraints-video.html >index 7765c4810598c903e2178f6bcc245cfd1cce0c12..02e112b4060b38b6e492e7eb442e18a4cea0823f 100644 >--- a/LayoutTests/fast/mediastream/apply-constraints-video.html >+++ b/LayoutTests/fast/mediastream/apply-constraints-video.html >@@ -30,39 +30,39 @@ > }, > { > message: "the 'max' constraint can't be satisfied, promise should reject and no settings should change.", >- constraint: { frameRate: {max: 6}, }, >+ constraint: { frameRate: {max: 8}, }, > expected: { frameRate: 30 }, > error: "frameRate", > }, > { > message: "the 'exact' constraint can be satisfied.", >- constraint: { width: { exact: 960 } }, >- expected: { width: 960, height: 540 }, >+ constraint: { width: { exact: 640 } }, >+ expected: { width: 640, height: 480 }, > }, > { > message: "the 'ideal' constraint can't be satisfied but the 'min' can, maximum value should be chosen.", > constraint: { width: {min: 300, ideal: 5000} }, >- expected: { width: 3840, height: 2160 }, >+ expected: { width: 1280, height: 720 }, > }, > { > message: "'ideal' and 'min' constraints can be satisfied, 'ideal' should be chosen.", >- constraint: { width: {min: 320, ideal: 1280}, height: {min: 480, ideal: 720}, }, >- expected: { width: 1280, height: 720 }, >+ constraint: { width: {min: 320, ideal: 640}, height: {min: 480, ideal: 720}, }, >+ expected: { width: 640, height: 720 }, > }, > { > message: "ideal width is greater than track capability, should be clamped to the maximum value.", > constraint: { width: 5000 }, >- expected: { width: 3840}, >+ expected: { width: 1280}, > }, > { > message: "all values are less than track capabilities, should be clamped to the minimum values.", >- constraint: { width: 160, height: 120, frameRate: 4 }, >- expected: { width: 320, height: 240, frameRate: 5 }, >+ constraint: { width: 100, height: 100, frameRate: 4 }, >+ expected: { width: 112, height: 112, frameRate: 5 }, > }, > { > message: "set frame rate, width and height should remain unchanged", > constraint: { frameRate: 20 }, >- expected: { width: 320, height: 240, frameRate: 20 }, >+ expected: { width: 112, height: 112, frameRate: 20 }, > }, > { > message: "illegal facing mode value should be ignored, height should change.",
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 178109
:
349603
|
349659
|
349669
|
349675
|
349772
|
349783