Attachment #441028 for bug #231353

View Details Raw Unified Return to Bug 231353

| Differences between



2021-10-10  Cameron McCormack  <heycam@apple.com>

        Override guessed color space for incoming H.264/265 streams
        https://bugs.webkit.org/show_bug.cgi?id=231353
        <rdar://problem/83969311>

        Reviewed by Youenn Fablet.

        When the VTDecompressionSession decodes incoming H.264/H.265 samples,
        it puts guessed color space attachments on the CVPixelBuffer, and the
        guesses are based on video resolution. WebRTC streams however default
        to being sRGB. So this patch overrides a guess with what we know the
        stream will be. (Once we support different color spaces in WebRTC
        video streams we'll need to adjust this.)

        * Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH264.mm:
        (overrideColorSpaceAttachmentsIfNeeded):
        (decompressionOutputCallback):
        * Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH265.mm:
        (overrideColorSpaceAttachmentsIfNeeded):
        (h265DecompressionOutputCallback):

2021-10-06  Youenn Fablet  <youenn@apple.com>

        ObjectiveC WebRTC frame buffers are autoreleased late, especially on Debug builds

Lines 41-46 @interface RTCVideoDecoderH264 () a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH264.mm_sec1
- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH264.mm +14 lines
41	- (void)setError:(OSStatus)error;	41	- (void)setError:(OSStatus)error;
42	@end	42	@end
43		43
		44	static void overrideColorSpaceAttachmentsIfNeeded(CVImageBufferRef imageBuffer) {
		45	auto guess = CVBufferGetAttachment(imageBuffer, (CFStringRef)@"ColorInfoGuessedBy", nullptr);
		46	if (!guess \|\| !CFEqual(guess, (CFStringRef)@"VideoToolbox"))
		47	return;
		48
		49	CVBufferRemoveAttachment(imageBuffer, kCVImageBufferCGColorSpaceKey);
		50	CVBufferSetAttachment(imageBuffer, kCVImageBufferColorPrimariesKey, kCVImageBufferColorPrimaries_ITU_R_709_2, kCVAttachmentMode_ShouldPropagate);
		51	CVBufferSetAttachment(imageBuffer, kCVImageBufferTransferFunctionKey, kCVImageBufferTransferFunction_sRGB, kCVAttachmentMode_ShouldPropagate);
		52	CVBufferSetAttachment(imageBuffer, kCVImageBufferYCbCrMatrixKey, kCVImageBufferYCbCrMatrix_ITU_R_709_2, kCVAttachmentMode_ShouldPropagate);
		53	CVBufferSetAttachment(imageBuffer, (CFStringRef)@"ColorInfoGuessedBy", (CFStringRef)@"RTCVideoDecoderH264", kCVAttachmentMode_ShouldPropagate);
		54	}
		55
44	// This is the callback function that VideoToolbox calls when decode is	56	// This is the callback function that VideoToolbox calls when decode is
45	// complete.	57	// complete.
46	void decompressionOutputCallback(void *decoderRef,	58	void decompressionOutputCallback(void *decoderRef,
Lines 57-62 void decompressionOutputCallback(void *decoderRef, a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH264.mm_sec2
57	return;	69	return;
58	}	70	}
59		71
		72	overrideColorSpaceAttachmentsIfNeeded(imageBuffer);
		73
60	std::unique_ptr<RTCFrameDecodeParams> decodeParams(reinterpret_cast<RTCFrameDecodeParams *>(params));	74	std::unique_ptr<RTCFrameDecodeParams> decodeParams(reinterpret_cast<RTCFrameDecodeParams *>(params));
61	// TODO(tkchin): Handle CVO properly.	75	// TODO(tkchin): Handle CVO properly.
62	RTCCVPixelBuffer *frameBuffer = [[RTCCVPixelBuffer alloc] initWithPixelBuffer:imageBuffer];	76	RTCCVPixelBuffer *frameBuffer = [[RTCCVPixelBuffer alloc] initWithPixelBuffer:imageBuffer];

Lines 38-43 @interface RTCVideoDecoderH265 () a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH265.mm_sec1
- a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH265.mm +14 lines
38	- (void)setError:(OSStatus)error;	38	- (void)setError:(OSStatus)error;
39	@end	39	@end
40		40
		41	static void overrideColorSpaceAttachmentsIfNeeded(CVImageBufferRef imageBuffer) {
		42	auto guess = CVBufferGetAttachment(imageBuffer, (CFStringRef)@"ColorInfoGuessedBy", nullptr);
		43	if (!guess \|\| !CFEqual(guess, (CFStringRef)@"VideoToolbox"))
		44	return;
		45
		46	CVBufferRemoveAttachment(imageBuffer, kCVImageBufferCGColorSpaceKey);
		47	CVBufferSetAttachment(imageBuffer, kCVImageBufferColorPrimariesKey, kCVImageBufferColorPrimaries_ITU_R_709_2, kCVAttachmentMode_ShouldPropagate);
		48	CVBufferSetAttachment(imageBuffer, kCVImageBufferTransferFunctionKey, kCVImageBufferTransferFunction_sRGB, kCVAttachmentMode_ShouldPropagate);
		49	CVBufferSetAttachment(imageBuffer, kCVImageBufferYCbCrMatrixKey, kCVImageBufferYCbCrMatrix_ITU_R_709_2, kCVAttachmentMode_ShouldPropagate);
		50	CVBufferSetAttachment(imageBuffer, (CFStringRef)@"ColorInfoGuessedBy", (CFStringRef)@"RTCVideoDecoderH265", kCVAttachmentMode_ShouldPropagate);
		51	}
		52
41	// This is the callback function that VideoToolbox calls when decode is	53	// This is the callback function that VideoToolbox calls when decode is
42	// complete.	54	// complete.
43	void h265DecompressionOutputCallback(void* decoderRef,	55	void h265DecompressionOutputCallback(void* decoderRef,
Lines 54-59 void h265DecompressionOutputCallback(void* decoderRef, a/Source/ThirdParty/libwebrtc/Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH265.mm_sec2
54	return;	66	return;
55	}	67	}
56		68
		69	overrideColorSpaceAttachmentsIfNeeded(imageBuffer);
		70
57	std::unique_ptr<RTCH265FrameDecodeParams> decodeParams(reinterpret_cast<RTCH265FrameDecodeParams*>(params));	71	std::unique_ptr<RTCH265FrameDecodeParams> decodeParams(reinterpret_cast<RTCH265FrameDecodeParams*>(params));
58	// TODO(tkchin): Handle CVO properly.	72	// TODO(tkchin): Handle CVO properly.
59	RTCCVPixelBuffer* frameBuffer =	73	RTCCVPixelBuffer* frameBuffer =

Return to Bug 231353

Lines 1-3 a/Source/ThirdParty/libwebrtc/ChangeLog_sec1
- a/Source/ThirdParty/libwebrtc/ChangeLog +22 lines
		1	2021-10-10 Cameron McCormack <heycam@apple.com>
		2
		3	Override guessed color space for incoming H.264/265 streams
		4	https://bugs.webkit.org/show_bug.cgi?id=231353
		5	<rdar://problem/83969311>
		6
		7	Reviewed by Youenn Fablet.
		8
		9	When the VTDecompressionSession decodes incoming H.264/H.265 samples,
		10	it puts guessed color space attachments on the CVPixelBuffer, and the
		11	guesses are based on video resolution. WebRTC streams however default
		12	to being sRGB. So this patch overrides a guess with what we know the
		13	stream will be. (Once we support different color spaces in WebRTC
		14	video streams we'll need to adjust this.)
		15
		16	* Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH264.mm:
		17	(overrideColorSpaceAttachmentsIfNeeded):
		18	(decompressionOutputCallback):
		19	* Source/webrtc/sdk/objc/components/video_codec/RTCVideoDecoderH265.mm:
		20	(overrideColorSpaceAttachmentsIfNeeded):
		21	(h265DecompressionOutputCallback):
		22
1	2021-10-06 Youenn Fablet <youenn@apple.com>	23	2021-10-06 Youenn Fablet <youenn@apple.com>
2		24
3	ObjectiveC WebRTC frame buffers are autoreleased late, especially on Debug builds	25	ObjectiveC WebRTC frame buffers are autoreleased late, especially on Debug builds