Bug 187408

Summary:	Layout Test imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker.html is flaky
Product:	WebKit	Reporter:	Dawei Fenton (:realdawei) <realdawei>
Component:	Tools / Tests	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	jiewen_tan, lforschler, ryanhaddad, tsavell, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Dawei Fenton (:realdawei) 2018-07-06 13:17:14 PDT

The following layout test is flaky on High Sierra Debug WK1 and Wk2

imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker.html

Probable cause:

Unknown, researching it. 

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2FWebCryptoAPI%2FgenerateKey%2Fsuccesses_RSA-PSS.https.any.worker.html

Sample Diff:
--- /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker-expected.txt
+++ /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/WebCryptoAPI/generateKey/successes_RSA-PSS.https.any.worker-actual.txt
@@ -1,3 +1,5 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: RSA-PSS, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) 
 PASS Success: generateKey({hash: SHA-1, modulusLength: 2048, name: RSA-PSS, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) 
@@ -32,7 +34,7 @@
 PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign]) 
 PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign]) 
 PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [verify, sign]) 
-PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) 
-PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) 
-PASS Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify]) 
+TIMEOUT Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [verify, sign]) Test timed out
+NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, false, [sign, verify, sign, sign, verify]) 
+NOTRUN Success: generateKey({hash: SHA-256, modulusLength: 2048, name: Rsa-pss, publicExponent: {0: 1, 1: 0, 2: 1}}, true, [sign, verify, sign, sign, verify])

Comment 1 Dawei Fenton (:realdawei) 2018-07-06 13:40:15 PDT

This test on average runs between 20-26 seconds which puts it on the borderline -- external conditions in the test runner likely cause it to go over from time to time. 

For this reason I marked as "Slow"

https://trac.webkit.org/changeset/233591/webkit

Comment 2 Jiewen Tan 2018-07-13 13:33:12 PDT

Test Expectations are updated.

Comment 3 Radar WebKit Bug Importer 2018-07-13 13:34:21 PDT

<rdar://problem/42178600>