Bug 155196
| Summary: | security/contentSecurityPolicy/video-with-blob-url-allowed-by-media-src-star.html and security/contentSecurityPolicy/video-with-data-url-allowed-by-media-src-star.html are flaky | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Ryan Haddad <ryanhaddad> |
| Component: | WebCore Misc. | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | ap, ayumi_kojima, bfulgham, clopez, dbates, jiewen_tan, repstein, webkit-bug-importer |
| Priority: | P2 | Keywords: | InRadar |
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: |
https://bugs.webkit.org/show_bug.cgi?id=154122 https://bugs.webkit.org/show_bug.cgi?id=222840 |
||
Ryan Haddad
Two media/video-with-* tests added with r197724 are flaky on El Capitan
media/video-with-blob-url-allowed-by-csp-media-src-star.html
media/video-with-data-url-allowed-by-csp-media-src-star.html
<https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK1%20(Tests)/r197791%20(4170)/results.html>
Flakiness dashboard links:
<https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#tests=media%2Fvideo-with-blob-url-allowed-by-csp-media-src-star.html>
<https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#tests=media%2Fvideo-with-data-url-allowed-by-csp-media-src-star.html>
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Ryan Haddad
At least in this example, it looks like the ACTUAL result is correct, and the EXPECTED result is incorrect for both tests:
<https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK1%20(Tests)/r197819%20(4188)/results.html>
Alexey Proskuryakov
The tests are super flaky, failing around half of the time. Dan, could you please take a look?
There was one failure on Yosemite too, so this may not be El Capitan specific.
Ryan Haddad
Dan should be able to look at this shortly. Skipped tests to reduce noise in <http://trac.webkit.org/projects/webkit/changeset/197890>
Daniel Bates
Skipped test LayoutTests/media/video-with-file-url-allowed-by-csp-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html (added in bug #157100) in <https://trac.webkit.org/r200180>
Ryan Haddad
(In reply to comment #4)
> Skipped test
> LayoutTests/media/video-with-file-url-allowed-by-csp-media-src-star-with-
> AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html (added
> in bug #157100) in <https://trac.webkit.org/r200180>
It appears that this test is also flaky on WK2
<http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fvideo-with-file-url-allowed-by-csp-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html>
Ryan Haddad
Moved skip(In reply to comment #5)
> It appears that this test is also flaky on WK2
> <http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.
> html#showAllRuns=true&tests=media%2Fvideo-with-file-url-allowed-by-csp-media-
> src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.
> html>
Moved skip to platform/mac/TestExpectations in <http://trac.webkit.org/projects/webkit/changeset/200199>
Radar WebKit Bug Importer
<rdar://problem/26846436>
Ryan Haddad
Also skipped security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star.html in https://trac.webkit.org/r202180
Alexey Proskuryakov
Updated test expectations in <http://trac.webkit.org/r202235>.
Changed from skipping to flaky, because there is no reason to not run these tests, and it's good to have results on the dashboard. Also added one more test from this group, which was making bots red.
Alexey Proskuryakov
Skipped security/contentSecurityPolicy/video-with-blob-url-allowed-by-media-src-star.html on WebKit1 only again, because is times out every time.
Russell Epstein
security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html appears to be flaky on iOS Simulator as well, with the same diff.
Carlos Alberto Lopez Perez
The same issue happens on GTK and WPE.
https://build.webkit.org/results/WPE-Linux-64-bit-Release-Tests/r275404%20(975)/security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-diffs.html
Updated expectations on r275423
ayumi_kojima
security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star.html is flaky on iOS-14-Simulator-WK2-Tests-EWS and the flaky failure is seen on the open source directory.
security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html mentioned above seems to be still flaky on the open source.
History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=security%2FcontentSecurityPolicy%2Fvideo-with-file-url-allowed-by-media-src-star.html&test=security%2FcontentSecurityPolicy%2Fvideo-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html
Marked expectations for those two tests here: https://trac.webkit.org/changeset/281566/webkit
ayumi_kojima
For security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html,
the same images failure is seen on iOS 15 Debug and continue seeing on all other platforms.
Updated test expectations : https://trac.webkit.org/changeset/283730/webkit