Bug 203733

Summary: TestWebKitAPI.EditorStateTests.TypingAttributesTextAlignmentStartEnd is flaky in iOS simulator
Product: WebKit Reporter: Aakash Jain <aakash_jain>
Component: Tools / TestsAssignee: Wenson Hsieh <wenson_hsieh>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, ap, bdakin, commit-queue, jbedard, megan_gardner, thorton, webkit-bot-watchers-bugzilla, webkit-bug-importer, wenson_hsieh
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Speculative fix none

Aakash Jain
Reported 2019-11-01 06:11:39 PDT
TestWebKitAPI.EditorStateTests.TypingAttributesTextAlignmentStartEnd seems flaky on iOS. In https://ews-build.webkit.org/#/builders/9/builds/11291, the test passed in run-api-tests step. However, in the immediately next retry step (re-run-api-tests), it failed.
Attachments
Speculative fix (2.61 KB, patch)
2019-11-01 09:01 PDT, Wenson Hsieh 		no flags
DetailsFormatted DiffDiff
View All Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2019-11-01 06:20:35 PDT
<rdar://problem/56814640>
Wenson Hsieh
Comment 2 2019-11-01 08:11:54 PDT
I’m unable to reproduce this failure locally (both by using --iterations 100, and by wrapping the test in a loop for 100 iterations and using --no-timeout).
Wenson Hsieh
Comment 3 2019-11-01 09:01:28 PDT
Created attachment 382584 [details] Speculative fix
Jonathan Bedard
Comment 4 2019-11-01 09:31:02 PDT
We had a string of failures awhile back on post-commit testers, https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.EditorStateTests.TypingAttributesTextAlignmentStartEnd&before_id=251111, but I doubt that's the same bug. My hunch here is that this is a timing problem and that our EWS bots are much faster than our trunk bots.
Wenson Hsieh
Comment 5 2019-11-01 09:36:11 PDT
(In reply to Jonathan Bedard from comment #4) > We had a string of failures awhile back on post-commit testers, > https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI. > EditorStateTests.TypingAttributesTextAlignmentStartEnd&before_id=251111, but > I doubt that's the same bug. > > My hunch here is that this is a timing problem and that our EWS bots are > much faster than our trunk bots. Those failures appear to be due to an unrelated assertion that was presumably caused by r251090 (and later fixed in a rollout in r251106). ~2000 API tests were all failing in tandem during that time.
Tim Horton
Comment 6 2019-11-01 10:23:45 PDT
Comment on attachment 382584 [details] Speculative fix View in context: https://bugs.webkit.org/attachment.cgi?id=382584&action=review > Tools/TestWebKitAPI/EditingTestHarness.mm:197 > + const NSTimeInterval loggingTimeout = 3; I think you should consider not having a timeout here, just depending on the test-wide timeout, to better integrate with e.g. --no-timeout
Wenson Hsieh
Comment 7 2019-11-01 10:28:40 PDT
(In reply to Tim Horton from comment #6) > Comment on attachment 382584 [details] > Speculative fix > > View in context: > https://bugs.webkit.org/attachment.cgi?id=382584&action=review > > > Tools/TestWebKitAPI/EditingTestHarness.mm:197 > > + const NSTimeInterval loggingTimeout = 3; > > I think you should consider not having a timeout here, just depending on the > test-wide timeout, to better integrate with e.g. --no-timeout (We chatted about this over Slack) The timeout here is just for logging, to make sure that something useful gets logged to stdout in the case where the test does timeout. This would still respect any --time-out option specified via the harness.
WebKit Commit Bot
Comment 8 2019-11-01 11:26:14 PDT
The commit-queue encountered the following flaky tests while processing attachment 382584 [details]: requestidlecallback/requestidlecallback-document-gc.html bug 203745 (author: rniwa@webkit.org) The commit-queue is continuing to process your patch.
WebKit Commit Bot
Comment 9 2019-11-01 11:27:07 PDT
Comment on attachment 382584 [details] Speculative fix Clearing flags on attachment: 382584 Committed r251931: <https://trac.webkit.org/changeset/251931>
WebKit Commit Bot
Comment 10 2019-11-01 11:27:08 PDT
All reviewed patches have been landed. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.