196006 2019-03-20 07:43:08 -0700 Flaky API Test TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 2019-05-17 20:46:57 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=196007 https://bugs.webkit.org/show_bug.cgi?id=196630 InRadar P2 Normal --- 1 aakash_jain wenson_hsieh aakash_jain ap lforschler pxlcoder ryanhaddad thorton tsavell webkit-bot-watchers-bugzilla webkit-bug-importer wenson_hsieh oldest_to_newest 1518882 0 aakash_jain 2019-03-20 07:43:08 -0700 TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput seems flaky. In https://ews-build.webkit-uat.org/#/builders/20/builds/2040, the test Failed in run-api-tests step. However, in the immediately next retry step (re-run-api-tests), it passed. 1518912 1 aakash_jain 2019-03-20 09:17:47 -0700 This test was added in https://trac.webkit.org/changeset/235201 1519915 2 aakash_jain 2019-03-21 14:00:32 -0700 Happened again in https://ews-build.webkit-uat.org/#/builders/20/builds/2198 TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 2019-03-21 12:08:07.471 TestWebKitAPI[32628:65791725] Displayed 7 for <input type='search' inputmode='tel' pattern=''>. Expected 5. 2019-03-21 12:08:09.257 TestWebKitAPI[32628:65791725] Displayed 5 for <input type='tel' inputmode='numeric' pattern=''>. Expected 2. /Volumes/Data/worker/iOS-11-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true /Volumes/Data/worker/iOS-11-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true 1520278 3 aakash_jain 2019-03-22 06:17:09 -0700 Happening very frequently, e.g.: https://ews-build.webkit-uat.org/#/builders/20/builds/2232 https://ews-build.webkit-uat.org/#/builders/20/builds/2224 https://ews-build.webkit-uat.org/#/builders/20/builds/2205 TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 2019-03-21 17:39:00.638 TestWebKitAPI[20388:66464848] Displayed 8 for <input type='search' inputmode='url' pattern=''>. Expected 3. /Volumes/Data/worker/iOS-12-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 2019-03-21 13:58:33.025 TestWebKitAPI[5458:274706] Displayed 3 for <input type='text' inputmode='' pattern=''>. Expected 0. /Volumes/Data/worker/iOS-12-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true 1520697 4 aakash_jain 2019-03-24 04:52:24 -0700 Can we prioritize this? I am working on EWS for API tests, and this flaky failure is creating problem (e.g.: https://ews-build.webkit-uat.org/#/builders/20/builds/2256). 1521038 5 wenson_hsieh 2019-03-25 14:41:42 -0700 I can repro this somewhat reliably by running the test for a dozen or so iterations in a loop. 1524193 6 aakash_jain 2019-04-04 04:41:36 -0700 Happening very frequently, Recent occurrences: https://ews-build.webkit-uat.org/#/builders/20/builds/2630 https://ews-build.webkit-uat.org/#/builders/20/builds/2624 https://ews-build.webkit-uat.org/#/builders/20/builds/2623 https://ews-build.webkit-uat.org/#/builders/20/builds/2616 https://ews-build.webkit-uat.org/#/builders/20/builds/2614 https://ews-build.webkit-uat.org/#/builders/20/builds/2612 https://ews-build.webkit-uat.org/#/builders/20/builds/2604 https://ews-build.webkit-uat.org/#/builders/20/builds/2599 https://ews-build.webkit-uat.org/#/builders/20/builds/2594 https://ews-build.webkit-uat.org/#/builders/20/builds/2593 https://ews-build.webkit-uat.org/#/builders/20/builds/2591 https://ews-build.webkit-uat.org/#/builders/20/builds/2588 https://ews-build.webkit-uat.org/#/builders/20/builds/2584 https://ews-build.webkit-uat.org/#/builders/20/builds/2564 https://ews-build.webkit-uat.org/#/builders/20/builds/2563 https://ews-build.webkit-uat.org/#/builders/20/builds/2554 https://ews-build.webkit-uat.org/#/builders/20/builds/2552 https://ews-build.webkit-uat.org/#/builders/20/builds/2551 https://ews-build.webkit-uat.org/#/builders/20/builds/2544 https://ews-build.webkit-uat.org/#/builders/20/builds/2535 https://ews-build.webkit-uat.org/#/builders/20/builds/2530 https://ews-build.webkit-uat.org/#/builders/20/builds/2521 https://ews-build.webkit-uat.org/#/builders/20/builds/2506 https://ews-build.webkit-uat.org/#/builders/20/builds/2497 https://ews-build.webkit-uat.org/#/builders/20/builds/2496 https://ews-build.webkit-uat.org/#/builders/20/builds/2492 https://ews-build.webkit-uat.org/#/builders/20/builds/2490 https://ews-build.webkit-uat.org/#/builders/20/builds/2480 https://ews-build.webkit-uat.org/#/builders/20/builds/2478 https://ews-build.webkit-uat.org/#/builders/20/builds/2473 https://ews-build.webkit-uat.org/#/builders/20/builds/2450 https://ews-build.webkit-uat.org/#/builders/20/builds/2446 https://ews-build.webkit-uat.org/#/builders/20/builds/2429 https://ews-build.webkit-uat.org/#/builders/20/builds/2426 https://ews-build.webkit-uat.org/#/builders/20/builds/2415 https://ews-build.webkit-uat.org/#/builders/20/builds/2413 https://ews-build.webkit-uat.org/#/builders/20/builds/2407 https://ews-build.webkit-uat.org/#/builders/20/builds/2403 https://ews-build.webkit-uat.org/#/builders/20/builds/2389 https://ews-build.webkit-uat.org/#/builders/20/builds/2385 https://ews-build.webkit-uat.org/#/builders/20/builds/2384 https://ews-build.webkit-uat.org/#/builders/20/builds/2376 https://ews-build.webkit-uat.org/#/builders/20/builds/2374 https://ews-build.webkit-uat.org/#/builders/20/builds/2366 https://ews-build.webkit-uat.org/#/builders/20/builds/2359 https://ews-build.webkit-uat.org/#/builders/20/builds/2335 https://ews-build.webkit-uat.org/#/builders/20/builds/2334 In following builds, this test was flaky in a manner that EWS thought that the patch being tested introduced this test failure (failed twice with patch, passed without patch): https://ews-build.webkit-uat.org/#/builders/20/builds/2498 https://ews-build.webkit-uat.org/#/builders/20/builds/2378 https://ews-build.webkit-uat.org/#/builders/20/builds/2337 https://ews-build.webkit-uat.org/#/builders/20/builds/2329 https://ews-build.webkit-uat.org/#/builders/20/builds/2311 https://ews-build.webkit-uat.org/#/builders/20/builds/2308 https://ews-build.webkit-uat.org/#/builders/20/builds/2256 1533505 7 webkit-bug-importer 2019-05-06 13:11:21 -0700 <rdar://problem/50509880> 1533529 8 aakash_jain 2019-05-06 13:33:26 -0700 This test was disabled in https://bugs.webkit.org/show_bug.cgi?id=196630. It would be nice to fix and re-enable it. 1537372 9 wenson_hsieh 2019-05-17 20:46:57 -0700 I reenabled this in https://trac.webkit.org/changeset/245338/webkit.