This has come in handy enough times over the last 2 months that it's worth committing. What this script does is read through the stdout of a test run (or runs) and determines which tests were run before some failing test. It then cross-references successful test runs, with the assumption that if a given test preceded a successful run of another test, it is unlikely to be the cause of the failure. We've been able to track down a few flakey test failures using this script, or a predecessor to it.
Created attachment 346782 [details] Patch
Comment on attachment 346782 [details] Patch Need to figure out why these tests failed on the bots but passed locally.
Created attachment 346807 [details] Patch
Comment on attachment 346807 [details] Patch Attachment 346807 [details] did not pass win-ews (win): Output: https://webkit-queues.webkit.org/results/8828955 New failing tests: http/tests/security/video-poster-cross-origin-crash2.html
Created attachment 346963 [details] Archive of layout-test-results from ews205 for win-future The attached test failures were seen while running run-webkit-tests on the win-ews. Bot: ews205 Port: win-future Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment on attachment 346807 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=346807&action=review > Tools/Scripts/webkitpy/flakiness_analyzer/flakiness_analyzer.py:153 > + index_to_print = int(raw_input('Pick which of the sequences (0-{}) to print: '.format(len(steps_candidates - 1)))) I hit a bug here when trying to test this locally. This should be: .format(len(steps_candidates) - 1))
Comment on attachment 346807 [details] Patch This has been requesting review for more than one year. If this is still needed, please rebase and re-request review.