51371 2010-12-20 19:30:16 -0800 commit-queue can't upload failure diffs from the first flake on a double-flake 2017-07-18 08:29:45 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 NEW P2 Normal --- 52048 1 eric webkit-unassigned abarth koz oldest_to_newest 324721 0 eric 2010-12-20 19:30:16 -0800 commit-queue can't upload failure diffs from the first flake on a double-flake def _test_patch(self): if self._patch.is_rollout(): return True if self._test(): return True first_failing_tests = self._failing_tests_from_last_run() if self._test(): self._report_flaky_tests(first_failing_tests) return True second_failing_tests = self._failing_tests_from_last_run() if first_failing_tests != second_failing_tests: self._report_flaky_tests(first_failing_tests + second_failing_tests) return False if self._build_and_test_without_patch(): raise self._script_error # The error from the previous ._test() run is real, report it. return False # Tree must be red, just retry later. Currently _failing_tests_from_last_run() only returns the test names (paths) for the failures. It does not bottle up the results information. So when we run the tests a second time, old-run-webkit-tests will delete the results from the first run. Then when we go to try and report the test as flaky, we don't have any results diff to upload. 324779 1 eric 2010-12-20 21:42:41 -0800 We already have a TestResult command in layout_tests/layout_package, so I'd rather not re-invent the wheel. But it doesn't look like it's meant for dealing with a single result from a single test. 324791 2 eric 2010-12-20 22:25:11 -0800 We still need better handling here. However, double-flake reporting was removed as part of bug 51272. 331884 3 eric 2011-01-10 13:23:33 -0800 Actually in fixing bug 52048 I realized that double-flakes were the *only* times we were able to upload failure diffs. :) However, once bug 52048 lands, we could (carefully) re-enable double-flake reporting and uploads would work.