50263 2010-11-30 13:00:54 -0800 Exception in flaky test reporting code in commit-queue 2010-12-20 20:35:00 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 RESOLVED DUPLICATE 51272 P2 Normal --- 51272 1 eric webkit-unassigned abarth oldest_to_newest 315254 0 eric 2010-11-30 13:00:54 -0800 Exception in flaky test reporting code in commit-queue From the log: === BUILD AGGREGATE TARGET All OF PROJECT DumpRenderTree WITH CONFIGURATION Release === Check dependencies ** BUILD SUCCEEDED ** Compiling Java tests make: Nothing to be done for `default'. Running tests from /Projects/CommitQueue/LayoutTests Testing 22075 test cases. accessibility ..................................................................................... animations ........................................................... animations/stop-animation-on-suspend.html -> failed Exiting early after 1 failures. 144 tests run. 44.81s total testing time 143 test cases (99%) succeeded 1 test case (<1%) had incorrect layout 1 test case (<1%) had stderr output Patch does not pass tests Traceback (most recent call last): File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/bot/queueengine.py", line 108, in run if not self._delegate.process_work_item(work_item): File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/commands/queues.py", line 265, in process_work_item if task.run(): File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/bot/commitqueuetask.py", line 188, in run if not self._test_patch(): File "/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/tool/bot/commitqueuetask.py", line 172, in _test_patch self._report_flaky_tests(first_failing_tests + second_failing_tests) TypeError: unsupported operand type(s) for +: 'NoneType' and 'list' Exception while preparing queue Sleeping until 2010-11-30 12:57:20 (2 mins). 315336 1 eric 2010-11-30 15:04:28 -0800 This would happen any time the tests fail on the first run, but we fail to get results from that failure. The code in question: def _test_patch(self): if self._patch.is_rollout(): return True if self._test(): return True first_failing_tests = self._failing_tests_from_last_run() if self._test(): self._report_flaky_tests(first_failing_tests) return True second_failing_tests = self._failing_tests_from_last_run() if first_failing_tests != second_failing_tests: self._report_flaky_tests(first_failing_tests + second_failing_tests) return False The fix is straightforward. We just make "first_failing_tests != second_failing_tests" into "first_failing_tests and (first_failing_tests != second_failing_tests)". However we might also want to add some special logging to track this case, as the layout tests shoudl not fail w/o us being able to get the results from the run! 324750 2 eric 2010-12-20 20:35:00 -0800 I expect this to be fixed by bug 51272. *** This bug has been marked as a duplicate of bug 51272 ***