59894 2011-05-01 14:22:13 -0700 waterfall or console doesn't report the number of failures for new-run-webkit-tests 2011-06-06 23:56:13 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) All All RESOLVED DUPLICATE 62178 P2 Normal --- 34984 1 rniwa webkit-unassigned abarth dpranke eric tony oldest_to_newest 396046 0 rniwa 2011-05-01 14:22:13 -0700 For old-run-webkit-tests, build.webkit.org/waterfall reports how many tests are failed. However, such information isn't available for bots that run new-run-webkit-tests. See http://build.webkit.org/builders/SnowLeopard%20Intel%20Release%20%28WebKit2%20Tests%29/builds/11263 http://build.webkit.org/builders/Leopard%20Intel%20Release%20%28NRWT%29/builds/70 396452 1 tony 2011-05-02 10:31:47 -0700 The number comes from parsing the std output of the script. Since NRWT has different output, there's no number. We could copy parser used by the chromium bots (that would also give us other stuff like listing the failing tests and linking to the flakiness dashboard). 396487 2 rniwa 2011-05-02 11:11:14 -0700 (In reply to comment #1) > The number comes from parsing the std output of the script. Since NRWT has different output, there's no number. > > We could copy parser used by the chromium bots (that would also give us other stuff like listing the failing tests and linking to the flakiness dashboard). I find the number of failures to be much more useful than a partial listing of failed list. 396496 3 tony 2011-05-02 11:19:50 -0700 (In reply to comment #2) > (In reply to comment #1) > > We could copy parser used by the chromium bots (that would also give us other stuff like listing the failing tests and linking to the flakiness dashboard). > > I find the number of failures to be much more useful than a partial listing of failed list. This is not an either-or situation :) The Chromium bots have both the number of failures and the list of failing tests. Perhaps you're saying that listing the failing tests makes it harder to scan and see the failed number? I think abarth was advocating having the list of failing tests so you don't have to make 2 more clicks (results dir + results.html) to figure out which test failed. This could be done as a separate patch. 396509 4 rniwa 2011-05-02 11:35:54 -0700 (In reply to comment #3) > This is not an either-or situation :) The Chromium bots have both the number of failures and the list of failing tests. Perhaps you're saying that listing the failing tests makes it harder to scan and see the failed number? Yeah. When there are many tests failing (20+), it's useful to have a number of tests failing so that when I skim through the list of builds, I can easily find where the number increased. > I think abarth was advocating having the list of failing tests so you don't have to make 2 more clicks (results dir + results.html) to figure out which test failed. This could be done as a separate patch. I think the list of failures will be useful when there are less than ~5 test failures. 416383 5 rniwa 2011-06-06 23:56:13 -0700 Oops, I forgot about this bug. Merging it to the bug 62178 since it already has a patch. *** This bug has been marked as a duplicate of bug 62178 ***