Bug 188795

Summary: REGRESSION(r234993): Test results page ignores TestExpectations, shows 1800 failures
Product: WebKit Reporter: Michael Catanzaro <mcatanzaro>
Component: Tools / TestsAssignee: Simon Fraser (smfr) <simon.fraser>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, bugs-noreply, commit-queue, lforschler, mcatanzaro, simon.fraser, thorton, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: PC   
OS: Linux   
Attachments:
Description Flags
Patch none

Description Michael Catanzaro 2018-08-21 06:24:03 PDT 
Since r234993 "Modernize results.html" our test results page has gone crazy. For example:

https://build.webkit.org/results/GTK Linux 64-bit Release (Tests)/r235110 (7847)/results.html

shows 1815 tests that failed pixel/audio diff. But the bot dashboard reports only 13 layout test failures. And looking at the text file, there were indeed only 13 failures:

https://build.webkit.org/builders/GTK Linux 64-bit Release (Tests)/builds/7847/steps/layout-test/logs/stdio

The problem seems to be that the platform TestExpectations file is now ignored, so all the expected failures are showing up as unexpected failures.

It seems this issue only affects GTK: the results for macOS, iOS, Windows, and WPE are all fine.
Comment 1 Michael Catanzaro 2018-08-21 06:25:48 PDT 
(In reply to Michael Catanzaro from comment #0)
> Since r234993 "Modernize results.html" our test results page has gone crazy.
> For example:
> 
> https://build.webkit.org/results/GTK Linux 64-bit Release (Tests)/r235110
> (7847)/results.html

Unrelated: it looks like Epiphany's code to percent-encode URLs when copied from the address bar is broken. :)
Comment 3 Simon Fraser (smfr) 2018-08-21 08:30:54 PDT 
It's showing expected failures; I'll fix.
Comment 4 Simon Fraser (smfr) 2018-08-21 08:41:59 PDT 
Created attachment 347643 [details]
Patch
Comment 5 WebKit Commit Bot 2018-08-21 11:08:32 PDT 
Comment on attachment 347643 [details]
Patch

Clearing flags on attachment: 347643

Committed r235126: <https://trac.webkit.org/changeset/235126>
Comment 6 WebKit Commit Bot 2018-08-21 11:08:34 PDT 
All reviewed patches have been landed.  Closing bug.
Comment 7 Radar WebKit Bug Importer 2018-08-21 11:09:23 PDT 
<rdar://problem/43568697>
Comment 8 Michael Catanzaro 2018-08-21 11:51:07 PDT 
Thanks Simon!