Bug 188280

Summary: Explain test name matching in run-api-tests help
Product: WebKit Reporter: Jonathan Bedard <jbedard>
Component: Tools / TestsAssignee: Jonathan Bedard <jbedard>
Status: RESOLVED FIXED    
Severity: Normal CC: ap, commit-queue, dbates, ews-watchlist, glenn, lforschler, rniwa, webkit-bug-importer
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=187893
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews205 for win-future
none
Patch
none
Archive of layout-test-results from ews206 for win-future
none
Patch for landing none

Description Jonathan Bedard 2018-08-02 15:24:35 PDT
After discussion on <https://bugs.webkit.org/show_bug.cgi?id=187893>, it seems that the help message in run-api-tests does not adequately explain how test matching works.
Comment 1 Jonathan Bedard 2018-08-02 15:43:52 PDT
Created attachment 346424 [details]
Patch
Comment 2 Daniel Bates 2018-08-02 19:23:44 PDT
Comment on attachment 346424 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=346424&action=review

> Tools/Scripts/webkitpy/api_tests/run_api_tests.py:128
> +                    'format <binary>.<suite>.<test> or <suite>.<test>. Note that in the case where a binary is not ' +

What are the binaries?
Comment 3 EWS Watchlist 2018-08-03 02:33:19 PDT
Comment on attachment 346424 [details]
Patch

Attachment 346424 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/8747593

New failing tests:
imported/blink/transitions/unprefixed-transform.html
Comment 4 EWS Watchlist 2018-08-03 02:33:30 PDT
Created attachment 346466 [details]
Archive of layout-test-results from ews205 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews205  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment 5 Jonathan Bedard 2018-08-03 16:02:33 PDT
(In reply to Daniel Bates from comment #2)
> Comment on attachment 346424 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=346424&action=review
> 
> > Tools/Scripts/webkitpy/api_tests/run_api_tests.py:128
> > +                    'format <binary>.<suite>.<test> or <suite>.<test>. Note that in the case where a binary is not ' +
> 
> What are the binaries?

If you would like the binary names in the help message, can we land <https://bugs.webkit.org/show_bug.cgi?id=187893> first so I don't have to duplicate them? That way the help message could grab from the static arrays, ensuring that if new binaries are added, the help message would automatically pick them up.
Comment 6 Jonathan Bedard 2018-08-20 15:54:20 PDT
Created attachment 347553 [details]
Patch
Comment 7 EWS Watchlist 2018-08-21 17:51:24 PDT
Comment on attachment 347553 [details]
Patch

Attachment 347553 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/8937142

New failing tests:
http/tests/security/canvas-remote-read-remote-video-blocked-no-crossorigin.html
Comment 8 EWS Watchlist 2018-08-21 17:51:36 PDT
Created attachment 347744 [details]
Archive of layout-test-results from ews206 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews206  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment 9 Daniel Bates 2018-08-22 22:50:00 PDT
Comment on attachment 347553 [details]
Patch

I don’t like this feature and would prefer to remove it than advertise it. If you are going to land this patch then please at least re-read and spell and grammar check what you wrote because it contains obvious errors and does not read well.
Comment 10 Jonathan Bedard 2018-08-23 09:00:49 PDT
Created attachment 347925 [details]
Patch for landing
Comment 11 WebKit Commit Bot 2018-08-23 09:40:00 PDT
Comment on attachment 347925 [details]
Patch for landing

Clearing flags on attachment: 347925

Committed r235227: <https://trac.webkit.org/changeset/235227>
Comment 12 WebKit Commit Bot 2018-08-23 09:40:02 PDT
All reviewed patches have been landed.  Closing bug.