Bug 188869

Summary:

API tests should output json results

Product:

WebKit

Reporter:

Jonathan Bedard <jbedard>

Component:

Tools / Tests

Assignee:

Jonathan Bedard <jbedard>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

aakash_jain, commit-queue, ews-watchlist, glenn, lforschler, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

See Also:

https://bugs.webkit.org/show_bug.cgi?id=205579

Attachments:

Description	Flags
Patch	none
Patch	none
Patch	none
Patch	none
Patch for landing	none

Jonathan Bedard

Reported 2018-08-22 16:39:38 PDT

Requested by Aakash Jain for adding API tests to EWS.

Attachments
Patch (4.56 KB, patch) 2018-08-22 16:47 PDT, Jonathan Bedard	no flags	Details Formatted Diff Diff
Patch (5.29 KB, patch) 2018-08-22 17:11 PDT, Jonathan Bedard	no flags	Details Formatted Diff Diff
Patch (5.55 KB, patch) 2018-08-23 14:24 PDT, Jonathan Bedard	no flags	Details Formatted Diff Diff
Patch (5.53 KB, patch) 2018-08-23 14:50 PDT, Jonathan Bedard	no flags	Details Formatted Diff Diff
Patch for landing (5.50 KB, patch) 2018-08-23 15:12 PDT, Jonathan Bedard	no flags	Details Formatted Diff Diff
Show Obsolete (4) View All Add attachment proposed patch, testcase, etc.

Jonathan Bedard

Comment 1 2018-08-22 16:39:49 PDT

<rdar://problem/43615652>

Jonathan Bedard

Comment 2 2018-08-22 16:47:48 PDT

Created attachment 347876 [details] Patch

Jonathan Bedard

Comment 3 2018-08-22 17:11:50 PDT

Created attachment 347879 [details] Patch

Aakash Jain

Comment 4 2018-08-22 17:36:28 PDT

Comment on attachment 347879 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=347879&action=review Thanks for the patch. Looks good, one comment below. > Tools/Scripts/webkitpy/api_tests/manager.py:208 > + result_dictionary[test] = {'status': 'Skipped', 'output': None} Do we need to add skipped tests in the json file? EWS doesn't need this, but I understand some other tools might use this information. If so, we should consider dividing the tests by keys. e.g.: { "skipped": [ {"test_name1": <test_log1>}, {"test_name2": <test_log2>} ], "failed": [ {"test_name3": <test_log3>} ], "crashed": [ {"test_name4": <test_log4>} ], "timedout": [ {"test_name5": <test_log5>} ] }

Jonathan Bedard

Comment 5 2018-08-23 08:45:20 PDT

Comment on attachment 347879 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=347879&action=review >> Tools/Scripts/webkitpy/api_tests/manager.py:208 >> + result_dictionary[test] = {'status': 'Skipped', 'output': None} > > Do we need to add skipped tests in the json file? EWS doesn't need this, but I understand some other tools might use this information. > > If so, we should consider dividing the tests by keys. e.g.: > > { > "skipped": [ {"test_name1": <test_log1>}, {"test_name2": <test_log2>} ], > "failed": [ {"test_name3": <test_log3>} ], > "crashed": [ {"test_name4": <test_log4>} ], > "timedout": [ {"test_name5": <test_log5>} ] > } We do not, but, for API tests, 'SKIPPED' does not mean 'DISABLED', it means that a signal was received forcing the program to exit early. In practice, this means that EWS should never see SKIPPED tests and if it did, something has gone terribly wrong.

Jonathan Bedard

Comment 6 2018-08-23 14:24:06 PDT

Created attachment 347952 [details] Patch

Jonathan Bedard

Comment 7 2018-08-23 14:50:08 PDT

Created attachment 347956 [details] Patch

Jonathan Bedard

Comment 8 2018-08-23 15:12:25 PDT

Created attachment 347960 [details] Patch for landing

WebKit Commit Bot

Comment 9 2018-08-23 15:50:58 PDT

Comment on attachment 347960 [details] Patch for landing Clearing flags on attachment: 347960 Committed r235252: <https://trac.webkit.org/changeset/235252>

WebKit Commit Bot

Comment 10 2018-08-23 15:51:00 PDT

All reviewed patches have been landed. Closing bug.

Radar WebKit Bug Importer

Comment 11 2018-08-23 15:51:26 PDT

<rdar://problem/43662254>

Note You need to log in before you can comment on or make changes to this bug.