| Summary: | API tests should output json results | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Jonathan Bedard <jbedard> | ||||||||||||
| Component: | Tools / Tests | Assignee: | Jonathan Bedard <jbedard> | ||||||||||||
| Status: | RESOLVED FIXED | ||||||||||||||
| Severity: | Normal | CC: | aakash_jain, commit-queue, ews-watchlist, glenn, lforschler, webkit-bug-importer | ||||||||||||
| Priority: | P2 | Keywords: | InRadar | ||||||||||||
| Version: | WebKit Nightly Build | ||||||||||||||
| Hardware: | Unspecified | ||||||||||||||
| OS: | Unspecified | ||||||||||||||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=205579 | ||||||||||||||
| Attachments: |
|
||||||||||||||
|
Description
Jonathan Bedard
2018-08-22 16:39:38 PDT
Created attachment 347876 [details]
Patch
Created attachment 347879 [details]
Patch
Comment on attachment 347879 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=347879&action=review Thanks for the patch. Looks good, one comment below. > Tools/Scripts/webkitpy/api_tests/manager.py:208 > + result_dictionary[test] = {'status': 'Skipped', 'output': None} Do we need to add skipped tests in the json file? EWS doesn't need this, but I understand some other tools might use this information. If so, we should consider dividing the tests by keys. e.g.: { "skipped": [ {"test_name1": <test_log1>}, {"test_name2": <test_log2>} ], "failed": [ {"test_name3": <test_log3>} ], "crashed": [ {"test_name4": <test_log4>} ], "timedout": [ {"test_name5": <test_log5>} ] } Comment on attachment 347879 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=347879&action=review >> Tools/Scripts/webkitpy/api_tests/manager.py:208 >> + result_dictionary[test] = {'status': 'Skipped', 'output': None} > > Do we need to add skipped tests in the json file? EWS doesn't need this, but I understand some other tools might use this information. > > If so, we should consider dividing the tests by keys. e.g.: > > { > "skipped": [ {"test_name1": <test_log1>}, {"test_name2": <test_log2>} ], > "failed": [ {"test_name3": <test_log3>} ], > "crashed": [ {"test_name4": <test_log4>} ], > "timedout": [ {"test_name5": <test_log5>} ] > } We do not, but, for API tests, 'SKIPPED' does not mean 'DISABLED', it means that a signal was received forcing the program to exit early. In practice, this means that EWS should never see SKIPPED tests and if it did, something has gone terribly wrong. Created attachment 347952 [details]
Patch
Created attachment 347956 [details]
Patch
Created attachment 347960 [details]
Patch for landing
Comment on attachment 347960 [details] Patch for landing Clearing flags on attachment: 347960 Committed r235252: <https://trac.webkit.org/changeset/235252> All reviewed patches have been landed. Closing bug. |