193595 2019-01-18 15:13:50 -0800 [build.webkit.org] api tests failures are incorrectly named as unit test failures 2019-01-18 16:53:25 -0800 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap commit-queue dbates ews-watchlist jlewis3 lforschler ryanhaddad tsavell webkit-bug-importer oldest_to_newest 1496057 0 aakash_jain 2019-01-18 15:13:50 -0800 api tests failures are incorrectly named as unit test failures by build.webkit.org e.g.: https://build.webkit.org/builders/Apple%20iOS%2012%20Simulator%20Debug%20WK2%20%28Tests%29/builds/1759/steps/run-api-tests says: "2 unit tests failed or timed out", However these are not unit tests, these are api tests. We should fix the build.webkit.org configuration to name it api tests instead of unit tests. 1496064 1 359547 aakash_jain 2019-01-18 15:22:18 -0800 Created attachment 359547 Proposed patch The original patch to add api test to build.webkit.org configuration named it unit tests: https://trac.webkit.org/changeset/89054/webkit 1496065 2 ews-watchlist 2019-01-18 15:24:11 -0800 Attachment 359547 did not pass style-queue: ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/steps_unittest.py:276: [RunAPITestsTest.assertFailures] Undefined variable 'RunAPITests' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/factories.py:93: [TestFactory.__init__] Undefined variable 'RunAPITests' [pylint/E0602] [5] Total errors found: 2 in 4 files If any of these errors are false positives, please file a bug against check-webkit-style. 1496070 3 lforschler 2019-01-18 15:37:49 -0800 lgtm, assuming the style errors are false. 1496086 4 359547 commit-queue 2019-01-18 16:10:24 -0800 Comment on attachment 359547 Proposed patch Clearing flags on attachment: 359547 Committed r240187: <https://trac.webkit.org/changeset/240187> 1496087 5 commit-queue 2019-01-18 16:10:26 -0800 All reviewed patches have been landed. Closing bug. 1496088 6 webkit-bug-importer 2019-01-18 16:11:29 -0800 <rdar://problem/47400323> 1496104 7 dbates 2019-01-18 16:53:25 -0800 po-tay-to po-tah-to run-api-tests is a misnomer. It has long since outgrown itself from its humble beginnings of being used to test WebKit API/SPI. It now runs WTF tests and WebCore tests. We typically use the term "API" to refer to the public interfaces that are available to customers. I would argue that "unit tests" is probably more accurate for what run-api-tests does. Oh well... 359547 2019-01-18 15:22:18 -0800 2019-01-18 16:10:24 -0800 Proposed patch build-webkit-rename-unit-to-api-test.patch text/plain 3952 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI0MDE4MSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDE5LTAxLTE4ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbYnVpbGQud2Via2l0Lm9yZ10gYXBpIHRlc3RzIGZhaWx1cmVzIGFyZSBp bmNvcnJlY3RseSBuYW1lZCBhcyB1bml0IHRlc3QgZmFpbHVyZXMKKyAgICAgICAgaHR0cHM6Ly9i dWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTE5MzU5NQorCisgICAgICAgIFJldmlld2Vk IGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgICogQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQu d2Via2l0Lm9yZy1jb25maWcvZmFjdG9yaWVzLnB5OiBSZW5hbWVkIFJ1blVuaXRUZXN0cyB0byBS dW5BUElUZXN0cy4KKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3Jn LWNvbmZpZy9zdGVwcy5weTogRGl0dG8uCisgICAgICAgICogQnVpbGRTbGF2ZVN1cHBvcnQvYnVp bGQud2Via2l0Lm9yZy1jb25maWcvc3RlcHNfdW5pdHRlc3QucHk6IERpdHRvLgorCiAyMDE5LTAx LTE4ICBDaHJpcyBEdW1leiAgPGNkdW1lekBhcHBsZS5jb20+CiAKICAgICAgICAgUmVncmVzc2lv bihQU09OKSBDb250ZW50IGJsb2NrZXJzIGFyZSBzb21ldGltZXMgbG9zdCBvbiBiYWNrIG5hdmln YXRpb24gY3Jvc3Mtc2l0ZQpJbmRleDogVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Vi a2l0Lm9yZy1jb25maWcvZmFjdG9yaWVzLnB5Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFRvb2xzL0J1aWxkU2xh dmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL2ZhY3Rvcmllcy5weQkocmV2aXNpb24g MjQwMTgxKQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25m aWcvZmFjdG9yaWVzLnB5CSh3b3JraW5nIGNvcHkpCkBAIC05MCw3ICs5MCw3IEBAIGNsYXNzIFRl c3RGYWN0b3J5KEZhY3RvcnkpOgogICAgICAgICAgICAgc2VsZi5hZGRTdGVwKHNlbGYuTGF5b3V0 VGVzdENsYXNzKCkpCiAKICAgICAgICAgaWYgcGxhdGZvcm0uc3RhcnRzd2l0aCgnd2luJykgb3Ig cGxhdGZvcm0uc3RhcnRzd2l0aCgnbWFjJykgb3IgcGxhdGZvcm0uc3RhcnRzd2l0aCgnaW9zLXNp bXVsYXRvcicpOgotICAgICAgICAgICAgc2VsZi5hZGRTdGVwKFJ1blVuaXRUZXN0cygpKQorICAg ICAgICAgICAgc2VsZi5hZGRTdGVwKFJ1bkFQSVRlc3RzKCkpCiAgICAgICAgIHNlbGYuYWRkU3Rl cChSdW5QeXRob25UZXN0cygpKQogICAgICAgICBzZWxmLmFkZFN0ZXAoUnVuUGVybFRlc3RzKCkp CiAgICAgICAgIHNlbGYuYWRkU3RlcChSdW5CaW5kaW5nc1Rlc3RzKCkpCkluZGV4OiBUb29scy9C dWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3JnLWNvbmZpZy9zdGVwcy5weQo9PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3JnLWNvbmZpZy9z dGVwcy5weQkocmV2aXNpb24gMjQwMTgxKQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVp bGQud2Via2l0Lm9yZy1jb25maWcvc3RlcHMucHkJKHdvcmtpbmcgY29weSkKQEAgLTUwMywxMiAr NTAzLDEyIEBAIGNsYXNzIFJ1bkRhc2hib2FyZFRlc3RzKFJ1bldlYktpdFRlc3RzKToKICAgICAg ICAgcmV0dXJuIFJ1bldlYktpdFRlc3RzLnN0YXJ0KHNlbGYpCiAKIAotY2xhc3MgUnVuVW5pdFRl c3RzKFRlc3RXaXRoRmFpbHVyZUNvdW50KToKK2NsYXNzIFJ1bkFQSVRlc3RzKFRlc3RXaXRoRmFp bHVyZUNvdW50KToKICAgICBuYW1lID0gInJ1bi1hcGktdGVzdHMiCi0gICAgZGVzY3JpcHRpb24g PSBbInVuaXQgdGVzdHMgcnVubmluZyJdCi0gICAgZGVzY3JpcHRpb25Eb25lID0gWyJ1bml0LXRl c3RzIl0KKyAgICBkZXNjcmlwdGlvbiA9IFsiYXBpIHRlc3RzIHJ1bm5pbmciXQorICAgIGRlc2Ny aXB0aW9uRG9uZSA9IFsiYXBpLXRlc3RzIl0KICAgICBjb21tYW5kID0gWyJweXRob24iLCAiLi9U b29scy9TY3JpcHRzL3J1bi1hcGktdGVzdHMiLCAiLS1uby1idWlsZCIsIFdpdGhQcm9wZXJ0aWVz KCItLSUoY29uZmlndXJhdGlvbilzIiksICItLXZlcmJvc2UiXQotICAgIGZhaWxlZFRlc3RzRm9y bWF0U3RyaW5nID0gIiVkIHVuaXQgdGVzdCVzIGZhaWxlZCBvciB0aW1lZCBvdXQiCisgICAgZmFp bGVkVGVzdHNGb3JtYXRTdHJpbmcgPSAiJWQgYXBpIHRlc3QlcyBmYWlsZWQgb3IgdGltZWQgb3V0 IgogCiAgICAgZGVmIHN0YXJ0KHNlbGYpOgogICAgICAgICBhcHBlbmRDdXN0b21CdWlsZEZsYWdz KHNlbGYsIHNlbGYuZ2V0UHJvcGVydHkoJ3BsYXRmb3JtJyksIHNlbGYuZ2V0UHJvcGVydHkoJ2Z1 bGxQbGF0Zm9ybScpKQpJbmRleDogVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0 Lm9yZy1jb25maWcvc3RlcHNfdW5pdHRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRT bGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvc3RlcHNfdW5pdHRlc3QucHkJKHJl dmlzaW9uIDI0MDE4MSkKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5v cmctY29uZmlnL3N0ZXBzX3VuaXR0ZXN0LnB5CSh3b3JraW5nIGNvcHkpCkBAIC0yNjAsMjAgKzI2 MCwyMCBAQCBjbGFzcyBSdW4zMmJpdEpTQ1Rlc3RzVGVzdCh1bml0dGVzdC5UZXN0CiAgICAgICAg IHNlbGYuYXNzZXJ0UmVzdWx0cyhTVUNDRVNTLCBbJ3dlYmtpdC0zMmJpdC1qc2MtdGVzdCddLCAw LCAgJyAgICAwIGZhaWx1cmVzIGZvdW5kLicpCiAKIAotY2xhc3MgUnVuVW5pdFRlc3RzVGVzdCh1 bml0dGVzdC5UZXN0Q2FzZSk6CitjbGFzcyBSdW5BUElUZXN0c1Rlc3QodW5pdHRlc3QuVGVzdENh c2UpOgogICAgIGRlZiBhc3NlcnRGYWlsdXJlcyhzZWxmLCBleHBlY3RlZF9mYWlsdXJlX2NvdW50 LCBzdGRpbyk6CiAgICAgICAgIGlmIGV4cGVjdGVkX2ZhaWx1cmVfY291bnQ6CiAgICAgICAgICAg ICByYyA9IDEKICAgICAgICAgICAgIGV4cGVjdGVkX3Jlc3VsdHMgPSBGQUlMVVJFCiAgICAgICAg ICAgICBwbHVyYWxfc3VmZml4ID0gIiIgaWYgZXhwZWN0ZWRfZmFpbHVyZV9jb3VudCA9PSAxIGVs c2UgInMiCi0gICAgICAgICAgICBleHBlY3RlZF90ZXh0ID0gJyVkIHVuaXQgdGVzdCVzIGZhaWxl ZCBvciB0aW1lZCBvdXQnICUgKGV4cGVjdGVkX2ZhaWx1cmVfY291bnQsIHBsdXJhbF9zdWZmaXgp CisgICAgICAgICAgICBleHBlY3RlZF90ZXh0ID0gJyVkIGFwaSB0ZXN0JXMgZmFpbGVkIG9yIHRp bWVkIG91dCcgJSAoZXhwZWN0ZWRfZmFpbHVyZV9jb3VudCwgcGx1cmFsX3N1ZmZpeCkKICAgICAg ICAgZWxzZToKICAgICAgICAgICAgIHJjID0gMAogICAgICAgICAgICAgZXhwZWN0ZWRfcmVzdWx0 cyA9IFNVQ0NFU1MKICAgICAgICAgICAgIGV4cGVjdGVkX3RleHQgPSAncnVuLWFwaS10ZXN0cycK IAogICAgICAgICBjbWQgPSBTdHViUmVtb3RlQ29tbWFuZChyYywgc3RkaW8pCi0gICAgICAgIHN0 ZXAgPSBSdW5Vbml0VGVzdHMoKQorICAgICAgICBzdGVwID0gUnVuQVBJVGVzdHMoKQogICAgICAg ICBzdGVwLmNvbW1hbmRDb21wbGV0ZShjbWQpCiAgICAgICAgIGFjdHVhbF9yZXN1bHRzID0gc3Rl cC5ldmFsdWF0ZUNvbW1hbmQoY21kKQogICAgICAgICBhY3R1YWxfZmFpbHVyZV9jb3VudCA9IHN0 ZXAuZmFpbGVkVGVzdENvdW50Cg==