227744 2021-07-07 07:46:53 -0700 [GTK] Add a new GTK layout tester bot to build.webkit.org that runs with --skip-failing-tests switch 2021-08-18 18:48:24 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=178472 https://bugs.webkit.org/show_bug.cgi?id=229270 InRadar P2 Normal --- 1 clopez clopez aakash_jain bugs-noreply dpino jbedard webkit-bug-importer oldest_to_newest 1775257 0 clopez 2021-07-07 07:46:53 -0700 We are trying to have a layout tests runner for the GTK port in the EWS, but we are experimenting there random flakiness and failures that are not reproducible on the GTK layout tester of build.webkit.org I think this is related with the different way of running the layout tests: - On build.webkit.org all known tests are scheduled for running (even those that are expected to crash or timeout). - On the EWS, the switch --skip-failing-tests is passed and that changes the order in which the tests are run because the test that are expected to fail are not scheduled for running. This different order of scheduling the test in the EWS I think it may cause different results sometimes or can be a source of undetected flakiness. So I think is a good idea to add a new tester for the GTK port for build.webkit.org that runs with this flag like in the EWS. I believe that will help us to easily discover and make easier doing gardening of this tests that fail or are flaky due to this different order of scheduling the tests because thanks to this bot we will get access to info about this runs in https://results.webkit.org and wktesthunter <https://github.com/clopez/webkit-testhunter> 1775258 1 433028 clopez 2021-07-07 07:50:12 -0700 Created attachment 433028 Patch 1776324 2 jbedard 2021-07-12 08:28:55 -0700 If I'm understanding the thought process here: You are adding a new queue so that the failures caused by test ordering are surfaced somewhere and can be investigated so that the tree can be kept clean enough for EWS to be effective. That generally seems reasonable, just a few thoughts here because this is a problem Apple's port have encountered too: - Such failures are usually indicative of a bug in either the test harness or the engine itself, tests *shouldn't* be able to effect the outcome of tests running after them, but this happens commonly enough that's it's something we explicitly check for any time we encounter a flakey test - When we've noticed this happening with many tests, it's often just a few bugs and you can make a huge difference by focusing on a single random case of test interdependence and fixing it - Test order is never consistent, so while this strategy is probably a good idea as a temporary measure to get EWS working, it's probably not going to be a great permanent solution. 1776332 3 433028 aakash_jain 2021-07-12 08:36:00 -0700 Comment on attachment 433028 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=433028&action=review > Tools/CISupport/build-webkit-org/config.json:438 > + "name": "GTK-Linux-64-bit-Release-Skip-Failing-EWS-Tests", "factory": "TestAllButJSCFactory", "builddir": "gtk-linux-64-release-skip-failing-ews-tests", Nit: I don't think builder name should have EWS in it. All it does is pass an additional argument which is already in the name. 1776379 4 clopez 2021-07-12 11:09:31 -0700 (In reply to Jonathan Bedard from comment #2) > If I'm understanding the thought process here: You are adding a new queue so > that the failures caused by test ordering are surfaced somewhere and can be > investigated so that the tree can be kept clean enough for EWS to be > effective. > > That generally seems reasonable, just a few thoughts here because this is a > problem Apple's port have encountered too: > - Such failures are usually indicative of a bug in either the test harness > or the engine itself, tests *shouldn't* be able to effect the outcome of > tests running after them, but this happens commonly enough that's it's > something we explicitly check for any time we encounter a flakey test > - When we've noticed this happening with many tests, it's often just a few > bugs and you can make a huge difference by focusing on a single random case > of test interdependence and fixing it > - Test order is never consistent, so while this strategy is probably a good > idea as a temporary measure to get EWS working, it's probably not going to > be a great permanent solution. Thanks for the thoughts. I agree with them. The main idea here is to use this bot as a temporary measure to get the EWS working with the pass rate that is expected. I think it can be also helpful to identify better those cases of test interdependences and help fixing them. 1776380 5 433332 clopez 2021-07-12 11:11:00 -0700 Created attachment 433332 Patch Remove EWS from the bot queue name 1776400 6 433332 clopez 2021-07-12 11:56:25 -0700 Comment on attachment 433332 Patch Clearing flags on attachment: 433332 Committed r279844 (239600@main): <https://commits.webkit.org/239600@main> 1776401 7 clopez 2021-07-12 11:56:30 -0700 All reviewed patches have been landed. Closing bug. 1776402 8 webkit-bug-importer 2021-07-12 11:57:15 -0700 <rdar://problem/80474599> 1776436 9 aakash_jain 2021-07-12 13:59:48 -0700 Restarted buildbot few hours back to pick up this change. Queue is live at: https://build.webkit.org/#/builders/GTK-Linux-64-bit-Release-Skip-Failing-Tests 433028 2021-07-07 07:50:12 -0700 2021-07-12 11:10:56 -0700 Patch bug-227744-20210707155010.patch text/plain 4295 clopez U3VidmVyc2lvbiBSZXZpc2lvbjogMjc5NjQwCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggZmUwNDJiNTUxNDg5NzI1MjEzNjQ2NzM3OWQzMmVjOThh N2M1NTQxZC4uYWExZjUwNzY3OWIxN2ZlMmViMGI1NDk3ZjY5MDYyZjViOGI4ZDM3NyAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE5 IEBACisyMDIxLTA3LTA3ICBDYXJsb3MgQWxiZXJ0byBMb3BleiBQZXJleiAgPGNsb3BlekBpZ2Fs aWEuY29tPgorCisgICAgICAgIFtHVEtdIEFkZCBhIG5ldyBHVEsgbGF5b3V0IHRlc3RlciBib3Qg dG8gYnVpbGQud2Via2l0Lm9yZyB0aGF0IHJ1bnMgd2l0aCAtLXNraXAtZmFpbGluZy10ZXN0cyBz d2l0Y2gKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTIy Nzc0NAorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIE9u IHRoZSBFV1MgdGhlIGxheW91dCB0ZXN0cyBydW4gd2l0aCB0aGlzIHN3aXRjaCAtLXNraXAtZmFp bGluZy10ZXN0cworICAgICAgICB3aGljaCBjaGFuZ2VzIHRoZSBvcmRlciBpbiB3aGljaCB0ZXN0 cyBhcmUgcnVuICh0aG9zZSBleHBlY3RlZCB0byBmYWlsCisgICAgICAgIGFyZSBub3Qgc2NoZWR1 bGVkKS4gSSB0aGluayB0aGlzIG1heSBiZSBjYXVzaW5nIGRpZmZlcmVudCByZXN1bHRzIG9uCisg ICAgICAgIG90aGVyIHRlc3RzIChsaWtlIHVuZXhwZWN0ZWQgZmFpbHVyZXMgb3IgdW5leHBlY3Rl ZCBmbGFraWVzKS4KKyAgICAgICAgSGF2aW5nIGEgYm90IGluIGJ1aWxkLndlYmtpdC5vcmcgcnVu bmluZyB3aXRoIHRoaXMgc3dpdGNoIHdpbGwgaGVscAorICAgICAgICB0byBkZXRlY3QgYW5kIGdh cmRlbiB0aG9zZSBjYXNlcy4KKworICAgICAgICAqIENJU3VwcG9ydC9idWlsZC13ZWJraXQtb3Jn L2NvbmZpZy5qc29uOgorCiAyMDIxLTA3LTA2ICBIYXJzaGlsIFJhdG51ICA8aHJhdG51QGFwcGxl LmNvbT4KIAogICAgICAgICBBZGQgbXlzZWxmIChIYXJzaGlsIFJhdG51KSB0byBjb250cmlidXRv cnMuanNvbgpkaWZmIC0tZ2l0IGEvVG9vbHMvQ0lTdXBwb3J0L2J1aWxkLXdlYmtpdC1vcmcvY29u ZmlnLmpzb24gYi9Ub29scy9DSVN1cHBvcnQvYnVpbGQtd2Via2l0LW9yZy9jb25maWcuanNvbgpp bmRleCBhM2M1MmNjYjQyNjE4ZWJmNjE1NjNjMTNiZDcxMzdhMjg0NTIwODFmLi5kNzM4OGQwODg1 NzcxNzE5NGQ5MTcxOWUyNWI1NDc3NzUzMmY0OGRhIDEwMDY0NAotLS0gYS9Ub29scy9DSVN1cHBv cnQvYnVpbGQtd2Via2l0LW9yZy9jb25maWcuanNvbgorKysgYi9Ub29scy9DSVN1cHBvcnQvYnVp bGQtd2Via2l0LW9yZy9jb25maWcuanNvbgpAQCAtMTEzLDYgKzExMyw3IEBACiAgICAgICAgICAg ICAgICAgICAgIHsgIm5hbWUiOiAiZ3RrLWxpbnV4LWJvdC0xNiIsICJwbGF0Zm9ybSI6ICJndGsi IH0sCiAgICAgICAgICAgICAgICAgICAgIHsgIm5hbWUiOiAiZ3RrLWxpbnV4LWJvdC0xNyIsICJw bGF0Zm9ybSI6ICJndGsiIH0sCiAgICAgICAgICAgICAgICAgICAgIHsgIm5hbWUiOiAiZ3RrLWxp bnV4LWJvdC0xOCIsICJwbGF0Zm9ybSI6ICJndGsiIH0sCisgICAgICAgICAgICAgICAgICAgIHsg Im5hbWUiOiAiZ3RrLWxpbnV4LWJvdC0xOSIsICJwbGF0Zm9ybSI6ICJndGsiIH0sCiAKICAgICAg ICAgICAgICAgICAgICAgeyAibmFtZSI6ICJqc2Nvbmx5LWxpbnV4LWlnYWxpYS1ib3QtMSIsICJw bGF0Zm9ybSI6ICJqc2Mtb25seSIgfSwKICAgICAgICAgICAgICAgICAgICAgeyAibmFtZSI6ICJq c2Nvbmx5LWxpbnV4LWlnYWxpYS1ib3QtMiIsICJwbGF0Zm9ybSI6ICJqc2Mtb25seSIgfSwKQEAg LTM1Miw3ICszNTMsOCBAQAogICAgICAgICAgICAgICAgICAgICAgICJuYW1lIjogIkdUSy1MaW51 eC02NC1iaXQtUmVsZWFzZS1CdWlsZCIsICJmYWN0b3J5IjogIkJ1aWxkQW5kR2VuZXJhdGVKU0NC dW5kbGVGYWN0b3J5IiwgImJ1aWxkZGlyIjogImd0ay1saW51eC02NC1yZWxlYXNlIiwKICAgICAg ICAgICAgICAgICAgICAgICAicGxhdGZvcm0iOiAiZ3RrIiwgImNvbmZpZ3VyYXRpb24iOiAicmVs ZWFzZSIsICJhcmNoaXRlY3R1cmVzIjogWyJ4ODZfNjQiXSwKICAgICAgICAgICAgICAgICAgICAg ICAidHJpZ2dlcnMiOiBbImd0ay1saW51eC02NC1yZWxlYXNlLXRlc3RzIiwgImd0ay1saW51eC02 NC1yZWxlYXNlLXRlc3RzLWpzIiwgImd0ay1saW51eC02NC1yZWxlYXNlLXRlc3RzLXdlYmRyaXZl ciIsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJndGstbGludXgtNjQtcmVs ZWFzZS13YXlsYW5kLXRlc3RzIiwgImd0ay1saW51eC02NC1yZWxlYXNlLXBlcmYtdGVzdHMiLCAi Z3RrLWxpbnV4LTY0LXJlbGVhc2UtZ3RrNC10ZXN0cyJdLAorICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAiZ3RrLWxpbnV4LTY0LXJlbGVhc2Utd2F5bGFuZC10ZXN0cyIsICJndGst bGludXgtNjQtcmVsZWFzZS1wZXJmLXRlc3RzIiwgImd0ay1saW51eC02NC1yZWxlYXNlLWd0azQt dGVzdHMiLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiZ3RrLWxpbnV4LTY0 LXJlbGVhc2Utc2tpcC1mYWlsaW5nLWV3cy10ZXN0cyJdLAogICAgICAgICAgICAgICAgICAgICAg ICJ3b3JrZXJuYW1lcyI6IFsiZ3RrLWxpbnV4LWJvdC0yIl0KICAgICAgICAgICAgICAgICAgICAg fSwKICAgICAgICAgICAgICAgICAgICAgewpAQCAtNDMyLDYgKzQzNCwxMiBAQAogICAgICAgICAg ICAgICAgICAgICAgICJwbGF0Zm9ybSI6ICJndGsiLCAiY29uZmlndXJhdGlvbiI6ICJyZWxlYXNl IiwgImFyY2hpdGVjdHVyZXMiOiBbIng4Nl82NCJdLAogICAgICAgICAgICAgICAgICAgICAgICJ3 b3JrZXJuYW1lcyI6IFsiZ3RrLWxpbnV4LWJvdC0xOCJdCiAgICAgICAgICAgICAgICAgICAgIH0s CisgICAgICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAgICAgICAibmFtZSI6ICJH VEstTGludXgtNjQtYml0LVJlbGVhc2UtU2tpcC1GYWlsaW5nLUVXUy1UZXN0cyIsICJmYWN0b3J5 IjogIlRlc3RBbGxCdXRKU0NGYWN0b3J5IiwgImJ1aWxkZGlyIjogImd0ay1saW51eC02NC1yZWxl YXNlLXNraXAtZmFpbGluZy1ld3MtdGVzdHMiLAorICAgICAgICAgICAgICAgICAgICAgICJwbGF0 Zm9ybSI6ICJndGsiLCAiY29uZmlndXJhdGlvbiI6ICJyZWxlYXNlIiwgImFyY2hpdGVjdHVyZXMi OiBbIng4Nl82NCJdLAorICAgICAgICAgICAgICAgICAgICAgICJhZGRpdGlvbmFsQXJndW1lbnRz IjogWyItLXNraXAtZmFpbGluZy10ZXN0cyJdLAorICAgICAgICAgICAgICAgICAgICAgICJ3b3Jr ZXJuYW1lcyI6IFsiZ3RrLWxpbnV4LWJvdC0xOSJdCisgICAgICAgICAgICAgICAgICAgIH0sCiAg ICAgICAgICAgICAgICAgICAgIHsKICAgICAgICAgICAgICAgICAgICAgICAibmFtZSI6ICJXaW5D YWlyby02NC1iaXQtV0tMLVJlbGVhc2UtQnVpbGQiLCAiZmFjdG9yeSI6ICJCdWlsZEZhY3Rvcnki LCAiYnVpbGRkaXIiOiAid2luY2Fpcm8td2tsLXJlbGVhc2UiLAogICAgICAgICAgICAgICAgICAg ICAgICJwbGF0Zm9ybSI6ICJ3aW5jYWlybyIsICJjb25maWd1cmF0aW9uIjogInJlbGVhc2UiLCAi YXJjaGl0ZWN0dXJlcyI6IFsieDg2XzY0Il0sCkBAIC02OTQsNiArNzAyLDkgQEAKICAgICAgICAg ICAgICAgICAgICAgeyAidHlwZSI6ICJUcmlnZ2VyYWJsZSIsICJuYW1lIjogImd0ay1saW51eC02 NC1yZWxlYXNlLXdheWxhbmQtdGVzdHMiLAogICAgICAgICAgICAgICAgICAgICAgICJidWlsZGVy TmFtZXMiOiBbIkdUSy1MaW51eC02NC1iaXQtUmVsZWFzZS1XYXlsYW5kLVRlc3RzIl0KICAgICAg ICAgICAgICAgICAgICAgfSwKKyAgICAgICAgICAgICAgICAgICAgeyAidHlwZSI6ICJUcmlnZ2Vy YWJsZSIsICJuYW1lIjogImd0ay1saW51eC02NC1yZWxlYXNlLXNraXAtZmFpbGluZy1ld3MtdGVz dHMiLAorICAgICAgICAgICAgICAgICAgICAgICJidWlsZGVyTmFtZXMiOiBbIkdUSy1MaW51eC02 NC1iaXQtUmVsZWFzZS1Ta2lwLUZhaWxpbmctRVdTLVRlc3RzIl0KKyAgICAgICAgICAgICAgICAg ICAgfSwKICAgICAgICAgICAgICAgICAgICAgeyAidHlwZSI6ICJUcmlnZ2VyYWJsZSIsICJuYW1l IjogIndpbmNhaXJvLXdrbC1yZWxlYXNlLXRlc3RzIiwKICAgICAgICAgICAgICAgICAgICAgICAi YnVpbGRlck5hbWVzIjogWyJXaW5DYWlyby02NC1iaXQtV0tMLVJlbGVhc2UtVGVzdHMiXQogICAg ICAgICAgICAgICAgICAgICB9LAo= 433332 2021-07-12 11:11:00 -0700 2021-07-12 12:44:54 -0700 Patch bug-227744-20210712191056.patch text/plain 4313 clopez U3VidmVyc2lvbiBSZXZpc2lvbjogMjc5ODM5CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggNTNmNzk4MjI3MjU2OWFjOTBkYTIyMTA3N2NhOTVkZjdm M2JmZDExYy4uMTM0NjgyZGY1MDA5OTBmMTNhNDc1MmM4MmYyZDU0MTM3OTc0MzRmNiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE5 IEBACisyMDIxLTA3LTA3ICBDYXJsb3MgQWxiZXJ0byBMb3BleiBQZXJleiAgPGNsb3BlekBpZ2Fs aWEuY29tPgorCisgICAgICAgIFtHVEtdIEFkZCBhIG5ldyBHVEsgbGF5b3V0IHRlc3RlciBib3Qg dG8gYnVpbGQud2Via2l0Lm9yZyB0aGF0IHJ1bnMgd2l0aCAtLXNraXAtZmFpbGluZy10ZXN0cyBz d2l0Y2gKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTIy Nzc0NAorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIE9u IHRoZSBFV1MgdGhlIGxheW91dCB0ZXN0cyBydW4gd2l0aCB0aGlzIHN3aXRjaCAtLXNraXAtZmFp bGluZy10ZXN0cworICAgICAgICB3aGljaCBjaGFuZ2VzIHRoZSBvcmRlciBpbiB3aGljaCB0ZXN0 cyBhcmUgcnVuICh0aG9zZSBleHBlY3RlZCB0byBmYWlsCisgICAgICAgIGFyZSBub3Qgc2NoZWR1 bGVkKS4gSSB0aGluayB0aGlzIG1heSBiZSBjYXVzaW5nIGRpZmZlcmVudCByZXN1bHRzIG9uCisg ICAgICAgIG90aGVyIHRlc3RzIChsaWtlIHVuZXhwZWN0ZWQgZmFpbHVyZXMgb3IgdW5leHBlY3Rl ZCBmbGFraWVzKS4KKyAgICAgICAgSGF2aW5nIGEgYm90IGluIGJ1aWxkLndlYmtpdC5vcmcgcnVu bmluZyB3aXRoIHRoaXMgc3dpdGNoIHdpbGwgaGVscAorICAgICAgICB0byBkZXRlY3QgYW5kIGdh cmRlbiB0aG9zZSBjYXNlcy4KKworICAgICAgICAqIENJU3VwcG9ydC9idWlsZC13ZWJraXQtb3Jn L2NvbmZpZy5qc29uOgorCiAyMDIxLTA3LTEyICBKZXIgTm9ibGUgIDxqZXIubm9ibGVAYXBwbGUu Y29tPgogCiAgICAgICAgIFtDb2NvYV0gSW5jb3JyZWN0IGRlcHJlY2F0aW9uIGRlY2xhcmF0aW9u IGZvciAtW1dLV2ViVmlldyBjbG9zZUFsbE1lZGlhUHJlc2VudGF0aW9uczpdCmRpZmYgLS1naXQg YS9Ub29scy9DSVN1cHBvcnQvYnVpbGQtd2Via2l0LW9yZy9jb25maWcuanNvbiBiL1Rvb2xzL0NJ U3VwcG9ydC9idWlsZC13ZWJraXQtb3JnL2NvbmZpZy5qc29uCmluZGV4IGEzYzUyY2NiNDI2MThl YmY2MTU2M2MxM2JkNzEzN2EyODQ1MjA4MWYuLmNlOGJlNGU3MDc4MjdiZGFhZDZlNzMwOWQ2Zjdj MGE1ODQ5NzFiNGEgMTAwNjQ0Ci0tLSBhL1Rvb2xzL0NJU3VwcG9ydC9idWlsZC13ZWJraXQtb3Jn L2NvbmZpZy5qc29uCisrKyBiL1Rvb2xzL0NJU3VwcG9ydC9idWlsZC13ZWJraXQtb3JnL2NvbmZp Zy5qc29uCkBAIC0xMTMsNiArMTEzLDcgQEAKICAgICAgICAgICAgICAgICAgICAgeyAibmFtZSI6 ICJndGstbGludXgtYm90LTE2IiwgInBsYXRmb3JtIjogImd0ayIgfSwKICAgICAgICAgICAgICAg ICAgICAgeyAibmFtZSI6ICJndGstbGludXgtYm90LTE3IiwgInBsYXRmb3JtIjogImd0ayIgfSwK ICAgICAgICAgICAgICAgICAgICAgeyAibmFtZSI6ICJndGstbGludXgtYm90LTE4IiwgInBsYXRm b3JtIjogImd0ayIgfSwKKyAgICAgICAgICAgICAgICAgICAgeyAibmFtZSI6ICJndGstbGludXgt Ym90LTE5IiwgInBsYXRmb3JtIjogImd0ayIgfSwKIAogICAgICAgICAgICAgICAgICAgICB7ICJu YW1lIjogImpzY29ubHktbGludXgtaWdhbGlhLWJvdC0xIiwgInBsYXRmb3JtIjogImpzYy1vbmx5 IiB9LAogICAgICAgICAgICAgICAgICAgICB7ICJuYW1lIjogImpzY29ubHktbGludXgtaWdhbGlh LWJvdC0yIiwgInBsYXRmb3JtIjogImpzYy1vbmx5IiB9LApAQCAtMzUyLDcgKzM1Myw4IEBACiAg ICAgICAgICAgICAgICAgICAgICAgIm5hbWUiOiAiR1RLLUxpbnV4LTY0LWJpdC1SZWxlYXNlLUJ1 aWxkIiwgImZhY3RvcnkiOiAiQnVpbGRBbmRHZW5lcmF0ZUpTQ0J1bmRsZUZhY3RvcnkiLCAiYnVp bGRkaXIiOiAiZ3RrLWxpbnV4LTY0LXJlbGVhc2UiLAogICAgICAgICAgICAgICAgICAgICAgICJw bGF0Zm9ybSI6ICJndGsiLCAiY29uZmlndXJhdGlvbiI6ICJyZWxlYXNlIiwgImFyY2hpdGVjdHVy ZXMiOiBbIng4Nl82NCJdLAogICAgICAgICAgICAgICAgICAgICAgICJ0cmlnZ2VycyI6IFsiZ3Rr LWxpbnV4LTY0LXJlbGVhc2UtdGVzdHMiLCAiZ3RrLWxpbnV4LTY0LXJlbGVhc2UtdGVzdHMtanMi LCAiZ3RrLWxpbnV4LTY0LXJlbGVhc2UtdGVzdHMtd2ViZHJpdmVyIiwKLSAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgImd0ay1saW51eC02NC1yZWxlYXNlLXdheWxhbmQtdGVzdHMi LCAiZ3RrLWxpbnV4LTY0LXJlbGVhc2UtcGVyZi10ZXN0cyIsICJndGstbGludXgtNjQtcmVsZWFz ZS1ndGs0LXRlc3RzIl0sCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJndGst bGludXgtNjQtcmVsZWFzZS13YXlsYW5kLXRlc3RzIiwgImd0ay1saW51eC02NC1yZWxlYXNlLXBl cmYtdGVzdHMiLCAiZ3RrLWxpbnV4LTY0LXJlbGVhc2UtZ3RrNC10ZXN0cyIsCisgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICJndGstbGludXgtNjQtcmVsZWFzZS1za2lwLWZhaWxp bmctdGVzdHMiXSwKICAgICAgICAgICAgICAgICAgICAgICAid29ya2VybmFtZXMiOiBbImd0ay1s aW51eC1ib3QtMiJdCiAgICAgICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAgICAg IHsKQEAgLTQzMiw2ICs0MzQsMTIgQEAKICAgICAgICAgICAgICAgICAgICAgICAicGxhdGZvcm0i OiAiZ3RrIiwgImNvbmZpZ3VyYXRpb24iOiAicmVsZWFzZSIsICJhcmNoaXRlY3R1cmVzIjogWyJ4 ODZfNjQiXSwKICAgICAgICAgICAgICAgICAgICAgICAid29ya2VybmFtZXMiOiBbImd0ay1saW51 eC1ib3QtMTgiXQogICAgICAgICAgICAgICAgICAgICB9LAorICAgICAgICAgICAgICAgICAgICB7 CisgICAgICAgICAgICAgICAgICAgICAgIm5hbWUiOiAiR1RLLUxpbnV4LTY0LWJpdC1SZWxlYXNl LVNraXAtRmFpbGluZy1UZXN0cyIsICJmYWN0b3J5IjogIlRlc3RBbGxCdXRKU0NGYWN0b3J5Iiwg ImJ1aWxkZGlyIjogImd0ay1saW51eC02NC1yZWxlYXNlLXNraXAtZmFpbGluZy10ZXN0cyIsCisg ICAgICAgICAgICAgICAgICAgICAgInBsYXRmb3JtIjogImd0ayIsICJjb25maWd1cmF0aW9uIjog InJlbGVhc2UiLCAiYXJjaGl0ZWN0dXJlcyI6IFsieDg2XzY0Il0sCisgICAgICAgICAgICAgICAg ICAgICAgImFkZGl0aW9uYWxBcmd1bWVudHMiOiBbIi0tc2tpcC1mYWlsaW5nLXRlc3RzIl0sCisg ICAgICAgICAgICAgICAgICAgICAgIndvcmtlcm5hbWVzIjogWyJndGstbGludXgtYm90LTE5Il0K KyAgICAgICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICAgICAgewogICAgICAgICAg ICAgICAgICAgICAgICJuYW1lIjogIldpbkNhaXJvLTY0LWJpdC1XS0wtUmVsZWFzZS1CdWlsZCIs ICJmYWN0b3J5IjogIkJ1aWxkRmFjdG9yeSIsICJidWlsZGRpciI6ICJ3aW5jYWlyby13a2wtcmVs ZWFzZSIsCiAgICAgICAgICAgICAgICAgICAgICAgInBsYXRmb3JtIjogIndpbmNhaXJvIiwgImNv bmZpZ3VyYXRpb24iOiAicmVsZWFzZSIsICJhcmNoaXRlY3R1cmVzIjogWyJ4ODZfNjQiXSwKQEAg LTY5NCw2ICs3MDIsOSBAQAogICAgICAgICAgICAgICAgICAgICB7ICJ0eXBlIjogIlRyaWdnZXJh YmxlIiwgIm5hbWUiOiAiZ3RrLWxpbnV4LTY0LXJlbGVhc2Utd2F5bGFuZC10ZXN0cyIsCiAgICAg ICAgICAgICAgICAgICAgICAgImJ1aWxkZXJOYW1lcyI6IFsiR1RLLUxpbnV4LTY0LWJpdC1SZWxl YXNlLVdheWxhbmQtVGVzdHMiXQogICAgICAgICAgICAgICAgICAgICB9LAorICAgICAgICAgICAg ICAgICAgICB7ICJ0eXBlIjogIlRyaWdnZXJhYmxlIiwgIm5hbWUiOiAiZ3RrLWxpbnV4LTY0LXJl bGVhc2Utc2tpcC1mYWlsaW5nLXRlc3RzIiwKKyAgICAgICAgICAgICAgICAgICAgICAiYnVpbGRl ck5hbWVzIjogWyJHVEstTGludXgtNjQtYml0LVJlbGVhc2UtU2tpcC1GYWlsaW5nLVRlc3RzIl0K KyAgICAgICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICAgICAgeyAidHlwZSI6ICJU cmlnZ2VyYWJsZSIsICJuYW1lIjogIndpbmNhaXJvLXdrbC1yZWxlYXNlLXRlc3RzIiwKICAgICAg ICAgICAgICAgICAgICAgICAiYnVpbGRlck5hbWVzIjogWyJXaW5DYWlyby02NC1iaXQtV0tMLVJl bGVhc2UtVGVzdHMiXQogICAgICAgICAgICAgICAgICAgICB9LAo=