67268 2011-08-31 01:01:47 -0700 [NRWT] New tests without expected files are reported as flakey tests 2011-10-12 20:17:54 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) All All RESOLVED FIXED P1 Major --- 34984 64812 69990 1 ossy rniwa abarth dpranke eric kkristof leandro loki ojan ossy rakuco rniwa webkit.review.bot zherczeg oldest_to_newest 459500 0 ossy 2011-08-31 01:01:47 -0700 I don't understand how can pass a test for the second run ... http://build.webkit.org/results/Qt%20Linux%20Release/r94155%20%2836977%29/results.html http://build.webkit.org/builders/Qt%20Linux%20Release/builds/36977/steps/layout-test/logs/stdio Unexpected flakiness: no expected results found (3) fast/multicol/overflow-across-columns.html = MISSING PASS fast/multicol/overflow-unsplittable.html = MISSING PASS fast/multicol/positive-leading.html = MISSING PASS A similar bug: https://bugs.webkit.org/show_bug.cgi?id=64812 459615 1 ossy 2011-08-31 07:33:33 -0700 More information: NRWT reports they are flakey when there are failing tests too, but it reports only missing when there aren't any failing test: Regressions: Unexpected no expected results found : (3) fast/multicol/overflow-across-columns.html = MISSING fast/multicol/overflow-unsplittable.html = MISSING fast/multicol/positive-leading.html = MISSING 459665 2 eric 2011-08-31 09:27:55 -0700 I could see how this might happen. Sad times. 459876 3 abarth 2011-08-31 13:46:14 -0700 Yep. Pretty easy to fix. 460374 4 zherczeg 2011-09-01 04:22:29 -0700 > Yep. Pretty easy to fix. Cool. When do you intend to? 460580 5 abarth 2011-09-01 10:32:37 -0700 > Cool. When do you intend to? Unfortunately, I've been dragged away to work on some other issues, but I hope to return to NRWT at some point. Eric is more actively working on these issues. Hopefully he'll be able to fix this issue soon. 464903 6 ossy 2011-09-09 10:58:15 -0700 Now 464906 7 ossy 2011-09-09 11:01:13 -0700 Very strange ... Missing tests can be flakey or missing when there aren't any failing test: missing: http://build.webkit.org/builders/Qt%20Linux%20Release/builds/37323/steps/layout-test/logs/stdio flakey: http://build.webkit.sed.hu/builders/x86-32%20Linux%20Qt%20Release%20WebKit2/builds/11534/steps/layout-test/logs/stdio 465293 8 ossy 2011-09-10 07:09:55 -0700 I don't understand this crazy NRWT ... I'm really thinking that we should switch back to the stable ORWT ... NRWT is so slower and so flakey. ------- http://build.webkit.org/builders/Qt%20Linux%20Release/builds/37349 - 3 flakes, rc=0 Unexpected flakiness: tests timed out (2) storage/open-database-creation-callback-isolated-world.html = TIMEOUT PASS storage/open-database-creation-callback.html = TIMEOUT PASS Unexpected flakiness: no expected results found (1) svg/custom/simple-text-double-shadow.svg = MISSING PASS ------- http://build.webkit.org/builders/Qt%20Linux%20Release/builds/37350 - 1 missing results, rc=1 Regressions: Unexpected no expected results found : (1) svg/custom/simple-text-double-shadow.svg = MISSING 465413 9 eric 2011-09-11 14:40:20 -0700 I fully support you switching back to ORWT for the Qt port. It's my job to convince you that NRWT is better, not my job to force you to switch. I'm happy to investigate this issue further on monday for you. But again, you should also feel welcome to switch back to ORWT until the Pros list is longer than the Cons list for Qt. 482918 10 110785 rniwa 2011-10-12 18:20:46 -0700 Created attachment 110785 Fixes the bug 482919 11 110786 rniwa 2011-10-12 18:21:30 -0700 Created attachment 110786 Revert some change in expected result 482937 12 110786 dpranke 2011-10-12 18:39:53 -0700 Comment on attachment 110786 Revert some change in expected result This patch looks okay, but it's definitely a hack that we're counting MISSING tests as flaky, and I officially declare tthe logic in summarize_results and get_failures to be too complicated, so we need to clean this up. Can you file a separate bug for this effort? 482946 13 110786 ojan 2011-10-12 18:51:10 -0700 Comment on attachment 110786 Revert some change in expected result View in context: https://bugs.webkit.org/attachment.cgi?id=110786&action=review Are there any tests for this that you can update? > Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:142 > + num_flaky += 1 Why would we increment num_flaky if we're no longer considering this flaky? 482947 14 ojan 2011-10-12 18:53:11 -0700 You might be able to test this through http://codesearch.google.com/codesearch#OAMlx_jo-ck/src/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/views/printing_unittest.py&l=416. 482950 15 dpranke 2011-10-12 18:57:27 -0700 (In reply to comment #13) > (From update of attachment 110786 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=110786&action=review > > Are there any tests for this that you can update? > I believe rniwa is adding a patch w/ a test. > > Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:142 > > + num_flaky += 1 > > Why would we increment num_flaky if we're no longer considering this flaky? You may have missed the IRC conversation I had w/ rniwa ... the jist of it is that we increment num_flaky in order to turn the buildbot orange instead of red. This is a hack and that needs to be cleaned up in the follow-on bug I have asked for. This behavior was previously agreed to (in bug 64812); this bug is just making that behavior actually work correctly. 482956 16 ojan 2011-10-12 19:04:40 -0700 (In reply to comment #15) > (In reply to comment #13) > > (From update of attachment 110786 [details] [details]) > > View in context: https://bugs.webkit.org/attachment.cgi?id=110786&action=review > > > Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:142 > > > + num_flaky += 1 > > > > Why would we increment num_flaky if we're no longer considering this flaky? > > You may have missed the IRC conversation I had w/ rniwa ... the jist of it is that we increment num_flaky in order to turn the buildbot orange instead of red. This is a hack and that needs to be cleaned up in the follow-on bug I have asked for. This behavior was previously agreed to (in bug 64812); this bug is just making that behavior actually work correctly. In that case, there should be a comment + FIXME explaining this. 482963 17 dpranke 2011-10-12 19:12:05 -0700 (In reply to comment #16) > (In reply to comment #15) > > (In reply to comment #13) > > > (From update of attachment 110786 [details] [details] [details]) > > > View in context: https://bugs.webkit.org/attachment.cgi?id=110786&action=review > > > > Tools/Scripts/webkitpy/layout_tests/controllers/manager.py:142 > > > > + num_flaky += 1 > > > > > > Why would we increment num_flaky if we're no longer considering this flaky? > > > > You may have missed the IRC conversation I had w/ rniwa ... the jist of it is that we increment num_flaky in order to turn the buildbot orange instead of red. This is a hack and that needs to be cleaned up in the follow-on bug I have asked for. This behavior was previously agreed to (in bug 64812); this bug is just making that behavior actually work correctly. > > In that case, there should be a comment + FIXME explaining this. Yup, that's exactly what I asked for in comment #12. 482966 18 110794 rniwa 2011-10-12 19:15:22 -0700 Created attachment 110794 Added a test 482977 19 110794 webkit.review.bot 2011-10-12 20:17:47 -0700 Comment on attachment 110794 Added a test Clearing flags on attachment: 110794 Committed r97339: <http://trac.webkit.org/changeset/97339> 482978 20 webkit.review.bot 2011-10-12 20:17:54 -0700 All reviewed patches have been landed. Closing bug. 110785 2011-10-12 18:20:46 -0700 2011-10-12 18:21:26 -0700 Fixes the bug bug-67268-20111012182045.patch text/plain 2534 rniwa SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDk3MzI4KQorKysgVG9vbHMvQ2hhbmdlTG9nCSh3b3JraW5nIGNvcHkpCkBAIC0xLDMg KzEsMTggQEAKKzIwMTEtMTAtMTIgIFJ5b3N1a2UgTml3YSAgPHJuaXdhQHdlYmtpdC5vcmc+CisK KyAgICAgICAgW05SV1RdIE5ldyB0ZXN0cyB3aXRob3V0IGV4cGVjdGVkIGZpbGVzIGFyZSByZXBv cnRlZCBhcyBmbGFrZXkgdGVzdHMKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hv d19idWcuY2dpP2lkPTY3MjY4CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISku CisKKyAgICAgICAgVGhlIGJ1ZyB3YXMgY2F1c2VkIGJ5IHN1bW1hcml6ZV9yZXN1bHRzJ3MgcmVn YXJkaW5nIGFsbCB0ZXN0cyB0aGF0IGhhdmUgYW4gZW50cnkgaW4gcmVzdWx0X3N1bW1hcnkKKyAg ICAgICAgYW5kIG5vIGVudHJ5IGluIHJldHJ5X3N1bW1hcnkgYXMgZmxha3kuIFNpbmNlIHdlIGRv bid0IHJldHJ5IG1pc3NpbmcgdGVzdHMsIHRoZXJlIHdpbGwgYmUgbm8gZW50cmllcworICAgICAg ICBmb3IgbWlzc2luZyByZXN1bHRzIGluIHJldHJ5X3N1bW1hcnkuCisKKyAgICAgICAgRml4ZWQg dGhlIGJ1ZyBieSB0cmVhdGluZyBtaXNzaW5nIHJlc3VsdHMgc2VwYXJhdGVseS4KKworICAgICAg ICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHk6 CisKIDIwMTEtMTAtMTIgIFNpbW9uIEZyYXNlciAgPHNpbW9uLmZyYXNlckBhcHBsZS5jb20+CiAK ICAgICAgICAgTWFueSBXSzIgc25hcHNob3RzIGFyZSBibGFuawpJbmRleDogVG9vbHMvU2NyaXB0 cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvY29udHJvbGxlcnMvbWFuYWdlci5weQo9PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9jb250cm9sbGVycy9tYW5h Z2VyLnB5CShyZXZpc2lvbiA5NzMxOSkKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0 X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHkJKHdvcmtpbmcgY29weSkKQEAgLTEzOCw2ICsx MzgsOSBAQCBkZWYgc3VtbWFyaXplX3Jlc3VsdHMocG9ydF9vYmosIGV4cGVjdGF0CiAgICAgICAg ICAgICAgICAgY29udGludWUKICAgICAgICAgZWxpZiByZXN1bHRfdHlwZSA9PSB0ZXN0X2V4cGVj dGF0aW9ucy5DUkFTSDoKICAgICAgICAgICAgIG51bV9yZWdyZXNzaW9ucyArPSAxCisgICAgICAg IGVsaWYgcmVzdWx0X3R5cGUgPT0gdGVzdF9leHBlY3RhdGlvbnMuTUlTU0lORzoKKyAgICAgICAg ICAgIG51bV9mbGFreSArPSAxCisgICAgICAgICAgICAjIEZJWE1FOiBicmVhayBNSVNTSU5HIHJl c3VsdHMgaW50byBhIGRpZmZlcmVudCBjYXRlZ29yeQogICAgICAgICBlbGlmIHRlc3RfbmFtZSBp biByZXN1bHRfc3VtbWFyeS51bmV4cGVjdGVkX3Jlc3VsdHM6CiAgICAgICAgICAgICBpZiB0ZXN0 X25hbWUgbm90IGluIHJldHJ5X3N1bW1hcnkudW5leHBlY3RlZF9yZXN1bHRzOgogICAgICAgICAg ICAgICAgIGFjdHVhbC5leHRlbmQoZXhwZWN0YXRpb25zLmdldF9leHBlY3RhdGlvbnNfc3RyaW5n KHRlc3RfbmFtZSkuc3BsaXQoIiAiKSkKQEAgLTE0Nyw5ICsxNTAsNiBAQCBkZWYgc3VtbWFyaXpl X3Jlc3VsdHMocG9ydF9vYmosIGV4cGVjdGF0CiAgICAgICAgICAgICAgICAgaWYgcmVzdWx0X3R5 cGUgIT0gcmV0cnlfcmVzdWx0X3R5cGU6CiAgICAgICAgICAgICAgICAgICAgIGFjdHVhbC5hcHBl bmQoa2V5d29yZHNbcmV0cnlfcmVzdWx0X3R5cGVdKQogICAgICAgICAgICAgICAgICAgICBudW1f Zmxha3kgKz0gMQotICAgICAgICAgICAgICAgICMgRklYTUU6IGJyZWFrIE1JU1NJTkcgcmVzdWx0 cyBpbnRvIGEgZGlmZmVyZW50IGNhdGVnb3J5Ci0gICAgICAgICAgICAgICAgZWxpZiAnTUlTU0lO RycgaW4gYWN0dWFsOgotICAgICAgICAgICAgICAgICAgICBudW1fZmxha3kgKz0gMQogICAgICAg ICAgICAgICAgIGVsc2U6CiAgICAgICAgICAgICAgICAgICAgIG51bV9yZWdyZXNzaW9ucyArPSAx CiAKSW5kZXg6IExheW91dFRlc3RzL2VkaXRpbmcvdW5kby9vcnBoYW5lZC1zZWxlY3Rpb24tY3Jh c2gtYnVnMzI4MjMtMS1leHBlY3RlZC50eHQKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gTGF5b3V0VGVzdHMvZWRp dGluZy91bmRvL29ycGhhbmVkLXNlbGVjdGlvbi1jcmFzaC1idWczMjgyMy0xLWV4cGVjdGVkLnR4 dAkocmV2aXNpb24gOTczMTkpCisrKyBMYXlvdXRUZXN0cy9lZGl0aW5nL3VuZG8vb3JwaGFuZWQt c2VsZWN0aW9uLWNyYXNoLWJ1ZzMyODIzLTEtZXhwZWN0ZWQudHh0CSh3b3JraW5nIGNvcHkpCkBA IC0xICsxIEBACi1QQVNTCitmZnNhUEFTUwo= 110786 2011-10-12 18:21:30 -0700 2011-10-12 19:15:17 -0700 Revert some change in expected result bug-67268-20111012182129.patch text/plain 2169 rniwa SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDk3MzI4KQorKysgVG9vbHMvQ2hhbmdlTG9nCSh3b3JraW5nIGNvcHkpCkBAIC0xLDMg KzEsMTggQEAKKzIwMTEtMTAtMTIgIFJ5b3N1a2UgTml3YSAgPHJuaXdhQHdlYmtpdC5vcmc+CisK KyAgICAgICAgW05SV1RdIE5ldyB0ZXN0cyB3aXRob3V0IGV4cGVjdGVkIGZpbGVzIGFyZSByZXBv cnRlZCBhcyBmbGFrZXkgdGVzdHMKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hv d19idWcuY2dpP2lkPTY3MjY4CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISku CisKKyAgICAgICAgVGhlIGJ1ZyB3YXMgY2F1c2VkIGJ5IHN1bW1hcml6ZV9yZXN1bHRzJ3MgcmVn YXJkaW5nIGFsbCB0ZXN0cyB0aGF0IGhhdmUgYW4gZW50cnkgaW4gcmVzdWx0X3N1bW1hcnkKKyAg ICAgICAgYW5kIG5vIGVudHJ5IGluIHJldHJ5X3N1bW1hcnkgYXMgZmxha3kuIFNpbmNlIHdlIGRv bid0IHJldHJ5IG1pc3NpbmcgdGVzdHMsIHRoZXJlIHdpbGwgYmUgbm8gZW50cmllcworICAgICAg ICBmb3IgbWlzc2luZyByZXN1bHRzIGluIHJldHJ5X3N1bW1hcnkuCisKKyAgICAgICAgRml4ZWQg dGhlIGJ1ZyBieSB0cmVhdGluZyBtaXNzaW5nIHJlc3VsdHMgc2VwYXJhdGVseS4KKworICAgICAg ICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHk6 CisKIDIwMTEtMTAtMTIgIFNpbW9uIEZyYXNlciAgPHNpbW9uLmZyYXNlckBhcHBsZS5jb20+CiAK ICAgICAgICAgTWFueSBXSzIgc25hcHNob3RzIGFyZSBibGFuawpJbmRleDogVG9vbHMvU2NyaXB0 cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvY29udHJvbGxlcnMvbWFuYWdlci5weQo9PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9jb250cm9sbGVycy9tYW5h Z2VyLnB5CShyZXZpc2lvbiA5NzMxOSkKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0 X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHkJKHdvcmtpbmcgY29weSkKQEAgLTEzOCw2ICsx MzgsOSBAQCBkZWYgc3VtbWFyaXplX3Jlc3VsdHMocG9ydF9vYmosIGV4cGVjdGF0CiAgICAgICAg ICAgICAgICAgY29udGludWUKICAgICAgICAgZWxpZiByZXN1bHRfdHlwZSA9PSB0ZXN0X2V4cGVj dGF0aW9ucy5DUkFTSDoKICAgICAgICAgICAgIG51bV9yZWdyZXNzaW9ucyArPSAxCisgICAgICAg IGVsaWYgcmVzdWx0X3R5cGUgPT0gdGVzdF9leHBlY3RhdGlvbnMuTUlTU0lORzoKKyAgICAgICAg ICAgIG51bV9mbGFreSArPSAxCisgICAgICAgICAgICAjIEZJWE1FOiBicmVhayBNSVNTSU5HIHJl c3VsdHMgaW50byBhIGRpZmZlcmVudCBjYXRlZ29yeQogICAgICAgICBlbGlmIHRlc3RfbmFtZSBp biByZXN1bHRfc3VtbWFyeS51bmV4cGVjdGVkX3Jlc3VsdHM6CiAgICAgICAgICAgICBpZiB0ZXN0 X25hbWUgbm90IGluIHJldHJ5X3N1bW1hcnkudW5leHBlY3RlZF9yZXN1bHRzOgogICAgICAgICAg ICAgICAgIGFjdHVhbC5leHRlbmQoZXhwZWN0YXRpb25zLmdldF9leHBlY3RhdGlvbnNfc3RyaW5n KHRlc3RfbmFtZSkuc3BsaXQoIiAiKSkKQEAgLTE0Nyw5ICsxNTAsNiBAQCBkZWYgc3VtbWFyaXpl X3Jlc3VsdHMocG9ydF9vYmosIGV4cGVjdGF0CiAgICAgICAgICAgICAgICAgaWYgcmVzdWx0X3R5 cGUgIT0gcmV0cnlfcmVzdWx0X3R5cGU6CiAgICAgICAgICAgICAgICAgICAgIGFjdHVhbC5hcHBl bmQoa2V5d29yZHNbcmV0cnlfcmVzdWx0X3R5cGVdKQogICAgICAgICAgICAgICAgICAgICBudW1f Zmxha3kgKz0gMQotICAgICAgICAgICAgICAgICMgRklYTUU6IGJyZWFrIE1JU1NJTkcgcmVzdWx0 cyBpbnRvIGEgZGlmZmVyZW50IGNhdGVnb3J5Ci0gICAgICAgICAgICAgICAgZWxpZiAnTUlTU0lO RycgaW4gYWN0dWFsOgotICAgICAgICAgICAgICAgICAgICBudW1fZmxha3kgKz0gMQogICAgICAg ICAgICAgICAgIGVsc2U6CiAgICAgICAgICAgICAgICAgICAgIG51bV9yZWdyZXNzaW9ucyArPSAx CiAK 110794 2011-10-12 19:15:22 -0700 2011-10-12 20:17:47 -0700 Added a test bug-67268-20111012191521.patch text/plain 4225 rniwa SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDk3MzI4KQorKysgVG9vbHMvQ2hhbmdlTG9nCSh3b3JraW5nIGNvcHkpCkBAIC0xLDMg KzEsMTkgQEAKKzIwMTEtMTAtMTIgIFJ5b3N1a2UgTml3YSAgPHJuaXdhQHdlYmtpdC5vcmc+CisK KyAgICAgICAgW05SV1RdIE5ldyB0ZXN0cyB3aXRob3V0IGV4cGVjdGVkIGZpbGVzIGFyZSByZXBv cnRlZCBhcyBmbGFrZXkgdGVzdHMKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hv d19idWcuY2dpP2lkPTY3MjY4CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISku CisKKyAgICAgICAgVGhlIGJ1ZyB3YXMgY2F1c2VkIGJ5IHN1bW1hcml6ZV9yZXN1bHRzJ3MgcmVn YXJkaW5nIGFsbCB0ZXN0cyB0aGF0IGhhdmUgYW4gZW50cnkgaW4gcmVzdWx0X3N1bW1hcnkKKyAg ICAgICAgYW5kIG5vIGVudHJ5IGluIHJldHJ5X3N1bW1hcnkgYXMgZmxha3kuIFNpbmNlIHdlIGRv bid0IHJldHJ5IG1pc3NpbmcgdGVzdHMsIHRoZXJlIHdpbGwgYmUgbm8gZW50cmllcworICAgICAg ICBmb3IgbWlzc2luZyByZXN1bHRzIGluIHJldHJ5X3N1bW1hcnkuCisKKyAgICAgICAgRml4ZWQg dGhlIGJ1ZyBieSB0cmVhdGluZyBtaXNzaW5nIHJlc3VsdHMgc2VwYXJhdGVseS4KKworICAgICAg ICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHk6 CisgICAgICAgICogU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0 c19pbnRlZ3JhdGlvbnRlc3QucHk6CisKIDIwMTEtMTAtMTIgIFNpbW9uIEZyYXNlciAgPHNpbW9u LmZyYXNlckBhcHBsZS5jb20+CiAKICAgICAgICAgTWFueSBXSzIgc25hcHNob3RzIGFyZSBibGFu awpJbmRleDogVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90 ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy93ZWJr aXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkJKHJl dmlzaW9uIDk3MzE5KQorKysgVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcnVu X3dlYmtpdF90ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkJKHdvcmtpbmcgY29weSkKQEAgLTQyNiw2 ICs0MjYsMjMgQEAgY2xhc3MgTWFpblRlc3QodW5pdHRlc3QuVGVzdENhc2UpOgogICAgICAgICBz ZWxmLmFzc2VydEZhbHNlKGVyci5lbXB0eSgpKQogICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHVz ZXIub3BlbmVkX3VybHMsIFsnL3RtcC9sYXlvdXQtdGVzdC1yZXN1bHRzL3Jlc3VsdHMuaHRtbCdd KQogCisgICAgZGVmIHRlc3RfbWlzc2luZ19hbmRfdW5leHBlY3RlZF9yZXN1bHRzKHNlbGYpOgor ICAgICAgICAjIFRlc3QgdGhhdCB3ZSB1cGRhdGUgZXhwZWN0YXRpb25zIGluIHBsYWNlLiBJZiB0 aGUgZXhwZWN0YXRpb24KKyAgICAgICAgIyBpcyBtaXNzaW5nLCB1cGRhdGUgdGhlIGV4cGVjdGVk IGdlbmVyaWMgbG9jYXRpb24uCisgICAgICAgIGZzID0gdW5pdF90ZXN0X2ZpbGVzeXN0ZW0oKQor ICAgICAgICByZXMsIG91dCwgZXJyLCBfID0gbG9nZ2luZ19ydW4oWyctLW5vLXNob3ctcmVzdWx0 cycsCisgICAgICAgICAgICAnZmFpbHVyZXMvZXhwZWN0ZWQvbWlzc2luZ19pbWFnZS5odG1sJywK KyAgICAgICAgICAgICdmYWlsdXJlcy91bmV4cGVjdGVkL21pc3NpbmdfdGV4dC5odG1sJywKKyAg ICAgICAgICAgICdmYWlsdXJlcy91bmV4cGVjdGVkL3RleHQtaW1hZ2UtY2hlY2tzdW0uaHRtbCdd LAorICAgICAgICAgICAgdGVzdHNfaW5jbHVkZWQ9VHJ1ZSwgZmlsZXN5c3RlbT1mcywgcmVjb3Jk X3Jlc3VsdHM9VHJ1ZSkKKyAgICAgICAgZmlsZV9saXN0ID0gZnMud3JpdHRlbl9maWxlcy5rZXlz KCkKKyAgICAgICAgZmlsZV9saXN0LnJlbW92ZSgnL3RtcC9sYXlvdXQtdGVzdC1yZXN1bHRzL3Rl c3RzX3J1bjAudHh0JykKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbHMocmVzLCAxKQorICAgICAg ICBleHBlY3RlZF90b2tlbiA9ICcidW5leHBlY3RlZCI6eyJ0ZXh0LWltYWdlLWNoZWNrc3VtLmh0 bWwiOnsiZXhwZWN0ZWQiOiJQQVNTIiwiYWN0dWFsIjoiVEVYVCJ9LCJtaXNzaW5nX3RleHQuaHRt bCI6eyJleHBlY3RlZCI6IlBBU1MiLCJpc19taXNzaW5nX3RleHQiOnRydWUsImFjdHVhbCI6Ik1J U1NJTkcifScKKyAgICAgICAganNvbl9zdHJpbmcgPSBmcy5yZWFkX3RleHRfZmlsZSgnL3RtcC9s YXlvdXQtdGVzdC1yZXN1bHRzL2Z1bGxfcmVzdWx0cy5qc29uJykKKyAgICAgICAgc2VsZi5hc3Nl cnRUcnVlKGpzb25fc3RyaW5nLmZpbmQoZXhwZWN0ZWRfdG9rZW4pICE9IC0xKQorICAgICAgICBz ZWxmLmFzc2VydFRydWUoanNvbl9zdHJpbmcuZmluZCgnIm51bV9mbGFreSI6MScpICE9IC0xKQor CiAgICAgZGVmIHRlc3RfY3Jhc2hfd2l0aF9zdGRlcnIoc2VsZik6CiAgICAgICAgIGZzID0gdW5p dF90ZXN0X2ZpbGVzeXN0ZW0oKQogICAgICAgICByZXMsIGJ1aWxkYm90X291dHB1dCwgcmVndWxh cl9vdXRwdXQsIHVzZXIgPSBsb2dnaW5nX3J1bihbCkluZGV4OiBUb29scy9TY3JpcHRzL3dlYmtp dHB5L2xheW91dF90ZXN0cy9jb250cm9sbGVycy9tYW5hZ2VyLnB5Cj09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFRv b2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHkJ KHJldmlzaW9uIDk3MzE5KQorKysgVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMv Y29udHJvbGxlcnMvbWFuYWdlci5weQkod29ya2luZyBjb3B5KQpAQCAtMTM4LDYgKzEzOCwxMiBA QCBkZWYgc3VtbWFyaXplX3Jlc3VsdHMocG9ydF9vYmosIGV4cGVjdGF0CiAgICAgICAgICAgICAg ICAgY29udGludWUKICAgICAgICAgZWxpZiByZXN1bHRfdHlwZSA9PSB0ZXN0X2V4cGVjdGF0aW9u cy5DUkFTSDoKICAgICAgICAgICAgIG51bV9yZWdyZXNzaW9ucyArPSAxCisgICAgICAgIGVsaWYg cmVzdWx0X3R5cGUgPT0gdGVzdF9leHBlY3RhdGlvbnMuTUlTU0lORzoKKyAgICAgICAgICAgICMg V2UgY291bnQgbWlzc2luZyByZXN1bHRzIGFzIGZsYWt5IG5vdCB0byB0dXJuIGJ1aWxkYm90IHJl ZAorICAgICAgICAgICAgIyBUaGlzIGlzIGEgaHVnZSBoYWNrIHNob3VsZCBiZSBmaXhlZCBieSBh ZGRpbmcgbmV3IGNhdGVnb3J5IGZvciBNSVNTSU5HIHJlc3VsdHMuCisgICAgICAgICAgICAjIFNl ZSBhbHNvOiBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9NjQ4MTIKKyAg ICAgICAgICAgIGlmIHRlc3RfbmFtZSBpbiByZXN1bHRfc3VtbWFyeS51bmV4cGVjdGVkX3Jlc3Vs dHM6CisgICAgICAgICAgICAgICAgbnVtX2ZsYWt5ICs9IDEKICAgICAgICAgZWxpZiB0ZXN0X25h bWUgaW4gcmVzdWx0X3N1bW1hcnkudW5leHBlY3RlZF9yZXN1bHRzOgogICAgICAgICAgICAgaWYg dGVzdF9uYW1lIG5vdCBpbiByZXRyeV9zdW1tYXJ5LnVuZXhwZWN0ZWRfcmVzdWx0czoKICAgICAg ICAgICAgICAgICBhY3R1YWwuZXh0ZW5kKGV4cGVjdGF0aW9ucy5nZXRfZXhwZWN0YXRpb25zX3N0 cmluZyh0ZXN0X25hbWUpLnNwbGl0KCIgIikpCkBAIC0xNDcsOSArMTUzLDYgQEAgZGVmIHN1bW1h cml6ZV9yZXN1bHRzKHBvcnRfb2JqLCBleHBlY3RhdAogICAgICAgICAgICAgICAgIGlmIHJlc3Vs dF90eXBlICE9IHJldHJ5X3Jlc3VsdF90eXBlOgogICAgICAgICAgICAgICAgICAgICBhY3R1YWwu YXBwZW5kKGtleXdvcmRzW3JldHJ5X3Jlc3VsdF90eXBlXSkKICAgICAgICAgICAgICAgICAgICAg bnVtX2ZsYWt5ICs9IDEKLSAgICAgICAgICAgICAgICAjIEZJWE1FOiBicmVhayBNSVNTSU5HIHJl c3VsdHMgaW50byBhIGRpZmZlcmVudCBjYXRlZ29yeQotICAgICAgICAgICAgICAgIGVsaWYgJ01J U1NJTkcnIGluIGFjdHVhbDoKLSAgICAgICAgICAgICAgICAgICAgbnVtX2ZsYWt5ICs9IDEKICAg ICAgICAgICAgICAgICBlbHNlOgogICAgICAgICAgICAgICAgICAgICBudW1fcmVncmVzc2lvbnMg Kz0gMQogCg==