139123 2014-12-01 05:38:54 -0800 REGRESSION: mozilla-tests.yaml/ecma/Date/15.9.5.8.js fails intermittently 2019-12-02 10:10:01 -0800 1 1 1 Unclassified WebKit JavaScriptCore 528+ (Nightly build) Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=138303 https://bugs.webkit.org/show_bug.cgi?id=142531 P1 Critical --- 1 ossy webkit-unassigned aakash_jain ap bfulgham cgarcia fpizlo ggaren guijemont gyuyoung.kim mark.lam mmaxfield msaboff ossy pmatos oldest_to_newest 1051419 0 ossy 2014-12-01 05:38:54 -0800 It fails everywhere in this time interval: - Apple Yosemite WK1: https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK1%20%28Tests%29/builds/985 - Apple Yosemite WK2: https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK2%20%28Tests%29/builds/982 - Apple Mavericks WK1: https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK1%20%28Tests%29/builds/10295 - Apple Mavericks WK2: https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK2%20%28Tests%29/builds/9718 - EFL: https://build.webkit.org/builders/EFL%20Linux%2064-bit%20Release%20WK2/builds/17969 - GTK: https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Tests%29/builds/4517 1051420 1 ossy 2014-12-01 05:39:55 -0800 mozilla-tests.yaml/ecma/Date/15.9.5.8.js.mozilla: (new Date(1446364102494)).getMonth() = 10 FAILED! expected: 9 mozilla-tests.yaml/ecma/Date/15.9.5.8.js.mozilla: (new Date(1446364102495)).getMonth() = 10 FAILED! expected: 9 mozilla-tests.yaml/ecma/Date/15.9.5.8.js.mozilla: (new Date(1446364102493)).getMonth() = 10 FAILED! expected: 9 1051421 2 ossy 2014-12-01 05:41:01 -0800 maybe it is related to bug138303 somehow, but it is unlikely. 1051487 3 ap 2014-12-01 10:57:23 -0800 The tests are still failing today 1051507 4 ossy 2014-12-01 11:29:10 -0800 strange, it must be a flakiness: Nov 30 23:46 - fail - https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK1%20%28Tests%29/builds/10295 Dec 01 05:26 - pass - https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK1%20%28Tests%29/builds/10296 Dec 01 06:59 - fail - https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK1%20%28Tests%29/builds/10297 Dec 01 10:40 - pass - https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK1%20%28Tests%29/builds/10298 1073264 5 ossy 2015-03-01 23:39:02 -0800 and again: - https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK2%20%28Tests%29/builds/11978 - https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK1%20%28Tests%29/builds/12570 - https://build.webkit.org/builders/EFL%20Linux%2064-bit%20Release%20WK2/builds/20120 - https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK1%20%28Tests%29/builds/3241 - https://build.webkit.org/builders/Apple%20Yosemite%20Release%20WK2%20%28Tests%29/builds/3228 1073265 6 ossy 2015-03-01 23:41:28 -0800 It seems this test fails only on the first day of months between 22:00-24:00. 1073343 7 ap 2015-03-02 09:27:36 -0800 The tests just failed again on all bots around 8am on March 2nd. 1073344 8 mmaxfield 2015-03-02 09:28:50 -0800 Perhaps related to daylight savings? 1073361 9 fpizlo 2015-03-02 10:14:08 -0800 (In reply to comment #8) > Perhaps related to daylight savings? Yes :-) 1073384 10 mmaxfield 2015-03-02 11:12:38 -0800 Looking into this now. 1073412 11 mmaxfield 2015-03-02 12:25:36 -0800 This test uses the current time, then performs date math on it in JS code, and compares to the same date math being performed by the platform. However, this date math is sensitive to daylight savings code. I propose that we perform the tests on hardcoded dates and not on "now" 1073415 12 fpizlo 2015-03-02 12:33:22 -0800 (In reply to comment #11) > This test uses the current time, then performs date math on it in JS code, > and compares to the same date math being performed by the platform. However, > this date math is sensitive to daylight savings code. I propose that we > perform the tests on hardcoded dates and not on "now" We could do that, but before we do, you should check what other coverage we have on the behavior of Date.now(). Also, have you checked if these tests overlap the more modern tests in sputnik and ietestcenter, or even Test262? Fixing these ancient Mozilla tests if there is a modern test with similar coverage is probably not worthwhile. 1073435 13 ap 2015-03-02 13:54:19 -0800 *** Bug 142180 has been marked as a duplicate of this bug. *** 1075585 14 ossy 2015-03-09 04:40:55 -0700 similar bug: bug138303 1075630 15 bfulgham 2015-03-09 09:56:17 -0700 The daylight-savings sensitivity is actually a useful thing to test. After all, real world code needs to deal with this stuff! However, I do think it would be better to have specific Daylight Savings Time tests so it would be very clear when these start failing that it is related to the time of year. What makes this more difficult is that the DST logic can change from year-to-year (based on law). But I agree that all of our stock date tests should use specific dates and times. To exercise 'now' we should probably place this in its own test with DST logic or special code to account for DST. 1075712 16 mmaxfield 2015-03-09 12:24:35 -0700 The consensus that Mark Lam and I came up with is that we should test all cases with specific test cases, but that we should not test "now." That way, we maintain our coverage without having possible future flakes if governments change DST behavior. 1075715 17 mark.lam 2015-03-09 12:28:41 -0700 (In reply to comment #16) > The consensus that Mark Lam and I came up with is that we should test all > cases with specific test cases, but that we should not test "now." That way, > we maintain our coverage without having possible future flakes if > governments change DST behavior. To clarify, I think Brent’s suggestion is a good one i.e. to put the DST test cases in a DST specific test. That makes it easier to understand at a glance what could have gone wrong. Brent’s suggestion is not in conflict with the approach Myles and I discussed. 1075721 18 fpizlo 2015-03-09 12:39:18 -0700 I'm opposed to removing these tests, as they are some of the more complex pieces of code that use Date and we should at least run them for crash test coverage. It shouldn't be hard to arrange for them to be run just for seeing that they don't crash. I haven't seen any comments regarding overlap with sputnik. Sputnik is a comprehensive and modern JS test suite that we run in LayoutTests. What we do about these tests depends on whether or not sputnik has Date.now tests. Have you looked at this, and what did you find? 1594664 19 aakash_jain 2019-12-02 10:10:01 -0800 This test flaked in https://ews-build.webkit.org/#/builders/26/builds/652 It failed in jscore-test, but passed on retry (in immediately next jscore-test-rerun step). mozilla-tests.yaml/ecma/Date/15.9.5.8.js.mozilla-dfg-eager-no-cjit-validate-phases: (new Date(1604216191497)).getMonth() = 10 FAILED! expected: 9 mozilla-tests.yaml/ecma/Date/15.9.5.8.js.mozilla-no-ftl: (new Date(1604216191382)).getMonth() = 10 FAILED! expected: 9