WebKit Bugzilla
New
Browse
Log In
×
Sign in with GitHub
or
Remember my login
Create Account
·
Forgot Password
Forgotten password account recovery
RESOLVED FIXED
38296
Import Sputnik JavaScript test suite
https://bugs.webkit.org/show_bug.cgi?id=38296
Summary
Import Sputnik JavaScript test suite
Alexey Proskuryakov
Reported
2010-04-28 17:21:54 PDT
Created
attachment 54639
[details]
translation script Some notes: - This contains both conformance tests that the web page runs, and other tests from source checkout. - It would be better to use run-javascriptcore-tests for these tests, but also more difficult to make a nice test runner. Someone can do that later. - Sputnik checkout contains tests that need to be processed before use. I've processed them to something like script-tests, but monolithic for better loading speed. - Sputnik tests often have several subtests, but bail out after the first error. There seems to be work in progress to distinguish hard failures from simple ones, and some tests use FAIL and ERROR distinctly - but since Google's own runner doesn't seem to distinguish those, I also didn't. - Several Unicode tests are disabled, because they are too slow due to iterating lots of code points. To import the tests, one needs to get a source checkout as below, edit sputnik.py, and run the attached script. - self.test_root = path.join(root, 'tests', 'Conformance') + self.test_root = path.join(root, 'tests')
Attachments
translation script
(3.82 KB, text/plain)
2010-04-28 17:21 PDT
,
Alexey Proskuryakov
no flags
Details
partial patch
(101.56 KB, patch)
2010-04-28 17:33 PDT
,
Alexey Proskuryakov
darin
: review+
Details
Formatted Diff
Diff
View All
Add attachment
proposed patch, testcase, etc.
Alexey Proskuryakov
Comment 1
2010-04-28 17:33:30 PDT
Created
attachment 54642
[details]
partial patch The complete patch is 23 megabytes, uploading a part.
Darin Adler
Comment 2
2010-04-28 17:36:03 PDT
Comment on
attachment 54642
[details]
partial patch rs=me Is there a way to reduce the total amount of boilerplate? Since this suite was made by the V8 team, I assume that the failures are JavaScriptCore-specific, so we need to expect success on the Chromium platform. How will we do that?
Alexey Proskuryakov
Comment 3
2010-04-28 17:45:15 PDT
> Is there a way to reduce the total amount of boilerplate?
It was my intentional choice to have monolithic tests for faster loads (but I didn't measure how much that helps). But I don't feel very strongly about either approach - hopefully, these tests will be moved to run-javascriptcore-tests in foreseeable future.
> we need to expect success on the Chromium platform. How will we do that?
Someone from Chromium team usually updates expectations later, since build.webkit.org doesn't list Chromium results, as far as I can tell. I'm quite worried about timeouts on slower bots though - there are several tests that took noticeable time on my Mac Pro. V8 fails more of these tests than JSC does, according to <
http://sputnik.googlelabs.com/compare
>.
Alexey Proskuryakov
Comment 4
2010-04-29 13:47:06 PDT
Committed <
http://trac.webkit.org/changeset/58534
>.
WebKit Review Bot
Comment 5
2010-04-29 14:07:48 PDT
http://trac.webkit.org/changeset/58534
might have broken GTK Linux 64-bit Release and Qt Linux Release The following changes are on the blame list:
http://trac.webkit.org/changeset/58533
http://trac.webkit.org/changeset/58534
Alexey Proskuryakov
Comment 6
2010-04-29 14:36:22 PDT
Added the whole directory to Skipped list, since there are mysterious (and different) failures on all platforms besides SnowLeopard.
Alexey Proskuryakov
Comment 7
2010-04-29 17:01:46 PDT
Paths in the test suite are way too long for svn on Windows. Subversion only supports up to 256 or 260 when used with relative paths, but up to 32K with absolute paths. I had to shorten the paths manually for checkout to work.
Alexey Proskuryakov
Comment 8
2010-04-29 17:11:36 PDT
Filed
http://code.google.com/p/sputniktests/issues/detail?id=25
Note
You need to
log in
before you can comment on or make changes to this bug.
Top of Page
Format For Printing
XML
Clone This Bug