Attachment 127303 Details for Bug 69444 – Patch

[patch] Patch

bug-69444-20120215200017.patch (text/plain), 4.21 KB, created by Dirk Pranke on 2012-02-15 20:00:18 PST

(hide)

Description:

Filename:

MIME Type:

Creator: Dirk Pranke

Created: 2012-02-15 20:00:18 PST

Size: 4.21 KB

patch

obsolete

>Subversion Revision: 107867
>diff --git a/Tools/ChangeLog b/Tools/ChangeLog
>index f85eee7afe9ed179e3427388513c75fc0d1ea691..a0308abf322c2dca734db7d7f663268c78ef27a6 100644
>--- a/Tools/ChangeLog
>+++ b/Tools/ChangeLog
>@@ -1,5 +1,25 @@
> 2012-02-15  Dirk Pranke  <dpranke@chromium.org>
> 
>+        The new run-webkit-tests needs to dump out pixel hash failures even if the pixel test passes.
>+        https://bugs.webkit.org/show_bug.cgi?id=69444
>+
>+        Reviewed by NOBODY (OOPS!).
>+
>+        Currently if a test's image checksum doesn't match the checksum
>+        in the baseline, but the images themselves match according to
>+        ImageDiff, new-run-webkit-tests ignores the problem. This is
>+        probably bad, but it's not yet clear what the right thing to do
>+        is. This patch will log a warning to stderr, at least (but the
>+        test will still pass).
>+
>+        * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
>+        (SingleTestRunner._compare_image):
>+        (SingleTestRunner._run_reftest):
>+        * Scripts/webkitpy/layout_tests/port/test.py:
>+        (unit_test_list):
>+
>+2012-02-15  Dirk Pranke  <dpranke@chromium.org>
>+
>         webkitpy: add a worker_args concept to start_worker()
>         https://bugs.webkit.org/show_bug.cgi?id=78572
> 
>diff --git a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
>index 89ae5bdcaea945e8cd2f7e93af766b4ec27940d3..c903cb3494a20e425ddbb03b9f0585c0953df69a 100644
>--- a/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
>+++ b/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
>@@ -264,6 +264,10 @@ class SingleTestRunner:
>             driver_output.image_diff = diff_result[0]
>             if driver_output.image_diff:
>                 failures.append(test_failures.FailureImageHashMismatch(diff_result[1]))
>+            else:
>+                # See https://bugs.webkit.org/show_bug.cgi?id=69444 for why this isn't a full failure.
>+                _log.warning('%s had mismatched checksums but matching images (exp %s, got %s)' %
>+                            (self._test_name, expected_driver_output.image_hash, driver_output.image_hash))
>         return failures
> 
>     def _run_reftest(self):
>@@ -272,8 +276,8 @@ class SingleTestRunner:
>         reference_output = None
>         test_result = None
> 
>-        # A reftest can have multiple match references and multiple mismatch references; 
>-        # the test fails if any mismatch matches and all of the matches don't match. 
>+        # A reftest can have multiple match references and multiple mismatch references;
>+        # the test fails if any mismatch matches and all of the matches don't match.
>         # To minimize the number of references we have to check, we run all of the mismatches first,
>         # then the matches, and short-circuit out as soon as we can.
>         # Note that sorting by the expectation sorts "!=" before "==" so this is easy to do.
>diff --git a/Tools/Scripts/webkitpy/layout_tests/port/test.py b/Tools/Scripts/webkitpy/layout_tests/port/test.py
>index b3aecfd2bd368286296bc439427c742387ee6d0f..3ddac1801b45831d3942e89bcf7ade960343565b 100644
>--- a/Tools/Scripts/webkitpy/layout_tests/port/test.py
>+++ b/Tools/Scripts/webkitpy/layout_tests/port/test.py
>@@ -100,8 +100,6 @@ class TestList(object):
> 
> def unit_test_list():
>     tests = TestList()
>-    tests.add('failures/expected/checksum.html',
>-              actual_checksum='checksum_fail-checksum')
>     tests.add('failures/expected/crash.html', crash=True)
>     tests.add('failures/expected/exception.html', exception=True)
>     tests.add('failures/expected/timeout.html', timeout=True)
>@@ -171,6 +169,10 @@ layer at (0,0) size 800x34
>               expected_checksum=None,
>               expected_image='tEXtchecksum\x00checksum_in_image-checksum')
> 
>+    # Note that here the checksums don't match but the images do, so this test passes.
>+    # See https://bugs.webkit.org/show_bug.cgi?id=69444 .
>+    tests.add('passes/checksum.html', actual_checksum='checksum_fail-checksum')
>+
>     # Text output files contain "\r\n" on Windows.  This may be
>     # helpfully filtered to "\r\r\n" by our Python/Cygwin tooling.
>     tests.add('passes/text.html',

Actions: View | Formatted Diff | Diff

Attachments on bug 69444: 127303 | 127305 | 127306 | 127506