Attachment 348883 Details for Bug 189293 – Patch

[patch] Patch

bug-189293-20180904182124.patch (text/plain), 3.81 KB, created by Simon Fraser (smfr) on 2018-09-04 18:21:25 PDT

(hide)

Description:

Filename:

MIME Type:

Creator: Simon Fraser (smfr)

Created: 2018-09-04 18:21:25 PDT

Size: 3.81 KB

patch

obsolete

>Subversion Revision: 235632
>diff --git a/Tools/ChangeLog b/Tools/ChangeLog
>index 78cb21285b155ee05075a60ecd848243b1042128..0f2edf0778413bfc86ea37b5e27f4b8b27d1a6e2 100644
>--- a/Tools/ChangeLog
>+++ b/Tools/ChangeLog
>@@ -1,3 +1,19 @@
>+2018-09-04  Simon Fraser  <simon.fraser@apple.com>
>+
>+        Log when leak detection changes the test result
>+        https://bugs.webkit.org/show_bug.cgi?id=189293
>+
>+        Reviewed by NOBODY (OOPS!).
>+        
>+        When you have an "Leak" expectation in TestExpectations, it's confusing to see a test unexpectedly pass,
>+        but then show up as an expected fail at the end (because leak detection happens at the end of a shard).
>+        So log when leak detection changes a test result.
>+
>+        * Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py:
>+        (LayoutTestRunner._annotate_results_with_additional_failures):
>+        * Scripts/webkitpy/layout_tests/models/test_expectations.py:
>+        (TestExpectations):
>+
> 2018-09-04  Chris Dumez  <cdumez@apple.com>
> 
>         Add process pool configuration flag to turn on automatic process pre-warming
>diff --git a/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py b/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py
>index 4b19974d9874dd826d3ab5d6db2b80eed99d1412..eb3cec29459eb165f80ff5cdf8738f390de6131b 100644
>--- a/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py
>+++ b/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_runner.py
>@@ -200,6 +200,10 @@ class LayoutTestRunner(object):
>             if existing_result and existing_result.test_number is not None:
>                 was_expected = self._expectations.matches_an_expected_result(new_result.test_name, existing_result.type, self._options.pixel_tests or existing_result.reftest_type, self._options.world_leaks)
>                 now_expected = self._expectations.matches_an_expected_result(new_result.test_name, new_result.type, self._options.pixel_tests or new_result.reftest_type, self._options.world_leaks)
>+                if was_expected != now_expected:
>+                    _log.warning('  %s -> test was a %s (%s) but is now a %s (%s)' % (new_result.test_name,
>+                        TestExpectations.EXPECTATION_DESCRIPTION[existing_result.type], 'expected' if was_expected else 'unexpected',
>+                        TestExpectations.EXPECTATION_DESCRIPTION[new_result.type], 'expected' if now_expected else 'unexpected'))
>                 run_results.change_result_to_failure(existing_result, new_result, was_expected, now_expected)
> 
>     def start_servers(self):
>diff --git a/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py b/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py
>index b3acc6725a74e4a3f8b938a8d0df48f87d699b20..47f01c98ce1fea834aea0ba375fb50f4b7d4a88b 100644
>--- a/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py
>+++ b/Tools/Scripts/webkitpy/layout_tests/models/test_expectations.py
>@@ -801,6 +801,19 @@ class TestExpectations(object):
>                     'leak': LEAK,
>                     'skip': SKIP}
> 
>+    # Singulars
>+    EXPECTATION_DESCRIPTION = {SKIP: 'skipped',
>+                                PASS: 'pass',
>+                                FAIL: 'failure',
>+                                IMAGE: 'image-only failure',
>+                                TEXT: 'text-only failure',
>+                                IMAGE_PLUS_TEXT: 'image and text failure',
>+                                AUDIO: 'audio failure',
>+                                CRASH: 'crash',
>+                                TIMEOUT: 'timeout',
>+                                MISSING: 'missing',
>+                                LEAK: 'leak'}
>+
>     # (aggregated by category, pass/fail/skip, type)
>     EXPECTATION_DESCRIPTIONS = {SKIP: 'skipped',
>                                 PASS: 'passes',

Actions: View | Formatted Diff | Diff

Attachments on bug 189293: 348883 | 349084