Attachment 191049 Details for Bug 90062 – Fixes the bug

[patch] Fixes the bug

bug-90062-20130301150909.patch (text/plain), 5.94 KB, created by Ryosuke Niwa on 2013-03-01 15:12:58 PST

(hide)

Description:

Filename:

MIME Type:

Creator: Ryosuke Niwa

Created: 2013-03-01 15:12:58 PST

Size: 5.94 KB

patch

obsolete

>Index: Tools/ChangeLog
>===================================================================
>--- Tools/ChangeLog	(revision 144510)
>+++ Tools/ChangeLog	(working copy)
>@@ -1,3 +1,26 @@
>+2013-03-01  Ryosuke Niwa  <rniwa@webkit.org>
>+
>+        Output numbers of run-perf-tests should be rounded in accordance with sample standard deviations
>+        https://bugs.webkit.org/show_bug.cgi?id=90062
>+
>+        Reviewed by NOBODY (OOPS!).
>+
>+        Round the sample standard deviation at 3 significant figures and show it in percentage with respect
>+        to the mean. Round other statistics based on the most significant digit of the sample standard deviation.
>+
>+        We show one more digit than the most significant digit in the sample standard deviation since it's
>+        helpful when doing back-of-the-envelop calculations using these numbers.
>+
>+        We also only round decimal points since it's not obvious at which point numbers are rounded when we have
>+        bunch of zeros. e.g. 123000 versus 123456 where 456 is the rounded portion.
>+
>+        * Scripts/webkitpy/performance_tests/perftest.py:
>+        (PerfTest.log_statistics.highest_decimal_place): Added.
>+        (PerfTest.log_statistics.round_to_sig_fig): Added.
>+        (PerfTest.log_statistics):
>+        (PerfTest.log_statistics.round_to_sig_fig):
>+        * Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py:
>+
> 2013-03-01  Ryosuke Niwa  <rniwa@webkit.org>
> 
>         Don't use legacy test names when returning results from PerfTest.run
>Index: Tools/Scripts/webkitpy/performance_tests/perftest.py
>===================================================================
>--- Tools/Scripts/webkitpy/performance_tests/perftest.py	(revision 144510)
>+++ Tools/Scripts/webkitpy/performance_tests/perftest.py	(working copy)
>@@ -1,4 +1,5 @@
> # Copyright (C) 2012 Google Inc. All rights reserved.
>+# Copyright (C) 2012, 2013 Apple Inc. All rights reserved.
> # Copyright (C) 2012 Zoltan Horvath, Adobe Systems Incorporated. All rights reserved.
> #
> # Redistribution and use in source and binary forms, with or without
>@@ -151,9 +152,22 @@ class PerfTest(object):
>         median = sorted_values[middle] if len(sorted_values) % 2 else (sorted_values[middle - 1] + sorted_values[middle]) / 2
>         stdev = math.sqrt(square_sum / (len(sorted_values) - 1)) if len(sorted_values) > 1 else 0
> 
>-        _log.info('RESULT %s= %s %s' % (test_name, mean, unit))
>-        _log.info('median= %s %s, stdev= %s %s, min= %s %s, max= %s %s' %
>-            (median, unit, stdev, unit, sorted_values[0], unit, sorted_values[-1], unit))
>+        def highest_decimal_place(x):
>+            return int(math.floor(math.log(x, 10)))
>+
>+        def round_to_sig_fig(x, sig_fig):
>+            return round(x, -highest_decimal_place(x) + sig_fig - 1)
>+
>+        stdev_decimal_place = -highest_decimal_place(stdev) + 1
>+        if stdev_decimal_place <= 0:
>+            round_to_stdev = lambda x: int(round(x))
>+        else:
>+            round_to_stdev = lambda x: round(x, stdev_decimal_place)
>+
>+        _log.info('RESULT %s= %s %s' % (test_name, round_to_stdev(mean), unit))
>+        _log.info('median= %s %s, stdev= %s %%, min= %s %s, max= %s %s' %
>+            (round_to_stdev(median), unit, round_to_sig_fig(stdev / mean * 100, 3),
>+            round_to_stdev(sorted_values[0]), unit, round_to_stdev(sorted_values[-1]), unit))
> 
>     _description_regex = re.compile(r'^Description: (?P<description>.*)$', re.IGNORECASE)
>     _metrics_regex = re.compile(r'^(?P<metric>Time|Malloc|JS Heap):')
>Index: Tools/Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py
>===================================================================
>--- Tools/Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py	(revision 144478)
>+++ Tools/Scripts/webkitpy/performance_tests/perftestsrunner_integrationtest.py	(working copy)
>@@ -1,4 +1,5 @@
> # Copyright (C) 2012 Google Inc. All rights reserved.
>+# Copyright (C) 2012, 2013 Apple Inc. All rights reserved.
> #
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions are
>@@ -86,8 +87,8 @@ max 1510 ms
> """
> 
>     output = """Running Bindings/event-target-wrapper.html (1 of 2)
>-RESULT Bindings: event-target-wrapper: Time= 1490.0 ms
>-median= 1488.0 ms, stdev= 15.13935 ms, min= 1471.0 ms, max= 1510.0 ms
>+RESULT Bindings: event-target-wrapper: Time= 1490 ms
>+median= 1488 ms, stdev= 1.02 %, min= 1471 ms, max= 1510 ms
> Finished: 0.1 s
> 
> """
>@@ -101,17 +102,17 @@ class SomeParserTestData:
> Ignoring warm-up run (1115)
> 
> Time:
>-values 1080, 1120, 1095, 1101, 1104 ms
>-avg 1100 ms
>-median 1101 ms
>-stdev 14.50861 ms
>-min 1080 ms
>-max 1120 ms
>+values 10.83, 11.20, 10.95, 11.01, 11.04 ms
>+avg 11.01 ms
>+median 11.01 ms
>+stdev 0.13501 ms
>+min 10.83 ms
>+max 11.20 ms
> """
> 
>     output = """Running Parser/some-parser.html (2 of 2)
>-RESULT Parser: some-parser: Time= 1100.0 ms
>-median= 1101.0 ms, stdev= 14.50861 ms, min= 1080.0 ms, max= 1120.0 ms
>+RESULT Parser: some-parser: Time= 11.01 ms
>+median= 11.01 ms, stdev= 1.23 %, min= 10.83 ms, max= 11.2 ms
> Finished: 0.1 s
> 
> """
>@@ -148,12 +149,12 @@ max 548000 bytes
> 
>     output = """Running 1 tests
> Running Parser/memory-test.html (1 of 1)
>-RESULT Parser: memory-test: Time= 1100.0 ms
>-median= 1101.0 ms, stdev= 14.50861 ms, min= 1080.0 ms, max= 1120.0 ms
>-RESULT Parser: memory-test: JSHeap= 830000.0 bytes
>-median= 829000.0 bytes, stdev= 13784.04875 bytes, min= 811000.0 bytes, max= 848000.0 bytes
>-RESULT Parser: memory-test: Malloc= 529000.0 bytes
>-median= 529000.0 bytes, stdev= 14124.44689 bytes, min= 511000.0 bytes, max= 548000.0 bytes
>+RESULT Parser: memory-test: Time= 1100 ms
>+median= 1101 ms, stdev= 1.32 %, min= 1080 ms, max= 1120 ms
>+RESULT Parser: memory-test: JSHeap= 830000 bytes
>+median= 829000 bytes, stdev= 1.66 %, min= 811000 bytes, max= 848000 bytes
>+RESULT Parser: memory-test: Malloc= 529000 bytes
>+median= 529000 bytes, stdev= 2.67 %, min= 511000 bytes, max= 548000 bytes
> Finished: 0.1 s
> """
>

Actions: View | Formatted Diff | Diff

Attachments on bug 90062: 149726 | 191049