Attachment 170694 Details for Bug 100030 – Patch

[patch] Patch

bug-100030-20121025203000.patch (text/plain), 8.81 KB, created by Mike West on 2012-10-25 11:31:27 PDT

(hide)

Description:

Filename:

MIME Type:

Creator: Mike West

Created: 2012-10-25 11:31:27 PDT

Size: 8.81 KB

patch

obsolete

>Subversion Revision: 132362
>diff --git a/Tools/ChangeLog b/Tools/ChangeLog
>index f2c031ee3ea83e2d934607a9edf4aed067e06b57..b5704b6f0f387076c1cde9a1cfe1b62e423bbfd6 100644
>--- a/Tools/ChangeLog
>+++ b/Tools/ChangeLog
>@@ -1,3 +1,38 @@
>+2012-10-25  Mike West  <mkwst@chromium.org>
>+
>+        run-perf-tests should have a --repeat option
>+        https://bugs.webkit.org/show_bug.cgi?id=100030
>+
>+        Reviewed by Dirk Pranke.
>+
>+        This patch adds two options to 'run-perf-tests': --repeat and
>+        --runtitle. The former sets the number of times the specified set of
>+        tests should be run, and the latter provides a mechanism for
>+        overriding the default column title for a run. I'm including them in
>+        the same patch in order to have the nice cooperative effect of
>+        iterating the column title for each run. That is, if a run title of
>+        "Clean Build" is provided, and it repeats 5 times, the titles of the
>+        five result columns will be "Clean Build #1", "Clean Build #2", and so
>+        on.
>+
>+        * Scripts/webkitpy/performance_tests/perftestsrunner.py:
>+        (PerfTestsRunner.__init__):
>+            Adds '_totalRuns' and '_currentRun' instance variables.
>+        (PerfTestsRunner._parse_args):
>+            Adds '--repeat' and '--runtitle' options to the runner.
>+        (PerfTestsRunner.run):
>+            Loop over '_run_tests_set' the number of times specified by '--repeat'.
>+        (PerfTestsRunner._generate_results):
>+        (PerfTestsRunner._upload_and_show_results):
>+            These two methods split '_generate_and_show_results'
>+            into its component tasks, allowing us to only upload the JSON file
>+            and show the results page if we've finished the final run.
>+        (PerfTestsRunner._generate_results_dict):
>+            Add 'webkit-runtitle' to the JSON result set for use in the template.
>+        (PerfTestsRunner._run_tests_set):
>+            If we're repeating the tests, note the current run in the log
>+            message: "Running [TEST NAME] (1 of 100) [Run 1 of 5]".
>+
> 2012-10-24  Jocelyn Turcotte  <jocelyn.turcotte@digia.com>
> 
>         [Qt] Move the QQuickWebViewExperimental pointer to QQuickWebViewPrivate
>diff --git a/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py b/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py
>index 42e0d96e11a65205d18c0698c28ce0f8c63c1f2c..6acef6dcde8b1170af2630e893c158a1e93e8177 100755
>--- a/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py
>+++ b/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py
>@@ -68,6 +68,8 @@ class PerfTestsRunner(object):
>         self._base_path = self._port.perf_tests_dir()
>         self._results = {}
>         self._timestamp = time.time()
>+        self._totalRuns = self._options.repeat
>+        self._currentRun = 0
> 
>     @staticmethod
>     def _parse_args(args=None):
>@@ -117,6 +119,10 @@ class PerfTestsRunner(object):
>                 help="Run replay tests."),
>             optparse.make_option("--force", dest="skipped", action="store_true", default=False,
>                 help="Run all tests, including the ones in the Skipped list."),
>+            optparse.make_option("--runtitle",
>+                help="Specify a name for the run. If provided, this title will replace the WebKit revision in the result page's column header."),
>+            optparse.make_option("--repeat", default=1, type="int",
>+                help="Specify how many times the set of tests should be run. Defaults to 1."),
>             ]
>         return optparse.OptionParser(option_list=(perf_option_list)).parse_args(args)
> 
>@@ -163,9 +169,15 @@ class PerfTestsRunner(object):
>             if not test.prepare(self._options.time_out_ms):
>                 return self.EXIT_CODE_BAD_PREPARATION
> 
>-        unexpected = self._run_tests_set(sorted(list(tests), key=lambda test: test.test_name()), self._port)
>-        if self._options.generate_results:
>-            exit_code = self._generate_and_show_results()
>+        while (self._currentRun < self._totalRuns):
>+            self._currentRun += 1
>+            unexpected = self._run_tests_set(sorted(list(tests), key=lambda test: test.test_name()), self._port)
>+            if self._options.generate_results:
>+                exit_code = self._generate_results()
>+                if exit_code:
>+                    return exit_code
>+        if self._currentRun == self._totalRuns:
>+            exit_code = self._upload_and_show_results()
>             if exit_code:
>                 return exit_code
> 
>@@ -177,7 +189,7 @@ class PerfTestsRunner(object):
>             return output_json_path
>         return self._host.filesystem.join(self._port.perf_results_directory(), self._DEFAULT_JSON_FILENAME)
> 
>-    def _generate_and_show_results(self):
>+    def _generate_results(self):
>         options = self._options
>         output_json_path = self._output_json_path()
>         output = self._generate_results_dict(self._timestamp, options.description, options.platform, options.builder_name, options.build_number)
>@@ -194,11 +206,12 @@ class PerfTestsRunner(object):
>         results_page_path = self._host.filesystem.splitext(output_json_path)[0] + '.html'
>         self._generate_output_files(output_json_path, results_page_path, output)
> 
>-        if options.test_results_server:
>-            if not self._upload_json(options.test_results_server, output_json_path):
>+    def _upload_and_show_results(self):
>+        if self._options.test_results_server:
>+            if not self._upload_json(options.test_results_server, self._output_json_path()):
>                 return self.EXIT_CODE_FAILED_UPLOADING
>-
>-        if options.show_results:
>+        if self._options.show_results:
>+            results_page_path = self._host.filesystem.splitext(self._output_json_path())[0] + '.html'
>             self._port.show_results_html_file(results_page_path)
> 
>     def _generate_results_dict(self, timestamp, description, platform, builder_name, build_number):
>@@ -208,6 +221,10 @@ class PerfTestsRunner(object):
>         for (name, path) in self._port.repository_paths():
>             scm = SCMDetector(self._host.filesystem, self._host.executive).detect_scm_system(path) or self._host.scm()
>             contents[name + '-revision'] = scm.svn_revision(path)
>+            if (self._options.runtitle):
>+                contents[name + '-runtitle'] = self._options.runtitle
>+                if (self._totalRuns > 1):
>+                    contents[name + '-runtitle'] += " #%d" % self._currentRun
> 
>         # FIXME: Add --branch or auto-detect the branch we're in
>         for key, value in {'timestamp': int(timestamp), 'branch': self._default_branch, 'platform': platform,
>@@ -297,8 +314,8 @@ class PerfTestsRunner(object):
>                 if not self._host.user.confirm("Ready to run test?"):
>                     driver.stop()
>                     return unexpected
>-
>-            _log.info('Running %s (%d of %d)' % (test.test_name(), expected + unexpected + 1, len(tests)))
>+            runs = " [Run %d of %d]" % (self._currentRun, self._totalRuns) if self._totalRuns > 1 else ""
>+            _log.info('Running %s (%d of %d)%s' % (test.test_name(), expected + unexpected + 1, len(tests), runs))
>             if self._run_single_test(test, driver):
>                 expected = expected + 1
>             else:
>diff --git a/PerformanceTests/ChangeLog b/PerformanceTests/ChangeLog
>index 360680399f04e51d3dbd036bcc3e3cf4d701adab..bfc3953dfb56ffeeebebe2930f28868d57a5acd7 100644
>--- a/PerformanceTests/ChangeLog
>+++ b/PerformanceTests/ChangeLog
>@@ -1,3 +1,15 @@
>+2012-10-25  Mike West  <mkwst@chromium.org>
>+
>+        run-perf-tests should have a --repeat option
>+        https://bugs.webkit.org/show_bug.cgi?id=100030
>+
>+        Reviewed by Dirk Pranke.
>+
>+        If the test result has a "run title", use it instead of the WebKit
>+        revision in the result's column header.
>+
>+        * resources/results-template.html:
>+
> 2012-10-24  Zoltan Horvath  <zoltan@webkit.org>
> 
>         REGRESSION(r131982): Skip SVG/SvgNestedUse.html performancetest since it is crashing
>diff --git a/PerformanceTests/resources/results-template.html b/PerformanceTests/resources/results-template.html
>index 9fdb7b9f6b17d7daa2a82527cbcf294a1b484022..73f4c873b96e9dcb346305f8eb2aaeebf02b5bf5 100644
>--- a/PerformanceTests/resources/results-template.html
>+++ b/PerformanceTests/resources/results-template.html
>@@ -204,8 +204,11 @@ function TestResult(associatedTest, result, associatedRun) {
> function TestRun(entry) {
>     this.description = function () { return entry['description']; }
>     this.webkitRevision = function () { return entry['webkit-revision']; }
>+    this.runTitle = function () { return entry['webkit-runtitle']; }
>     this.label = function () {
>         var label = 'r' + this.webkitRevision();
>+        if (this.runTitle())
>+            label = this.runTitle();
>         if (this.description())
>             label += ' &dash; ' + this.description();
>         return label;

Actions: View | Formatted Diff | Diff

Attachments on bug 100030: 170614 | 170691 | 170694 | 191921 | 192059