Bug 183156

Few examples of failures while talking to bugzilla: https://webkit-queues.webkit.org/results/6681924 https://webkit-queues.webkit.org/results/6681223 https://webkit-queues.webkit.org/results/6681840

e.g.: 2018-02-25 12:28:08,404 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334583&action=edit 2018-02-25 12:28:08,866 - Fetching: https://bugs.webkit.org/show_bug.cgi?id=183109&ctype=xml&excludefield=attachmentdata 2018-02-25 12:28:09,186 - Running: webkit-patch --status-host=webkit-queues.webkit.org --bot-id=webkit-misc-style apply-attachment --no-update --non-interactive 334583 2018-02-25 12:28:26,507 - Applied patch 2018-02-25 12:28:27,929 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334583&action=edit Traceback (most recent call last): File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/bot/queueengine.py", line 103, in run if not self._delegate.process_work_item(work_item): File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/queues.py", line 452, in process_work_item passed = self.review_patch(patch) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/queues.py", line 476, in review_patch style_check_succeeded = task.run() File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/bot/stylequeuetask.py", line 75, in run self._apply_watch_list() File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/bot/stylequeuetask.py", line 66, in _apply_watch_list "Unabled to apply watchlist") File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/bot/patchanalysistask.py", line 97, in _run_command if not self.validate(): File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/bot/stylequeuetask.py", line 39, in validate self._patch = self._delegate.refetch_patch(self._patch) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/commands/queues.py", line 511, in refetch_patch return self._tool.bugs.fetch_attachment(patch.id()) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/net/bugzilla/bugzilla.py", line 518, in fetch_attachment bug_id = self.bug_id_for_attachment_id(attachment_id) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/net/bugzilla/bugzilla.py", line 508, in bug_id_for_attachment_id page = self.browser.open(attachment_url) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_mechanize.py", line 203, in open return self._mech_open(url, data, timeout=timeout) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_mechanize.py", line 230, in _mech_open response = UserAgentBase.open(self, request, data) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_opener.py", line 193, in open response = urlopen(self, req, data) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 344, in _open '_open', req) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 332, in _call_chain result = func(*args) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 1170, in https_open return self.do_open(conn_factory, req) File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_urllib2_fork.py", line 1118, in do_open raise URLError(err) URLError: <urlopen error [Errno 60] Operation timed out>

Created attachment 334665 [details] Proposed patch

iOS EWS for this patch failed with exactly same network issue which would be fixed by this patch.

Aakash asked me to look this over this afternoon; I took a look at NetworkTransaction's implementation and the use of lambdas here and everything looks like it should work as described by Aakash. Unofficial r=me.

Comment on attachment 334665 [details] Proposed patch Clearing flags on attachment: 334665 Committed r229064: <https://trac.webkit.org/changeset/229064>

All reviewed patches have been landed. Closing bug.

<rdar://problem/37945984>

This seems to help significantly. Noticing the logs, all the patches below would have failed ews with network exceptions, which now worked properly. ews109: 2018-02-28 09:55:14,708 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334748&action=edit 2018-02-28 09:55:22,578 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 10:54:30,971 - Fetching: https://bugs.webkit.org/show_bug.cgi?id=183191&ctype=xml&excludefield=attachmentdata 2018-02-28 10:55:45,975 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 10:55:56,258 - Started processing patch 2018-02-28 11:30:09,177 - Fetching: https://bugs.webkit.org/show_bug.cgi?id=183184&ctype=xml&excludefield=attachmentdata 2018-02-28 11:30:17,102 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 11:30:27,401 - Started processing patch 2018-02-28 11:30:28,085 - Fetching: https://bugs.webkit.org/show_bug.cgi?id=183184&ctype=xml&excludefield=attachmentdata 2018-02-28 11:30:36,017 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 11:30:46,312 - Running: webkit-patch --status-host=webkit-queues.webkit.org --bot-id=ews109 clean --port=ios-device --architecture=arm64 ews108: 2018-02-28 06:33:50,141 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334742&action=edit 2018-02-28 06:35:05,177 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 06:35:15,187 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334742&action=edit 2018-02-28 06:36:30,202 - Received URLError: [Errno 60] Operation timed out. Retrying in 15.0 seconds... 2018-02-28 06:36:45,211 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334742&action=edit 2018-02-28 08:56:08,825 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334369&action=edit 2018-02-28 08:57:23,847 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 08:57:33,857 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334369&action=edit 2018-02-28 11:50:43,646 - Fetching: https://bugs.webkit.org/show_bug.cgi?id=183218&ctype=xml&excludefield=attachmentdata 2018-02-28 11:50:51,515 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 11:51:01,759 - Running: webkit-patch --status-host=webkit-queues.webkit.org --bot-id=ews108 clean --port=ios-device --architecture=arm64 2018-02-28 12:59:05,387 - Applied patch 2018-02-28 12:59:05,788 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334759&action=edit 2018-02-28 12:59:13,791 - Received URLError: [Errno 60] Operation timed out. Retrying in 10 seconds... 2018-02-28 12:59:23,799 - Fetching: https://bugs.webkit.org/attachment.cgi?id=334759&action=edit

Note that this added retry for some of the (frequent) network transactions with Bugzilla. We should add retry for ALL the network transaction with Bugzilla. Tracked in https://bugs.webkit.org/show_bug.cgi?id=183222