<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugs.webkit.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.4.1"
          urlbase="https://bugs.webkit.org/"
          
          maintainer="admin@webkit.org"
>

    <bug>
          <bug_id>107451</bug_id>
          
          <creation_ts>2013-01-21 08:00:36 -0800</creation_ts>
          <short_desc>[SOUP] Use SoupSession instead of SoupSessionAsync</short_desc>
          <delta_ts>2018-12-03 19:57:00 -0800</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>1</classification_id>
          <classification>Unclassified</classification>
          <product>WebKit</product>
          <component>Platform</component>
          <version>528+ (Nightly build)</version>
          <rep_platform>Unspecified</rep_platform>
          <op_sys>Unspecified</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords>InRadar</keywords>
          <priority>P2</priority>
          <bug_severity>Normal</bug_severity>
          <target_milestone>---</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Dominik Röttsches (drott)">d-r</reporter>
          <assigned_to name="Michael Catanzaro">mcatanzaro</assigned_to>
          <cc>berto</cc>
    
    <cc>bugs-noreply</cc>
    
    <cc>cgarcia</cc>
    
    <cc>commit-queue</cc>
    
    <cc>danw</cc>
    
    <cc>ews-watchlist</cc>
    
    <cc>gustavo</cc>
    
    <cc>mcatanzaro</cc>
    
    <cc>mrobinson</cc>
    
    <cc>webkit-bug-importer</cc>
    
    <cc>youennf</cc>
    
    <cc>zan</cc>
          

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>812154</commentid>
    <comment_count>0</comment_count>
    <who name="Dominik Röttsches (drott)">d-r</who>
    <bug_when>2013-01-21 08:00:36 -0800</bug_when>
    <thetext>As the subject says, Dan suggested in
https://bugs.webkit.org/show_bug.cgi?id=104684#c3
to use the unified SoupSession.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>985148</commentid>
    <comment_count>1</comment_count>
    <who name="youenn fablet">youennf</who>
    <bug_when>2014-02-27 07:14:12 -0800</bug_when>
    <thetext>Minimum required version of libsoup should be 2.44.1 due to libsoup bug #707711</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>985176</commentid>
    <comment_count>2</comment_count>
    <who name="Martin Robinson">mrobinson</who>
    <bug_when>2014-02-27 08:23:56 -0800</bug_when>
    <thetext>I think we decided that it was simpler to just the asynchronous SoupSession for synchronous requests. We have been working out all the bugs and the code is completely shared between the different paths, something that might be harder with another type of SoupSession.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1483731</commentid>
    <comment_count>3</comment_count>
    <who name="Michael Catanzaro">mcatanzaro</who>
    <bug_when>2018-11-30 14:18:18 -0800</bug_when>
    <thetext>With glib-networking 2.57.1, WebKit is no longer able to load TLS error pages. The problem is a network process deadlock caused by a change in how glib-networking performs certificate verification. Previously it verified certificates *after* the TLS handshake had completed, which was weirdly late, but previously not problematic. But now that TLS 1.3 exists, application data can be sent before certificate verification occurs, which is no good. So I moved verification to occur during the handshake. I needed to do this regardless because I need to add a new callback in GnuTLS for another feature. This introduced a deadlock in WebKit:

 * glib-networking detects an unacceptable certificate, emits accept-certificate signal
 * NetworkDataTaskSoup::tlsConnectionAcceptCertificate calls NetworkDataTaskSoup::invalidateAndCancel calls NetworkDataTaskSoup::clearRequest
 * NetworkDataTaskSoup::clearRequest calls soup_session_cancel_message

The problem is that, in the deprecated SoupSessionAsync used by WebKit, cancellation is always *synchronous* despite the name of the class. So soup_session_cancel_message winds up doing its thing to close everything out, and that eventually ends up in a synchronous call to g_tls_connection_gnutls_close. The close operation can&apos;t proceed until the TLS handshake is finished, and the handshake is blocked waiting for WebKit to return from its accept-certificate handler. So the close operation winds up polling forever waiting for the handshake to finish (the handshake normally runs on a secondary thread, just except for accept-certificate). Deadlock. I&apos;ve attached a backtrace showing the callstack at the time of deadlock.

We can try to fix this in glib-networking or in WebKit. I&apos;ve actually stalled on this for several weeks just because I&apos;m not sure which approach is best. WebKit&apos;s use of soup_session_cancel_message seems *really* innocuous and even for someone familiar with the libsoup API, it&apos;s really unexpected that using it in the accept-certificate handler could cause a deadlock, so that favored fixing it in glib-networking. That could be done in several ways, e.g. queuing the close operation to occur at a later time and returning a fake success exit status, or returning an error code instead of polling until the handshake completes.

Ultimately, I decided glib-networking is already much too complicated, and solving this in WebKit is so much easier. All we need to do is switch from SoupSessionAsync to the modern SoupSession API. Ironically, with modern SoupSession, cancellation is asynchronous, unlike SoupSessionAsync, so the problem goes away. Solving this in WebKit has the major disadvantage that other applications could be broken by the behavior change, but I&apos;ve used Debian codesearch to verify that WebKit is the only software likely to be affected.

The only changes required in WebKit are adjustments for the new default property values. Most of the properties we used to set explicitly are now defaults and should be removed. Additionally, SoupSession has default timeouts, which we want to override to allow NetworkDataTaskSoup to implement its own timeouts.

Some final notes:

 * We already require libsoup 2.42, the first version to introduce the modern SoupSession, so there is no dependency bump required. Youenn noted that this version is kinda broken, but it will build.
 * This deadlock bug is super annoying, so if we wind up accepting this solution, it would be nice to do a 2.23.2 release.
 * There is a moderately-high risk of unexpected regression. Do not backport!</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1483732</commentid>
    <comment_count>4</comment_count>
      <attachid>356235</attachid>
    <who name="Michael Catanzaro">mcatanzaro</who>
    <bug_when>2018-11-30 14:18:58 -0800</bug_when>
    <thetext>Created attachment 356235
Backtrace showing the deadlock</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1483744</commentid>
    <comment_count>5</comment_count>
      <attachid>356239</attachid>
    <who name="Michael Catanzaro">mcatanzaro</who>
    <bug_when>2018-11-30 14:27:31 -0800</bug_when>
    <thetext>Created attachment 356239
Patch</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1483750</commentid>
    <comment_count>6</comment_count>
    <who name="Michael Catanzaro">mcatanzaro</who>
    <bug_when>2018-11-30 14:37:14 -0800</bug_when>
    <thetext>BTW Dan had favored hiding this change in glib-networking, which we can still try to do. This WebKit change is far simpler, though.

There is another glib-networking behavior change that cannot be hidden: certificate verification will now stall forever if the thread that started the handshake is not iterating the main context that was the thread-default main context at the time of the operation that triggered the handshake. This caused a couple glib-networking tests to break. That is much less worrisome, though, because unlike the soup_session_cancel_message deadlock, it&apos;s quite unlikely that any real application would be affected by that change.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1484273</commentid>
    <comment_count>7</comment_count>
      <attachid>356239</attachid>
    <who name="Carlos Garcia Campos">cgarcia</who>
    <bug_when>2018-12-03 00:00:08 -0800</bug_when>
    <thetext>Comment on attachment 356239
Patch

I think this is good to stop using deprecated API, but the actual bug should be fixed in glib-networking too IMO.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1484402</commentid>
    <comment_count>8</comment_count>
    <who name="Michael Catanzaro">mcatanzaro</who>
    <bug_when>2018-12-03 10:43:36 -0800</bug_when>
    <thetext>Problem is glib-networking is actually functioning as designed. It&apos;s only a bug once you combine three different libraries with different expectations. That&apos;s why I hesitated so long to decide what to do. You could say that a design change is needed in glib-networking to change this there: either to make the close call return a fake success code and queue the real close to occur later, or else to fail the close call with an error.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1484404</commentid>
    <comment_count>9</comment_count>
    <who name="Michael Catanzaro">mcatanzaro</who>
    <bug_when>2018-12-03 10:45:16 -0800</bug_when>
    <thetext>(In reply to Michael Catanzaro from comment #8)
&gt; You could say that a design change is needed in glib-networking to change this there: either to
&gt; make the close call return a fake success code and queue the real close to
&gt; occur later, or else to fail the close call with an error.

And neither of those seem like great ideas to me, though either one would be OK and doable if needed.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1484412</commentid>
    <comment_count>10</comment_count>
      <attachid>356239</attachid>
    <who name="WebKit Commit Bot">commit-queue</who>
    <bug_when>2018-12-03 11:11:37 -0800</bug_when>
    <thetext>Comment on attachment 356239
Patch

Clearing flags on attachment: 356239

Committed r238805: &lt;https://trac.webkit.org/changeset/238805&gt;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1484413</commentid>
    <comment_count>11</comment_count>
    <who name="WebKit Commit Bot">commit-queue</who>
    <bug_when>2018-12-03 11:11:39 -0800</bug_when>
    <thetext>All reviewed patches have been landed.  Closing bug.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1484415</commentid>
    <comment_count>12</comment_count>
    <who name="Radar WebKit Bug Importer">webkit-bug-importer</who>
    <bug_when>2018-12-03 11:12:31 -0800</bug_when>
    <thetext>&lt;rdar://problem/46424873&gt;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>1484681</commentid>
    <comment_count>13</comment_count>
    <who name="Michael Catanzaro">mcatanzaro</who>
    <bug_when>2018-12-03 19:57:00 -0800</bug_when>
    <thetext>(In reply to Michael Catanzaro from comment #8)
&gt; or else to fail the close call with an error.

This actually works well, fixing the hang without requiring any changes in WebKit. The test I wrote for this isn&apos;t working quite right though. If I can fix my test, I&apos;ll land that change.

This SoupSessionAsync -&gt; SoupSession change is desirable regardless, of course.</thetext>
  </long_desc>
      
          <attachment
              isobsolete="0"
              ispatch="0"
              isprivate="0"
          >
            <attachid>356235</attachid>
            <date>2018-11-30 14:18:58 -0800</date>
            <delta_ts>2018-11-30 14:18:58 -0800</delta_ts>
            <desc>Backtrace showing the deadlock</desc>
            <filename>backtrace</filename>
            <type>text/plain</type>
            <size>7240</size>
            <attacher name="Michael Catanzaro">mcatanzaro</attacher>
            
              <data encoding="base64">IzAgIDB4MDAwMDdmZTFhMmY3NDNkMSBpbiBwb2xsICgpIGZyb20gL2xpYjY0L2xpYmMuc28uNgoj
MSAgMHgwMDAwN2ZlMWEzNzM0NDgwIGluIGdfcG9sbCAoZmRzPWZkc0BlbnRyeT0weDdmZmRhODFj
MmUxMCwgbmZkcz1uZmRzQGVudHJ5PTEsIAogICAgdGltZW91dD10aW1lb3V0QGVudHJ5PS0xKSBh
dCAvaG9tZS9tY2F0YW56YXJvL1Byb2plY3RzL2dsaWIvZ2xpYi9ncG9sbC5jOjEyNAojMiAgMHgw
MDAwN2ZlMTVlYzhhZTZlIGluIGNsYWltX29wIChnbnV0bHM9Z251dGxzQGVudHJ5PTB4MThlMzUz
MCwgCiAgICBvcD1vcEBlbnRyeT1HX1RMU19DT05ORUNUSU9OX0dOVVRMU19PUF9DTE9TRV9CT1RI
LCB0aW1lb3V0PXRpbWVvdXRAZW50cnk9LTEsIAogICAgY2FuY2VsbGFibGU9Y2FuY2VsbGFibGVA
ZW50cnk9MHgwLCBlcnJvcj1lcnJvckBlbnRyeT0weDApCiAgICBhdCAuLi8uLi8uLi8uLi9Qcm9q
ZWN0cy9nbGliLW5ldHdvcmtpbmcvdGxzL2dudXRscy9ndGxzY29ubmVjdGlvbi1nbnV0bHMuYzo4
MDYKIzMgIDB4MDAwMDdmZTE1ZWM4YmQxNyBpbiBnX3Rsc19jb25uZWN0aW9uX2dudXRsc19jbG9z
ZV9pbnRlcm5hbCAoc3RyZWFtPTxvcHRpbWl6ZWQgb3V0PiwgCiAgICBkaXJlY3Rpb249ZGlyZWN0
aW9uQGVudHJ5PShHX1RMU19ESVJFQ1RJT05fUkVBRCB8IEdfVExTX0RJUkVDVElPTl9XUklURSks
IHRpbWVvdXQ9dGltZW91dEBlbnRyeT0tMSwgCiAgICBjYW5jZWxsYWJsZT0weDAsIGVycm9yPTB4
MCkKICAgIGF0IC4uLy4uLy4uLy4uL1Byb2plY3RzL2dsaWItbmV0d29ya2luZy90bHMvZ251dGxz
L2d0bHNjb25uZWN0aW9uLWdudXRscy5jOjI4MjkKIzQgIDB4MDAwMDdmZTE1ZWM4YmZmMCBpbiBn
X3Rsc19jb25uZWN0aW9uX2dudXRsc19jbG9zZSAoc3RyZWFtPTxvcHRpbWl6ZWQgb3V0PiwgCiAg
ICBjYW5jZWxsYWJsZT08b3B0aW1pemVkIG91dD4sIGVycm9yPTxvcHRpbWl6ZWQgb3V0PikKICAg
IGF0IC4uLy4uLy4uLy4uL1Byb2plY3RzL2dsaWItbmV0d29ya2luZy90bHMvZ251dGxzL2d0bHNj
b25uZWN0aW9uLWdudXRscy5jOjI4OTUKIzUgIDB4MDAwMDdmZTFhMzhjMDExNCBpbiBnX2lvX3N0
cmVhbV9jbG9zZSAoc3RyZWFtPTB4MThlMzUzMCwgY2FuY2VsbGFibGU9Y2FuY2VsbGFibGVAZW50
cnk9MHgwLCAKICAgIGVycm9yPWVycm9yQGVudHJ5PTB4MCkgYXQgL2hvbWUvbWNhdGFuemFyby9Q
cm9qZWN0cy9nbGliL2dpby9naW9zdHJlYW0uYzo0MjIKIzYgIDB4MDAwMDdmZTFhMzYwMzUxZiBp
biBkaXNjb25uZWN0X2ludGVybmFsIChzb2NrPXNvY2tAZW50cnk9MHgxNGYzMjgwLCBjbG9zZT1j
bG9zZUBlbnRyeT0xKQogICAgYXQgLi4vLi4vLi4vLi4vUHJvamVjdHMvbGlic291cC9saWJzb3Vw
L3NvdXAtc29ja2V0LmM6MTkwCiM3ICAweDAwMDA3ZmUxYTM2MDViZjQgaW4gc291cF9zb2NrZXRf
ZGlzY29ubmVjdCAoc29jaz1zb2NrQGVudHJ5PTB4MTRmMzI4MCkKICAgIGF0IC4uLy4uLy4uLy4u
L1Byb2plY3RzL2xpYnNvdXAvbGlic291cC9zb3VwLXNvY2tldC5jOjE1OTMKIzggIDB4MDAwMDdm
ZTFhMzVkZjAwYSBpbiBzb3VwX2Nvbm5lY3Rpb25fZGlzY29ubmVjdCAoY29ubj1jb25uQGVudHJ5
PTB4MTZjMjU1MCkKICAgIGF0IC4uLy4uLy4uLy4uL1Byb2plY3RzL2xpYnNvdXAvbGlic291cC9z
b3VwLWNvbm5lY3Rpb24uYzo1ODYKIzkgIDB4MDAwMDdmZTFhMzVkZThiYyBpbiBzb3VwX2Nvbm5l
Y3Rpb25fc2V0X3N0YXRlIChjb25uPTB4MTZjMjU1MCwgCiAgICBzdGF0ZT1zdGF0ZUBlbnRyeT1T
T1VQX0NPTk5FQ1RJT05fSURMRSkKICAgIGF0IC4uLy4uLy4uLy4uL1Byb2plY3RzL2xpYnNvdXAv
bGlic291cC9zb3VwLWNvbm5lY3Rpb24uYzo2NzYKIzEwIDB4MDAwMDdmZTFhMzVmY2EyMyBpbiBz
b3VwX3Nlc3Npb25fdW5xdWV1ZV9pdGVtIChzZXNzaW9uPXNlc3Npb25AZW50cnk9MHgxMzdkMTAw
LCAKICAgIGl0ZW09aXRlbUBlbnRyeT0weDE3MTNmMjApIGF0IC4uLy4uLy4uLy4uL1Byb2plY3Rz
L2xpYnNvdXAvbGlic291cC9zb3VwLXNlc3Npb24uYzoxNDg5CiMxMSAweDAwMDA3ZmUxYTM2MDBm
NmYgaW4gc291cF9zZXNzaW9uX3Byb2Nlc3NfcXVldWVfaXRlbSAoc2Vzc2lvbj1zZXNzaW9uQGVu
dHJ5PTB4MTM3ZDEwMCwgCiAgICBpdGVtPWl0ZW1AZW50cnk9MHgxNzEzZjIwLCBzaG91bGRfY2xl
YW51cD1zaG91bGRfY2xlYW51cEBlbnRyeT0weDAsIGxvb3A9bG9vcEBlbnRyeT0wKQogICAgYXQg
Li4vLi4vLi4vLi4vUHJvamVjdHMvbGlic291cC9saWJzb3VwL3NvdXAtc2Vzc2lvbi5jOjIwNDAK
IzEyIDB4MDAwMDdmZTFhMzYwMjg1OSBpbiBzb3VwX3Nlc3Npb25fYXN5bmNfY2FuY2VsX21lc3Nh
Z2UgKHNlc3Npb249MHgxMzdkMTAwLCBtc2c9MHgxOGYyY2EwLCAKICAgIHN0YXR1c19jb2RlPTxv
cHRpbWl6ZWQgb3V0PikgYXQgLi4vLi4vLi4vLi4vUHJvamVjdHMvbGlic291cC9saWJzb3VwL3Nv
dXAtc2Vzc2lvbi1hc3luYy5jOjEzMQojMTMgMHgwMDAwN2ZlMWEzNWZlZmNlIGluIHNvdXBfc2Vz
c2lvbl9jYW5jZWxfbWVzc2FnZSAoc2Vzc2lvbj0weDEzN2QxMDAsIG1zZz1tc2dAZW50cnk9MHgx
OGYyY2EwLCAKICAgIHN0YXR1c19jb2RlPXN0YXR1c19jb2RlQGVudHJ5PTEpIGF0IC4uLy4uLy4u
Ly4uL1Byb2plY3RzL2xpYnNvdXAvbGlic291cC9zb3VwLXNlc3Npb24uYzoyNDY4CiMxNCAweDAw
MDA3ZmUxYTdlMWFjZTYgaW4gV2ViS2l0OjpOZXR3b3JrRGF0YVRhc2tTb3VwOjpjYW5jZWwgKHRo
aXM9dGhpc0BlbnRyeT0weDdmZTE1ZWJmY2MwMCkKICAgIGF0IERlcml2ZWRTb3VyY2VzL0Zvcndh
cmRpbmdIZWFkZXJzL3d0Zi9EdW1iUHRyVHJhaXRzLmg6NDEKIzE1IDB4MDAwMDdmZTFhN2UxYWQy
OSBpbiBXZWJLaXQ6Ok5ldHdvcmtEYXRhVGFza1NvdXA6OmludmFsaWRhdGVBbmRDYW5jZWwgKHRo
aXM9MHg3ZmUxNWViZmNjMDApCiAgICBhdCAvaG9tZS9tY2F0YW56YXJvL1Byb2plY3RzL1dlYktp
dC9Tb3VyY2UvV2ViS2l0L05ldHdvcmtQcm9jZXNzL3NvdXAvTmV0d29ya0RhdGFUYXNrU291cC5j
cHA6MjY2CiMxNiAweDAwMDA3ZmUxYTdlMWFlZmMgaW4gV2ViS2l0OjpOZXR3b3JrRGF0YVRhc2tT
b3VwOjp0bHNDb25uZWN0aW9uQWNjZXB0Q2VydGlmaWNhdGUgKAogICAgdGhpcz0weDdmZTE1ZWJm
Y2MwMCwgY2VydGlmaWNhdGU9MHgxNWJmNTcwLCB0bHNFcnJvcnM9R19UTFNfQ0VSVElGSUNBVEVf
VU5LTk9XTl9DQSkKICAgIGF0IC9ob21lL21jYXRhbnphcm8vUHJvamVjdHMvV2ViS2l0L1NvdXJj
ZS9XZWJLaXQvTmV0d29ya1Byb2Nlc3Mvc291cC9OZXR3b3JrRGF0YVRhc2tTb3VwLmNwcDo0MzIK
IzE3IDB4MDAwMDdmZTFhN2UxYjA2ZiBpbiBXZWJLaXQ6Ok5ldHdvcmtEYXRhVGFza1NvdXA6OnRs
c0Nvbm5lY3Rpb25BY2NlcHRDZXJ0aWZpY2F0ZUNhbGxiYWNrICgKICAgIHRhc2s9MHg3ZmUxNWVi
ZmNjMDAsIGVycm9ycz1HX1RMU19DRVJUSUZJQ0FURV9VTktOT1dOX0NBLCBjZXJ0aWZpY2F0ZT0w
eDE1YmY1NzAsIAogICAgY29ubmVjdGlvbj0weDE4ZTM1MzApCiAgICBhdCAvaG9tZS9tY2F0YW56
YXJvL1Byb2plY3RzL1dlYktpdC9Tb3VyY2UvV2ViS2l0L05ldHdvcmtQcm9jZXNzL3NvdXAvTmV0
d29ya0RhdGFUYXNrU291cC5jcHA6NDIwCiMxOCBXZWJLaXQ6Ok5ldHdvcmtEYXRhVGFza1NvdXA6
OnRsc0Nvbm5lY3Rpb25BY2NlcHRDZXJ0aWZpY2F0ZUNhbGxiYWNrIChjb25uZWN0aW9uPTB4MThl
MzUzMCwgCiAgICBjZXJ0aWZpY2F0ZT0weDE1YmY1NzAsIGVycm9ycz1HX1RMU19DRVJUSUZJQ0FU
RV9VTktOT1dOX0NBLCB0YXNrPTB4N2ZlMTVlYmZjYzAwKQogICAgYXQgL2hvbWUvbWNhdGFuemFy
by9Qcm9qZWN0cy9XZWJLaXQvU291cmNlL1dlYktpdC9OZXR3b3JrUHJvY2Vzcy9zb3VwL05ldHdv
cmtEYXRhVGFza1NvdXAuY3BwOjQwOQojMTkgMHgwMDAwN2ZlMWEyMWNkYWNlIGluIGZmaV9jYWxs
X3VuaXg2NCAoKSBhdCAuLi9zcmMveDg2L3VuaXg2NC5TOjc2CiMyMCAweDAwMDA3ZmUxYTIxY2Q0
OGYgaW4gZmZpX2NhbGwgKGNpZj1jaWZAZW50cnk9MHg3ZmZkYTgxYzMzZDAsIAogICAgZm49Zm5A
ZW50cnk9MHg3ZmUxYTdlMWIwMDAgPFdlYktpdDo6TmV0d29ya0RhdGFUYXNrU291cDo6dGxzQ29u
bmVjdGlvbkFjY2VwdENlcnRpZmljYXRlQ2FsbGJhY2soX0dUbHNDb25uZWN0aW9uKiwgX0dUbHND
ZXJ0aWZpY2F0ZSosIEdUbHNDZXJ0aWZpY2F0ZUZsYWdzLCBXZWJLaXQ6Ok5ldHdvcmtEYXRhVGFz
a1NvdXAqKT4sIAogICAgcnZhbHVlPTxvcHRpbWl6ZWQgb3V0PiwgYXZhbHVlPWF2YWx1ZUBlbnRy
eT0weDdmZmRhODFjMzJjMCkgYXQgLi4vc3JjL3g4Ni9mZmk2NC5jOjUyNQojMjEgMHgwMDAwN2Zl
MWEzODA2OGIxIGluIGdfY2Nsb3N1cmVfbWFyc2hhbF9nZW5lcmljX3ZhIChjbG9zdXJlPTB4MTRl
Yjc5MCwgCiAgICByZXR1cm5fdmFsdWU9MHg3ZmZkYTgxYzM1NDAsIGluc3RhbmNlPTxvcHRpbWl6
ZWQgb3V0PiwgYXJnc19saXN0PTxvcHRpbWl6ZWQgb3V0PiwgCiAgICBtYXJzaGFsX2RhdGE9PG9w
dGltaXplZCBvdXQ+LCBuX3BhcmFtcz0yLCBwYXJhbV90eXBlcz0weDEzMmIxOTApCiAgICBhdCAv
aG9tZS9tY2F0YW56YXJvL1Byb2plY3RzL2dsaWIvZ29iamVjdC9nY2xvc3VyZS5jOjE2MTQKIzIy
IDB4MDAwMDdmZTFhMzgwNWQ3MiBpbiBfZ19jbG9zdXJlX2ludm9rZV92YSAoY2xvc3VyZT1jbG9z
dXJlQGVudHJ5PTB4MTRlYjc5MCwgCiAgICByZXR1cm5fdmFsdWU9cmV0dXJuX3ZhbHVlQGVudHJ5
PTB4N2ZmZGE4MWMzNTQwLCBpbnN0YW5jZT1pbnN0YW5jZUBlbnRyeT0weDE4ZTM1MzAsIAogICAg
YXJncz1hcmdzQGVudHJ5PTB4N2ZmZGE4MWMzNjM4LCBuX3BhcmFtcz0yLCBwYXJhbV90eXBlcz0w
eDEzMmIxOTApCiAgICBhdCAvaG9tZS9tY2F0YW56YXJvL1Byb2plY3RzL2dsaWIvZ29iamVjdC9n
Y2xvc3VyZS5jOjg3MwojMjMgMHgwMDAwN2ZlMWEzODIxYTJmIGluIGdfc2lnbmFsX2VtaXRfdmFs
aXN0IChpbnN0YW5jZT0weDE4ZTM1MzAsIHNpZ25hbF9pZD08b3B0aW1pemVkIG91dD4sIAogICAg
ZGV0YWlsPTAsIHZhcl9hcmdzPXZhcl9hcmdzQGVudHJ5PTB4N2ZmZGE4MWMzNjM4KQogICAgYXQg
L2hvbWUvbWNhdGFuemFyby9Qcm9qZWN0cy9nbGliL2dvYmplY3QvZ3NpZ25hbC5jOjMzMDAKIzI0
IDB4MDAwMDdmZTFhMzgyMjlhZSBpbiBnX3NpZ25hbF9lbWl0IChpbnN0YW5jZT08b3B0aW1pemVk
IG91dD4sIHNpZ25hbF9pZD08b3B0aW1pemVkIG91dD4sIAogICAgZGV0YWlsPWRldGFpbEBlbnRy
eT0wKSBhdCAvaG9tZS9tY2F0YW56YXJvL1Byb2plY3RzL2dsaWIvZ29iamVjdC9nc2lnbmFsLmM6
MzQ0NwojMjUgMHgwMDAwN2ZlMWEzOGVkYTgxIGluIGdfdGxzX2Nvbm5lY3Rpb25fZW1pdF9hY2Nl
cHRfY2VydGlmaWNhdGUgKGNvbm49PG9wdGltaXplZCBvdXQ+LCAKICAgIHBlZXJfY2VydD1wZWVy
X2NlcnRAZW50cnk9MHgxNWJmNTcwLCBlcnJvcnM9ZXJyb3JzQGVudHJ5PUdfVExTX0NFUlRJRklD
QVRFX1VOS05PV05fQ0EpCiAgICBhdCAvaG9tZS9tY2F0YW56YXJvL1Byb2plY3RzL2dsaWIvZ2lv
L2d0bHNjb25uZWN0aW9uLmM6ODY1CiMyNiAweDAwMDA3ZmUxNWVjOGMyMWIgaW4gYWNjZXB0X3Bl
ZXJfY2VydGlmaWNhdGUgKGdudXRscz1nbnV0bHNAZW50cnk9MHgxOGUzNTMwLCAKICAgIHBlZXJf
Y2VydGlmaWNhdGU9MHgxNWJmNTcwLCBwZWVyX2NlcnRpZmljYXRlX2Vycm9ycz1HX1RMU19DRVJU
SUZJQ0FURV9VTktOT1dOX0NBKQogICAgYXQgLi4vLi4vLi4vLi4vUHJvamVjdHMvZ2xpYi1uZXR3
b3JraW5nL3Rscy9nbnV0bHMvZ3Rsc2Nvbm5lY3Rpb24tZ251dGxzLmM6MTg0OQojMjcgMHgwMDAw
N2ZlMTVlYzhjMjkxIGluIGFjY2VwdF9jZXJ0aWZpY2F0ZV9jYiAodXNlcl9kYXRhPXVzZXJfZGF0
YUBlbnRyeT0weDE4ZTM1MzApCiAgICBhdCAuLi8uLi8uLi8uLi9Qcm9qZWN0cy9nbGliLW5ldHdv
cmtpbmcvdGxzL2dudXRscy9ndGxzY29ubmVjdGlvbi1nbnV0bHMuYzoxODY5CiMyOCAweDAwMDA3
ZmUxYTM3MjE4MjQgaW4gZ19pZGxlX2Rpc3BhdGNoIChzb3VyY2U9MHgxNjUwZjkwLCAKICAgIGNh
bGxiYWNrPTB4N2ZlMTVlYzhjMjNjIDxhY2NlcHRfY2VydGlmaWNhdGVfY2I+LCB1c2VyX2RhdGE9
MHgxOGUzNTMwKQogICAgYXQgL2hvbWUvbWNhdGFuemFyby9Qcm9qZWN0cy9nbGliL2dsaWIvZ21h
aW4uYzo1NjI0CiMyOSAweDAwMDA3ZmUxYTM3MjM5YjUgaW4gZ19tYWluX2Rpc3BhdGNoIChjb250
ZXh0PWNvbnRleHRAZW50cnk9MHgxMzA0YTYwKQogICAgYXQgL2hvbWUvbWNhdGFuemFyby9Qcm9q
ZWN0cy9nbGliL2dsaWIvZ21haW4uYzozMTkwCiMzMCAweDAwMDA3ZmUxYTM3MjRlMTIgaW4gZ19t
YWluX2NvbnRleHRfZGlzcGF0Y2ggKGNvbnRleHQ9Y29udGV4dEBlbnRyeT0weDEzMDRhNjApCiAg
ICBhdCAvaG9tZS9tY2F0YW56YXJvL1Byb2plY3RzL2dsaWIvZ2xpYi9nbWFpbi5jOjM4NTUKIzMx
IDB4MDAwMDdmZTFhMzcyNGY2ZSBpbiBnX21haW5fY29udGV4dF9pdGVyYXRlIChjb250ZXh0PTB4
MTMwNGE2MCwgYmxvY2s9YmxvY2tAZW50cnk9MSwgCiAgICBkaXNwYXRjaD1kaXNwYXRjaEBlbnRy
eT0xLCBzZWxmPXNlbGZAZW50cnk9MHgxMzEwZTkwKQogICAgYXQgL2hvbWUvbWNhdGFuemFyby9Q
cm9qZWN0cy9nbGliL2dsaWIvZ21haW4uYzozOTI4CiMzMiAweDAwMDA3ZmUxYTM3MjUyZjUgaW4g
Z19tYWluX2xvb3BfcnVuIChsb29wPWxvb3BAZW50cnk9MHgxMzA0YjUwKQogICAgYXQgL2hvbWUv
bWNhdGFuemFyby9Qcm9qZWN0cy9nbGliL2dsaWIvZ21haW4uYzo0MTI0CiMzMyAweDAwMDA3ZmUx
YTcxZDJhYzAgaW4gV1RGOjpSdW5Mb29wOjpydW4gKCkKICAgIGF0IC9ob21lL21jYXRhbnphcm8v
UHJvamVjdHMvV2ViS2l0L1NvdXJjZS9XVEYvd3RmL2dsaWIvUnVuTG9vcEdMaWIuY3BwOjk2CiMz
NCAweDAwMDA3ZmUxYTdlMjJhZTkgaW4gV2ViS2l0OjpDaGlsZFByb2Nlc3NNYWluPFdlYktpdDo6
TmV0d29ya1Byb2Nlc3MsIFdlYktpdDo6TmV0d29ya1Byb2Nlc3NNYWluPiAoCiAgICBhcmdjPTxv
cHRpbWl6ZWQgb3V0PiwgYXJndj0weDdmZmRhODFjM2EzOCkKICAgIGF0IC9ob21lL21jYXRhbnph
cm8vUHJvamVjdHMvV2ViS2l0L1NvdXJjZS9XZWJLaXQvU2hhcmVkL3VuaXgvQ2hpbGRQcm9jZXNz
TWFpbi5oOjQxCiMzNSAweDAwMDA3ZmUxYTJlYTY0MTMgaW4gX19saWJjX3N0YXJ0X21haW4gKCkg
ZnJvbSAvbGliNjQvbGliYy5zby42CiMzNiAweDAwMDAwMDAwMDA0MDBiYWUgaW4gX3N0YXJ0ICgp
Cg==
</data>

          </attachment>
          <attachment
              isobsolete="0"
              ispatch="1"
              isprivate="0"
          >
            <attachid>356239</attachid>
            <date>2018-11-30 14:27:31 -0800</date>
            <delta_ts>2018-12-03 11:11:37 -0800</delta_ts>
            <desc>Patch</desc>
            <filename>bug-107451-20181130162730.patch</filename>
            <type>text/plain</type>
            <size>4506</size>
            <attacher name="Michael Catanzaro">mcatanzaro</attacher>
            
              <data encoding="base64">U3VidmVyc2lvbiBSZXZpc2lvbjogMjM4NjcwCmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViQ29yZS9D
aGFuZ2VMb2cgYi9Tb3VyY2UvV2ViQ29yZS9DaGFuZ2VMb2cKaW5kZXggYTgxZGRlMzQ1YjgwNjg1
MWRlYmExMWU1N2RmN2QzMjUzMjQzNWJkMi4uMDA0OWFhZmY3M2NlNTZhMDBmZjIzZjcyMzE4ZmJm
NWQ2NjNlYWM0MSAxMDA2NDQKLS0tIGEvU291cmNlL1dlYkNvcmUvQ2hhbmdlTG9nCisrKyBiL1Nv
dXJjZS9XZWJDb3JlL0NoYW5nZUxvZwpAQCAtMSwzICsxLDQzIEBACisyMDE4LTExLTMwICBNaWNo
YWVsIENhdGFuemFybyAgPG1jYXRhbnphcm9AaWdhbGlhLmNvbT4KKworICAgICAgICBbU09VUF0g
VXNlIFNvdXBTZXNzaW9uIGluc3RlYWQgb2YgU291cFNlc3Npb25Bc3luYworICAgICAgICBodHRw
czovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MTA3NDUxCisKKyAgICAgICAgUmV2
aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgV2l0aCBnbGliLW5ldHdvcmtpbmcg
Mi41Ny4xLCBXZWJLaXQgaXMgbm8gbG9uZ2VyIGFibGUgdG8gbG9hZCBUTFMgZXJyb3IgcGFnZXMu
IFRoZSBwcm9ibGVtCisgICAgICAgIGlzIGEgbmV0d29yayBwcm9jZXNzIGRlYWRsb2NrIGNhdXNl
ZCBieSBhIGNoYW5nZSBpbiBob3cgZ2xpYi1uZXR3b3JraW5nIHBlcmZvcm1zIGNlcnRpZmljYXRl
CisgICAgICAgIHZlcmlmaWNhdGlvbi4gUHJldmlvdXNseSBpdCB2ZXJpZmllZCBjZXJ0aWZpY2F0
ZXMgKmFmdGVyKiB0aGUgVExTIGhhbmRzaGFrZSBoYWQgY29tcGxldGVkLAorICAgICAgICB3aGlj
aCB3YXMgd2VpcmRseSBsYXRlLCBidXQgcHJldmlvdXNseSBub3QgcHJvYmxlbWF0aWMuIEJ1dCBu
b3cgdGhhdCBUTFMgMS4zIGV4aXN0cywKKyAgICAgICAgYXBwbGljYXRpb24gZGF0YSBjYW4gYmUg
c2VudCBiZWZvcmUgY2VydGlmaWNhdGUgdmVyaWZpY2F0aW9uIG9jY3Vycywgd2hpY2ggaXMgbm8g
Z29vZC4gU28gSQorICAgICAgICBtb3ZlZCB2ZXJpZmljYXRpb24gdG8gb2NjdXIgZHVyaW5nIHRo
ZSBoYW5kc2hha2UuIEkgbmVlZGVkIHRvIGRvIHRoaXMgcmVnYXJkbGVzcyBiZWNhdXNlIEkKKyAg
ICAgICAgbmVlZCB0byBhZGQgYSBuZXcgY2FsbGJhY2sgaW4gR251VExTIGZvciBhbm90aGVyIGZl
YXR1cmUuIFRoaXMgaW50cm9kdWNlZCBhIGRlYWRsb2NrIGluCisgICAgICAgIFdlYktpdDoKKwor
ICAgICAgICAgLSBnbGliLW5ldHdvcmtpbmcgZGV0ZWN0cyBhbiB1bmFjY2VwdGFibGUgY2VydGlm
aWNhdGUsIGVtaXRzIGFjY2VwdC1jZXJ0aWZpY2F0ZSBzaWduYWwKKyAgICAgICAgIC0gTmV0d29y
a0RhdGFUYXNrU291cDo6dGxzQ29ubmVjdGlvbkFjY2VwdENlcnRpZmljYXRlIGNhbGxzCisgICAg
ICAgICAgIE5ldHdvcmtEYXRhVGFza1NvdXA6OmludmFsaWRhdGVBbmRDYW5jZWwgY2FsbHMgTmV0
d29ya0RhdGFUYXNrU291cDo6Y2xlYXJSZXF1ZXN0CisgICAgICAgICAtIE5ldHdvcmtEYXRhVGFz
a1NvdXA6OmNsZWFyUmVxdWVzdCBjYWxscyBzb3VwX3Nlc3Npb25fY2FuY2VsX21lc3NhZ2UKKwor
ICAgICAgICBUaGUgcHJvYmxlbSBpcyB0aGF0LCBpbiB0aGUgZGVwcmVjYXRlZCBTb3VwU2Vzc2lv
bkFzeW5jIHVzZWQgYnkgV2ViS2l0LCBjYW5jZWxsYXRpb24gaXMKKyAgICAgICAgYWx3YXlzICpz
eW5jaHJvbm91cyogZGVzcGl0ZSB0aGUgbmFtZSBvZiB0aGUgY2xhc3MuIFNvIHNvdXBfc2Vzc2lv
bl9jYW5jZWxfbWVzc2FnZSB3aW5kcyB1cAorICAgICAgICBkb2luZyBpdHMgdGhpbmcgdG8gY2xv
c2UgZXZlcnl0aGluZyBvdXQsIGFuZCB0aGF0IGV2ZW50dWFsbHkgZW5kcyB1cCBpbiBhIHN5bmNo
cm9ub3VzIGNhbGwKKyAgICAgICAgdG8gZ190bHNfY29ubmVjdGlvbl9nbnV0bHNfY2xvc2UuIFRo
ZSBjbG9zZSBvcGVyYXRpb24gY2FuJ3QgcHJvY2VlZCB1bnRpbCB0aGUgVExTIGhhbmRzaGFrZQor
ICAgICAgICBpcyBmaW5pc2hlZCwgYW5kIHRoZSBoYW5kc2hha2UgaXMgYmxvY2tlZCB3YWl0aW5n
IGZvciBXZWJLaXQgdG8gcmV0dXJuIGZyb20gaXRzCisgICAgICAgIGFjY2VwdC1jZXJ0aWZpY2F0
ZSBoYW5kbGVyLiBTbyB0aGUgY2xvc2Ugb3BlcmF0aW9uIHdpbmRzIHVwIHBvbGxpbmcgZm9yZXZl
ciB3YWl0aW5nIGZvciB0aGUKKyAgICAgICAgaGFuZHNoYWtlIHRvIGZpbmlzaC4gRGVhZGxvY2su
CisKKyAgICAgICAgVGhlIG9ubHkgY2hhbmdlcyByZXF1aXJlZCBpbiBXZWJLaXQgdG8gdXNlIHRo
ZSBtb2Rlcm4gU291cFNlc3Npb24gYXJlIGFkanVzdG1lbnRzIGZvciB0aGUKKyAgICAgICAgbmV3
IGRlZmF1bHQgcHJvcGVydHkgdmFsdWVzLiBNb3N0IG9mIHRoZSBwcm9wZXJ0aWVzIHdlIHVzZWQg
dG8gc2V0IGV4cGxpY2l0bHkgYXJlIG5vdworICAgICAgICBkZWZhdWx0cyBhbmQgc2hvdWxkIGJl
IHJlbW92ZWQuIEFkZGl0aW9uYWxseSwgU291cFNlc3Npb24gaGFzIGRlZmF1bHQgdGltZW91dHMs
IHdoaWNoIHdlCisgICAgICAgIHdhbnQgdG8gb3ZlcnJpZGUgdG8gYWxsb3cgTmV0d29ya0RhdGFU
YXNrU291cCB0byBpbXBsZW1lbnQgaXRzIG93biB0aW1lb3V0cy4KKworICAgICAgICBObyBuZXcg
dGVzdHMgYmVjYXVzZSB0aGlzIGlzIGNvdmVyZWQgYnkgVGVzdFNTTCAod2hpY2ggd291bGQgYmUg
ZmFpbGluZyBpZiBydW4gd2l0aCB0aGUKKyAgICAgICAgbmV3ZXIgZ2xpYi1uZXR3b3JraW5nKS4K
KworICAgICAgICAqIHBsYXRmb3JtL25ldHdvcmsvc291cC9Tb3VwTmV0d29ya1Nlc3Npb24uY3Bw
OgorICAgICAgICAoV2ViQ29yZTo6U291cE5ldHdvcmtTZXNzaW9uOjpTb3VwTmV0d29ya1Nlc3Np
b24pOgorCiAyMDE4LTExLTI5ICBaYWxhbiBCdWp0YXMgIDx6YWxhbkBhcHBsZS5jb20+CiAKICAg
ICAgICAgW0xGQ11bQkZDXVtRdWlya10gV2lkdGggZG9lcyBub3QgbmVlZCBzdHJldGNoaW5nIHF1
aXJrLgpkaWZmIC0tZ2l0IGEvU291cmNlL1dlYkNvcmUvcGxhdGZvcm0vbmV0d29yay9zb3VwL1Nv
dXBOZXR3b3JrU2Vzc2lvbi5jcHAgYi9Tb3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS9uZXR3b3JrL3Nv
dXAvU291cE5ldHdvcmtTZXNzaW9uLmNwcAppbmRleCAxM2VjYjViMGY3NjNjMjAxY2YxN2E0MTYx
MmE3M2YwOWE4ZjVhMDc4Li5mMjRlMjY1Yzk2ZWFmYjVkZTQzNzFhMWQyMjYzOGNjNzE4ZTY5NTE2
IDEwMDY0NAotLS0gYS9Tb3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS9uZXR3b3JrL3NvdXAvU291cE5l
dHdvcmtTZXNzaW9uLmNwcAorKysgYi9Tb3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS9uZXR3b3JrL3Nv
dXAvU291cE5ldHdvcmtTZXNzaW9uLmNwcApAQCAtMTA1LDcgKzEwNSw3IEBAIHN0YXRpYyBIYXNo
TWFwPFN0cmluZywgSG9zdFRMU0NlcnRpZmljYXRlU2V0LCBBU0NJSUNhc2VJbnNlbnNpdGl2ZUhh
c2g+JiBjbGllbnRDCiB9CiAKIFNvdXBOZXR3b3JrU2Vzc2lvbjo6U291cE5ldHdvcmtTZXNzaW9u
KFBBTDo6U2Vzc2lvbklEIHNlc3Npb25JRCwgU291cENvb2tpZUphciogY29va2llSmFyKQotICAg
IDogbV9zb3VwU2Vzc2lvbihhZG9wdEdSZWYoc291cF9zZXNzaW9uX2FzeW5jX25ldygpKSkKKyAg
ICA6IG1fc291cFNlc3Npb24oYWRvcHRHUmVmKHNvdXBfc2Vzc2lvbl9uZXcoKSkpCiB7CiAgICAg
Ly8gVmFsdWVzIHRha2VuIGZyb20gaHR0cDovL3d3dy5icm93c2Vyc2NvcGUub3JnLyBmb2xsb3dp
bmcKICAgICAvLyB0aGUgcnVsZSAiRG8gV2hhdCBFdmVyeSBPdGhlciBNb2Rlcm4gQnJvd3NlciBJ
cyBEb2luZyIuIFRoZXkgc2VlbQpAQCAtMTIzLDEzICsxMjMsMTAgQEAgU291cE5ldHdvcmtTZXNz
aW9uOjpTb3VwTmV0d29ya1Nlc3Npb24oUEFMOjpTZXNzaW9uSUQgc2Vzc2lvbklELCBTb3VwQ29v
a2llSmFyKgogICAgIGdfb2JqZWN0X3NldChtX3NvdXBTZXNzaW9uLmdldCgpLAogICAgICAgICBT
T1VQX1NFU1NJT05fTUFYX0NPTk5TLCBtYXhDb25uZWN0aW9ucywKICAgICAgICAgU09VUF9TRVNT
SU9OX01BWF9DT05OU19QRVJfSE9TVCwgbWF4Q29ubmVjdGlvbnNQZXJIb3N0LAotICAgICAgICBT
T1VQX1NFU1NJT05fQUREX0ZFQVRVUkVfQllfVFlQRSwgU09VUF9UWVBFX0NPTlRFTlRfREVDT0RF
UiwKKyAgICAgICAgU09VUF9TRVNTSU9OX1RJTUVPVVQsIDAsCisgICAgICAgIFNPVVBfU0VTU0lP
Tl9JRExFX1RJTUVPVVQsIDAsCiAgICAgICAgIFNPVVBfU0VTU0lPTl9BRERfRkVBVFVSRV9CWV9U
WVBFLCBTT1VQX1RZUEVfQ09OVEVOVF9TTklGRkVSLAotICAgICAgICBTT1VQX1NFU1NJT05fQURE
X0ZFQVRVUkVfQllfVFlQRSwgU09VUF9UWVBFX1BST1hZX1JFU09MVkVSX0RFRkFVTFQsCiAgICAg
ICAgIFNPVVBfU0VTU0lPTl9BRERfRkVBVFVSRSwgamFyLmdldCgpLAotICAgICAgICBTT1VQX1NF
U1NJT05fVVNFX1RIUkVBRF9DT05URVhULCBUUlVFLAotICAgICAgICBTT1VQX1NFU1NJT05fU1NM
X1VTRV9TWVNURU1fQ0FfRklMRSwgVFJVRSwKLSAgICAgICAgU09VUF9TRVNTSU9OX1NTTF9TVFJJ
Q1QsIFRSVUUsCiAgICAgICAgIG51bGxwdHIpOwogCiAgICAgc2V0dXBDdXN0b21Qcm90b2NvbHMo
KTsK
</data>

          </attachment>
      

    </bug>

</bugzilla>