Bug 190258

Summary: [GTK] build fails with python 3 if LANG and LC_TYPE are unset
Product: WebKit Reporter: Mike Gorse <mgorse>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: annulen, bugs-noreply, commit-queue, ews-watchlist, keith_miller, lforschler, mark.lam, mcatanzaro, mgorse, msaboff, saam, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Local Build   
Hardware: Unspecified   
OS: Linux   
Attachments:
Description Flags
Proposed patch.
annulen: review+
Updated patch. none

Mike Gorse
Reported 2018-10-03 12:52:02 PDT
Building the gtk port still fails using Python 3 if LANG or LC_TYPE do not specify UTF-8. I'm seeing tracebacks such as the following: Traceback (most recent call last): File "/home/mgorse/src/WebKit/Source/JavaScriptCore/Scripts/generateIntlCanonicalizeLanguage.py", line 197, in <module> registry.parse(registryFile) File "/home/mgorse/src/WebKit/Source/JavaScriptCore/Scripts/generateIntlCanonicalizeLanguage.py", line 63, in parse for line in file: File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 866: ordinal not in range(128)
Attachments
Proposed patch. (4.31 KB, patch)
2018-10-03 13:05 PDT, Mike Gorse 		annulen: review+
DetailsFormatted DiffDiff
Updated patch. (4.37 KB, patch)
2018-10-04 16:59 PDT, Mike Gorse 		no flags
DetailsFormatted DiffDiff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.
Mike Gorse
Comment 1 2018-10-03 13:05:08 PDT
Created attachment 351542 [details] Proposed patch. Explicitly specify UTF-8 encoding for Python 3 in some places, rather than defaulting to the encoding specified in the environment.
Mark Lam
Comment 2 2018-10-03 13:22:58 PDT
Comment on attachment 351542 [details] Proposed patch. LGTM but let's wait for the EWS bots to be green before committing.
Konstantin Tokarev
Comment 3 2018-10-03 13:53:03 PDT
Comment on attachment 351542 [details] Proposed patch. View in context: https://bugs.webkit.org/attachment.cgi?id=351542&action=review > Source/JavaScriptCore/Scripts/cssmin.py:51 > + sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='UTF-8') These lines duplicate code above
Mike Gorse
Comment 4 2018-10-03 18:21:09 PDT
(In reply to Konstantin Tokarev from comment #3) > Comment on attachment 351542 [details] > Proposed patch. > > View in context: > https://bugs.webkit.org/attachment.cgi?id=351542&action=review > > > Source/JavaScriptCore/Scripts/cssmin.py:51 > > + sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='UTF-8') > > These lines duplicate code above I think you're referring to this line: sys.stdin = io.TextIOWrapper(sys.stdin.buffer, encoding='UTF-8') My patch adds similar handling for sys.stdout.
Konstantin Tokarev
Comment 5 2018-10-03 18:59:49 PDT
Comment on attachment 351542 [details] Proposed patch. View in context: https://bugs.webkit.org/attachment.cgi?id=351542&action=review >>> Source/JavaScriptCore/Scripts/cssmin.py:51 >>> + sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='UTF-8') >> >> These lines duplicate code above > > I think you're referring to this line: > sys.stdin = io.TextIOWrapper(sys.stdin.buffer, encoding='UTF-8') > My patch adds similar handling for sys.stdout. Oh, sorry. Still I think it would be better to avoid duplication of 'sys.version_info[0] >= 3' and 'import io' to prevent misreads
Mike Gorse
Comment 6 2018-10-04 16:59:17 PDT
Created attachment 351643 [details] Updated patch. Updated cssmin.py to remove duplicated if check and import.
Michael Catanzaro
Comment 7 2018-10-05 02:02:09 PDT
If you keep the previous reviewed by line when you upload it (either by adding it manually or by applying it with 'webkit-patch apply-from-bug') then any committer would be able to commit it without waiting for Konstantin again.
WebKit Commit Bot
Comment 8 2018-10-05 03:56:39 PDT
Comment on attachment 351643 [details] Updated patch. Clearing flags on attachment: 351643 Committed r236870: <https://trac.webkit.org/changeset/236870>
WebKit Commit Bot
Comment 9 2018-10-05 03:56:41 PDT
All reviewed patches have been landed. Closing bug.
Radar WebKit Bug Importer
Comment 10 2018-10-05 03:57:33 PDT
<rdar://problem/45038771>
Note You need to log in before you can comment on or make changes to this bug.