WebKit Bugzilla
Attachment 369198 Details for
Bug 197622
: Rewrite generate-xcfilelists in Python
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Address style issues. The remaining two are bogus.
bug-197622-20190506163503.patch (text/plain), 121.54 KB, created by
Keith Rollin
on 2019-05-06 16:35:04 PDT
(
hide
)
Description:
Address style issues. The remaining two are bogus.
Filename:
MIME Type:
Creator:
Keith Rollin
Created:
2019-05-06 16:35:04 PDT
Size:
121.54 KB
patch
obsolete
>Subversion Revision: 244987 >diff --git a/Tools/ChangeLog b/Tools/ChangeLog >index ef56ed36d9f96a02540ca79040f97a3825ef1a0e..4e34b08743951cad61f5b7fa981a099283481697 100644 >--- a/Tools/ChangeLog >+++ b/Tools/ChangeLog >@@ -1,3 +1,155 @@ >+2019-05-06 Keith Rollin <krollin@apple.com> >+ >+ Rewrite generate-xcfilelists in Python >+ https://bugs.webkit.org/show_bug.cgi?id=197622 >+ <rdar://problem/50508222> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ For a number of reasons, rewrite generate-xcfilelists in Python: >+ >+ - The previous bash script was large and unmaintainable. >+ - We have more internal expertise with Python than with bash. >+ - The bash script used temporary files in /tmp that got stranded and >+ were owned by root, making them awkward to clear out. >+ <rdar://problem/49490262> >+ - We needed to complete support for engineers that built with Xcode >+ rather than the command line. >+ - There are some bugs leading to mildly corrupted .xcfilelist files. >+ It's not apparent the source of the bugs, but one general reason >+ might be due to approaches born of using bash. Rewriting in Python >+ may clear these up. >+ >+ * Scripts/generate-xcfilelists: >+ * Scripts/webkitpy/generate_xcfilelists_lib/__init__.py: Added. >+ * Scripts/webkitpy/generate_xcfilelists_lib/application.py: Added. >+ (Application): >+ (Application.__init__): >+ (Application.run): >+ (Application._initialize): >+ (Application._initialize.collect_attributes): >+ (Application._create_parser): >+ (_validate_args): >+ (_validate_args.matches): >+ (_validate_args.test_item_validity): >+ (_cmd_generate): >+ (_cmd_check): >+ (_cmd_subgenerate): >+ (_cmd_help): >+ (_do_generate): >+ (_do_merge): >+ (_report_results): >+ (_report_merge_results): >+ (_report_remediation_steps): >+ (_report_remediation_steps.add_to_message): >+ (_any_have_errors): >+ (_any_have_actions): >+ (_log_progress): >+ (_log_results): >+ (_get_generate_xcfilelists_script_path): >+ (_get_root_dir): >+ (_get_opensource_dir): >+ (_get_build_scripts_dir): >+ (_get_extract_dependencies_from_makefile_script): >+ (_get_xcode_built_products_dir): >+ (_getenv): >+ * Scripts/webkitpy/generate_xcfilelists_lib/generators.py: Added. >+ (BaseGenerator): >+ (BaseGenerator.__init__): >+ (BaseGenerator.__str__): >+ (BaseGenerator.generate): >+ (BaseGenerator._sublaunch_under_xcode): >+ (BaseGenerator.subgenerate): >+ (BaseGenerator.merge): >+ (BaseGenerator.pickle_to_file): >+ (BaseGenerator.has_action): >+ (BaseGenerator.has_error): >+ (BaseGenerator.is_valid): >+ (BaseGenerator.report_error): >+ (BaseGenerator._generate_derived): >+ (BaseGenerator._merge_derived): >+ (BaseGenerator._generate_unified): >+ (BaseGenerator._merge_unified): >+ (BaseGenerator._replace): >+ (BaseGenerator._unexpand): >+ (BaseGenerator._find_added_lines): >+ (BaseGenerator._merge_added_lines): >+ (BaseGenerator._get_file_lines): >+ (BaseGenerator._getenv): >+ (BaseGenerator._get_project_file_path): >+ (BaseGenerator._get_project_dir): >+ (BaseGenerator._get_project_file_name): >+ (BaseGenerator._get_project_name): >+ (BaseGenerator._get_sym_root): >+ (BaseGenerator._get_obj_root): >+ (BaseGenerator._get_shared_precomps_dir): >+ (BaseGenerator._get_build_dirs): >+ (BaseGenerator._get_build_dirs.define_xcode_build_dirs): >+ (BaseGenerator._get_build_dirs.define_xcode_build_dirs.read_xcode_user_default): >+ (BaseGenerator._get_build_dirs.define_command_line_build_dirs): >+ (BaseGenerator._get_derived_sources_dir): >+ (BaseGenerator._get_xcfilelist_dir): >+ (BaseGenerator._get_input_derived_xcfilelist_project_path): >+ (BaseGenerator._get_output_derived_xcfilelist_project_path): >+ (BaseGenerator._get_input_unified_xcfilelist_project_path): >+ (BaseGenerator._get_output_unified_xcfilelist_project_path): >+ (BaseGenerator._get_generate_derived_sources_script): >+ (BaseGenerator._get_generate_unified_sources_script): >+ (JavaScriptCoreGenerator): >+ (JavaScriptCoreGenerator._get_project_file_path): >+ (JavaScriptCoreGenerator._get_generate_derived_sources_script): >+ (JavaScriptCoreGenerator._get_generate_unified_sources_script): >+ (WebCoreGenerator): >+ (WebCoreGenerator._get_project_file_path): >+ (WebCoreGenerator._get_generate_derived_sources_script): >+ (WebCoreGenerator._get_generate_unified_sources_script): >+ (WebKitGenerator): >+ (WebKitGenerator._get_project_file_path): >+ (WebKitGenerator._get_derived_sources_dir): >+ (WebKitGenerator._get_generate_derived_sources_script): >+ (WebKitGenerator._get_generate_unified_sources_script): >+ (DumpRenderTreeGenerator): >+ (DumpRenderTreeGenerator._get_project_file_path): >+ (DumpRenderTreeGenerator._get_generate_derived_sources_script): >+ (WebKitTestRunnerGenerator): >+ (WebKitTestRunnerGenerator._get_project_file_path): >+ (WebKitTestRunnerGenerator._get_generate_derived_sources_script): >+ * Scripts/webkitpy/generate_xcfilelists_lib/sdk.py: Added. >+ (SDK): >+ (SDK.__init__): >+ (SDK.__repr__): >+ (SDK.as_xcode_specification): >+ (SDK.get_preferred_sdk_for_platform): >+ (SDK._parse_sdk): >+ * Scripts/webkitpy/generate_xcfilelists_lib/util.py: Added. >+ (debug_log): >+ (LogEntryHelper): >+ (LogEntryHelper.__init__): >+ (LogEntryHelper.log_entry): >+ (LogEntryHelper.log_result): >+ (LogEntryHelper.log_exception): >+ (LogEntryHelper._print): >+ (LogEntryExit): >+ (LogEntryExit._show_debug_logging): >+ (LogEntryExitClass): >+ (LogEntryExitClass._show_debug_logging): >+ (LogEntryExitGlobal): >+ (LogEntryExitGlobal._show_debug_logging): >+ (subprocess_run): >+ (is_running_under_xcode): >+ (AttributeSaver): >+ (AttributeSaver.__init__): >+ (AttributeSaver.__enter__): >+ (AttributeSaver.__exit__): >+ (QuickExit): >+ (InvalidCommandError): >+ (InvalidArgumentError): >+ (InvalidConfigurationError): >+ (CalledProcessError): >+ (CalledProcessError.__str__): >+ (hash_string_for_path): >+ (hash_string_for_path.convert_to_string): >+ > 2019-05-06 Keith Rollin <krollin@apple.com> > > Temporarily disable generate-xcfilelists >diff --git a/Tools/Scripts/generate-xcfilelists b/Tools/Scripts/generate-xcfilelists >index 0d869780a7c8f9792c7768b072783c953bcbb01f..8b5cb0393badff9464c28dc121ea9ad51eb25b33 100755 >--- a/Tools/Scripts/generate-xcfilelists >+++ b/Tools/Scripts/generate-xcfilelists >@@ -1,6 +1,7 @@ >-#!/bin/bash >+#!/usr/bin/env python >+# -*- coding: utf-8 -*- > # >-# Copyright (C) 2018-2019 Apple Inc. All rights reserved. >+# Copyright (C) 2019 Apple Inc. All rights reserved. > # > # Redistribution and use in source and binary forms, with or without > # modification, are permitted provided that the following conditions >@@ -47,1330 +48,27 @@ > # > # As part of its operation, this script can sub-launch itself under Xcode. It > # does this in order to establish the context in which builds occur. In >-# particular, it does this in order to set needed environment variables. In >-# order to protect against environment variable name collision, all variable >-# names in this script start with "GX_". >- >- >-function stdout() >-{ >- (( ${GX_QUIET} )) || echo "$@" >&1 >-} >- >- >-function stderr() >-{ >- echo "$@" >&2 >-} >- >- >-function remove_extra_spaces() >-{ >- echo $1 >-} >- >-function print_option() >-{ >- local GX_OPTION="$1" >- local GX_MESSAGE="$(remove_extra_spaces "$2")" >- >- local GX_LINE >- local GX_LINE_WIDTH=90 >- local GX_OPTION_WIDTH=17 >- >- if (( ${#GX_OPTION} <= GX_OPTION_WIDTH )) >- then >- local GX_MESSAGE_CATENATED=$(printf " %-${GX_OPTION_WIDTH}s %s" "$GX_OPTION" "$GX_MESSAGE") >- if (( ${#GX_MESSAGE_CATENATED} <= GX_LINE_WIDTH )) >- then >- stderr "${GX_MESSAGE_CATENATED}" >- return >- fi >- >- local GX_MESSAGE_FIRST_LINE="${GX_MESSAGE_CATENATED:0:${GX_LINE_WIDTH}}" # Get the first 80 characters >- GX_MESSAGE_FIRST_LINE="${GX_MESSAGE_FIRST_LINE% *}" # Trim back to the last space >- GX_MESSAGE="${GX_MESSAGE_CATENATED:${#GX_MESSAGE_FIRST_LINE}+1}" # Get the rest after that space >- >- stderr "${GX_MESSAGE_FIRST_LINE}" >- else >- stderr " ${GX_OPTION}" >- fi >- >- while IFS='' read -r GX_LINE >- do >- GX_LINE=$(printf " %${GX_OPTION_WIDTH}s %s" "" "$GX_LINE") >- stderr "${GX_LINE}" >- done < <(echo "${GX_MESSAGE}" | fold -w $(( GX_LINE_WIDTH - GX_OPTION_WIDTH - 3)) -s) >-} >- >- >-function join_by() >-{ >- local GX_DELIM=$1 >- shift >- >- echo -n "$1" >- shift >- >- printf "%s" "${@/#/${GX_DELIM}}" >-} >- >- >-function usage() >-{ >- local GX_JOINED_PROJECT_TAGS=$(join_by ", " "${GX_PROJECT_TAGS[@]}") >- >- stderr "Usage: $(basename "${GX_ME}") [OPTION...] COMMAND [PARAMETER...]" >- stderr "" >- stderr "Generate or check .xcfilelist files" >- stderr "" >- stderr "Commands:" >- stderr "" >- print_option "generate" "Generate a complete and up-to-date set of >- .xcfilelist files and copy them to their appropriate places in the >- project directories." >- print_option "generate-xcode" "Generate .xcfilelist files for the current >- target configuration an copy them to their appropriate places in the >- project directories. Invoked from withing Xcode to update the current >- build if needed." >- print_option "check" "Generate a complete and up-to-date set of .xcfilelist >- files and compare them to their counterparts in the project >- directories." >- print_option "check-xcode" "Generate .xcfilelist files for the current >- target configuration and see if any files appear that aren't in the >- checked-in .xcfilelist files. Invoked from within Xcode to validate the >- current build." >- print_option "subgenerate" "Generate an .xcfilelist file for a particular >- combination of project, platform, and configuration. This operation is >- performed in the context of an Xcode build in order to inherit the same >- environment as that build." >- print_option "help" "Print this text and exit." >- stderr "" >- stderr "Options:" >- stderr "" >- print_option "--project <project>" "Specify which project for which to >- generate .xcfilelist files or to check. Possible values are >- (${GX_JOINED_PROJECT_TAGS}). Default is to iterate over all projects." >- print_option "--platform <platform>" "Specify which platform for which to >- generate .xcfilelist files or to check. Possible values are (macosx, >- iphoneos). Default is to iterate over all platforms, filtered to those >- platforms that a particular project supports (e.g., you can't specify >- 'iphoneos' for WebKitTestRunner)." >- print_option "--configuration <configuration>" "Specify which configuration >- for which to generate .xcfilelist files or to check, Possible values >- are (release, debug). Default is to iterate over all configurations." >- print_option "--xcode" "Specify that existing build output can be found in >- the directory created by the Xcode IDE (in ~/Library/Developer/Xcode)." >- print_option "--webkitbuild" "Specify that existing build output can be >- found in the directory used by the WebKit make files or the >- build-webkit script (that is, in WebKitBuild or in the directory >- identified by WEBKIT_OUTPUTDIR). This option is the default, so it >- doesn't really get you anything by specifying this option (or by my >- implementing it)." >- print_option "--nocleanup" "Disable cleaning up temporary files after a >- subgenerate operation. Normally these are cleaned up, but if >- 'subgenerate' is part of an overall 'generate' operation, then the >- 'generate' operation needs to keep around the temporary files until it >- can complete aggregating them." >- stderr "" >- print_option "--dry-run" "When running through the requested process, don't >- make any material changes. For example, for 'generate', create the new >- .xcfilelist files, but don't copy them into their final locations" >- print_option "--debug" "Provide verbose output." >- print_option "--quiet" "Don't print any std output." >- print_option "--help" "Print this text and exit." >- stderr "" >-} >- >- >-function cleanup() >-{ >- if (( ${GX_DO_CLEANUP} )) && (( ${GX_DEFERRED_EXIT_CODE} == 0 )) >- then >- rm -rf "${GX_TEMP}" &> /dev/null >- fi >-} >- >- >-function my_exit() >-{ >- local GX_ERR=$1 >- exit $GX_ERR >-} >- >- >-function die() >-{ >- local GX_ERR=$2 >- [[ "$GX_ERR" == "" ]] && GX_ERR=1 >- stderr "### (${FUNCNAME[@]}) $1" >- my_exit $GX_ERR >-} >- >- >-function log_debug() >-{ >- (( "${GX_DEBUG}" > 0 )) && stderr "GXCF: $@" >-} >- >- >-function log_debug_var() >-{ >- local GX_VAR_NAME=$1 >- local GX_IS_ARRAY=$(declare -p ${GX_VAR_NAME} 2> /dev/null | grep -q '^declare -a' && echo 1 || echo 0) >- >- if (( $GX_IS_ARRAY )) >- then >- log_debug $(printf "%-47s = (%s)" $GX_VAR_NAME "$(eval echo \$\{${GX_VAR_NAME}\[\@\]\})") >- else >- log_debug $(printf "%-47s = %s" $GX_VAR_NAME "$(eval echo \$\{${GX_VAR_NAME}\})") >- fi >-} >- >- >-function log_callstack_and_parameters() >-{ >- if (( "${GX_DEBUG}" > "1" )) >- then >- local GX_ARGS=("$@") >- local GX_CALLSTACK=("${FUNCNAME[@]}") >- unset GX_CALLSTACK[0] >- stderr "GXCF: (${GX_CALLSTACK[@]}): ${GX_ARGS[@]}" >- fi >-} >- >- >-function log_progress() >-{ >- stdout "GXCF: $@" >-} >- >- >-function log_progress_long() >-{ >- local GX_MESSAGE="$(remove_extra_spaces "$1")" >- local GX_LINE >- while IFS='' read -r GX_LINE >- do >- log_progress $GX_LINE >- done < <(echo "${GX_MESSAGE}" | fold -w 86 -s) >-} >- >- >-function normalize_directory_path() >-{ >- log_callstack_and_parameters "$@" >- >- [[ -z "$1" ]] && { echo "$1"; return; } >- cd "$1" 2> /dev/null || { echo "$1"; return; } >- pwd -P >-} >- >- >-function normalize_file_path() >-{ >- log_callstack_and_parameters "$@" >- >- [[ -z "$1" ]] && { echo "$1"; return; } >- echo $(normalize_directory_path "$(dirname "$1")")/$(basename "$1") >-} >- >- >-function call() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_FN="$1" >- [[ "$(type -t ${GX_FN})" == "function" ]] || return $? >- eval "${GX_FN}" >-} >- >- >-function set_project_specific_settings_for_JavaScriptCore() >-{ >- log_callstack_and_parameters "$@" >- >- GX_PS_PROJECT_FILE_PATH="${GX_OPENSOURCE_DIR}/Source/JavaScriptCore/JavaScriptCore.xcodeproj" # Can't use PROJECT_FILE_PATH because this is used pre-Xcode. >- GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-derived-sources.sh" >- GX_PS_UNIFIEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-unified-sources.sh" >- GX_PS_PLATFORM_NAMES=(macosx iphoneos) >-} >- >- >-function set_project_specific_settings_for_WebCore() >-{ >- log_callstack_and_parameters "$@" >- >- GX_PS_PROJECT_FILE_PATH="${GX_OPENSOURCE_DIR}/Source/WebCore/WebCore.xcodeproj" # Can't use PROJECT_FILE_PATH because this is used pre-Xcode. >- GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-derived-sources.sh" >- GX_PS_UNIFIEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-unified-sources.sh" >- GX_PS_PLATFORM_NAMES=(macosx iphoneos) >-} >- >- >-function set_project_specific_settings_for_WebKit() >-{ >- log_callstack_and_parameters "$@" >- >- GX_PS_PROJECT_FILE_PATH="${GX_OPENSOURCE_DIR}/Source/WebKit/WebKit.xcodeproj" # Can't use PROJECT_FILE_PATH because this is used pre-Xcode. >- GX_PS_DERIVED_SOURCES_DIR="${BUILT_PRODUCTS_DIR}/DerivedSources/WebKit2" >- GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-derived-sources.sh" >- GX_PS_UNIFIEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-unified-sources.sh" >- GX_PS_PLATFORM_NAMES=(macosx iphoneos) >-} >- >- >-function set_project_specific_settings_for_DumpRenderTree() >-{ >- log_callstack_and_parameters "$@" >- >- GX_PS_PROJECT_FILE_PATH="${GX_OPENSOURCE_DIR}/Tools/DumpRenderTree/DumpRenderTree.xcodeproj" # Can't use PROJECT_FILE_PATH because this is used pre-Xcode. >- GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-derived-sources.sh" >- GX_PS_PLATFORM_NAMES=(macosx) >-} >- >- >-function set_project_specific_settings_for_WebKitTestRunner() >-{ >- log_callstack_and_parameters "$@" >- >- GX_PS_PROJECT_FILE_PATH="${GX_OPENSOURCE_DIR}/Tools/WebKitTestRunner/WebKitTestRunner.xcodeproj" # Can't use PROJECT_FILE_PATH because this is used pre-Xcode. >- GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH="Scripts/generate-derived-sources.sh" >- GX_PS_PLATFORM_NAMES=(macosx) >-} >- >- >-function set_project_specific_settings() >-{ >- log_callstack_and_parameters "$@" >- >- # Unset the variables that the set_project_specific_settings_for_FOO >- # functions set. We need to do this in case the function for project BAR >- # doesn't define one or more of them, causing it to inherit the settings >- # for a previous project FOO. >- # >- # To achieve this unsetting, we prefix all the variables we want to unset >- # with "GX_PS_" ("PS for "project specific"). We can then gather all >- # variables that start with this prefix and unset them. >- >- unset ${!GX_PS_*} >- >- [[ -n "${GX_PROJECT_TAG}" ]] || die "GX_PROJECT_TAG is not defined." >- >- call set_project_specific_settings_for_${GX_PROJECT_TAG} || die "Could not set build variables for ${GX_PROJECT_TAG}." >-} >- >- >-function set_common_project_settings() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_EFFECTIVE_PROJECT_FILE_PATH="${GX_PS_PROJECT_FILE_PATH}" >- local GX_EFFECTIVE_PROJECT_DIR=$(dirname "${GX_EFFECTIVE_PROJECT_FILE_PATH}") >- >- [[ -n "${GX_EFFECTIVE_PROJECT_FILE_PATH}" ]] || die "GX_EFFECTIVE_PROJECT_FILE_PATH is not defined." >- [[ -d "${GX_EFFECTIVE_PROJECT_FILE_PATH}" ]] || die "${GX_EFFECTIVE_PROJECT_FILE_PATH} is not a directory." >- [[ -n "${GX_EFFECTIVE_PROJECT_DIR}" ]] || die "GX_EFFECTIVE_PROJECT_DIR is not defined." >- [[ -d "${GX_EFFECTIVE_PROJECT_DIR}" ]] || die "${GX_EFFECTIVE_PROJECT_DIR} is not a directory." >- >- # Get the paths to the scripts that drive the generation of the >- # .xcfilelists or the data going into the .xcfilelists. >- >- GX_DERIVEDSOURCES_GENERATOR="${GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH:+${GX_EFFECTIVE_PROJECT_DIR}/${GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH}}" >- GX_UNIFIEDSOURCES_GENERATOR="${GX_PS_UNIFIEDSOURCES_GENERATOR_RELATIVE_PATH:+${GX_EFFECTIVE_PROJECT_DIR}/${GX_PS_UNIFIEDSOURCES_GENERATOR_RELATIVE_PATH}}" >- >- [[ -n "${GX_DERIVEDSOURCES_GENERATOR}" ]] && { [[ -f "${GX_DERIVEDSOURCES_GENERATOR}" ]] || die "${GX_DERIVEDSOURCES_GENERATOR} does not exist."; } >- [[ -n "${GX_UNIFIEDSOURCES_GENERATOR}" ]] && { [[ -f "${GX_UNIFIEDSOURCES_GENERATOR}" ]] || die "${GX_UNIFIEDSOURCES_GENERATOR} does not exist."; } >- >- # Get the paths to the locations of the final .xcfilelist files. >- >- local GX_BASE_PATH="${GX_EFFECTIVE_PROJECT_DIR}/${GX_PS_XCFILELIST_RELATIVE_PATH}" >- GX_INPUT_DERIVED_XCFILELIST_PROJECT_PATH="${GX_BASE_PATH}/DerivedSources-input.xcfilelist" >- GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_PATH="${GX_BASE_PATH}/DerivedSources-output.xcfilelist" >- GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_PATH="${GX_BASE_PATH}/UnifiedSources-output.xcfilelist" >- >- # Get the paths to the locations of the temporary .xcfilelist files (the >- # ones being built up before being moved to their final locations). >- >- GX_INPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH="${GX_TEMP}/${GX_INPUT_DERIVED_XCFILELIST_PROJECT_PATH}" >- GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH="${GX_TEMP}/${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_PATH}" >- GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_TEMP_PATH="${GX_TEMP}/${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_PATH}" >- >- # Collect the default per-project commands to execute during the generate, >- # merge, and compare stages. >- >- GX_SUBGENERATE_ACTIONS=() >- GX_MERGE_ACTIONS=() >- GX_CHECK_ACTIONS=() >- >- if [[ -n "${GX_DERIVEDSOURCES_GENERATOR}" ]] >- then >- GX_SUBGENERATE_ACTIONS+=(generate_derivedsources_xcfilelists) >- GX_MERGE_ACTIONS+=(merge_derivedsources_xcfilelists) >- GX_CHECK_ACTIONS+=(check_derivedsources_xcfilelists) >- fi >- >- if [[ -n "${GX_UNIFIEDSOURCES_GENERATOR}" ]] >- then >- GX_SUBGENERATE_ACTIONS+=(generate_unifiedsources_xcfilelists) >- GX_MERGE_ACTIONS+=(merge_unifiedsources_xcfilelists) >- GX_CHECK_ACTIONS+=(check_unifiedsources_xcfilelists) >- fi >- >- [[ -z "${GX_PS_DERIVED_SOURCES_DIR}" ]] && GX_PS_DERIVED_SOURCES_DIR="${BUILT_PRODUCTS_DIR}/DerivedSources/${PROJECT_NAME}" >- >- GX_TEMP_INPUT_XCFILELIST="${GX_TEMP}/input.xcfilelist" >- GX_TEMP_OUTPUT_XCFILELIST="${GX_TEMP}/output.xcfilelist" >-} >- >- >-function dump_project_settings() >-{ >- log_callstack_and_parameters "$@" >- >- log_debug_var BUILD_SCRIPTS_DIR >- log_debug_var BUILT_PRODUCTS_DIR >- log_debug_var CONFIGURATION >- log_debug_var DERIVED_SOURCES_DIR >- log_debug_var JAVASCRIPTCORE_PRIVATE_HEADERS_DIR >- log_debug_var PROJECT_DIR >- log_debug_var PROJECT_FILE_PATH >- log_debug_var PROJECT_NAME >- log_debug_var WEBCORE_PRIVATE_HEADERS_DIR >- log_debug_var WEBKIT2_PRIVATE_HEADERS_DIR >- log_debug >- log_debug_var GX_PS_PROJECT_FILE_PATH >- log_debug_var GX_PS_DERIVEDSOURCES_GENERATOR_RELATIVE_PATH >- log_debug_var GX_PS_UNIFIEDSOURCES_GENERATOR_RELATIVE_PATH >- log_debug_var GX_PS_XCFILELIST_RELATIVE_PATH >- log_debug_var GX_PS_PLATFORM_NAMES >- log_debug >- log_debug_var GX_PROJECT_TAG >- log_debug_var GX_PLATFORM_NAME >- log_debug_var GX_CONFIGURATION >- log_debug >- log_debug_var GX_EFFECTIVE_PROJECT_FILE_PATH >- log_debug_var GX_EFFECTIVE_PROJECT_DIR >- log_debug >- log_debug_var GX_DERIVEDSOURCES_GENERATOR >- log_debug_var GX_UNIFIEDSOURCES_GENERATOR >- log_debug >- log_debug_var GX_INPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH >- log_debug_var GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH >- log_debug_var GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_TEMP_PATH >- log_debug >- log_debug_var GX_INPUT_DERIVED_XCFILELIST_PROJECT_PATH >- log_debug_var GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_PATH >- log_debug_var GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_PATH >- log_debug >- log_debug_var GX_PS_DERIVED_SOURCES_DIR >- log_debug >- log_debug_var GX_SUBGENERATE_ACTIONS >- log_debug_var GX_MERGE_ACTIONS >- log_debug_var GX_CHECK_ACTIONS >-} >- >- >-function set_project_settings() >-{ >- log_callstack_and_parameters "$@" >- >- [[ -z "${GX_PROJECT_TAG}" ]] && return >- >- set_project_specific_settings >- set_common_project_settings >- dump_project_settings >-} >- >- >-function ensure_directories_exist() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_DIR >- for GX_DIR in "$@" >- do >- log_debug "Creating ${GX_DIR}" >- mkdir -p "${GX_DIR}" &> /dev/null || die "Unable to create "${GX_DIR}"." >- done >-} >- >- >-function create_empty_files() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_FILE >- for GX_FILE in "$@" >- do >- ensure_directories_exist "$(dirname "${GX_FILE}")" >- >- log_debug "Creating ${GX_FILE}" >- echo -n '' > "${GX_FILE}" >- done >-} >- >- >-function append_xcfilelist_header() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_FILE >- for GX_FILE in "$@" >- do >- echo '# This file is generated by the generate-xcfilelists script.' >> "${GX_FILE}" >- done >-} >- >- >-function sort_and_unique_in_place() >-{ >- sort -u "$1" -o "$1" >-} >- >- >-function replace() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_FILE="$1" >- local GX_PATTERN="$2" >- local GX_REPLACEMENT="$3" >- >- [[ -n "${GX_FILE}" ]] || die "GX_FILE is not defined." >- [[ -f "${GX_FILE}" ]] || die "${GX_FILE} does not exist." >- >- sed -E -e "s|${GX_PATTERN}|${GX_REPLACEMENT}|" -i '' "${GX_FILE}" >-} >- >- >-function unexpand() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_FILE="$1" >- local GX_VAR="$2" >- local GX_VALUE=$(eval echo \$${GX_VAR}) >- >- [[ -n "${GX_VALUE}" ]] && replace "${GX_FILE}" "^${GX_VALUE}/" "\$\($GX_VAR\)/" >-} >- >- >-function find_additions() >-{ >- # Find additions and removals so that we can report them. >- # >- # UPDATE: We can't really report removals. It's possible -- even >- # overwhelmingly likely -- that we are generating .xcfilelist information >- # for only a subset of the possible configurations for which we build. For >- # example, a developer may be building for macOS and iOS, but not watchOS. >- # In situations like this, we'll be generating the .xcfilelist for those >- # two platforms, but any files specific to watchOS will not be discovered >- # and won't be included in the .xcfilelists. If we were to report those >- # watchOS files as files that should be removed from the .xcfilelists, then >- # the watchOS build may fail. So, for now, let any obsolete files >- # accumulate in the .xcfilelist files; they're mostly harmless. >- >- local GX_NEW="$1" >- local GX_ORIG="$2" >- local GX_ADDITIONS="$3" >- >- [[ -f "${GX_NEW}" ]] || die "${GX_NEW} does not exist." >- [[ -f "${GX_ORIG}" ]] || die "${GX_ORIG} does not exist." >- >- comm -2 -3 <(sort "${GX_NEW}") <(sort "${GX_ORIG}") > "${GX_ADDITIONS}" >-} >- >- >-function get_sdks() >-{ >- log_callstack_and_parameters "$@" >- >- (( ${#GX_SDKS[@]} )) && return >- >- GX_SDKS=($( >- xcodebuild -showsdks \ >- | grep -e '\s-sdk\s' \ >- | sed -E \ >- -e 's/.*-sdk ([^[:digit:]]+)$/\1/' \ >- -e 's/.*-sdk ([^[:digit:]]+)([[:digit:]\.]+)$/\1/' \ >- -e 's/.*-sdk ([^[:digit:]]+)([[:digit:]\.]+)([^[:digit:]]+)$/\1.\3/' \ >- | sort -u >- )) >- >- (( ${#GX_SDKS[@]} )) || die "Unable to find any SDKs." >-} >- >- >-function get_canonical_configuration() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_PROVISIONAL_CONFIGURATION=$(echo "$1" | tr '[:upper:]' '[:lower:]') >- local GX_RESULT=$2 >- >- [[ "${GX_PROVISIONAL_CONFIGURATION}" == "" ]] && { eval $GX_RESULT=""; return; } >- >- [[ "${GX_PROVISIONAL_CONFIGURATION}" == "debug" ]] && { eval $GX_RESULT="Debug"; return; } >- [[ "${GX_PROVISIONAL_CONFIGURATION}" == "release" ]] && { eval $GX_RESULT="Release"; return; } >- [[ "${GX_PROVISIONAL_CONFIGURATION}" == "production" ]] && { eval $GX_RESULT="Production"; return; } >- [[ "${GX_PROVISIONAL_CONFIGURATION}" == "profiling" ]] && { eval $GX_RESULT="Profiling"; return; } >- >- die "Unrecognized configuration: $1" >-} >- >- >-function get_canonical_platform_name() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_PROVISIONAL_PLATFORM_NAME=$(echo "$1" | tr '[:upper:]' '[:lower:]') >- local GX_RESULT=$2 >- >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "" ]] && { eval $GX_RESULT=""; return; } >- >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "ios" ]] && { eval $GX_RESULT="iphoneos"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "iphone" ]] && { eval $GX_RESULT="iphoneos"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "ipad" ]] && { eval $GX_RESULT="iphoneos"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "iphoneos" ]] && { eval $GX_RESULT="iphoneos"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "iphonesimulator" ]] && { eval $GX_RESULT="iphonesimulator"; return; } >- >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "mac" ]] && { eval $GX_RESULT="macosx"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "osx" ]] && { eval $GX_RESULT="macosx"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "macos" ]] && { eval $GX_RESULT="macosx"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "macosx" ]] && { eval $GX_RESULT="macosx"; return; } >- >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "tvos" ]] && { eval $GX_RESULT="appletvos"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "appletvos" ]] && { eval $GX_RESULT="appletvos"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "tvsimulator" ]] && { eval $GX_RESULT="appletvsimulator"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "appletvsimulator" ]] && { eval $GX_RESULT="appletvsimulator"; return; } >- >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "watchos" ]] && { eval $GX_RESULT="watchos"; return; } >- [[ "${GX_PROVISIONAL_PLATFORM_NAME}" == "watchsimulator" ]] && { eval $GX_RESULT="watchsimulator"; return; } >- >- die "Unrecognized platform name: $1" >-} >- >- >-function get_sdk_name() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_SDK=$1 >- local GX_RESULT=$2 >- get_canonical_platform_name "${GX_SDK}" GX_SDK >- local GX_INTERNAL_SDK="${GX_SDK}.internal" >- >- get_sdks >- >- # Prefer an internal SDK if one exists. >- >- [[ " ${GX_SDKS[@]} " =~ " ${GX_INTERNAL_SDK} " ]] && { eval $GX_RESULT="${GX_INTERNAL_SDK}"; return; } >- [[ " ${GX_SDKS[@]} " =~ " ${GX_SDK} " ]] && { eval $GX_RESULT="${GX_SDK}"; return; } >- >- die "Unsupported SDK: ${GX_SDK}." >-} >- >- >-function invoke_each_action() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_ACTION >- for GX_ACTION in "$@" >- do >- eval "${GX_ACTION}" >- done >-} >- >- >-function for_each_configuration() >-{ >- log_callstack_and_parameters "$@" >- >- if [[ -z "${GX_CONFIGURATION}" ]] >- then >- for GX_CONFIGURATION in "${GX_CONFIGURATIONS[@]}" >- do >- eval "$@" >- done >- >- GX_CONFIGURATION= >- else >- eval "$@" >- fi >-} >- >- >-function for_each_platform() >-{ >- log_callstack_and_parameters "$@" >- >- if [[ -z "${GX_PLATFORM_NAME}" ]] >- then >- for GX_PLATFORM_NAME in "${GX_PS_PLATFORM_NAMES[@]}" >- do >- eval "$@" >- done >- >- GX_PLATFORM_NAME= >- else >- eval "$@" >- fi >-} >- >- >-function for_each_project() >-{ >- log_callstack_and_parameters "$@" >- >- if [[ -z "${GX_PROJECT_TAG}" ]] >- then >- for GX_PROJECT_TAG in "${GX_PROJECT_TAGS[@]}" >- do >- set_project_settings >- eval "$@" >- done >- >- GX_PROJECT_TAG= >- else >- set_project_settings >- eval "$@" >- fi >-} >- >- >-function generate_derivedsources_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- create_empty_files "${GX_TEMP_INPUT_XCFILELIST}" "${GX_TEMP_OUTPUT_XCFILELIST}" >- >- local GX_DERIVEDSOURCES_EXTRACTOR="${GX_HERE}/extract-dependencies-from-makefile" >- >- [[ -n "${GX_DERIVEDSOURCES_GENERATOR}" ]] || die "GX_DERIVEDSOURCES_GENERATOR is not defined." >- [[ -x "${GX_DERIVEDSOURCES_GENERATOR}" ]] || die "${GX_DERIVEDSOURCES_GENERATOR} does not exist or is not executable." >- [[ -x "${GX_DERIVEDSOURCES_EXTRACTOR}" ]] || die "${GX_DERIVEDSOURCES_EXTRACTOR} does not exist or is not executable." >- >- log_debug "Creating derived .xcfilelists for ${PROJECT_NAME}/${PLATFORM_NAME}/${CONFIGURATION}" >- >- "${GX_DERIVEDSOURCES_GENERATOR}" \ >- NO_SUPPLEMENTAL_FILES=1 \ >- --no-builtin-rules \ >- --dry-run \ >- --always-make \ >- --debug=abvijm all \ >- 1> "${GX_TEMP}/std.out" 2> "${GX_TEMP}/std.err" >- local GX_RESULT=$? >- >- if (( ${GX_RESULT} )) >- then >- sed -E -e 's/^/GXCF: /' < "${GX_TEMP}/std.err" >- die "Error generating derived sources: error = ${GX_RESULT}" ${GX_RESULT} >- fi >- >- cat "${GX_TEMP}/std.out" \ >- | "${GX_DERIVEDSOURCES_EXTRACTOR}" \ >- --input "${GX_TEMP_INPUT_XCFILELIST}" \ >- --output "${GX_TEMP_OUTPUT_XCFILELIST}" >- local GX_RESULT=$? >- >- (( ${GX_RESULT} )) && die "Error extracting dependencies from DerivedSources.make: error = ${GX_RESULT}" >- [[ -f "${GX_TEMP_INPUT_XCFILELIST}" ]] || die "${GX_TEMP_INPUT_XCFILELIST} was not generated." >- [[ -f "${GX_TEMP_OUTPUT_XCFILELIST}" ]] || die "${GX_TEMP_OUTPUT_XCFILELIST} was not generated." >- >- replace "${GX_TEMP_INPUT_XCFILELIST}" '^WebCore/' '$(PROJECT_DIR)/' >- replace "${GX_TEMP_INPUT_XCFILELIST}" '^JavaScriptCore/' '$(PROJECT_DIR)/' >- replace "${GX_TEMP_INPUT_XCFILELIST}" '^JavaScriptCorePrivateHeaders/' '$(JAVASCRIPTCORE_PRIVATE_HEADERS_DIR)/' >- replace "${GX_TEMP_INPUT_XCFILELIST}" '^WebKit2PrivateHeaders/' '$(WEBKIT2_PRIVATE_HEADERS_DIR)/' >- unexpand "${GX_TEMP_INPUT_XCFILELIST}" PROJECT_DIR >- unexpand "${GX_TEMP_INPUT_XCFILELIST}" WEBKITADDITIONS_HEADERS_FOLDER_PATH >- unexpand "${GX_TEMP_INPUT_XCFILELIST}" WEBCORE_PRIVATE_HEADERS_DIR >- unexpand "${GX_TEMP_INPUT_XCFILELIST}" WEBKIT2_PRIVATE_HEADERS_DIR >- unexpand "${GX_TEMP_INPUT_XCFILELIST}" JAVASCRIPTCORE_PRIVATE_HEADERS_DIR >- unexpand "${GX_TEMP_INPUT_XCFILELIST}" BUILT_PRODUCTS_DIR >- >- replace "${GX_TEMP_OUTPUT_XCFILELIST}" "^" "${GX_PS_DERIVED_SOURCES_DIR}/" >- unexpand "${GX_TEMP_OUTPUT_XCFILELIST}" BUILT_PRODUCTS_DIR >- >- merge_xcfilelists_helper "${GX_TEMP_INPUT_XCFILELIST}" "${GX_INPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" >- merge_xcfilelists_helper "${GX_TEMP_OUTPUT_XCFILELIST}" "${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" >-} >- >- >-function generate_unifiedsources_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- create_empty_files "${GX_TEMP_OUTPUT_XCFILELIST}" >- >- [[ -n "${GX_UNIFIEDSOURCES_GENERATOR}" ]] || die "GX_UNIFIEDSOURCES_GENERATOR is not defined." >- [[ -x "${GX_UNIFIEDSOURCES_GENERATOR}" ]] || die "${GX_UNIFIEDSOURCES_GENERATOR} does not exist or is not executable." >- >- local GX_BUILD_SCRIPTS_DIR="${GX_OPENSOURCE_DIR}/Source/WTF/Scripts" >- [[ -d "${GX_BUILD_SCRIPTS_DIR}" ]] || die "${GX_BUILD_SCRIPTS_DIR} does not exist or is not a directory." >- [[ -f "${GX_BUILD_SCRIPTS_DIR}/generate-unified-source-bundles.rb" ]] || die "${GX_BUILD_SCRIPTS_DIR}/generate-unified-source-bundles.rb does not exist or is not a file." >- >- log_debug "Creating unified .xcfilelists for ${PROJECT_NAME}/${PLATFORM_NAME}/${CONFIGURATION}" >- >- BUILD_SCRIPTS_DIR="${GX_BUILD_SCRIPTS_DIR}" \ >- "${GX_UNIFIEDSOURCES_GENERATOR}" \ >- --generate-xcfilelists \ >- --output-xcfilelist-path "${GX_TEMP_OUTPUT_XCFILELIST}" >- GX_RESULT=$? >- >- (( ${GX_RESULT} )) && die "Error generating unified sources: error = ${GX_RESULT}" >- [[ -f "${GX_TEMP_OUTPUT_XCFILELIST}" ]] || die "${GX_TEMP_OUTPUT_XCFILELIST} was not generated." >- >- unexpand "${GX_TEMP_OUTPUT_XCFILELIST}" BUILT_PRODUCTS_DIR >- >- merge_xcfilelists_helper "${GX_TEMP_OUTPUT_XCFILELIST}" "${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_TEMP_PATH}" >-} >- >- >-function subgenerate_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- invoke_each_action "${GX_SUBGENERATE_ACTIONS[@]}" >-} >- >- >-function sublaunch_under_xcode() >-{ >- log_callstack_and_parameters "$@" >- >- # Sublaunch the script for the given project, for each platform for which >- # the project is built, and for Debug and Release configurations. >- >- local GX_ARGS=("$@") >- (( ${GX_DRY_RUN} )) && GX_ARGS+=("--dry-run") >- (( ${GX_DEBUG} )) && { for n in $(seq $GX_DEBUG); do GX_ARGS+=("--debug"); done; } >- (( ${GX_QUIET} )) && GX_ARGS+=("--quiet") >- >- local GX_SDK_NAME >- get_sdk_name "${GX_PLATFORM_NAME}" GX_SDK_NAME >- >- log_debug "Sublaunching for: ${GX_PROJECT_TAG}/${GX_PLATFORM_NAME}/${GX_CONFIGURATION}" >- >- local GX_XCODE_PARAMETERS=( >- -project "${GX_PS_PROJECT_FILE_PATH}" >- -sdk "${GX_SDK_NAME}" >- -configuration "${GX_CONFIGURATION}" >- -target "Apply Configuration to XCFileLists" >- ) >- >- if (( ${GX_USE_WEBKITBUILD_BUILD_OUTPUT} )) >- then >- local GX_WEBKITBUILD_DIR="${WEBKIT_OUTPUTDIR:-${GX_OPENSOURCE_DIR}/WebKitBuild}" >- [[ -d "${GX_WEBKITBUILD_DIR}" ]] || die "${GX_WEBKITBUILD_DIR} does not exist." >- >- GX_XCODE_PARAMETERS+=( >- SYMROOT="${GX_WEBKITBUILD_DIR}" >- OBJROOT="${GX_WEBKITBUILD_DIR}" >- SHARED_PRECOMPS_DIR="${GX_WEBKITBUILD_DIR}/PrecompiledHeaders" >- ) >- fi >- >- if (( $GX_DEBUG > 0 )) >- then >- WK_SUBLAUNCH_SCRIPT_PARAMETERS=("${GX_ARGS[@]}") xcodebuild "${GX_XCODE_PARAMETERS[@]}" >- else >- WK_SUBLAUNCH_SCRIPT_PARAMETERS=("${GX_ARGS[@]}") xcodebuild "${GX_XCODE_PARAMETERS[@]}" | grep '^\.\.\. ' >- fi >- >- local GX_RESULT=${PIPESTATUS[0]} >- (( ${GX_RESULT} )) && die "Error sub-launching under xcode: error = ${GX_RESULT}" >-} >- >- >-function reset_project_specific_temp_files() >-{ >- log_callstack_and_parameters "$@" >- >- create_empty_files \ >- "${GX_INPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" "${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" \ >- "${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_TEMP_PATH}" >-} >- >- >-function generate_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- reset_project_specific_temp_files >- >- for_each_platform \ >- for_each_configuration \ >- sublaunch_under_xcode \ >- "${GX_SUBEXECUTE_SCRIPT}" subgenerate --nocleanup --project "${GX_PROJECT_TAG}" >-} >- >- >-function merge_xcfilelists_helper() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_SOURCE="$1" >- local GX_DEST="$2" >- local GX_ADDITIONS="${GX_TEMP}/diff_added.tmp" >- >- append_xcfilelist_header "${GX_SOURCE}" >- sort_and_unique_in_place "${GX_SOURCE}" >- >- find_additions "${GX_SOURCE}" "${GX_DEST}" "${GX_ADDITIONS}" >- >- if [[ -s "${GX_ADDITIONS}" ]] >- then >- cat "${GX_SOURCE}" >> "${GX_DEST}" >- sort_and_unique_in_place "${GX_DEST}" >- return 0 >- fi >- >- return 1 >-} >- >- >-function merge_derivedsources_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- log_debug "Merging ${GX_INPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH} into ${GX_INPUT_DERIVED_XCFILELIST_PROJECT_PATH}" >- log_debug "Merging ${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH} into ${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_PATH}" >- >- if (( ! ${GX_DRY_RUN} )) >- then >- merge_xcfilelists_helper "${GX_INPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" "${GX_INPUT_DERIVED_XCFILELIST_PROJECT_PATH}" && GX_DEFERRED_EXIT_CODE=3 >- log_debug "GX_DEFERRED_EXIT_CODE = $GX_DEFERRED_EXIT_CODE" >- >- merge_xcfilelists_helper "${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" "${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_PATH}" && GX_DEFERRED_EXIT_CODE=3 >- log_debug "GX_DEFERRED_EXIT_CODE = $GX_DEFERRED_EXIT_CODE" >- fi >-} >- >- >-function merge_unifiedsources_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- log_debug "Merging ${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_TEMP_PATH} into ${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_PATH}" >- >- if (( ! ${GX_DRY_RUN} )) >- then >- merge_xcfilelists_helper "${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_TEMP_PATH}" "${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_PATH}" && GX_DEFERRED_EXIT_CODE=3 >- log_debug "GX_DEFERRED_EXIT_CODE = $GX_DEFERRED_EXIT_CODE" >- fi >-} >- >- >-function merge_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- invoke_each_action "${GX_MERGE_ACTIONS[@]}" >- >- log_debug "GX_DEFERRED_EXIT_CODE = $GX_DEFERRED_EXIT_CODE" >-} >- >- >-function check_xcfilelists_helper() >-{ >- log_callstack_and_parameters "$@" >- >- local GX_NEW="$1" >- local GX_ORIG="$2" >- local GX_ADDITIONS="${GX_TEMP}/diff_added.tmp" >- >- log_debug "Checking ${GX_NEW} against ${GX_ORIG}" >- >- find_additions "${GX_NEW}" "${GX_ORIG}" "${GX_ADDITIONS}" >- >- if [[ -s "${GX_ADDITIONS}" ]] >- then >- log_progress >- log_progress "------------------------------------------------------------------------------" >- log_progress "Found added files for ${GX_ORIG}:" >- log_progress "------------------------------------------------------------------------------" >- >- local GX_LINE >- while IFS='' read -r GX_LINE >- do >- log_progress "${GX_LINE}" >- done < "${GX_ADDITIONS}" >- >- log_progress "------------------------------------------------------------------------------" >- >- GX_DEFERRED_EXIT_CODE=2 >- fi >-} >- >- >-function check_derivedsources_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- check_xcfilelists_helper "${GX_INPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" "${GX_INPUT_DERIVED_XCFILELIST_PROJECT_PATH}" >- check_xcfilelists_helper "${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_TEMP_PATH}" "${GX_OUTPUT_DERIVED_XCFILELIST_PROJECT_PATH}" >-} >- >- >-function check_unifiedsources_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- check_xcfilelists_helper "${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_TEMP_PATH}" "${GX_OUTPUT_UNIFIED_XCFILELIST_PROJECT_PATH}" >-} >- >- >-function check_xcfilelists() >-{ >- log_callstack_and_parameters "$@" >- >- invoke_each_action "${GX_CHECK_ACTIONS[@]}" >-} >- >- >-function report_merge_results() >-{ >- log_callstack_and_parameters "$@" >- >- log_debug "GX_DEFERRED_EXIT_CODE = $GX_DEFERRED_EXIT_CODE" >- >- if (( ${GX_DEFERRED_EXIT_CODE} )) >- then >- log_progress >- >- local GX_MESSAGE='".xcfilelist" files tell the build system what files >- are consumed and produced by the "Run Script" build phases in >- Xcode. At least one of these .xcfilelist files was out of date and >- has been updated. You now need to restart your build.' >- >- log_progress_long "${GX_MESSAGE}" >- fi >-} >- >- >-function report_remediation_steps() >-{ >- log_callstack_and_parameters "$@" >- >- if (( ${GX_DEFERRED_EXIT_CODE} )) >- then >- log_progress >- >- # We can either be called from within an xcodebuild context (meaning >- # that we are being called during a build to validate the .xcfilelists >- # for the current build configuration) or from the command line (to >- # validate all .xcfilelists). We'll determine which of these is the >- # case by checking $GX_PROJECT_TAG (which will have been set via the >- # --project <TAG> option passed in to us when we're invoked in this >- # context). >- # >- # Further, if called from within xcodebuild, it's important to know if >- # we're being called from within Xcode or being invoked by a Makefile. >- # If the former, the output/build files we are looking for are found in >- # ~/Library/Developer/Xcode/DerivedData. Otherwise, they'll be found in >- # OpenSource/WebKitBuild (or whatever $WEBKIT_OUTPUTDIR points to). We >- # need to distinguish between these so that we can tell the user how to >- # invoke us in a way that finds the right output/build files. >- # >- # If invoked from the command line, we really can't tell where the >- # output files are, so give the user advice in this area. >- >- local GX_MESSAGE='".xcfilelist" files tell the build system what files >- are consumed and produced by the "Run Script" build phases in >- Xcode. At least one of these .xcfilelist files are out of date and >- need to be regenerated. Regenerate these files by running >- `Tools/Scripts/generate-xcfilelists generate' >- >- if [[ -n "${GX_PROJECT_TAG}" ]] >- then >- local GX_MESSAGE+=" --project ${GX_PROJECT_TAG}" >- if [[ "${SYMROOT}" =~ Library/Developer/Xcode/DerivedData ]] >- then >- local GX_MESSAGE+=" --xcode" >- fi >- else >- if (( ${GX_USE_XCODE_BUILD_OUTPUT} )) >- then >- local GX_MESSAGE+=" --xcode" >- fi >- fi >- >- local GX_MESSAGE+='`, or manually add the file or files shown above to >- the indicated .xcfilelist files. Then restart your build. When >- submitting, include the updated .xcfilelist files.' >- >- log_progress_long "${GX_MESSAGE}" >- fi >-} >- >- >-function do_generate() >-{ >- # Invoked from the command line to generate .xcfilelist files. Use any >- # specified project, platform, and/or configuration. Any of those that >- # aren't specified results in our iterating over all possible values for >- # the unspecified value. >- >- log_callstack_and_parameters "$@" >- >- if [[ -z "${GX_PROJECT_TAG}" ]] >- then >- log_progress "=== Generating all .xcfilelists ===" >- else >- log_progress "=== Generating .xcfilelists for ${GX_PROJECT_TAG} ===" >- fi >- >- for_each_project \ >- generate_xcfilelists >-} >- >- >-function do_merge() >-{ >- # Implicitly invoked for generate and generate-xcode operations to move the >- # temporary results into the Xcode project. >- >- log_callstack_and_parameters "$@" >- >- if [[ -z "${GX_PROJECT_TAG}" ]] >- then >- log_progress "=== Merging all .xcfilelists into place ===" >- else >- log_progress "=== Merging .xcfilelists for ${GX_PROJECT_TAG} ===" >- fi >- >- for_each_project \ >- merge_xcfilelists >- >- log_debug "GX_DEFERRED_EXIT_CODE = $GX_DEFERRED_EXIT_CODE" >-} >- >- >-function do_check() >-{ >- # Implicitly invoked for check and check-xcode operations to check the >- # temporary results against the Xcode project. >- >- log_callstack_and_parameters "$@" >- >- # Being invoked from the command-line to check everything. >- >- if [[ -z "${GX_PROJECT_TAG}" ]] >- then >- log_progress "=== Checking all .xcfilelists ===" >- else >- log_progress "=== Checking .xcfilelists for ${GX_PROJECT_TAG} ===" >- fi >- >- for_each_project \ >- check_xcfilelists >-} >- >- >-function do_subgenerate() >-{ >- # Invoked from within an Xcode context to generate .xcfilelist files. Use >- # the project, platform, an configuration established in the environment. >- >- log_callstack_and_parameters "$@" >- >- [[ -n "${GX_PROJECT_TAG}" ]] || die "GX_PROJECT_TAG is not defined." >- [[ -n "${PROJECT_NAME}" ]] || die "subgenerate should only be invoked in an Xcode context." >- >- log_progress "=== Generating .xcfilelists for ${PROJECT_NAME}/${PLATFORM_NAME}/${CONFIGURATION} ===" >- >- set_project_settings >- >- # We're called during generate-xcode, check-xcode, and subgenerate >- # operations. We need to reset our project-specific temp files for the >- # first two, but not the last one. >- >- (( ${GX_DO_SUBGENERATE} )) || reset_project_specific_temp_files >- >- subgenerate_xcfilelists >-} >- >- >-function main() >-{ >- log_callstack_and_parameters "$@" >- >- log_debug_var GX_ME >- log_debug_var GX_HERE >- log_debug_var GX_ROOT_DIR >- log_debug_var GX_OPENSOURCE_DIR >- log_debug_var GX_TEMP >- log_debug_var GX_PROJECT_TAGS >- log_debug_var GX_CONFIGURATIONS >- log_debug_var GX_SUBEXECUTE_SCRIPT >- log_debug_var GX_DO_GENERATE >- log_debug_var GX_DO_GENERATE_XCODE >- log_debug_var GX_DO_CHECK >- log_debug_var GX_DO_CHECK_XCODE >- log_debug_var GX_DO_SUBGENERATE >- log_debug_var GX_DO_CLEANUP >- log_debug_var GX_USE_XCODE_BUILD_OUTPUT >- log_debug_var GX_USE_WEBKITBUILD_BUILD_OUTPUT >- log_debug_var GX_DRY_RUN >- log_debug_var GX_DEBUG >- log_debug_var GX_QUIET >- log_debug_var GX_ORIG_ARGS >- >- if (( ${GX_DO_HELP} )) >- then >- usage >- my_exit 0 >- fi >- >- if (( GX_DO_GENERATE + GX_DO_GENERATE_XCODE + GX_DO_CHECK + GX_DO_CHECK_XCODE + GX_DO_SUBGENERATE + GX_DO_HELP < 1 )) >- then >- stderr "### One of generate, generate-xcode, check, check-xcode, subgenerate, or --help must be specified." >- my_exit 1 >- fi >- >- if (( GX_DO_GENERATE + GX_DO_GENERATE_XCODE + GX_DO_CHECK + GX_DO_CHECK_XCODE + GX_DO_SUBGENERATE + GX_DO_HELP > 1 )) >- then >- stderr "### Only one of generate, generate-xcode, check, check-xcode, subgenerate, or --help can be specified." >- my_exit 1 >- fi >- >- [[ -z "${GX_PROJECT_TAG}" || ( " ${GX_PROJECT_TAGS[@]} " =~ " ${GX_PROJECT_TAG} " ) ]] || { die "Unrecognized project: ${GX_PROJECT_TAG}"; } >- get_canonical_platform_name "${GX_PLATFORM_NAME}" GX_PLATFORM_NAME >- get_canonical_configuration "${GX_CONFIGURATION}" GX_CONFIGURATION >- >- if (( ${GX_DO_GENERATE} )) >- then >- do_generate >- call post_generate_hook >- >- do_merge >- call post_merge_hook >- >- report_merge_results >- elif (( ${GX_DO_GENERATE_XCODE} )) >- then >- do_subgenerate >- call post_generate_hook >- >- do_merge >- call post_merge_hook >- >- report_merge_results >- elif (( ${GX_DO_CHECK} )) >- then >- do_generate >- call post_generate_hook >- >- do_check >- call post_check_hook >- >- report_remediation_steps >- elif (( ${GX_DO_CHECK_XCODE} )) >- then >- do_subgenerate >- call post_generate_hook >- >- do_check >- call post_check_hook >- >- report_remediation_steps >- elif (( ${GX_DO_SUBGENERATE} )) >- then >- do_subgenerate >- else >- stderr "### No subcommand provided" >- usage >- my_exit 1 >- fi >- >- my_exit ${GX_DEFERRED_EXIT_CODE} >-} >- >- >-# Initialize and sanity check >- >-GX_ORIG_ARGS=("$@") >- >-GX_DO_GENERATE=0 >-GX_DO_GENERATE_XCODE=0 >-GX_DO_CHECK=0 >-GX_DO_CHECK_XCODE=0 >-GX_DO_SUBGENERATE=0 >-GX_DO_HELP=0 >-GX_DO_CLEANUP=0 >-GX_USE_XCODE_BUILD_OUTPUT=0 >-GX_USE_WEBKITBUILD_BUILD_OUTPUT=1 >-GX_DRY_RUN=0 >-GX_DEBUG=0 >-GX_QUIET=0 >- >-GX_DEFERRED_EXIT_CODE=0 >-GX_PROJECT_TAG= >-GX_PLATFORM_NAME= >-GX_CONFIGURATION= >-GX_SDKS=() >- >-GX_ME=$(normalize_file_path "${BASH_SOURCE[0]}") >-GX_HERE=$(dirname "${GX_ME}") >-GX_ROOT_DIR=$(normalize_directory_path "${GX_HERE}/../../..") >-GX_OPENSOURCE_DIR="${GX_ROOT_DIR}/OpenSource" >-GX_TEMP=$(normalize_directory_path /tmp/generate-xcfilelists) >-GX_PROJECT_TAGS=(JavaScriptCore WebCore WebKit DumpRenderTree WebKitTestRunner) >-GX_CONFIGURATIONS=(Release Debug) >-GX_SUBEXECUTE_SCRIPT="${GX_ME}" >- >-[[ -n "${GX_ROOT_DIR}" ]] || die "Could not find GX_ROOT_DIR." >-[[ -n "${GX_OPENSOURCE_DIR}" ]] || die "Could not find GX_OPENSOURCE_DIR." >-[[ -d "${GX_ROOT_DIR}" ]] || die "${GX_ROOT_DIR} does not exist." >-[[ -d "${GX_OPENSOURCE_DIR}" ]] || die "${GX_OPENSOURCE_DIR} does not exist." >- >-trap cleanup EXIT >- >-ensure_directories_exist "${GX_TEMP}" >- >- >-# Process command-line parameters. >- >-while [[ "${1:+x}" ]] >-do >- case "${1}" in >- generate) GX_DO_GENERATE=1 ;; >- generate-xcode) GX_DO_GENERATE_XCODE=1 ;; >- check) GX_DO_CHECK=1 ;; >- check-xcode) GX_DO_CHECK_XCODE=1 ;; >- subgenerate) GX_DO_SUBGENERATE=1 ;; >- help) GX_DO_HELP=1 ;; >- >- --project) shift; GX_PROJECT_TAG="$1" ;; >- --platform) shift; GX_PLATFORM_NAME="$1" ;; >- --configuration)shift; GX_CONFIGURATION="$1" ;; >- --xcode) GX_USE_XCODE_BUILD_OUTPUT=1; GX_USE_WEBKITBUILD_BUILD_OUTPUT=0 ;; >- --webkitbuild) GX_USE_XCODE_BUILD_OUTPUT=0; GX_USE_WEBKITBUILD_BUILD_OUTPUT=1 ;; >- --nocleanup) GX_DO_CLEANUP=0 ;; >- >- -n | --dry-run) GX_DRY_RUN=1 ;; >- -d | --debug) GX_DEBUG=$(( $GX_DEBUG + 1 )) ;; >- -q | --quiet) GX_QUIET=1 ;; >- -h | --help) GX_DO_HELP=1 ;; >- >- *) stderr "### Unknown command: $1" >- usage >- my_exit 1 ;; >- esac >- shift >-done >- >- >-GX_SOURCED=$([[ "$0" == "${BASH_SOURCE[@]}" ]] && echo 0 || echo 1) >-if (( ! ${GX_SOURCED} )) >-then >- GX_INTERNAL_ME="${GX_ROOT_DIR}/Internal/Tools/Scripts/generate-xcfilelists" >- if [[ -x "${GX_INTERNAL_ME}" ]] >- then >- "${GX_INTERNAL_ME}" "${GX_ORIG_ARGS[@]}" >- else >- main >- fi >-fi >+# particular, it does this in order to set needed environment variables. >+ >+import os >+import sys >+ >+# Relaunch ourselves if the Internal version of generator-xcfilelists exists. >+ >+ROOT_DIR = os.path.dirname( # Remove "OpenSource" >+ os.path.dirname( # Remove "Tools" >+ os.path.dirname( # Remove "Scripts" >+ os.path.dirname( # Remove script name >+ os.path.abspath(__file__))))) >+INTERNAL_SCRIPTS = os.path.join(ROOT_DIR, "Internal", "Tools", "Scripts") >+INTERNAL_GENERATE_XCFILELISTS = os.path.join(INTERNAL_SCRIPTS, os.path.basename(__file__)) >+if os.path.exists(INTERNAL_GENERATE_XCFILELISTS): >+ os.execve(INTERNAL_GENERATE_XCFILELISTS, >+ [INTERNAL_GENERATE_XCFILELISTS] + sys.argv[1:], >+ os.environ) >+ assert False >+ >+from webkitpy.generate_xcfilelists_lib.application import Application >+ >+if __name__ == "__main__": >+ sys.exit(Application(__file__).run()) >diff --git a/Tools/Scripts/webkitpy/generate_xcfilelists_lib/__init__.py b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/__init__.py >new file mode 100644 >index 0000000000000000000000000000000000000000..46d8cebac4e93608cef28f2e17c4973191339a28 >--- /dev/null >+++ b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/__init__.py >@@ -0,0 +1,26 @@ >+#!/usr/bin/env python >+# -*- coding: utf-8 -*- >+# >+# Copyright (C) 2019 Apple Inc. All rights reserved. >+# >+# Redistribution and use in source and binary forms, with or without >+# modification, are permitted provided that the following conditions >+# are met: >+# >+# 1. Redistributions of source code must retain the above copyright >+# notice, this list of conditions and the following disclaimer. >+# 2. Redistributions in binary form must reproduce the above copyright >+# notice, this list of conditions and the following disclaimer in the >+# documentation and/or other materials provided with the distribution. >+# >+# THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >diff --git a/Tools/Scripts/webkitpy/generate_xcfilelists_lib/application.py b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/application.py >new file mode 100644 >index 0000000000000000000000000000000000000000..e4477b85055d79892e38832fa35bd5dc7f4f7680 >--- /dev/null >+++ b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/application.py >@@ -0,0 +1,460 @@ >+#!/usr/bin/env python >+# -*- coding: utf-8 -*- >+# >+# Copyright (C) 2019 Apple Inc. All rights reserved. >+# >+# Redistribution and use in source and binary forms, with or without >+# modification, are permitted provided that the following conditions >+# are met: >+# >+# 1. Redistributions of source code must retain the above copyright >+# notice, this list of conditions and the following disclaimer. >+# 2. Redistributions in binary form must reproduce the above copyright >+# notice, this list of conditions and the following disclaimer in the >+# documentation and/or other materials provided with the distribution. >+# >+# THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ >+# Application represents the main operation of the script. It's a singleton, >+# created by main() and then invoked to run everything. This class parses the >+# command-line options, validates them, creates and invokes BaseGenerators, >+# reports the results, and handles the catching and reporting of any >+# exceptions/errors. >+ >+from __future__ import print_function >+ >+import argparse >+import itertools >+import os >+import sys >+import textwrap >+import traceback >+ >+from functools import reduce >+ >+import webkitpy.generate_xcfilelists_lib.generators as Generators >+import webkitpy.generate_xcfilelists_lib.util as util >+ >+ >+EX_GENERAL_ERROR = 1 # General error >+EX_ACTION_REQUIRED = 2 # Returned when script determines that the generated .xcfilelist files have changed. >+ >+ >+class Application(object): >+ >+ __slots__ = ( >+ "command_file", >+ "parser", "cmd_line_args", >+ "dispatch", "project_specific_generators", >+ "supported_project_tags", "supported_platforms", "supported_configurations") >+ >+ # Aliases for platforms, in the form (alias, canonical). These are handy >+ # when using the script from the command line and you don't remember if >+ # it's "ios" or "iphoneos, or "tvos" or "appletvos". This list of aliases >+ # will let you use any of those. >+ # >+ # This is implemented as a list of tuples as opposed to a dict in order to >+ # better support cases where the input does not match the capitalization >+ # provided here. >+ >+ platform_aliases = ( >+ ("ios", "iphoneos"), >+ ("iphone", "iphoneos"), >+ ("simulator", "iphonesimulator"), >+ ("sim", "iphonesimulator"), >+ ("mac", "macosx"), >+ ("macos", "macosx"), >+ ("osx", "macosx"), >+ # ("", "iosmac"), >+ ("tvos", "appletvos"), >+ ("tv", "appletvos"), >+ ("tvsimulator", "appletvsimulator"), >+ ("watch", "watchos"), >+ # ("", "watchsimulator"), >+ ) >+ >+ @util.LogEntryExit >+ def __init__(self, command_file): >+ self.command_file = os.path.realpath(command_file) >+ self.parser = None >+ self.cmd_line_args = None >+ >+ self.dispatch = { >+ "generate": self._cmd_generate, >+ "generate-xcode": self._cmd_generate, >+ "check": self._cmd_check, >+ "check-xcode": self._cmd_check, >+ "subgenerate": self._cmd_subgenerate, >+ "help": self._cmd_help, >+ } >+ >+ self.project_specific_generators = { >+ "JavaScriptCore": Generators.JavaScriptCoreGenerator, >+ "WebCore": Generators.WebCoreGenerator, >+ "WebKit": Generators.WebKitGenerator, >+ "DumpRenderTree": Generators.DumpRenderTreeGenerator, >+ "WebKitTestRunner": Generators.WebKitTestRunnerGenerator, >+ } >+ >+ self.supported_project_tags = None >+ self.supported_platforms = None >+ self.supported_configurations = None >+ >+ @util.LogEntryExit >+ def run(self): >+ try: >+ self._initialize() >+ >+ self.parser = self._create_parser() >+ self.cmd_line_args = args = self.parser.parse_args() >+ >+ if args.help: >+ return self._cmd_help(os.EX_OK) >+ >+ self._validate_args(args) >+ >+ try: >+ func = self.dispatch[args.command] >+ except KeyError as e: >+ raise util.InvalidCommandError(args.command) >+ >+ return func() >+ >+ except util.InvalidArgumentError as e: >+ print("### Invalid argument: {}".format(e)) >+ return self._cmd_help(os.EX_USAGE) >+ >+ except util.InvalidCommandError as e: >+ if e.args: >+ print("### Invalid command: {}".format(e)) >+ else: >+ print("### Missing command") >+ return self._cmd_help(os.EX_USAGE) >+ >+ except util.QuickExit as e: >+ return e.args[0] >+ >+ except BaseException as e: >+ traceback.print_exc() >+ return os.EX_SOFTWARE >+ >+ # Perform some post __init__ initialization. This is performed after >+ # __init__ so that we can respond to any information provided in any >+ # sub-class's __init__. >+ >+ @util.LogEntryExit >+ def _initialize(self): >+ def collect_attributes(key): >+ configurations = set() >+ for project_tag in self.project_specific_generators: >+ configurations |= set(key(self.project_specific_generators[project_tag])) >+ return sorted(list(configurations)) >+ >+ self.supported_project_tags = sorted(list(self.project_specific_generators.keys())) >+ self.supported_platforms = collect_attributes(lambda gen_cls: gen_cls.VALID_PLATFORMS) >+ self.supported_configurations = collect_attributes(lambda gen_cls: gen_cls.VALID_CONFIGURATIONS) >+ >+ @util.LogEntryExit >+ def _create_parser(self): >+ parser = argparse.ArgumentParser(add_help=False, >+ formatter_class=argparse.RawDescriptionHelpFormatter, >+ description="""\ >+Generate or check .xcfilelist files. One of the following commands must be >+specified on the command-line: >+ >+ generate Generate a complete and up-to-date set of .xcfilelist >+ files and copy them to their appropriate places in the >+ project directories. >+ generate-xcode Alias for 'generate'. Obsolete. >+ check Generate a complete and up-to-date set of .xcfilelist >+ files and compare them to their counterparts in the >+ project directories. >+ check-xcode Alias for 'check'. Obsolete. >+ subgenerate [Internal use only] Generate an .xcfilelist file for a >+ particular combination of project, platform, and >+ configuration. This operation is performed in the >+ context of an Xcode build in order to inherit the same >+ environment as that build. >+ help Print this text and exit.""") >+ >+ parser.add_argument("command", action="store", nargs="?", >+ help=argparse.SUPPRESS) >+ >+ parser.add_argument("--project", action="append", dest="project_tags", >+ metavar="<PROJECT>", help="""\ >+ Specify which project or projects for which to generate >+ .xcfilelist files or to check. Possible values are >+ ({}). Can be specified more than once. Default is to >+ iterate over all projects.""".format( >+ ", ".join(self.supported_project_tags))) >+ parser.add_argument("--platform", action="append", dest="platforms", >+ metavar="<PLATFORM>", help="""\ >+ Specify which platform or platforms for which to >+ generate .xcfilelist files or to check. Possible values >+ are ({}, plus common aliases). Can be specified more >+ than once. Default is to iterate over all platforms, >+ filtered to those platforms that a particular project >+ supports (e.g., you can't specify 'iphoneos' for >+ WebKitTestRunner).""".format( >+ ", ".join(self.supported_platforms))) >+ parser.add_argument("--configuration", action="append", >+ dest="configurations", metavar="<CONFIGURATION>", help="""\ >+ Specify which configuration or configurations for which >+ to generate .xcfilelist files or to check. Possible >+ values are ({}). Can be specified more than once. >+ Default is to iterate over all >+ configurations.""".format( >+ ", ".join(self.supported_configurations))) >+ parser.add_argument("--xcode", metavar="<WORKSPACE>", help="""\ >+ If the existing build output was created by building >+ with Xcode, specify the path to the workspace that was >+ used.""") >+ parser.add_argument("--pickle-file", help="""\ >+ [Internal use only] Name of the file used to store >+ results to be transported out from the Xcode execution >+ environment out to an outer later. This parameter is >+ only used with the 'subgenerate' command.""") >+ parser.add_argument("-d", "--debug", action="store_true", help="""\ >+ Provide verbose output.""") >+ parser.add_argument("--debug-file", help="""\ >+ [Internal use only] Name of the file to which to write >+ debug information. Used when this script sub-launches >+ itself and needs to collect the debug information from >+ the sub-launched instance. Not normally used when this >+ script is invoked from the command-line or Xcode.""") >+ parser.add_argument("-q", "--quiet", action="store_true", help="""\ >+ Don't print any standard output.""") >+ parser.add_argument("-h", "--help", action="store_true", help="""\ >+ Print this text and exit.""") >+ >+ return parser >+ >+ @util.LogEntryExit >+ def _validate_args(self, args): >+ def matches(a, b): >+ return a.casefold() == b.casefold() if util.PY3 else a.lower() == b.lower() >+ >+ def test_item_validity(our_items, valid_items, collection_name, aliases=None): >+ if our_items is None: >+ return valid_items >+ >+ output_items = list() >+ used_items = list() >+ >+ for our_item, valid_item in itertools.product(our_items, valid_items): >+ if matches(our_item, valid_item): >+ output_items.append(valid_item) >+ used_items.append(our_item) >+ elif aliases: >+ for alias, canonical in aliases: >+ if canonical == valid_item and matches(our_item, alias): >+ output_items.append(canonical) >+ used_items.append(our_item) >+ >+ unused_items = set(our_items) - set(used_items) >+ for unused_item in unused_items: >+ raise util.InvalidArgumentError("the {} \"{}\" is not supported.".format(collection_name, unused_item)) >+ >+ return list(set(output_items)) >+ >+ args.project_tags = test_item_validity(args.project_tags, self.supported_project_tags, "project") >+ args.platforms = test_item_validity(args.platforms, self.supported_platforms, "platform", self.platform_aliases) >+ args.configurations = test_item_validity(args.configurations, self.supported_configurations, "configuration") >+ >+ if not args.command: >+ raise util.InvalidCommandError() >+ >+ if args.command == "generate_xcode" or args.command == "check_xcode" or args.command == "subgenerate": >+ assert len(self.cmd_line_args.project_tags) == 1 >+ assert len(self.cmd_line_args.platforms) == 1 >+ assert len(self.cmd_line_args.configurations) == 1 >+ >+ @util.LogEntryExit >+ def _cmd_generate(self): >+ generators = self._do_generate() >+ generators = self._do_merge(generators) >+ return self._report_results(generators) >+ >+ @util.LogEntryExit >+ def _cmd_check(self): >+ generators = self._do_generate() >+ return self._report_results(generators) >+ >+ @util.LogEntryExit >+ def _cmd_subgenerate(self): >+ generators = self._do_generate() >+ with open(self.cmd_line_args.pickle_file, "wb") as f: >+ for generator in generators: >+ generator.pickle_to_file(f) >+ return os.EX_OK >+ >+ @util.LogEntryExit >+ def _cmd_help(self, status=os.EX_OK): >+ self.parser.print_help() >+ return status >+ >+ @util.LogEntryExit >+ def _do_generate(self): >+ generators = [] >+ >+ for triple in itertools.product( >+ self.cmd_line_args.project_tags, >+ self.cmd_line_args.platforms, >+ self.cmd_line_args.configurations): >+ generator = self.project_specific_generators[triple[0]](self, *triple) >+ try: >+ if not generator.is_valid(): >+ continue >+ >+ self._log_progress("Generating .xcfilelists for {}/{}/{}".format(*triple)) >+ >+ if not util.is_running_under_xcode(): >+ new_generators = generator.generate() >+ generators.extend(new_generators) >+ else: >+ generator.subgenerate() >+ generators.append(generator) >+ except BaseException as e: >+ # TODO: Turn the traceback into a string, and then allow this >+ # field to be pickled and printed by the calling context. >+ (generator.ex_type, generator.ex_value, generator.ex_traceback) = sys.exc_info() >+ if generator.has_error(): >+ raise util.QuickExit(self._report_results([generator])) >+ >+ return generators >+ >+ @util.LogEntryExit >+ def _do_merge(self, generators): >+ if self._any_have_errors(generators): >+ return generators >+ >+ for generator in generators: >+ if generator.has_action(): >+ self._log_progress("Merging .xcfilelists for {}/{}/{}".format(*generator.triple)) >+ generator.merge() >+ >+ return generators >+ >+ @util.LogEntryExit >+ def _report_results(self, generators): >+ generators_with_errors = [generator for generator in generators if generator.has_error()] >+ if generators_with_errors: >+ for generator in generators_with_errors: >+ generator.report_error() >+ return EX_GENERAL_ERROR >+ >+ generators_with_actions = [generator for generator in generators if generator.has_action()] >+ if generators_with_actions: >+ if self.cmd_line_args.command == "generate" or self.cmd_line_args.command == "generate-xcode": >+ self._report_merge_results(generators_with_actions) >+ else: >+ self._report_remediation_steps(generators_with_actions) >+ return EX_ACTION_REQUIRED >+ >+ return os.EX_OK >+ >+ @util.LogEntryExit >+ def _report_merge_results(self, generators): >+ message = textwrap.wrap( >+ "\".xcfilelist\" files tell the build system what files are " + >+ "consumed and produced by the \"Run Script\" build phases in " + >+ "Xcode. At least one of these .xcfilelist files was out of date " + >+ "and has been updated. You now need to restart your build.", 90) >+ >+ self._log_results("") >+ for line in message: >+ self._log_results(line) >+ >+ @util.LogEntryExit >+ def _report_remediation_steps(self, generators): >+ message = textwrap.wrap("One or more \".xcfilelist\" files are out of date. Regenerate them by running the following commands:", 90) >+ message.append("") >+ >+ def add_to_message(generator, message): >+ if generator.has_action(): >+ message.append(" `Tools/Scripts/generate-xcfilelists generate --project {} --platform {} --configuration {}{}`\n".format( >+ generator.project_tag, generator.platform, generator.configuration, >+ " --xcode {}".format(self.cmd_line_args.xcode) if self.cmd_line_args.xcode else "")) >+ return message >+ >+ for generator in generators: >+ message = add_to_message(generator, message) >+ >+ for line in message: >+ self._log_results(line) >+ >+ @util.LogEntryExit >+ def _any_have_errors(self, generators): >+ return reduce(lambda acc, generator: acc or generator.has_error(), generators, None) >+ >+ @util.LogEntryExit >+ def _any_have_actions(self, generators): >+ return reduce(lambda acc, generator: acc or generator.has_action(), generators, None) >+ >+ @util.LogEntryExit >+ def _log_progress(self, message): >+ if not self.cmd_line_args.quiet: >+ print("### {}".format(message)) >+ >+ @util.LogEntryExit >+ def _log_results(self, message): >+ if not self.cmd_line_args.quiet: >+ print("{}".format(message)) >+ >+ # Return the path to the script to sublaunch. >+ >+ @util.LogEntryExit >+ def _get_generate_xcfilelists_script_path(self): >+ return self.command_file >+ >+ # Return the parent of the WebKit check-out directory. >+ >+ @util.LogEntryExit >+ def _get_root_dir(self): >+ return os.path.dirname( # Remove "OpenSource" >+ os.path.dirname( # Remove "Tools" >+ os.path.dirname( # Remove "Scripts" >+ os.path.dirname( # Remove script name >+ self._get_generate_xcfilelists_script_path())))) >+ >+ # Return the path to the WebKit check-out directory. >+ >+ @util.LogEntryExit >+ def _get_opensource_dir(self): >+ return os.path.join(self._get_root_dir(), "OpenSource") >+ >+ # Return the path to the directory containing supporting build scripts. >+ >+ @util.LogEntryExit >+ def _get_build_scripts_dir(self): >+ return os.path.join(self._get_opensource_dir(), "Source", "WTF", "Scripts") >+ >+ # Return the path to a supporting build script. >+ >+ @util.LogEntryExit >+ def _get_extract_dependencies_from_makefile_script(self): >+ return os.path.join(self._get_opensource_dir(), "Tools", "Scripts", "extract-dependencies-from-makefile") >+ >+ # Return $(BUILT_PRODUCTS_DIR) >+ # aka $(CONFIGURATION_BUILD_DIR) >+ # aka $(BUILD_DIR)/$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME) >+ >+ @util.LogEntryExit >+ def _get_xcode_built_products_dir(self): >+ assert util.is_running_under_xcode() >+ return self._getenv("BUILT_PRODUCTS_DIR") >+ >+ # Return the named environment variable. >+ @util.LogEntryExit >+ def _getenv(self, variable_name): >+ return os.environ.get(variable_name) >diff --git a/Tools/Scripts/webkitpy/generate_xcfilelists_lib/generators.py b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/generators.py >new file mode 100644 >index 0000000000000000000000000000000000000000..159f45006c4f8eb25a1799022286eba0a2e857f2 >--- /dev/null >+++ b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/generators.py >@@ -0,0 +1,708 @@ >+#!/usr/bin/env python >+# -*- coding: utf-8 -*- >+# >+# Copyright (C) 2019 Apple Inc. All rights reserved. >+# >+# Redistribution and use in source and binary forms, with or without >+# modification, are permitted provided that the following conditions >+# are met: >+# >+# 1. Redistributions of source code must retain the above copyright >+# notice, this list of conditions and the following disclaimer. >+# 2. Redistributions in binary form must reproduce the above copyright >+# notice, this list of conditions and the following disclaimer in the >+# documentation and/or other materials provided with the distribution. >+# >+# THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ >+# BaseGenerator is the base class for generating new .xcfilelist content. Most >+# of the implementation is provided in this base class. Subclasses are created >+# to provide project-specific information, such as the path to the project file >+# and valid platform and configuration information. >+# >+# Instances of this class are create for each project/platform/configuration >+# triple. For each triple, .xcfilelist content is generated and compared to the >+# file that ultimately will contain that information. Any new lines are >+# determined and remembered for later addition to that destination file. >+# >+# Instances of this class can operate in either of two contexts. First, it's >+# possible for Xcode projects to invoke this script in order to keep their >+# .xcfilelist files up-to-date. In that case, this script starts out running in >+# the context of Xcode and it can just go ahead and generate the new content. >+# Alternatively, it's possible for someone to invoke this script from the >+# command line. In that case, this script needs to sublaunch itself in order to >+# expose itself to the environment variables that Xcode establishes during >+# builds. This script thus starts out as a free-standing script, creates a >+# BaseGenerator for the .xcfilelist content it needs to create, sub-launches >+# itself, and then creates another BaseGenerator to do the actual generation. >+# When this inner instance of the script exits, it writes the results of the >+# BaseGenerators to a temporary file. When the outer instance of the script >+# resumes, it reads the information from that temporary file. That information >+# is actually just a serialized (pickled) BaseGenerator instance and is >+# restored as such. This restored BaseGenerator now replaces the original >+# BaseGenerator in the outer script that caused it to be created, since the >+# inner one is the one with all the xcfilelist information. >+ >+from __future__ import print_function >+ >+import os >+import pickle >+import tempfile >+import traceback >+ >+import webkitpy.generate_xcfilelists_lib.sdk as sdk >+import webkitpy.generate_xcfilelists_lib.util as util >+ >+ >+class BaseGenerator(object): >+ __slots__ = [ >+ "application", "project_tag", "platform", "configuration", "triple", >+ "ex_type", "ex_value", "ex_traceback", >+ "added_lines_input_derived", "added_lines_output_derived", "added_lines_input_unified", "added_lines_output_unified", >+ "cached_build_dirs"] >+ >+ VALID_PLATFORMS = None >+ VALID_CONFIGURATIONS = None >+ >+ @util.LogEntryExit >+ def __init__(self, application, project_tag, platform, configuration): >+ self.application = application >+ self.project_tag = project_tag >+ self.platform = platform >+ self.configuration = configuration >+ self.triple = (project_tag, platform, configuration) >+ >+ self.ex_type = None >+ self.ex_value = None >+ self.ex_traceback = None >+ >+ self.added_lines_input_derived = None >+ self.added_lines_output_derived = None >+ self.added_lines_input_unified = None >+ self.added_lines_output_unified = None >+ >+ self.cached_build_dirs = None >+ >+ def __str__(self): >+ return "project_tag = {}, platform = {}, configuration = {}, ex_type = {}, ex_value = {}, ex_traceback = {}, added_lines_input_derived = {}, added_lines_output_derived = {}, added_lines_input_unified = {}, added_lines_output_unified = {}, cached_build_dirs = {}".format( >+ self.project_tag, self.platform, self.configuration, >+ self.ex_type, self.ex_value, self.ex_traceback, >+ self.added_lines_input_derived, self.added_lines_output_derived, self.added_lines_input_unified, self.added_lines_output_unified, >+ self.cached_build_dirs) >+ >+ # Generate new .xcfilist list contents and return any new lines. This >+ # generation is performed by relaunching ourselves under Xcode so that we >+ # can operate in the context of its environment variables. When we relaunch >+ # ourselves, we do so in a way that invokes this class's subgenerate() >+ # method. Any results from subgenerate are written to a temporary file that >+ # we set up here. When the relaunched instance completes, we reads those >+ # results from the temporary file. >+ >+ @util.LogEntryExit >+ def generate(self): >+ with tempfile.NamedTemporaryFile() as pickle_file, tempfile.NamedTemporaryFile() as debug_file: >+ sublaunch_args = [ >+ "PATH=\"{}:${{PATH}}\"".format(self._getenv("PATH")), # Xcode will "sanitize" PATH, such that Python is no longer on it, so get /usr/bin back in PATH. >+ self.application._get_generate_xcfilelists_script_path(), >+ "subgenerate", >+ "--project", self.project_tag, >+ "--platform", self.platform, >+ "--configuration", self.configuration, >+ "--pickle-file", pickle_file.name] >+ if self.application.cmd_line_args.debug: >+ sublaunch_args += [ >+ "--debug", >+ "--debug-file", debug_file.name] >+ if self.application.cmd_line_args.quiet: >+ sublaunch_args.append("--quiet") >+ >+ try: >+ self._sublaunch_under_xcode(sublaunch_args) >+ finally: >+ # If xcodebuild returns an error, we want to at least display the >+ # debugging information. >+ if self.application.cmd_line_args.debug: >+ for line in debug_file: >+ util.debug_log("{}".format(line.rstrip())) >+ >+ generators = [] >+ while True: >+ try: >+ generator = pickle.load(pickle_file) >+ generator.application = self.application >+ generators.append(generator) >+ except EOFError as e: >+ break >+ return generators >+ >+ # Relaunch this script under Xcode. This is performed by launching Xcode, >+ # pointing it to the project that we are processing as well as a build >+ # target that will re-execute us as a custom build step. >+ >+ @util.LogEntryExit >+ def _sublaunch_under_xcode(self, sublaunch_args): >+ xcode_parameters = [ >+ "xcodebuild", >+ "-project", self._get_project_file_path(), >+ "-sdk", sdk.SDK.get_preferred_sdk_for_platform(self.platform).as_xcode_specification(), >+ "-configuration", self.configuration, >+ "-target", "Apply Configuration to XCFileLists", >+ "SYMROOT={}".format(self._get_sym_root()), >+ "OBJROOT={}".format(self._get_obj_root()), >+ "SHARED_PRECOMPS_DIR={}".format(self._get_shared_precomps_dir())] >+ >+ # TODO: sublaunch_args will contain the path to the script to >+ # sublaunch. There might be a problem if there's a space in that path. >+ util.subprocess_run(xcode_parameters, >+ env={"WK_SUBLAUNCH_SCRIPT_PARAMETERS": " ".join(sublaunch_args)}) >+ >+ # Generate the .xcfilelist content. Save the results internally as sets of >+ # new lines to be added to the .xcfilelist files. These new lines can be >+ # merged into those files if the user specified a "generate" command, or >+ # can be reported in a "check" command. >+ >+ @util.LogEntryExit >+ def subgenerate(self): >+ self._generate_derived() >+ self._generate_unified() >+ >+ # Merge any saved added lines to their ultimate destinations. >+ >+ @util.LogEntryExit >+ def merge(self): >+ self._merge_derived() >+ self._merge_unified() >+ >+ @util.LogEntryExit >+ def pickle_to_file(self, f): >+ # We don't want to pickle the application reference >+ # We can't seem to pickle ex_traceback: PicklingError: Can't pickle <type 'traceback'>: it's not found as __builtin__.traceback >+ with util.AttributeSaver(self, "application"), util.AttributeSaver(self, "ex_traceback"): >+ pickle.dump(self, f, pickle.HIGHEST_PROTOCOL) >+ >+ # Return whether or not any new lines for any .xcfilelist files were >+ # discovered. >+ >+ @util.LogEntryExit >+ def has_action(self): >+ return (self.added_lines_input_derived or >+ self.added_lines_output_derived or >+ self.added_lines_input_unified or >+ self.added_lines_output_unified) >+ >+ # If any errors occur during the generation of .xcfilelist content, they >+ # are remembered internally for later processing. This function returns >+ # whether or not any such error occurred with this generator. >+ >+ @util.LogEntryExit >+ def has_error(self): >+ return self.ex_type >+ >+ # Return whether or not the combination of project, platform, and >+ # configuration is valid (which is to say, if, for example, the given >+ # project can be built for the given platform and configuration). >+ >+ @util.LogEntryExit >+ def is_valid(self): >+ return (self.platform in self.__class__.VALID_PLATFORMS and >+ self.configuration in self.__class__.VALID_CONFIGURATIONS) >+ >+ # If an error/exception occurred and was recorded (that is, if has_error() >+ # returns true), report that error to the console. >+ >+ @util.LogEntryExit >+ def report_error(self): >+ try: >+ if self.ex_value: >+ raise self.ex_value >+ except KeyboardInterrupt: >+ print("### Canceled") >+ except util.CalledProcessError as e: >+ print("### Error {} calling subprocess: {}".format(e.args[0], e.args[1])) >+ if e.args[2]: >+ print("### stdout = {}".format(e.args[2])) >+ if e.args[3]: >+ print("### stderr = {}".format(e.args[3])) >+ except util.InvalidConfigurationError as e: >+ print("### Invalid configuration: {}".format(e)) >+ return os.EX_USAGE >+ except BaseException: >+ traceback.print_exception(self.ex_type, self.ex_value, self.ex_traceback) >+ return os.EX_SOFTWARE >+ >+ # Generate .xcfilelist content for the "Generate Derived Sources" build >+ # phase. >+ >+ @util.LogEntryExit >+ def _generate_derived(self): >+ script = self._get_generate_derived_sources_script() >+ if not script: >+ return >+ >+ with tempfile.NamedTemporaryFile() as input, tempfile.NamedTemporaryFile() as output: >+ (stdout, stderr) = util.subprocess_run( >+ [script, >+ "NO_SUPPLEMENTAL_FILES=1", >+ "--no-builtin-rules", >+ "--dry-run", >+ "--always-make", >+ "--debug=abvijm", >+ "all"]) >+ stdout = stdout.encode() if isinstance(stdout, str) else stdout >+ (stdout, stderr) = util.subprocess_run( >+ [self.application._get_extract_dependencies_from_makefile_script(), >+ "--input", input.name, >+ "--output", output.name], >+ input=stdout) >+ >+ self._replace(input.name, "^JavaScriptCore/", "$(PROJECT_DIR)/") >+ self._replace(input.name, "^JavaScriptCorePrivateHeaders/", "$(JAVASCRIPTCORE_PRIVATE_HEADERS_DIR)/") >+ self._replace(input.name, "^WebCore/", "$(PROJECT_DIR)/") >+ self._replace(input.name, "^WebKit2PrivateHeaders/", "$(WEBKIT2_PRIVATE_HEADERS_DIR)/") >+ >+ self._unexpand(input.name, "JAVASCRIPTCORE_PRIVATE_HEADERS_DIR") >+ self._unexpand(input.name, "PROJECT_DIR") >+ self._unexpand(input.name, "WEBCORE_PRIVATE_HEADERS_DIR") >+ self._unexpand(input.name, "WEBKIT2_PRIVATE_HEADERS_DIR") >+ self._unexpand(input.name, "WEBKITADDITIONS_HEADERS_FOLDER_PATH") >+ self._unexpand(input.name, "BUILT_PRODUCTS_DIR") # Do this last, since it's a prefix of some other variables and will "intercept" them if earlier than them. >+ >+ self._replace(output.name, "^", self._get_derived_sources_dir() + "/") >+ self._unexpand(output.name, "BUILT_PRODUCTS_DIR") >+ >+ self.added_lines_input_derived = self._find_added_lines(input.name, self._get_input_derived_xcfilelist_project_path()) >+ self.added_lines_output_derived = self._find_added_lines(output.name, self._get_output_derived_xcfilelist_project_path()) >+ >+ @util.LogEntryExit >+ def _merge_derived(self): >+ self._merge_added_lines(self.added_lines_input_derived, self._get_input_derived_xcfilelist_project_path()) >+ self._merge_added_lines(self.added_lines_output_derived, self._get_output_derived_xcfilelist_project_path()) >+ >+ # Generate .xcfilelist content for the "Generate Unified Sources" build >+ # phase. >+ >+ @util.LogEntryExit >+ def _generate_unified(self): >+ script = self._get_generate_unified_sources_script() >+ if not script: >+ return >+ >+ with tempfile.NamedTemporaryFile() as output: >+ >+ # We need to define BUILD_SCRIPTS_DIR so that the bash script we're >+ # invoking can find the ruby script that it invokes. If we don't >+ # define BUILD_SCRIPTS_DIR, the bash script we're invoking with try >+ # to create one, but it will create an incorrect value for our >+ # purposes, leading to dire results. >+ env = os.environ.copy() >+ env["BUILD_SCRIPTS_DIR"] = self.application._get_build_scripts_dir() >+ >+ util.subprocess_run( >+ [script, >+ "--generate-xcfilelists", >+ "--output-xcfilelist-path", output.name], >+ env=env) >+ >+ self._unexpand(output.name, "BUILT_PRODUCTS_DIR") >+ >+ self.added_lines_input_unified = self._find_added_lines(None, self._get_input_unified_xcfilelist_project_path()) >+ self.added_lines_output_unified = self._find_added_lines(output.name, self._get_output_unified_xcfilelist_project_path()) >+ >+ @util.LogEntryExit >+ def _merge_unified(self): >+ self._merge_added_lines(self.added_lines_input_unified, self._get_input_unified_xcfilelist_project_path()) >+ self._merge_added_lines(self.added_lines_output_unified, self._get_output_unified_xcfilelist_project_path()) >+ >+ # Utility for post-processing the initial .xcfilelist content. Used to >+ # replace text in the file. >+ >+ @util.LogEntryExit >+ def _replace(self, file_name, to_replace, replace_with): >+ util.subprocess_run([ >+ "sed", "-E", "-e", >+ "s|{}|{}|".format(to_replace, replace_with), >+ "-i", "''", file_name]) >+ >+ # Utility for post-processing the initial .xcfilelist content. Used to >+ # replace file path segments with the variables that represent those path >+ # segments. >+ >+ @util.LogEntryExit >+ def _unexpand(self, file_name, variable_name): >+ to_replace = self._getenv(variable_name) >+ if not to_replace: >+ return >+ >+ self._replace(file_name, "^{}/".format(to_replace), "$({})/".format(variable_name)) >+ >+ # Given a source file with new .xcfilelist content and a dest file that >+ # contains the original/previous .xcfilelist content (that is, likely the >+ # file that's checked into the repo), determine what, if any, new lines >+ # there are in source that aren't in dest. >+ >+ @util.LogEntryExit >+ def _find_added_lines(self, source, dest): >+ if not source: >+ return set() >+ source_lines = set(source) if isinstance(source, list) else self._get_file_lines(source) >+ dest_lines = set(dest) if isinstance(dest, list) else self._get_file_lines(dest) >+ delta_lines = source_lines - dest_lines >+ return delta_lines >+ >+ # Bottleneck routine for taking a set of new lines of .xcfilelist content >+ # and adding them to their ultimate file/destination. >+ >+ @util.LogEntryExit >+ def _merge_added_lines(self, added_lines, dest): >+ if not added_lines: >+ return >+ dest_lines = self._get_file_lines(dest) >+ merged_lines = sorted(set(added_lines) | dest_lines) >+ merged_lines = [line + "\n" for line in merged_lines if line and not line.startswith("#")] >+ merged_lines[0:0] = ["# This file is generated by the generate-xcfilelists script.\n"] >+ with open(dest, "w") as f: >+ f.writelines(merged_lines) >+ >+ # Utility to read a file and results the contents as a set of lines with >+ # EOLs removed. >+ >+ @util.LogEntryExit >+ def _get_file_lines(self, file): >+ try: >+ with open(file, "r") as f: >+ return set([line.strip() for line in f]) >+ except: >+ return set() >+ >+ # Wrapper to return environment variable values. >+ >+ @util.LogEntryExit >+ def _getenv(self, variable_name): >+ return os.environ.get(variable_name) >+ >+ # Return the path to the project file (the *.xcodeproj "file). >+ >+ @util.LogEntryExit >+ def _get_project_file_path(self): >+ assert False, """\ >+ Override this to return full path to the project file (e.g., >+ ".../Source/JavaScriptCore/JavaScriptCode.xcodeproj").""" >+ >+ # Return the path to the directory containing the project file and its >+ # supporting files and directories (e.g., ".../Source/JavaScriptCore"). >+ >+ @util.LogEntryExit >+ def _get_project_dir(self): >+ return os.path.dirname(self._get_project_file_path()) >+ >+ # Return the project file name (e.g., "JavaScriptCore.xcodeproj"). >+ >+ @util.LogEntryExit >+ def _get_project_file_name(self): >+ return os.path.basename(self._get_project_file_path()) >+ >+ # Return the project name (e.g., "JavaScriptCore"). >+ >+ @util.LogEntryExit >+ def _get_project_name(self): >+ return os.path.splitext(self._get_project_file_name())[0] >+ >+ # Return the path to the build output directory to use when the user has >+ # indicated that they are building from the command line. Be sure to >+ # support default command-line users, command-line users that set >+ # WEBKIT_OUTPUTDIR, default Xcode users, and people like Jeff Miller who >+ # configure a custom build output location for Xcode. >+ >+ @util.LogEntryExit >+ def _get_sym_root(self): >+ return self._get_build_dirs()[0] >+ >+ @util.LogEntryExit >+ def _get_obj_root(self): >+ return self._get_build_dirs()[1] >+ >+ @util.LogEntryExit >+ def _get_shared_precomps_dir(self): >+ return os.path.join(self._get_build_dirs()[1], "PrecompiledHeaders") >+ >+ # From Xcode; >+ # >+ # export SYMROOT =/Users/keith/Library/Developer/Xcode/DerivedData/Safari-bmjhivzkbpxamlajyexvkivfjbmb/Build/Products >+ # export OBJROOT =/Users/keith/Library/Developer/Xcode/DerivedData/Safari-bmjhivzkbpxamlajyexvkivfjbmb/Build/Intermediates.noindex >+ # export SHARED_PRECOMPS_DIR=/Users/keith/Library/Developer/Xcode/DerivedData/Safari-bmjhivzkbpxamlajyexvkivfjbmb/Build/Intermediates.noindex/PrecompiledHeaders >+ # >+ # From command-line: >+ # >+ # export SYMROOT =/Volumes/Data/dev/webkit/OpenSource/WebKitBuild >+ # export OBJROOT =/Volumes/Data/dev/webkit/OpenSource/WebKitBuild >+ # export SHARED_PRECOMPS_DIR=/Volumes/Data/dev/webkit/OpenSource/WebKitBuild/PrecompiledHeaders >+ >+ @util.LogEntryExit >+ def _get_build_dirs(self): >+ def define_xcode_build_dirs(self): >+ # Delete any spurious ~/Library/Preferences/xcodebuild.plist, as this >+ # file will interfere with any preferences set in the IDE. This >+ # .plist file shouldn't really ever exist, so nuking it doesn't >+ # cause any problems. >+ >+ xcodebuild_plist = os.path.join(os.path.expanduser("~"), "Library", "Preferences", "xcodebuild.plist") >+ try: >+ os.unlink(xcodebuild_plist) >+ except: >+ pass >+ >+ def read_xcode_user_default(key): >+ try: >+ (stdout, stderr) = util.subprocess_run(["defaults", "read", "com.apple.dt.Xcode", key]) >+ return stdout.strip() >+ except util.CalledProcessError: >+ return None >+ >+ # The following is based on the logic in determineBaseProductDir() >+ # in webkitdirs.pm and https://pewpewthespells.com/blog/xcode_build_locations.html. >+ >+ # Get the base directory for the build output. This will be some >+ # default location (e.g., ~/Library/Developer/Xcode/DerivedData), >+ # an absolute path, or a project-relative path. >+ >+ ide_custom_derived_data_location = read_xcode_user_default("IDECustomDerivedDataLocation") >+ >+ # Path not specified; use the default. >+ >+ if not ide_custom_derived_data_location: >+ base_dir = os.path.join(os.path.expanduser("~"), "Library", "Developer", "Xcode", "DerivedData") >+ >+ # An absolute path is specified; use it. >+ >+ elif os.path.isabs(ide_custom_derived_data_location): >+ base_dir = ide_custom_derived_data_location >+ >+ # A relative path is specified; append it to the project path. >+ >+ else: >+ base_dir = os.path.join(self._get_project_dir(), ide_custom_derived_data_location) >+ >+ # Get the specification for how the build output should be stored >+ # withing that base directory. This will be some unique directory >+ # based on the hash of the project file path (e.g., >+ # "Safari-sdlfkhasalksdjfhsdfhlksf"), some shared directory (e.g., >+ # "Build"), or even something that might be an absolute path. >+ >+ ide_build_location_style = read_xcode_user_default("IDEBuildLocationStyle") >+ >+ # Create a unique directory within the base directory based on >+ # project name and hash of its full path. >+ # >+ # IDEBuildLocationStyle = Unique; # This is the default if not specified. >+ >+ if ide_build_location_style == "Unique" or not ide_build_location_style: >+ workspace = os.path.abspath(self.application.cmd_line_args.xcode) >+ build_dir = os.path.join( >+ base_dir, >+ os.path.splitext(os.path.basename(workspace))[0] + "-" + util.hash_string_for_path(workspace), >+ "Build") >+ products_dir = os.path.join(build_dir, "Products") >+ intermediates_dir = os.path.join(build_dir, "Intermediates.noindex") >+ >+ # Use a shared subdirectory; use the specified directory name. >+ # >+ # IDEBuildLocationStyle = Shared; >+ # IDESharedBuildFolderName = Build; # Relative to DerivedDataLocation >+ >+ elif ide_build_location_style == "Shared": >+ build_dir = os.path.join(base_dir, read_xcode_user_default("IDESharedBuildFolderName")) >+ products_dir = os.path.join(build_dir, "Products") >+ intermediates_dir = os.path.join(build_dir, "Intermediates.noindex") >+ >+ # Use the saved products and intermediates paths and either use >+ # them as relative to the DerivedData directory or the project >+ # folder, or just use them as absolute addresses. >+ # >+ # IDEBuildLocationStyle = Custom; >+ >+ elif ide_build_location_style == "Custom": >+ ide_build_location_type = read_xcode_user_default("IDECustomBuildLocationType") >+ >+ products_dir = read_xcode_user_default("IDECustomBuildProductsPath") >+ intermediates_dir = read_xcode_user_default("IDECustomBuildIntermediatesPath") >+ >+ # IDECustomBuildLocationType = RelativeToDerivedData; >+ # IDECustomBuildIntermediatesPath = "Build/Intermediates.noindex"; >+ # IDECustomBuildProductsPath = "Build/Products"; >+ # IDECustomIndexStorePath = "Index/DataStore"; >+ >+ if ide_build_location_type == "RelativeToDerivedData": >+ products_dir = os.path.join(base_dir, products_dir) >+ intermediates_dir = os.path.join(base_dir, intermediates_dir) >+ >+ # IDECustomBuildLocationType = RelativeToWorkspace; >+ # IDECustomBuildIntermediatesPath = "Build/Intermediates.noindex"; >+ # IDECustomBuildProductsPath = "Build/Products"; >+ # IDECustomIndexStorePath = "Index/DataStore"; >+ >+ elif ide_build_location_type == "RelativeToWorkspace": >+ base_dir = self._get_project_dir(), >+ products_dir = os.path.join(base_dir, products_dir) >+ intermediates_dir = os.path.join(base_dir, intermediates_dir) >+ >+ # IDECustomBuildLocationType = Absolute; >+ # IDECustomBuildIntermediatesPath = "/Users/joedeveloper/Desktop/Build/Intermediates.noindex"; >+ # IDECustomBuildProductsPath = "/Users/joedeveloper/Desktop/Build/Products"; >+ # IDECustomIndexStorePath = "/Users/joedeveloper/Desktop/Index/DataStore"; >+ # IDECustomDerivedDataLocation = "/Users/joedeveloper/Library/Developer/Xcode/DerivedData"; >+ >+ elif ide_build_location_type == "Absolute": >+ pass >+ >+ else: >+ assert False, "Unknown/unsupported location type" >+ else: >+ assert False, "Unknown/unsupported style" >+ >+ return (products_dir, intermediates_dir) >+ >+ def define_command_line_build_dirs(self): >+ products_dir = self._getenv("WEBKIT_OUTPUTDIR") >+ if not products_dir: >+ products_dir = os.path.join(self.application._get_opensource_dir(), "WebKitBuild") >+ return (products_dir, products_dir) >+ >+ if not self.cached_build_dirs and self.application.cmd_line_args.xcode: >+ self.cached_build_dirs = define_xcode_build_dirs(self) >+ if not self.cached_build_dirs: >+ self.cached_build_dirs = define_command_line_build_dirs(self) >+ return self.cached_build_dirs >+ >+ # Return the location of the DerivedSources directory. Conventionally, we >+ # use $BUILT_PRODUCTS/DerivedSources/$PROJECT_NAME, but we may some day use >+ # $DERIVED_FILE_DIR aka $DERIVED_FILES_DIR aka $DERIVED_SOURCES_DIR, when >+ # the projects have been updated to use them. >+ >+ @util.LogEntryExit >+ def _get_derived_sources_dir(self): >+ return os.path.join(self.application._get_xcode_built_products_dir(), "DerivedSources", self._get_project_name()) >+ >+ # Return the location for the xcfilelists. >+ >+ @util.LogEntryExit >+ def _get_xcfilelist_dir(self): >+ return self._get_project_dir() >+ >+ # Return the paths to the actual xcfilelists. >+ >+ @util.LogEntryExit >+ def _get_input_derived_xcfilelist_project_path(self): >+ return os.path.join(self._get_xcfilelist_dir(), "DerivedSources-input.xcfilelist") >+ >+ @util.LogEntryExit >+ def _get_output_derived_xcfilelist_project_path(self): >+ return os.path.join(self._get_xcfilelist_dir(), "DerivedSources-output.xcfilelist") >+ >+ @util.LogEntryExit >+ def _get_input_unified_xcfilelist_project_path(self): >+ return os.path.join(self._get_xcfilelist_dir(), "UnifiedSources-input.xcfilelist") >+ >+ @util.LogEntryExit >+ def _get_output_unified_xcfilelist_project_path(self): >+ return os.path.join(self._get_xcfilelist_dir(), "UnifiedSources-output.xcfilelist") >+ >+ # Return the paths to the scripts that generate the derived sources. >+ >+ @util.LogEntryExit >+ def _get_generate_derived_sources_script(self): >+ return None >+ >+ @util.LogEntryExit >+ def _get_generate_unified_sources_script(self): >+ return None >+ >+ >+class JavaScriptCoreGenerator(BaseGenerator): >+ VALID_PLATFORMS = ("macosx", "iosmac", "iphoneos", "watchos") >+ VALID_CONFIGURATIONS = ("Debug", "Release", "Production", "Profiling") >+ >+ @util.LogEntryExit >+ def _get_project_file_path(self): >+ return os.path.join(self.application._get_opensource_dir(), "Source", "JavaScriptCore", "JavaScriptCore.xcodeproj") >+ >+ @util.LogEntryExit >+ def _get_generate_derived_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-derived-sources.sh") >+ >+ @util.LogEntryExit >+ def _get_generate_unified_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-unified-sources.sh") >+ >+ >+class WebCoreGenerator(BaseGenerator): >+ VALID_PLATFORMS = ("macosx", "iosmac", "iphoneos", "watchos") >+ VALID_CONFIGURATIONS = ("Debug", "Release", "Production") >+ >+ @util.LogEntryExit >+ def _get_project_file_path(self): >+ return os.path.join(self.application._get_opensource_dir(), "Source", "WebCore", "WebCore.xcodeproj") >+ >+ @util.LogEntryExit >+ def _get_generate_derived_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-derived-sources.sh") >+ >+ @util.LogEntryExit >+ def _get_generate_unified_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-unified-sources.sh") >+ >+ >+class WebKitGenerator(BaseGenerator): >+ VALID_PLATFORMS = ("macosx", "iosmac", "iphoneos", "watchos") >+ VALID_CONFIGURATIONS = ("Debug", "Release", "Production") >+ >+ @util.LogEntryExit >+ def _get_project_file_path(self): >+ return os.path.join(self.application._get_opensource_dir(), "Source", "WebKit", "WebKit.xcodeproj") >+ >+ @util.LogEntryExit >+ def _get_derived_sources_dir(self): >+ return os.path.join(self.application._get_xcode_built_products_dir(), "DerivedSources", "WebKit2") >+ >+ @util.LogEntryExit >+ def _get_generate_derived_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-derived-sources.sh") >+ >+ @util.LogEntryExit >+ def _get_generate_unified_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-unified-sources.sh") >+ >+ >+class DumpRenderTreeGenerator(BaseGenerator): >+ VALID_PLATFORMS = ("macosx", ) >+ VALID_CONFIGURATIONS = ("Debug", "Release", "Production") >+ >+ @util.LogEntryExit >+ def _get_project_file_path(self): >+ return os.path.join(self.application._get_opensource_dir(), "Tools", "DumpRenderTree", "DumpRenderTree.xcodeproj") >+ >+ @util.LogEntryExit >+ def _get_generate_derived_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-derived-sources.sh") >+ >+ >+class WebKitTestRunnerGenerator(BaseGenerator): >+ VALID_PLATFORMS = ("macosx", ) >+ VALID_CONFIGURATIONS = ("Debug", "Release", "Production") >+ >+ @util.LogEntryExit >+ def _get_project_file_path(self): >+ return os.path.join(self.application._get_opensource_dir(), "Tools", "WebKitTestRunner", "WebKitTestRunner.xcodeproj") >+ >+ @util.LogEntryExit >+ def _get_generate_derived_sources_script(self): >+ return os.path.join(self._get_project_dir(), "Scripts", "generate-derived-sources.sh") >diff --git a/Tools/Scripts/webkitpy/generate_xcfilelists_lib/sdk.py b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/sdk.py >new file mode 100644 >index 0000000000000000000000000000000000000000..c6fbd96b933def45e58dda6a909cc4bd9f4b35d9 >--- /dev/null >+++ b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/sdk.py >@@ -0,0 +1,122 @@ >+#!/usr/bin/env python >+# -*- coding: utf-8 -*- >+# >+# Copyright (C) 2019 Apple Inc. All rights reserved. >+# >+# Redistribution and use in source and binary forms, with or without >+# modification, are permitted provided that the following conditions >+# are met: >+# >+# 1. Redistributions of source code must retain the above copyright >+# notice, this list of conditions and the following disclaimer. >+# 2. Redistributions in binary form must reproduce the above copyright >+# notice, this list of conditions and the following disclaimer in the >+# documentation and/or other materials provided with the distribution. >+# >+# THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ >+# Class for representing SDK information, in the form of Platform, Version, and >+# whether or not it's an Internal SDK. This class also contains a factory >+# method for producing an SDK instance from a "specification" string such as >+# "macOS15.3Internal", like you would get from 'xcodebuild -showsdks". >+ >+import re >+ >+import webkitpy.generate_xcfilelists_lib.util as util >+ >+ >+class SDK(object): >+ __slots__ = ["xcode_specification", "platform", "version", "internal"] >+ >+ @util.LogEntryExit >+ def __init__(self, xcode_specification, platform, version, internal): >+ super(SDK, self).__init__() >+ self.xcode_specification = xcode_specification >+ self.platform = platform >+ self.version = version >+ self.internal = internal >+ >+ def __repr__(self): >+ return "SDK({}, {}, {}, {})".format( >+ self.xcode_specification, >+ self.platform, >+ self.version, >+ self.internal) >+ >+ @util.LogEntryExit >+ def as_xcode_specification(self): >+ return "{}.internal".format(self.platform) if self.internal else self.platform >+ >+ # @util.LogEntryExitClass >+ @classmethod >+ def get_preferred_sdk_for_platform(cls, platform): >+ "Prefer the internal SDK, if it exists" >+ >+ preferred_sdk = cls._parse_sdk(platform) >+ >+ (stdout, stderr) = util.subprocess_run(["xcodebuild", "-showsdks"]) >+ >+ for line in stdout.splitlines(): >+ m = re.match(r".*-sdk (.*)", line) >+ if m: >+ this_sdk = cls._parse_sdk(m.group(1)) >+ >+ if preferred_sdk.platform != this_sdk.platform: >+ continue >+ >+ # Both have version information, so compare versions. If the >+ # versions are equal, prefer the Internal sdk. Otherwise, keep >+ # the one with the higher version. >+ >+ if preferred_sdk.version and this_sdk.version: >+ if float(preferred_sdk.version) == float(this_sdk.version): >+ if this_sdk.internal: >+ preferred_sdk = this_sdk >+ elif float(preferred_sdk.version) < float(this_sdk.version): >+ preferred_sdk = this_sdk >+ >+ # The preferred sdk does not have a version but the prospective >+ # one does, so keep the prospective one. >+ >+ elif not preferred_sdk.version and this_sdk.version: >+ preferred_sdk = this_sdk >+ >+ # The preferred sdk has a version but the prospective one does >+ # not, so keep the preferred one. >+ >+ elif preferred_sdk.version and not this_sdk.version: >+ pass >+ >+ # Neither has version information; prefer the current one if >+ # it's internal. >+ >+ elif not preferred_sdk.version and not this_sdk.version: >+ if this_sdk.internal: >+ preferred_sdk = this_sdk >+ >+ return preferred_sdk >+ >+ # @util.LogEntryExitClass >+ @classmethod >+ def _parse_sdk(cls, xcode_specification): >+ # letters-and-dots, followed by optional numbers-and-dots, followed by >+ # an optional "internal" >+ m = re.match(r"([a-zA-Z.]+[a-zA-Z])([0-9.]+[0-9])?\.?(internal)?", xcode_specification) >+ if not m: >+ return SDK(xcode_specification, None, None, None) >+ >+ platform = m.group(1) >+ version = m.group(2) >+ internal = m.group(3) >+ >+ return SDK(xcode_specification, platform, version, internal) >diff --git a/Tools/Scripts/webkitpy/generate_xcfilelists_lib/util.py b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/util.py >new file mode 100644 >index 0000000000000000000000000000000000000000..76186376be4238a9ad7af39ca604f75420f94d30 >--- /dev/null >+++ b/Tools/Scripts/webkitpy/generate_xcfilelists_lib/util.py >@@ -0,0 +1,274 @@ >+#!/usr/bin/env python >+# -*- coding: utf-8 -*- >+# >+# Copyright (C) 2019 Apple Inc. All rights reserved. >+# >+# Redistribution and use in source and binary forms, with or without >+# modification, are permitted provided that the following conditions >+# are met: >+# >+# 1. Redistributions of source code must retain the above copyright >+# notice, this list of conditions and the following disclaimer. >+# 2. Redistributions in binary form must reproduce the above copyright >+# notice, this list of conditions and the following disclaimer in the >+# documentation and/or other materials provided with the distribution. >+# >+# THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY >+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE >+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR >+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, >+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, >+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR >+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY >+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT >+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE >+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >+ >+from __future__ import print_function >+ >+import hashlib >+import os >+import struct >+import subprocess >+import sys >+import traceback >+ >+PY3 = sys.version_info.major >= 3 >+ >+# Gather information about our debugging environment right now. Do this before >+# executing any "main" code so that our debugging preferences are in place by >+# the time we get there and we can debug from main() on down as opposed to >+# parse_args() on down. >+ >+SHOW_DEBUG_LOGGING = "-d" in sys.argv or "--debug" in sys.argv >+ >+if SHOW_DEBUG_LOGGING: >+ >+ DEBUG_LOGGING_FILE = None >+ for index, arg in enumerate(sys.argv): >+ if arg == "--debug-file": >+ if index + 1 < len(sys.argv): >+ DEBUG_LOGGING_FILE = sys.argv[index + 1] >+ >+ # Bottleneck function for printing debugging lines to either the console or >+ # the screen, as appropriate. >+ >+ if DEBUG_LOGGING_FILE: >+ >+ def debug_log(msg): >+ with open(DEBUG_LOGGING_FILE, "a") as f: >+ print(msg, file=f) >+ else: >+ >+ def debug_log(msg): >+ print(msg) >+ >+ # Context Manager class for logging information about function entry/exit. >+ # On entry, the function name is logged along with its parameters. On exit, >+ # the function name is logged with its result. If an exception occurs, the >+ # function name is logged with the exception. In all cases, the logging is >+ # indented according to the level of the function in the backtrace. >+ # >+ # Versions of these classes exist for instance methods, class methods, and >+ # global functions so that instance and class information can be extracted >+ # and displayed. >+ >+ class LogEntryHelper(object): >+ __slots__ = ["indent", "class_name", "function_name", "type"] >+ >+ def __init__(self, func, type): >+ tb = traceback.extract_stack() >+ self.indent = " " * 2 * (len(tb) - 3) >+ self.class_name = None >+ self.function_name = func.__name__ >+ self.type = type >+ >+ def log_entry(self, args, kwargs): >+ if self.type == "instance": >+ self.class_name = args[0].__class__.__name__ + "." >+ args = args[1:] >+ elif self.type == "class": >+ self.class_name = args[0].__name__ + "." >+ args = args[1:] >+ else: >+ self.class_name = "" >+ >+ self._print("args={}, kwargs={}".format(args, kwargs)) >+ >+ def log_result(self, result): >+ if hasattr(result, '__iter__') and not isinstance(result, str): >+ for line in result: >+ self._print("result={}".format(line)) >+ else: >+ self._print("result={}".format(result)) >+ >+ def log_exception(self, exc): >+ self._print("exception={}".format(exc)) >+ >+ def _print(self, msg): >+ debug_log("{}{}{}: {}".format(self.indent, self.class_name, self.function_name, msg)) >+ >+ def LogEntryExit(func): >+ def _show_debug_logging(*args, **kwargs): >+ helper = LogEntryHelper(func, "instance") >+ helper.log_entry(args, kwargs) >+ try: >+ result = func(*args, **kwargs) >+ helper.log_result(result) >+ return result >+ except BaseException as e: >+ helper.log_exception(e) >+ raise >+ return _show_debug_logging >+ >+ def LogEntryExitClass(func): >+ def _show_debug_logging(*args, **kwargs): >+ helper = LogEntryHelper(func, "class") >+ helper.log_entry(args, kwargs) >+ try: >+ result = func(*args, **kwargs) >+ helper.log_result(result) >+ return result >+ except BaseException as e: >+ helper.log_exception(e) >+ raise >+ return _show_debug_logging >+ >+ def LogEntryExitGlobal(func): >+ def _show_debug_logging(*args, **kwargs): >+ helper = LogEntryHelper(func, None) >+ helper.log_entry(args, kwargs) >+ try: >+ result = func(*args, **kwargs) >+ helper.log_result(result) >+ return result >+ except BaseException as e: >+ helper.log_exception(e) >+ raise >+ return _show_debug_logging >+ >+else: >+ >+ def debug_log(msg): >+ pass >+ >+ def LogEntryExit(func): >+ return func >+ >+ def LogEntryExitClass(func): >+ return func >+ >+ def LogEntryExitGlobal(func): >+ return func >+ >+ >+# Utility function for operating similar to subprocess.run() in Python 3. One >+# difference is that the result is a 2-tuple with stdout and stderr, rather >+# than a 3-tuple that includes returncode. For our purposes, if returncode is >+# non-zero, we raise an exception. >+ >+@LogEntryExitGlobal >+def subprocess_run(args, **kwargs): >+ kwargs["stdout"] = subprocess.PIPE >+ kwargs["stderr"] = subprocess.PIPE >+ input = None >+ if "input" in kwargs: >+ input = kwargs["input"] >+ del kwargs["input"] >+ kwargs["stdin"] = subprocess.PIPE >+ process = subprocess.Popen(args, **kwargs) >+ (stdout, stderr) = process.communicate(input=input) >+ stdout = stdout.decode() if isinstance(stdout, bytes) else stdout >+ stderr = stderr.decode() if isinstance(stderr, bytes) else stderr >+ if process.returncode: >+ raise CalledProcessError(process.returncode, args[0], stdout, stderr) >+ return (stdout, stderr) >+ >+ >+# Utility function to allow us to verify that we're running under Xcode or not. >+# For example, if we are not, then we need to make sure that we don't try to >+# access Xcode-specific environment variables. >+ >+@LogEntryExitGlobal >+def is_running_under_xcode(): >+ return os.environ.get("XCODE_INSTALL_PATH") >+ >+ >+# Context Manager for saving the value of an object's attribute, setting it to >+# a new value, and then restoring the original value. >+ >+class AttributeSaver: >+ def __init__(self, obj, attribute, value=None): >+ self.obj = obj >+ self.attribute = attribute >+ self.old_value = getattr(self.obj, self.attribute) >+ self.new_value = value >+ >+ def __enter__(self): >+ setattr(self.obj, self.attribute, self.new_value) >+ >+ def __exit__(self, exc_type, exc_value, traceback): >+ setattr(self.obj, self.attribute, self.old_value) >+ return None >+ >+ >+# Some Exceptions >+ >+class QuickExit(Exception): >+ pass >+ >+ >+class InvalidCommandError(Exception): >+ pass >+ >+ >+class InvalidArgumentError(Exception): >+ pass >+ >+ >+class InvalidConfigurationError(Exception): >+ pass >+ >+ >+# subprocess.CalledProcessError has problems with being pickled, which is >+# something that we do to it. In particular, when unpickled, it throws an >+# exception, and so CalledProcessError get's turned into an exception saying >+# "__init__() takes at least 3 arguments (1 given)". Address this by creating >+# our own CalledProcessError that's a little more generic. >+ >+class CalledProcessError(Exception): >+ def __str__(self): >+ returncode = self.args[0] if len(self.args) > 0 else None >+ command = self.args[1] if len(self.args) > 1 else None >+ stdout = self.args[2] if len(self.args) > 2 else None >+ stderr = self.args[3] if len(self.args) > 3 else None >+ >+ if stderr: >+ return "Command '{}' returned non-zero exit status {}: {}".format(command, returncode, stderr) >+ elif stdout: >+ return "Command '{}' returned non-zero exit status {}: {}".format(command, returncode, stdout) >+ else: >+ return "Command '{}' returned non-zero exit status {}".format(command, returncode) >+ >+ >+# The default location for Xcode's "DerivedData" (build output and intermediate >+# files) is a unique directory in ~/Library/Developer/Xcode/DerivedData with a >+# name incorporating a hash based on the full path of the project or workspace >+# being built. The following function takes that path and returns the >+# corresponding hash. >+# >+# The algorithm is adopted from the following article: >+# >+# <https://pewpewthespells.com/blog/xcode_deriveddata_hashes.pdf> >+ >+def hash_string_for_path(path): >+ def convert_to_string(n): >+ s = '' >+ for _ in range(0, 14): >+ (n, r) = divmod(n, 26) >+ s = chr(r + 97) + s >+ return s >+ >+ (part1, part2) = struct.unpack(">QQ", hashlib.md5(path.encode()).digest()) >+ return convert_to_string(part1) + convert_to_string(part2)
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 197622
:
369193
|
369198
|
369208
|
369249
|
369432
|
369694
|
369994