WebKit Bugzilla
Attachment 358629 Details for
Bug 193252
: Move ResourceLoadStatistics files from UIProcess to NetworkProcess
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Patch for landing
bug-193252-20190108132531.patch (text/plain), 342.81 KB, created by
Brent Fulgham
on 2019-01-08 13:25:32 PST
(
hide
)
Description:
Patch for landing
Filename:
MIME Type:
Creator:
Brent Fulgham
Created:
2019-01-08 13:25:32 PST
Size:
342.81 KB
patch
obsolete
>Subversion Revision: 239705 >diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog >index 1319dde179a3c47038f74df138fd494f2b5740ec..533f101f8ae3ec16f3bb999e0f06dae345c6ed7a 100644 >--- a/Source/WebKit/ChangeLog >+++ b/Source/WebKit/ChangeLog >@@ -1,3 +1,44 @@ >+2019-01-08 Brent Fulgham <bfulgham@apple.com> >+ >+ Move ResourceLoadStatistics files from UIProcess to NetworkProcess >+ https://bugs.webkit.org/show_bug.cgi?id=193252 >+ <rdar://problem/47125401> >+ >+ Reviewed by Alex Christensen. >+ >+ This patch is the first part of a refactoring to move the ResourceLoadStatistics logic from the UIProcess to the NetworkProcess. >+ >+ This patch moves code into different folders and adjusts necessary build files, but does not change where the code executes. These >+ changes have no impact on behavior or test results. >+ >+ I also modified a few files to add missing include files uncovered by the unified build system. >+ >+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp: Renamed from Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.cpp. >+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h: Renamed from Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.h. >+ * NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp: Renamed from Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.cpp. >+ * NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.h: Renamed from Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.h. >+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp: Renamed from Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp. >+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h: Renamed from Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.h. >+ * NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp: Renamed from Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.cpp. >+ * NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.h: Renamed from Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.h. >+ * NetworkProcess/Downloads/PendingDownload.cpp: >+ * NetworkProcess/Downloads/PendingDownload.h: >+ * PlatformMac.cmake: >+ * PlatformWin.cmake: >+ * Shared/PersistencyUtils.cpp: Renamed from Source/WebKit/UIProcess/PersistencyUtils.cpp. >+ * Shared/PersistencyUtils.h: Renamed from Source/WebKit/UIProcess/PersistencyUtils.h. >+ * Sources.txt: >+ * SourcesCocoa.txt: >+ * SourcesGTK.txt: >+ * SourcesWPE.txt: >+ * UIProcess/API/C/WKWebsiteDataStoreRef.cpp: >+ (WKWebsiteDataStoreSetWebAuthenticationMockConfiguration): Add missing WebKit:: scope. >+ * UIProcess/UserMediaPermissionRequestManagerProxy.cpp: Add missing include. >+ * UIProcess/WebDataListSuggestionsDropdown.cpp: Ditto. >+ * UIProcess/mac/DisplayLink.cpp: >+ * UnifiedSources-input.xcfilelist: >+ * WebKit.xcodeproj/project.pbxproj: >+ > 2019-01-07 Dean Jackson <dino@apple.com> > > Turn on Pointer Events by default for iOS >diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..19c38e1c47ced6082578e7d1938151fd50f754fd >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp >@@ -0,0 +1,1294 @@ >+/* >+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+#include "ResourceLoadStatisticsMemoryStore.h" >+ >+#include "Logging.h" >+#include "PluginProcessManager.h" >+#include "PluginProcessProxy.h" >+#include "ResourceLoadStatisticsPersistentStorage.h" >+#include "WebProcessProxy.h" >+#include "WebResourceLoadStatisticsTelemetry.h" >+#include "WebsiteDataStore.h" >+#include <WebCore/KeyedCoding.h> >+#include <WebCore/ResourceLoadStatistics.h> >+#include <wtf/CallbackAggregator.h> >+#include <wtf/DateMath.h> >+#include <wtf/MathExtras.h> >+#include <wtf/text/StringBuilder.h> >+ >+namespace WebKit { >+using namespace WebCore; >+ >+constexpr unsigned statisticsModelVersion { 14 }; >+constexpr unsigned maxNumberOfRecursiveCallsInRedirectTraceBack { 50 }; >+constexpr Seconds minimumStatisticsProcessingInterval { 5_s }; >+constexpr unsigned operatingDatesWindow { 30 }; >+constexpr unsigned maxImportance { 3 }; >+static const char* debugStaticPrevalentResource { "3rdpartytestwebkit.org" }; >+ >+#if !RELEASE_LOG_DISABLED >+static String domainsToString(const Vector<String>& domains) >+{ >+ StringBuilder builder; >+ for (auto& domain : domains) { >+ if (!builder.isEmpty()) >+ builder.appendLiteral(", "); >+ builder.append(domain); >+ } >+ return builder.toString(); >+} >+#endif >+ >+class OperatingDate { >+public: >+ OperatingDate() = default; >+ >+ static OperatingDate fromWallTime(WallTime time) >+ { >+ double ms = time.secondsSinceEpoch().milliseconds(); >+ int year = msToYear(ms); >+ int yearDay = dayInYear(ms, year); >+ int month = monthFromDayInYear(yearDay, isLeapYear(year)); >+ int monthDay = dayInMonthFromDayInYear(yearDay, isLeapYear(year)); >+ >+ return OperatingDate { year, month, monthDay }; >+ } >+ >+ static OperatingDate today() >+ { >+ return OperatingDate::fromWallTime(WallTime::now()); >+ } >+ >+ Seconds secondsSinceEpoch() const >+ { >+ return Seconds { dateToDaysFrom1970(m_year, m_month, m_monthDay) * secondsPerDay }; >+ } >+ >+ bool operator==(const OperatingDate& other) const >+ { >+ return m_monthDay == other.m_monthDay && m_month == other.m_month && m_year == other.m_year; >+ } >+ >+ bool operator<(const OperatingDate& other) const >+ { >+ return secondsSinceEpoch() < other.secondsSinceEpoch(); >+ } >+ >+ bool operator<=(const OperatingDate& other) const >+ { >+ return secondsSinceEpoch() <= other.secondsSinceEpoch(); >+ } >+ >+private: >+ OperatingDate(int year, int month, int monthDay) >+ : m_year(year) >+ , m_month(month) >+ , m_monthDay(monthDay) >+ { } >+ >+ int m_year { 0 }; >+ int m_month { 0 }; // [0, 11]. >+ int m_monthDay { 0 }; // [1, 31]. >+}; >+ >+static Vector<OperatingDate> mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates) >+{ >+ if (existingDates.isEmpty()) >+ return WTFMove(newDates); >+ >+ Vector<OperatingDate> mergedDates(existingDates.size() + newDates.size()); >+ >+ // Merge the two sorted vectors of dates. >+ std::merge(existingDates.begin(), existingDates.end(), newDates.begin(), newDates.end(), mergedDates.begin()); >+ // Remove duplicate dates. >+ removeRepeatedElements(mergedDates); >+ >+ // Drop old dates until the Vector size reaches operatingDatesWindow. >+ while (mergedDates.size() > operatingDatesWindow) >+ mergedDates.remove(0); >+ >+ return mergedDates; >+} >+ >+struct StatisticsLastSeen { >+ String topPrivatelyOwnedDomain; >+ WallTime lastSeen; >+}; >+ >+static void pruneResources(HashMap<String, WebCore::ResourceLoadStatistics>& statisticsMap, Vector<StatisticsLastSeen>& statisticsToPrune, size_t& numberOfEntriesToPrune) >+{ >+ if (statisticsToPrune.size() > numberOfEntriesToPrune) { >+ std::sort(statisticsToPrune.begin(), statisticsToPrune.end(), [](const StatisticsLastSeen& a, const StatisticsLastSeen& b) { >+ return a.lastSeen < b.lastSeen; >+ }); >+ } >+ >+ for (size_t i = 0, end = std::min(numberOfEntriesToPrune, statisticsToPrune.size()); i != end; ++i, --numberOfEntriesToPrune) >+ statisticsMap.remove(statisticsToPrune[i].topPrivatelyOwnedDomain); >+} >+ >+static unsigned computeImportance(const ResourceLoadStatistics& resourceStatistic) >+{ >+ unsigned importance = maxImportance; >+ if (!resourceStatistic.isPrevalentResource) >+ importance -= 1; >+ if (!resourceStatistic.hadUserInteraction) >+ importance -= 2; >+ return importance; >+} >+ >+ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue) >+ : m_store(store) >+ , m_workQueue(workQueue) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+#if PLATFORM(COCOA) >+ registerUserDefaultsIfNeeded(); >+#endif >+ includeTodayAsOperatingDateIfNecessary(); >+ >+ m_workQueue->dispatchAfter(5_s, [weakThis = makeWeakPtr(*this)] { >+ if (weakThis) >+ weakThis->calculateAndSubmitTelemetry(); >+ }); >+} >+ >+ResourceLoadStatisticsMemoryStore::~ResourceLoadStatisticsMemoryStore() >+{ >+ ASSERT(!RunLoop::isMain()); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setPersistentStorage(ResourceLoadStatisticsPersistentStorage& persistentStorage) >+{ >+ m_persistentStorage = makeWeakPtr(persistentStorage); >+} >+ >+void ResourceLoadStatisticsMemoryStore::calculateAndSubmitTelemetry() const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (m_parameters.shouldSubmitTelemetry) >+ WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*this); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setNotifyPagesWhenDataRecordsWereScanned(bool value) >+{ >+ ASSERT(!RunLoop::isMain()); >+ m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned = value; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value) >+{ >+ ASSERT(!RunLoop::isMain()); >+ m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval = value; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setShouldSubmitTelemetry(bool value) >+{ >+ ASSERT(!RunLoop::isMain()); >+ m_parameters.shouldSubmitTelemetry = value; >+} >+ >+void ResourceLoadStatisticsMemoryStore::removeDataRecords(CompletionHandler<void()>&& callback) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (!shouldRemoveDataRecords()) { >+ callback(); >+ return; >+ } >+ >+#if ENABLE(NETSCAPE_PLUGIN_API) >+ m_activePluginTokens.clear(); >+ for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses()) >+ m_activePluginTokens.add(plugin->pluginProcessToken()); >+#endif >+ >+ auto prevalentResourceDomains = topPrivatelyControlledDomainsToRemoveWebsiteDataFor(); >+ if (prevalentResourceDomains.isEmpty()) { >+ callback(); >+ return; >+ } >+ >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "About to remove data records for %{public}s.", domainsToString(prevalentResourceDomains).utf8().data()); >+#endif >+ >+ setDataRecordsBeingRemoved(true); >+ >+ RunLoop::main().dispatch([prevalentResourceDomains = crossThreadCopy(prevalentResourceDomains), callback = WTFMove(callback), weakThis = makeWeakPtr(*this), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef()] () mutable { >+ WebProcessProxy::deleteWebsiteDataForTopPrivatelyControlledDomainsInAllPersistentDataStores(WebResourceLoadStatisticsStore::monitoredDataTypes(), WTFMove(prevalentResourceDomains), shouldNotifyPagesWhenDataRecordsWereScanned, [callback = WTFMove(callback), weakThis = WTFMove(weakThis), workQueue = workQueue.copyRef()](const HashSet<String>& domainsWithDeletedWebsiteData) mutable { >+ workQueue->dispatch([topDomains = crossThreadCopy(domainsWithDeletedWebsiteData), callback = WTFMove(callback), weakThis = WTFMove(weakThis)] () mutable { >+ if (!weakThis) { >+ callback(); >+ return; >+ } >+ for (auto& prevalentResourceDomain : topDomains) { >+ auto& statistic = weakThis->ensureResourceStatisticsForPrimaryDomain(prevalentResourceDomain); >+ ++statistic.dataRecordsRemoved; >+ } >+ weakThis->setDataRecordsBeingRemoved(false); >+ callback(); >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(weakThis->m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Done removing data records."); >+#endif >+ }); >+ }); >+ }); >+} >+ >+unsigned ResourceLoadStatisticsMemoryStore::recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(const WebCore::ResourceLoadStatistics& resourceStatistic, HashSet<String>& domainsThatHaveRedirectedTo, unsigned numberOfRecursiveCalls) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (numberOfRecursiveCalls >= maxNumberOfRecursiveCallsInRedirectTraceBack) { >+ // Model version 14 invokes a deliberate re-classification of the whole set. >+ if (statisticsModelVersion != 14) >+ ASSERT_NOT_REACHED(); >+ RELEASE_LOG(ResourceLoadStatistics, "Hit %u recursive calls in redirect backtrace. Returning early.", maxNumberOfRecursiveCallsInRedirectTraceBack); >+ return numberOfRecursiveCalls; >+ } >+ >+ numberOfRecursiveCalls++; >+ >+ for (auto& subresourceUniqueRedirectFromDomain : resourceStatistic.subresourceUniqueRedirectsFrom.values()) { >+ auto mapEntry = m_resourceStatisticsMap.find(subresourceUniqueRedirectFromDomain); >+ if (mapEntry == m_resourceStatisticsMap.end() || mapEntry->value.isPrevalentResource) >+ continue; >+ if (domainsThatHaveRedirectedTo.add(mapEntry->value.highLevelDomain).isNewEntry) >+ numberOfRecursiveCalls = recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(mapEntry->value, domainsThatHaveRedirectedTo, numberOfRecursiveCalls); >+ } >+ for (auto& topFrameUniqueRedirectFromDomain : resourceStatistic.topFrameUniqueRedirectsFrom.values()) { >+ auto mapEntry = m_resourceStatisticsMap.find(topFrameUniqueRedirectFromDomain); >+ if (mapEntry == m_resourceStatisticsMap.end() || mapEntry->value.isPrevalentResource) >+ continue; >+ if (domainsThatHaveRedirectedTo.add(mapEntry->value.highLevelDomain).isNewEntry) >+ numberOfRecursiveCalls = recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(mapEntry->value, domainsThatHaveRedirectedTo, numberOfRecursiveCalls); >+ } >+ >+ return numberOfRecursiveCalls; >+} >+ >+void ResourceLoadStatisticsMemoryStore::markAsPrevalentIfHasRedirectedToPrevalent(WebCore::ResourceLoadStatistics& resourceStatistic) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (resourceStatistic.isPrevalentResource) >+ return; >+ >+ for (auto& subresourceDomainRedirectedTo : resourceStatistic.subresourceUniqueRedirectsTo.values()) { >+ auto mapEntry = m_resourceStatisticsMap.find(subresourceDomainRedirectedTo); >+ if (mapEntry != m_resourceStatisticsMap.end() && mapEntry->value.isPrevalentResource) { >+ setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >+ return; >+ } >+ } >+ >+ for (auto& topFrameDomainRedirectedTo : resourceStatistic.topFrameUniqueRedirectsTo.values()) { >+ auto mapEntry = m_resourceStatisticsMap.find(topFrameDomainRedirectedTo); >+ if (mapEntry != m_resourceStatisticsMap.end() && mapEntry->value.isPrevalentResource) { >+ setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >+ return; >+ } >+ } >+} >+ >+bool ResourceLoadStatisticsMemoryStore::isPrevalentDueToDebugMode(ResourceLoadStatistics& resourceStatistic) >+{ >+ if (!m_debugModeEnabled) >+ return false; >+ >+ return resourceStatistic.highLevelDomain == debugStaticPrevalentResource || resourceStatistic.highLevelDomain == m_debugManualPrevalentResource; >+} >+ >+void ResourceLoadStatisticsMemoryStore::processStatisticsAndDataRecords() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval) { >+ for (auto& resourceStatistic : m_resourceStatisticsMap.values()) { >+ if (isPrevalentDueToDebugMode(resourceStatistic)) >+ setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >+ else if (!resourceStatistic.isVeryPrevalentResource) { >+ markAsPrevalentIfHasRedirectedToPrevalent(resourceStatistic); >+ auto currentPrevalence = resourceStatistic.isPrevalentResource ? ResourceLoadPrevalence::High : ResourceLoadPrevalence::Low; >+ auto newPrevalence = m_resourceLoadStatisticsClassifier.calculateResourcePrevalence(resourceStatistic, currentPrevalence); >+ if (newPrevalence != currentPrevalence) >+ setPrevalentResource(resourceStatistic, newPrevalence); >+ } >+ } >+ } >+ >+ removeDataRecords([this, weakThis = makeWeakPtr(*this)] { >+ ASSERT(!RunLoop::isMain()); >+ if (!weakThis) >+ return; >+ >+ pruneStatisticsIfNeeded(); >+ if (m_persistentStorage) >+ m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::No); >+ >+ if (!m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned) >+ return; >+ >+ RunLoop::main().dispatch([] { >+ WebProcessProxy::notifyPageStatisticsAndDataRecordsProcessed(); >+ }); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::hasStorageAccess(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >+ if (shouldBlockAndPurgeCookies(subFrameStatistic)) { >+ completionHandler(false); >+ return; >+ } >+ >+ if (!shouldBlockAndKeepCookies(subFrameStatistic)) { >+ completionHandler(true); >+ return; >+ } >+ >+ RunLoop::main().dispatch([store = makeRef(m_store), subFramePrimaryDomain = subFramePrimaryDomain.isolatedCopy(), topFramePrimaryDomain = topFramePrimaryDomain.isolatedCopy(), frameID, pageID, completionHandler = WTFMove(completionHandler)]() mutable { >+ store->callHasStorageAccessForFrameHandler(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)](bool result) mutable { >+ store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler), result] () mutable { >+ completionHandler(result); >+ }); >+ }); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::requestStorageAccess(String&& subFramePrimaryDomain, String&& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >+ if (shouldBlockAndPurgeCookies(subFrameStatistic)) { >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Cannot grant storage access to %{public}s since its cookies are blocked in third-party contexts and it has not received user interaction as first-party.", subFramePrimaryDomain.utf8().data()); >+#endif >+ completionHandler(StorageAccessStatus::CannotRequestAccess); >+ return; >+ } >+ >+ if (!shouldBlockAndKeepCookies(subFrameStatistic)) { >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "No need to grant storage access to %{public}s since its cookies are not blocked in third-party contexts.", subFramePrimaryDomain.utf8().data()); >+#endif >+ completionHandler(StorageAccessStatus::HasAccess); >+ return; >+ } >+ >+ auto userWasPromptedEarlier = promptEnabled && hasUserGrantedStorageAccessThroughPrompt(subFrameStatistic, topFramePrimaryDomain); >+ if (promptEnabled && !userWasPromptedEarlier) { >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "About to ask the user whether they want to grant storage access to %{public}s under %{public}s or not.", subFramePrimaryDomain.utf8().data(), topFramePrimaryDomain.utf8().data()); >+#endif >+ completionHandler(StorageAccessStatus::RequiresUserPrompt); >+ return; >+ } else if (userWasPromptedEarlier) { >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Storage access was granted to %{public}s under %{public}s.", subFramePrimaryDomain.utf8().data(), topFramePrimaryDomain.utf8().data()); >+#endif >+ } >+ >+ subFrameStatistic.timesAccessedAsFirstPartyDueToStorageAccessAPI++; >+ >+ grantStorageAccessInternal(WTFMove(subFramePrimaryDomain), WTFMove(topFramePrimaryDomain), frameID, pageID, userWasPromptedEarlier, [completionHandler = WTFMove(completionHandler)] (bool wasGrantedAccess) mutable { >+ completionHandler(wasGrantedAccess ? StorageAccessStatus::HasAccess : StorageAccessStatus::CannotRequestAccess); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain) >+{ >+ ASSERT(primaryDomainInNeedOfStorageAccess != openerPrimaryDomain); >+ ASSERT(!RunLoop::isMain()); >+ >+ if (primaryDomainInNeedOfStorageAccess == openerPrimaryDomain) >+ return; >+ >+ auto& domainInNeedOfStorageAccessStatistic = ensureResourceStatisticsForPrimaryDomain(primaryDomainInNeedOfStorageAccess); >+ auto cookiesBlockedAndPurged = shouldBlockAndPurgeCookies(domainInNeedOfStorageAccessStatistic); >+ >+ // The domain already has access if its cookies are not blocked. >+ if (!cookiesBlockedAndPurged && !shouldBlockAndKeepCookies(domainInNeedOfStorageAccessStatistic)) >+ return; >+ >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "[Temporary combatibility fix] Storage access was granted for %{public}s under opener page from %{public}s, with user interaction in the opened window.", primaryDomainInNeedOfStorageAccess.utf8().data(), openerPrimaryDomain.utf8().data()); >+#endif >+ grantStorageAccessInternal(WTFMove(primaryDomainInNeedOfStorageAccess), WTFMove(openerPrimaryDomain), WTF::nullopt, openerPageID, false, [](bool) { }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto subFramePrimaryDomain = ResourceLoadStatistics::primaryDomain(subFrameHost); >+ auto topFramePrimaryDomain = ResourceLoadStatistics::primaryDomain(topFrameHost); >+ if (userWasPromptedNow) { >+ auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >+ ASSERT(subFrameStatistic.hadUserInteraction); >+ subFrameStatistic.storageAccessUnderTopFrameOrigins.add(topFramePrimaryDomain); >+ } >+ grantStorageAccessInternal(WTFMove(subFramePrimaryDomain), WTFMove(topFramePrimaryDomain), frameID, pageID, userWasPromptedNow, WTFMove(completionHandler)); >+} >+ >+void ResourceLoadStatisticsMemoryStore::grantStorageAccessInternal(String&& subFramePrimaryDomain, String&& topFramePrimaryDomain, Optional<uint64_t> frameID, uint64_t pageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&& callback) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (subFramePrimaryDomain == topFramePrimaryDomain) { >+ callback(true); >+ return; >+ } >+ >+ // FIXME: Remove m_storageAccessPromptsEnabled check if prompting is no longer experimental. >+ if (userWasPromptedNowOrEarlier && m_storageAccessPromptsEnabled) { >+ auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >+ ASSERT(subFrameStatistic.hadUserInteraction); >+ ASSERT(subFrameStatistic.storageAccessUnderTopFrameOrigins.contains(topFramePrimaryDomain)); >+ subFrameStatistic.mostRecentUserInteractionTime = WallTime::now(); >+ } >+ >+ RunLoop::main().dispatch([subFramePrimaryDomain = subFramePrimaryDomain.isolatedCopy(), topFramePrimaryDomain = topFramePrimaryDomain.isolatedCopy(), frameID, pageID, store = makeRef(m_store), callback = WTFMove(callback)]() mutable { >+ store->callGrantStorageAccessHandler(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, [callback = WTFMove(callback), store = store.copyRef()](bool value) mutable { >+ store->statisticsQueue().dispatch([callback = WTFMove(callback), value] () mutable { >+ callback(value); >+ }); >+ }); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::grandfatherExistingWebsiteData(CompletionHandler<void()>&& callback) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), callback = WTFMove(callback), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef()] () mutable { >+ // FIXME: This method being a static call on WebProcessProxy is wrong. >+ // It should be on the data store that this object belongs to. >+ WebProcessProxy::topPrivatelyControlledDomainsWithWebsiteData(WebResourceLoadStatisticsStore::monitoredDataTypes(), shouldNotifyPagesWhenDataRecordsWereScanned, [weakThis = WTFMove(weakThis), callback = WTFMove(callback), workQueue = workQueue.copyRef()] (HashSet<String>&& topPrivatelyControlledDomainsWithWebsiteData) mutable { >+ workQueue->dispatch([weakThis = WTFMove(weakThis), topDomains = crossThreadCopy(topPrivatelyControlledDomainsWithWebsiteData), callback = WTFMove(callback)] () mutable { >+ if (!weakThis) { >+ callback(); >+ return; >+ } >+ >+ for (auto& topPrivatelyControlledDomain : topDomains) { >+ auto& statistic = weakThis->ensureResourceStatisticsForPrimaryDomain(topPrivatelyControlledDomain); >+ statistic.grandfathered = true; >+ } >+ weakThis->m_endOfGrandfatheringTimestamp = WallTime::now() + weakThis->m_parameters.grandfatheringTime; >+ if (weakThis->m_persistentStorage) >+ weakThis->m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::Yes); >+ callback(); >+ weakThis->logTestingEvent("Grandfathered"_s); >+ }); >+ }); >+ }); >+} >+ >+Vector<String> ResourceLoadStatisticsMemoryStore::ensurePrevalentResourcesForDebugMode() >+{ >+ if (!m_debugModeEnabled) >+ return { }; >+ >+ Vector<String> primaryDomainsToBlock; >+ primaryDomainsToBlock.reserveInitialCapacity(2); >+ >+ auto& staticSesourceStatistic = ensureResourceStatisticsForPrimaryDomain(debugStaticPrevalentResource); >+ setPrevalentResource(staticSesourceStatistic, ResourceLoadPrevalence::High); >+ primaryDomainsToBlock.uncheckedAppend(debugStaticPrevalentResource); >+ >+ if (!m_debugManualPrevalentResource.isEmpty()) { >+ auto& manualResourceStatistic = ensureResourceStatisticsForPrimaryDomain(m_debugManualPrevalentResource); >+ setPrevalentResource(manualResourceStatistic, ResourceLoadPrevalence::High); >+ primaryDomainsToBlock.uncheckedAppend(m_debugManualPrevalentResource); >+ } >+ >+ return primaryDomainsToBlock; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setResourceLoadStatisticsDebugMode(bool enable) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_debugModeEnabled = enable; >+ m_debugLoggingEnabled = enable; >+ >+ ensurePrevalentResourcesForDebugMode(); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setPrevalentResourceForDebugMode(const String& domain) >+{ >+ if (!m_debugModeEnabled) >+ return; >+ >+ m_debugManualPrevalentResource = domain; >+ auto& resourceStatistic = ensureResourceStatisticsForPrimaryDomain(domain); >+ setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >+ >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "Did set %{public}s as prevalent resource for the purposes of ITP Debug Mode.", domain.utf8().data()); >+#endif >+} >+ >+void ResourceLoadStatisticsMemoryStore::scheduleStatisticsProcessingRequestIfNecessary() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_pendingStatisticsProcessingRequestIdentifier = ++m_lastStatisticsProcessingRequestIdentifier; >+ m_workQueue->dispatchAfter(minimumStatisticsProcessingInterval, [this, weakThis = makeWeakPtr(*this), statisticsProcessingRequestIdentifier = *m_pendingStatisticsProcessingRequestIdentifier] { >+ if (!weakThis) >+ return; >+ >+ if (!m_pendingStatisticsProcessingRequestIdentifier || *m_pendingStatisticsProcessingRequestIdentifier != statisticsProcessingRequestIdentifier) { >+ // This request has been canceled. >+ return; >+ } >+ >+ updateCookieBlocking([]() { }); >+ processStatisticsAndDataRecords(); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::cancelPendingStatisticsProcessingRequest() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_pendingStatisticsProcessingRequestIdentifier = WTF::nullopt; >+} >+ >+void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, const String& sourcePrimaryDomain, const String& targetHost, const String& mainFrameHost, bool isRedirect, bool isMainFrame) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ bool areTargetAndMainFrameDomainsAssociated = targetPrimaryDomain == mainFramePrimaryDomain; >+ bool areTargetAndSourceDomainsAssociated = targetPrimaryDomain == sourcePrimaryDomain; >+ >+ bool statisticsWereUpdated = false; >+ if (!isMainFrame && targetHost != mainFrameHost && !(areTargetAndMainFrameDomainsAssociated || areTargetAndSourceDomainsAssociated)) { >+ auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain); >+ targetStatistics.lastSeen = ResourceLoadStatistics::reduceTimeResolution(WallTime::now()); >+ if (targetStatistics.subframeUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry) >+ statisticsWereUpdated = true; >+ } >+ >+ if (isRedirect && !areTargetAndSourceDomainsAssociated) { >+ if (isMainFrame) { >+ auto& redirectingOriginStatistics = ensureResourceStatisticsForPrimaryDomain(sourcePrimaryDomain); >+ if (redirectingOriginStatistics.topFrameUniqueRedirectsTo.add(targetPrimaryDomain).isNewEntry) >+ statisticsWereUpdated = true; >+ auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain); >+ if (targetStatistics.topFrameUniqueRedirectsFrom.add(sourcePrimaryDomain).isNewEntry) >+ statisticsWereUpdated = true; >+ } else { >+ auto& redirectingOriginStatistics = ensureResourceStatisticsForPrimaryDomain(sourcePrimaryDomain); >+ if (redirectingOriginStatistics.subresourceUniqueRedirectsTo.add(targetPrimaryDomain).isNewEntry) >+ statisticsWereUpdated = true; >+ auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain); >+ if (targetStatistics.subresourceUniqueRedirectsFrom.add(sourcePrimaryDomain).isNewEntry) >+ statisticsWereUpdated = true; >+ } >+ } >+ >+ if (statisticsWereUpdated) >+ scheduleStatisticsProcessingRequestIfNecessary(); >+} >+ >+void ResourceLoadStatisticsMemoryStore::logUserInteraction(const String& primaryDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >+ statistics.hadUserInteraction = true; >+ statistics.mostRecentUserInteractionTime = WallTime::now(); >+} >+ >+void ResourceLoadStatisticsMemoryStore::clearUserInteraction(const String& primaryDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >+ statistics.hadUserInteraction = false; >+ statistics.mostRecentUserInteractionTime = { }; >+} >+ >+bool ResourceLoadStatisticsMemoryStore::hasHadUserInteraction(const String& primaryDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >+ return mapEntry == m_resourceStatisticsMap.end() ? false: hasHadUnexpiredRecentUserInteraction(mapEntry->value); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setPrevalentResource(WebCore::ResourceLoadStatistics& resourceStatistic, ResourceLoadPrevalence newPrevalence) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ resourceStatistic.isPrevalentResource = true; >+ resourceStatistic.isVeryPrevalentResource = newPrevalence == ResourceLoadPrevalence::VeryHigh; >+ HashSet<String> domainsThatHaveRedirectedTo; >+ recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(resourceStatistic, domainsThatHaveRedirectedTo, 0); >+ for (auto& domain : domainsThatHaveRedirectedTo) { >+ auto mapEntry = m_resourceStatisticsMap.find(domain); >+ if (mapEntry == m_resourceStatisticsMap.end()) >+ continue; >+ ASSERT(!mapEntry->value.isPrevalentResource); >+ mapEntry->value.isPrevalentResource = true; >+ } >+} >+ >+String ResourceLoadStatisticsMemoryStore::dumpResourceLoadStatistics() const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ StringBuilder result; >+ result.appendLiteral("Resource load statistics:\n\n"); >+ for (auto& mapEntry : m_resourceStatisticsMap.values()) >+ result.append(mapEntry.toString()); >+ return result.toString(); >+} >+ >+bool ResourceLoadStatisticsMemoryStore::isPrevalentResource(const String& primaryDomain) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >+ return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource; >+} >+ >+bool ResourceLoadStatisticsMemoryStore::isVeryPrevalentResource(const String& primaryDomain) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >+ return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource && mapEntry->value.isVeryPrevalentResource; >+} >+ >+bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubresourceUnder(const String& subresourcePrimaryDomain, const String& topFramePrimaryDomain) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto mapEntry = m_resourceStatisticsMap.find(subresourcePrimaryDomain); >+ return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subresourceUnderTopFrameOrigins.contains(topFramePrimaryDomain); >+} >+ >+bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubFrameUnder(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto mapEntry = m_resourceStatisticsMap.find(subFramePrimaryDomain); >+ return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subframeUnderTopFrameOrigins.contains(topFramePrimaryDomain); >+} >+ >+bool ResourceLoadStatisticsMemoryStore::isRegisteredAsRedirectingTo(const String& hostRedirectedFromPrimaryDomain, const String& hostRedirectedToPrimaryDomain) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto mapEntry = m_resourceStatisticsMap.find(hostRedirectedFromPrimaryDomain); >+ return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subresourceUniqueRedirectsTo.contains(hostRedirectedToPrimaryDomain); >+} >+ >+void ResourceLoadStatisticsMemoryStore::clearPrevalentResource(const String& primaryDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >+ statistics.isPrevalentResource = false; >+ statistics.isVeryPrevalentResource = false; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setGrandfathered(const String& primaryDomain, bool value) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >+ statistics.grandfathered = value; >+} >+ >+bool ResourceLoadStatisticsMemoryStore::isGrandfathered(const String& primaryDomain) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >+ return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.grandfathered; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setSubframeUnderTopFrameOrigin(const String& primarySubFrameDomain, const String& primaryTopFrameDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubFrameDomain); >+ statistics.subframeUnderTopFrameOrigins.add(primaryTopFrameDomain); >+ // For consistency, make sure we also have a statistics entry for the top frame domain. >+ ensureResourceStatisticsForPrimaryDomain(primaryTopFrameDomain); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setSubresourceUnderTopFrameOrigin(const String& primarySubresourceDomain, const String& primaryTopFrameDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomain); >+ statistics.subresourceUnderTopFrameOrigins.add(primaryTopFrameDomain); >+ // For consistency, make sure we also have a statistics entry for the top frame domain. >+ ensureResourceStatisticsForPrimaryDomain(primaryTopFrameDomain); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectTo(const String& primarySubresourceDomain, const String& primaryRedirectDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomain); >+ statistics.subresourceUniqueRedirectsTo.add(primaryRedirectDomain); >+ // For consistency, make sure we also have a statistics entry for the redirect domain. >+ ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectFrom(const String& primarySubresourceDomain, const String& primaryRedirectDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomain); >+ statistics.subresourceUniqueRedirectsFrom.add(primaryRedirectDomain); >+ // For consistency, make sure we also have a statistics entry for the redirect domain. >+ ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectTo(const String& topFramePrimaryDomain, const String& primaryRedirectDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(topFramePrimaryDomain); >+ statistics.topFrameUniqueRedirectsTo.add(primaryRedirectDomain); >+ // For consistency, make sure we also have a statistics entry for the redirect domain. >+ ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectFrom(const String& topFramePrimaryDomain, const String& primaryRedirectDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(topFramePrimaryDomain); >+ statistics.topFrameUniqueRedirectsFrom.add(primaryRedirectDomain); >+ // For consistency, make sure we also have a statistics entry for the redirect domain. >+ ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setTimeToLiveUserInteraction(Seconds seconds) >+{ >+ ASSERT(!RunLoop::isMain()); >+ ASSERT(seconds >= 0_s); >+ >+ m_parameters.timeToLiveUserInteraction = seconds; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setMinimumTimeBetweenDataRecordsRemoval(Seconds seconds) >+{ >+ ASSERT(!RunLoop::isMain()); >+ ASSERT(seconds >= 0_s); >+ >+ m_parameters.minimumTimeBetweenDataRecordsRemoval = seconds; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setGrandfatheringTime(Seconds seconds) >+{ >+ ASSERT(!RunLoop::isMain()); >+ ASSERT(seconds >= 0_s); >+ >+ m_parameters.grandfatheringTime = seconds; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setCacheMaxAgeCap(Seconds seconds) >+{ >+ ASSERT(!RunLoop::isMain()); >+ ASSERT(seconds >= 0_s); >+ >+ m_parameters.cacheMaxAgeCapTime = seconds; >+ updateCacheMaxAgeCap(); >+} >+ >+void ResourceLoadStatisticsMemoryStore::updateCacheMaxAgeCap() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.cacheMaxAgeCapTime] () { >+ store->setCacheMaxAgeCap(seconds, [] { }); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setAgeCapForClientSideCookies(Seconds seconds) >+{ >+ ASSERT(!RunLoop::isMain()); >+ ASSERT(seconds >= 0_s); >+ >+ m_parameters.clientSideCookiesAgeCapTime = seconds; >+ updateClientSideCookiesAgeCap(); >+} >+ >+void ResourceLoadStatisticsMemoryStore::updateClientSideCookiesAgeCap() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+#if ENABLE(RESOURCE_LOAD_STATISTICS) >+ RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.clientSideCookiesAgeCapTime] () { >+ if (auto* websiteDataStore = store->websiteDataStore()) >+ websiteDataStore->setAgeCapForClientSideCookies(seconds, [] { }); >+ }); >+#endif >+} >+ >+bool ResourceLoadStatisticsMemoryStore::shouldRemoveDataRecords() const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (m_dataRecordsBeingRemoved) >+ return false; >+ >+#if ENABLE(NETSCAPE_PLUGIN_API) >+ for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses()) { >+ if (!m_activePluginTokens.contains(plugin->pluginProcessToken())) >+ return true; >+ } >+#endif >+ >+ return !m_lastTimeDataRecordsWereRemoved || MonotonicTime::now() >= (m_lastTimeDataRecordsWereRemoved + m_parameters.minimumTimeBetweenDataRecordsRemoval); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setDataRecordsBeingRemoved(bool value) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_dataRecordsBeingRemoved = value; >+ if (m_dataRecordsBeingRemoved) >+ m_lastTimeDataRecordsWereRemoved = MonotonicTime::now(); >+} >+ >+ResourceLoadStatistics& ResourceLoadStatisticsMemoryStore::ensureResourceStatisticsForPrimaryDomain(const String& primaryDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ return m_resourceStatisticsMap.ensure(primaryDomain, [&primaryDomain] { >+ return ResourceLoadStatistics(primaryDomain); >+ }).iterator->value; >+} >+ >+std::unique_ptr<KeyedEncoder> ResourceLoadStatisticsMemoryStore::createEncoderFromData() const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto encoder = KeyedEncoder::encoder(); >+ encoder->encodeUInt32("version", statisticsModelVersion); >+ encoder->encodeDouble("endOfGrandfatheringTimestamp", m_endOfGrandfatheringTimestamp.secondsSinceEpoch().value()); >+ >+ encoder->encodeObjects("browsingStatistics", m_resourceStatisticsMap.begin(), m_resourceStatisticsMap.end(), [](KeyedEncoder& encoderInner, const auto& origin) { >+ origin.value.encode(encoderInner); >+ }); >+ >+ encoder->encodeObjects("operatingDates", m_operatingDates.begin(), m_operatingDates.end(), [](KeyedEncoder& encoderInner, OperatingDate date) { >+ encoderInner.encodeDouble("date", date.secondsSinceEpoch().value()); >+ }); >+ >+ return encoder; >+} >+ >+void ResourceLoadStatisticsMemoryStore::mergeWithDataFromDecoder(KeyedDecoder& decoder) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ unsigned versionOnDisk; >+ if (!decoder.decodeUInt32("version", versionOnDisk)) >+ return; >+ >+ if (versionOnDisk > statisticsModelVersion) { >+ WTFLogAlways("Found resource load statistics on disk with model version %u whereas the highest supported version is %u. Resetting.", versionOnDisk, statisticsModelVersion); >+ return; >+ } >+ >+ double endOfGrandfatheringTimestamp; >+ if (decoder.decodeDouble("endOfGrandfatheringTimestamp", endOfGrandfatheringTimestamp)) >+ m_endOfGrandfatheringTimestamp = WallTime::fromRawSeconds(endOfGrandfatheringTimestamp); >+ else >+ m_endOfGrandfatheringTimestamp = { }; >+ >+ Vector<ResourceLoadStatistics> loadedStatistics; >+ bool succeeded = decoder.decodeObjects("browsingStatistics", loadedStatistics, [versionOnDisk](KeyedDecoder& decoderInner, ResourceLoadStatistics& statistics) { >+ return statistics.decode(decoderInner, versionOnDisk); >+ }); >+ >+ if (!succeeded) >+ return; >+ >+ mergeStatistics(WTFMove(loadedStatistics)); >+ updateCookieBlocking([]() { }); >+ >+ Vector<OperatingDate> operatingDates; >+ succeeded = decoder.decodeObjects("operatingDates", operatingDates, [](KeyedDecoder& decoder, OperatingDate& date) { >+ double value; >+ if (!decoder.decodeDouble("date", value)) >+ return false; >+ >+ date = OperatingDate::fromWallTime(WallTime::fromRawSeconds(value)); >+ return true; >+ }); >+ >+ if (!succeeded) >+ return; >+ >+ m_operatingDates = mergeOperatingDates(m_operatingDates, WTFMove(operatingDates)); >+} >+ >+void ResourceLoadStatisticsMemoryStore::clear(CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_resourceStatisticsMap.clear(); >+ m_operatingDates.clear(); >+ >+ auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler)); >+ >+ removeAllStorageAccess([callbackAggregator = callbackAggregator.copyRef()] { }); >+ >+ auto primaryDomainsToBlock = ensurePrevalentResourcesForDebugMode(); >+ updateCookieBlockingForDomains(primaryDomainsToBlock, [callbackAggregator = callbackAggregator.copyRef()] { }); >+} >+ >+bool ResourceLoadStatisticsMemoryStore::wasAccessedAsFirstPartyDueToUserInteraction(const ResourceLoadStatistics& current, const ResourceLoadStatistics& updated) const >+{ >+ if (!current.hadUserInteraction && !updated.hadUserInteraction) >+ return false; >+ >+ auto mostRecentUserInteractionTime = std::max(current.mostRecentUserInteractionTime, updated.mostRecentUserInteractionTime); >+ >+ return updated.lastSeen <= mostRecentUserInteractionTime + 24_h; >+} >+ >+void ResourceLoadStatisticsMemoryStore::mergeStatistics(Vector<ResourceLoadStatistics>&& statistics) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ for (auto& statistic : statistics) { >+ auto result = m_resourceStatisticsMap.ensure(statistic.highLevelDomain, [&statistic] { >+ return WTFMove(statistic); >+ }); >+ if (!result.isNewEntry) { >+ if (wasAccessedAsFirstPartyDueToUserInteraction(result.iterator->value, statistic)) >+ result.iterator->value.timesAccessedAsFirstPartyDueToUserInteraction++; >+ result.iterator->value.merge(statistic); >+ } >+ } >+} >+ >+bool ResourceLoadStatisticsMemoryStore::shouldBlockAndKeepCookies(const ResourceLoadStatistics& statistic) >+{ >+ return statistic.isPrevalentResource && statistic.hadUserInteraction; >+} >+ >+bool ResourceLoadStatisticsMemoryStore::shouldBlockAndPurgeCookies(const ResourceLoadStatistics& statistic) >+{ >+ return statistic.isPrevalentResource && !statistic.hadUserInteraction; >+} >+ >+bool ResourceLoadStatisticsMemoryStore::hasUserGrantedStorageAccessThroughPrompt(const ResourceLoadStatistics& statistic, const String& firstPartyPrimaryDomain) >+{ >+ return statistic.storageAccessUnderTopFrameOrigins.contains(firstPartyPrimaryDomain); >+} >+ >+static void debugLogDomainsInBatches(const char* action, const Vector<String>& domains) >+{ >+#if !RELEASE_LOG_DISABLED >+ static const auto maxNumberOfDomainsInOneLogStatement = 50; >+ if (domains.isEmpty()) >+ return; >+ >+ if (domains.size() <= maxNumberOfDomainsInOneLogStatement) { >+ RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for: %{public}s.", action, domainsToString(domains).utf8().data()); >+ return; >+ } >+ >+ Vector<String> batch; >+ batch.reserveInitialCapacity(maxNumberOfDomainsInOneLogStatement); >+ auto batchNumber = 1; >+ unsigned numberOfBatches = std::ceil(domains.size() / static_cast<float>(maxNumberOfDomainsInOneLogStatement)); >+ >+ for (auto& domain : domains) { >+ if (batch.size() == maxNumberOfDomainsInOneLogStatement) { >+ RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data()); >+ batch.shrink(0); >+ ++batchNumber; >+ } >+ batch.append(domain); >+ } >+ if (!batch.isEmpty()) >+ RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data()); >+#else >+ UNUSED_PARAM(action); >+ UNUSED_PARAM(domains); >+#endif >+} >+ >+void ResourceLoadStatisticsMemoryStore::updateCookieBlocking(CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ Vector<String> domainsToBlock; >+ for (auto& resourceStatistic : m_resourceStatisticsMap.values()) { >+ if (resourceStatistic.isPrevalentResource) >+ domainsToBlock.append(resourceStatistic.highLevelDomain); >+ } >+ >+ if (domainsToBlock.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ if (m_debugLoggingEnabled && !domainsToBlock.isEmpty()) >+ debugLogDomainsInBatches("block", domainsToBlock); >+ >+ RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), store = makeRef(m_store), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable { >+ store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [weakThis = WTFMove(weakThis), store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable { >+ store->statisticsQueue().dispatch([weakThis = WTFMove(weakThis), completionHandler = WTFMove(completionHandler)]() mutable { >+ completionHandler(); >+ if (!weakThis) >+ return; >+#if !RELEASE_LOG_DISABLED >+ RELEASE_LOG_INFO_IF(weakThis->m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Done updating cookie blocking."); >+#endif >+ }); >+ }); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::updateCookieBlockingForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ RunLoop::main().dispatch([store = makeRef(m_store), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable { >+ store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable { >+ store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable { >+ completionHandler(); >+ }); >+ }); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::clearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (domains.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ RunLoop::main().dispatch([store = makeRef(m_store), domains = crossThreadCopy(domains)] { >+ store->callRemoveDomainsHandler(domains); >+ }); >+ >+ completionHandler(); >+} >+ >+void ResourceLoadStatisticsMemoryStore::processStatistics(const Function<void(const ResourceLoadStatistics&)>& processFunction) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ for (auto& resourceStatistic : m_resourceStatisticsMap.values()) >+ processFunction(resourceStatistic); >+} >+ >+bool ResourceLoadStatisticsMemoryStore::hasHadUnexpiredRecentUserInteraction(ResourceLoadStatistics& resourceStatistic) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (resourceStatistic.hadUserInteraction && hasStatisticsExpired(resourceStatistic)) { >+ // Drop privacy sensitive data because we no longer need it. >+ // Set timestamp to 0 so that statistics merge will know >+ // it has been reset as opposed to its default -1. >+ resourceStatistic.mostRecentUserInteractionTime = { }; >+ resourceStatistic.storageAccessUnderTopFrameOrigins.clear(); >+ resourceStatistic.hadUserInteraction = false; >+ } >+ >+ return resourceStatistic.hadUserInteraction; >+} >+ >+Vector<String> ResourceLoadStatisticsMemoryStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ bool shouldCheckForGrandfathering = m_endOfGrandfatheringTimestamp > WallTime::now(); >+ bool shouldClearGrandfathering = !shouldCheckForGrandfathering && m_endOfGrandfatheringTimestamp; >+ >+ if (shouldClearGrandfathering) >+ m_endOfGrandfatheringTimestamp = { }; >+ >+ Vector<String> prevalentResources; >+ for (auto& statistic : m_resourceStatisticsMap.values()) { >+ if (statistic.isPrevalentResource && !hasHadUnexpiredRecentUserInteraction(statistic) && (!shouldCheckForGrandfathering || !statistic.grandfathered)) >+ prevalentResources.append(statistic.highLevelDomain); >+ >+ if (shouldClearGrandfathering && statistic.grandfathered) >+ statistic.grandfathered = false; >+ } >+ >+ return prevalentResources; >+} >+ >+void ResourceLoadStatisticsMemoryStore::includeTodayAsOperatingDateIfNecessary() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto today = OperatingDate::today(); >+ if (!m_operatingDates.isEmpty() && today <= m_operatingDates.last()) >+ return; >+ >+ while (m_operatingDates.size() >= operatingDatesWindow) >+ m_operatingDates.remove(0); >+ >+ m_operatingDates.append(today); >+} >+ >+bool ResourceLoadStatisticsMemoryStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic) const >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (m_operatingDates.size() >= operatingDatesWindow) { >+ if (OperatingDate::fromWallTime(resourceStatistic.mostRecentUserInteractionTime) < m_operatingDates.first()) >+ return true; >+ } >+ >+ // If we don't meet the real criteria for an expired statistic, check the user setting for a tighter restriction (mainly for testing). >+ if (m_parameters.timeToLiveUserInteraction) { >+ if (WallTime::now() > resourceStatistic.mostRecentUserInteractionTime + m_parameters.timeToLiveUserInteraction.value()) >+ return true; >+ } >+ >+ return false; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setMaxStatisticsEntries(size_t maximumEntryCount) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_parameters.maxStatisticsEntries = maximumEntryCount; >+} >+ >+void ResourceLoadStatisticsMemoryStore::setPruneEntriesDownTo(size_t pruneTargetCount) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_parameters.pruneEntriesDownTo = pruneTargetCount; >+} >+ >+void ResourceLoadStatisticsMemoryStore::pruneStatisticsIfNeeded() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (m_resourceStatisticsMap.size() <= m_parameters.maxStatisticsEntries) >+ return; >+ >+ ASSERT(m_parameters.pruneEntriesDownTo <= m_parameters.maxStatisticsEntries); >+ >+ size_t numberOfEntriesLeftToPrune = m_resourceStatisticsMap.size() - m_parameters.pruneEntriesDownTo; >+ ASSERT(numberOfEntriesLeftToPrune); >+ >+ Vector<StatisticsLastSeen> resourcesToPrunePerImportance[maxImportance + 1]; >+ for (auto& resourceStatistic : m_resourceStatisticsMap.values()) >+ resourcesToPrunePerImportance[computeImportance(resourceStatistic)].append({ resourceStatistic.highLevelDomain, resourceStatistic.lastSeen }); >+ >+ for (unsigned importance = 0; numberOfEntriesLeftToPrune && importance <= maxImportance; ++importance) >+ pruneResources(m_resourceStatisticsMap, resourcesToPrunePerImportance[importance], numberOfEntriesLeftToPrune); >+ >+ ASSERT(!numberOfEntriesLeftToPrune); >+} >+ >+void ResourceLoadStatisticsMemoryStore::resetParametersToDefaultValues() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_parameters = { }; >+} >+ >+void ResourceLoadStatisticsMemoryStore::logTestingEvent(const String& event) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ RunLoop::main().dispatch([store = makeRef(m_store), event = event.isolatedCopy()] { >+ store->logTestingEvent(event); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setLastSeen(const String& primaryDomain, Seconds seconds) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >+ statistics.lastSeen = WallTime::fromRawSeconds(seconds.seconds()); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setPrevalentResource(const String& primaryDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& resourceStatistic = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >+ setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >+} >+ >+void ResourceLoadStatisticsMemoryStore::setVeryPrevalentResource(const String& primaryDomain) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto& resourceStatistic = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >+ setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::VeryHigh); >+} >+ >+void ResourceLoadStatisticsMemoryStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(!RunLoop::isMain()); >+ RunLoop::main().dispatch([store = makeRef(m_store), completionHandler = WTFMove(completionHandler)]() mutable { >+ store->removeAllStorageAccess([store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable { >+ store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable { >+ completionHandler(); >+ }); >+ }); >+ }); >+} >+ >+void ResourceLoadStatisticsMemoryStore::didCreateNetworkProcess() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ updateCookieBlocking([]() { }); >+ updateCacheMaxAgeCap(); >+ updateClientSideCookiesAgeCap(); >+} >+ >+} // namespace WebKit >diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h >new file mode 100644 >index 0000000000000000000000000000000000000000..f3a9a7b96a00d85d0c1ea7b1caa61c72bd1db0c4 >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h >@@ -0,0 +1,203 @@ >+/* >+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#include "ResourceLoadStatisticsClassifier.h" >+#include "WebResourceLoadStatisticsStore.h" >+#include <wtf/CompletionHandler.h> >+#include <wtf/Vector.h> >+#include <wtf/WeakPtr.h> >+#include <wtf/WorkQueue.h> >+ >+#if HAVE(CORE_PREDICTION) >+#include "ResourceLoadStatisticsClassifierCocoa.h" >+#endif >+ >+namespace WebCore { >+class KeyedDecoder; >+class KeyedEncoder; >+struct ResourceLoadStatistics; >+} >+ >+namespace WebKit { >+ >+class OperatingDate; >+class ResourceLoadStatisticsPersistentStorage; >+ >+// This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue. >+class ResourceLoadStatisticsMemoryStore : public CanMakeWeakPtr<ResourceLoadStatisticsMemoryStore> { >+public: >+ ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore&, WorkQueue&); >+ ~ResourceLoadStatisticsMemoryStore(); >+ >+ void setPersistentStorage(ResourceLoadStatisticsPersistentStorage&); >+ >+ void clear(CompletionHandler<void()>&&); >+ bool isEmpty() const { return m_resourceStatisticsMap.isEmpty(); } >+ >+ std::unique_ptr<WebCore::KeyedEncoder> createEncoderFromData() const; >+ void mergeWithDataFromDecoder(WebCore::KeyedDecoder&); >+ >+ void mergeStatistics(Vector<WebCore::ResourceLoadStatistics>&&); >+ void processStatistics(const Function<void(const WebCore::ResourceLoadStatistics&)>&) const; >+ >+ void updateCookieBlocking(CompletionHandler<void()>&&); >+ void updateCookieBlockingForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&&); >+ void clearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&&); >+ >+ void includeTodayAsOperatingDateIfNecessary(); >+ void processStatisticsAndDataRecords(); >+ >+ void requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain); >+ void removeAllStorageAccess(CompletionHandler<void()>&&); >+ >+ void grandfatherExistingWebsiteData(CompletionHandler<void()>&&); >+ void cancelPendingStatisticsProcessingRequest(); >+ >+ bool isRegisteredAsSubresourceUnder(const String& subresourcePrimaryDomain, const String& topFramePrimaryDomain) const; >+ bool isRegisteredAsSubFrameUnder(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain) const; >+ bool isRegisteredAsRedirectingTo(const String& hostRedirectedFromPrimaryDomain, const String& hostRedirectedToPrimaryDomain) const; >+ >+ void clearPrevalentResource(const String& primaryDomain); >+ String dumpResourceLoadStatistics() const; >+ bool isPrevalentResource(const String& primaryDomain) const; >+ bool isVeryPrevalentResource(const String& primaryDomain) const; >+ void setPrevalentResource(const String& primaryDomain); >+ void setVeryPrevalentResource(const String& primaryDomain); >+ >+ void setGrandfathered(const String& primaryDomain, bool value); >+ bool isGrandfathered(const String& primaryDomain) const; >+ >+ void setSubframeUnderTopFrameOrigin(const String& primarySubFrameDomain, const String& primaryTopFrameDomain); >+ void setSubresourceUnderTopFrameOrigin(const String& primarySubresourceDomain, const String& primaryTopFrameDomain); >+ void setSubresourceUniqueRedirectTo(const String& primarySubresourceDomain, const String& primaryRedirectDomain); >+ void setSubresourceUniqueRedirectFrom(const String& primarySubresourceDomain, const String& primaryRedirectDomain); >+ void setTopFrameUniqueRedirectTo(const String& topFramePrimaryDomain, const String& primaryRedirectDomain); >+ void setTopFrameUniqueRedirectFrom(const String& topFramePrimaryDomain, const String& primaryRedirectDomain); >+ >+ void logTestingEvent(const String&); >+ >+ void setMaxStatisticsEntries(size_t maximumEntryCount); >+ void setPruneEntriesDownTo(size_t pruneTargetCount); >+ void resetParametersToDefaultValues(); >+ >+ void calculateAndSubmitTelemetry() const; >+ >+ void setNotifyPagesWhenDataRecordsWereScanned(bool); >+ void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool); >+ void setShouldSubmitTelemetry(bool); >+ void setTimeToLiveUserInteraction(Seconds); >+ void setMinimumTimeBetweenDataRecordsRemoval(Seconds); >+ void setGrandfatheringTime(Seconds); >+ void setResourceLoadStatisticsDebugMode(bool); >+ bool isDebugModeEnabled() const { return m_debugModeEnabled; }; >+ void setPrevalentResourceForDebugMode(const String& domain); >+ >+ void hasStorageAccess(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&&); >+ void requestStorageAccess(String&& subFramePrimaryDomain, String&& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&); >+ void grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&); >+ >+ void logFrameNavigation(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, const String& sourcePrimaryDomain, const String& targetHost, const String& mainFrameHost, bool isRedirect, bool isMainFrame); >+ void logUserInteraction(const String& primaryDomain); >+ >+ void clearUserInteraction(const String& primaryDomain); >+ bool hasHadUserInteraction(const String& primaryDomain); >+ >+ void setLastSeen(const String& primaryDomain, Seconds); >+ >+ void didCreateNetworkProcess(); >+ >+private: >+ static bool shouldBlockAndKeepCookies(const WebCore::ResourceLoadStatistics&); >+ static bool shouldBlockAndPurgeCookies(const WebCore::ResourceLoadStatistics&); >+ static bool hasUserGrantedStorageAccessThroughPrompt(const WebCore::ResourceLoadStatistics&, const String& firstPartyPrimaryDomain); >+ bool hasHadUnexpiredRecentUserInteraction(WebCore::ResourceLoadStatistics&) const; >+ bool hasStatisticsExpired(const WebCore::ResourceLoadStatistics&) const; >+ bool wasAccessedAsFirstPartyDueToUserInteraction(const WebCore::ResourceLoadStatistics& current, const WebCore::ResourceLoadStatistics& updated) const; >+ void setPrevalentResource(WebCore::ResourceLoadStatistics&, ResourceLoadPrevalence); >+ unsigned recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(const WebCore::ResourceLoadStatistics&, HashSet<String>& domainsThatHaveRedirectedTo, unsigned numberOfRecursiveCalls) const; >+ void setStorageAccessPromptsEnabled(bool enabled) { m_storageAccessPromptsEnabled = enabled; } >+ bool shouldRemoveDataRecords() const; >+ void setDebugLogggingEnabled(bool enabled) { m_debugLoggingEnabled = enabled; } >+ void setDataRecordsBeingRemoved(bool); >+ void scheduleStatisticsProcessingRequestIfNecessary(); >+ void grantStorageAccessInternal(String&& subFrameHost, String&& topFrameHost, Optional<uint64_t> frameID, uint64_t pageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&&); >+ void markAsPrevalentIfHasRedirectedToPrevalent(WebCore::ResourceLoadStatistics&); >+ bool isPrevalentDueToDebugMode(WebCore::ResourceLoadStatistics&); >+ Vector<String> ensurePrevalentResourcesForDebugMode(); >+ void removeDataRecords(CompletionHandler<void()>&&); >+ void pruneStatisticsIfNeeded(); >+ WebCore::ResourceLoadStatistics& ensureResourceStatisticsForPrimaryDomain(const String&); >+ Vector<String> topPrivatelyControlledDomainsToRemoveWebsiteDataFor(); >+ void setCacheMaxAgeCap(Seconds); >+ void updateCacheMaxAgeCap(); >+ void setAgeCapForClientSideCookies(Seconds); >+ void updateClientSideCookiesAgeCap(); >+ >+#if PLATFORM(COCOA) >+ void registerUserDefaultsIfNeeded(); >+#endif >+ >+ struct Parameters { >+ size_t pruneEntriesDownTo { 800 }; >+ size_t maxStatisticsEntries { 1000 }; >+ Optional<Seconds> timeToLiveUserInteraction; >+ Seconds minimumTimeBetweenDataRecordsRemoval { 1_h }; >+ Seconds grandfatheringTime { 24_h * 7 }; >+ Seconds cacheMaxAgeCapTime { 24_h * 7 }; >+ Seconds clientSideCookiesAgeCapTime { 24_h * 7 }; >+ bool shouldNotifyPagesWhenDataRecordsWereScanned { false }; >+ bool shouldClassifyResourcesBeforeDataRecordsRemoval { true }; >+ bool shouldSubmitTelemetry { true }; >+ }; >+ >+ WebResourceLoadStatisticsStore& m_store; >+ Ref<WorkQueue> m_workQueue; >+ WeakPtr<ResourceLoadStatisticsPersistentStorage> m_persistentStorage; >+ HashMap<String, WebCore::ResourceLoadStatistics> m_resourceStatisticsMap; >+#if HAVE(CORE_PREDICTION) >+ ResourceLoadStatisticsClassifierCocoa m_resourceLoadStatisticsClassifier; >+#else >+ ResourceLoadStatisticsClassifier m_resourceLoadStatisticsClassifier; >+#endif >+#if ENABLE(NETSCAPE_PLUGIN_API) >+ HashSet<uint64_t> m_activePluginTokens; >+#endif >+ Parameters m_parameters; >+ Vector<OperatingDate> m_operatingDates; >+ WallTime m_endOfGrandfatheringTimestamp; >+ bool m_debugLoggingEnabled { false }; >+ bool m_debugModeEnabled { false }; >+ String m_debugManualPrevalentResource; >+ bool m_storageAccessPromptsEnabled { false }; >+ bool m_dataRecordsBeingRemoved { false }; >+ MonotonicTime m_lastTimeDataRecordsWereRemoved; >+ >+ uint64_t m_lastStatisticsProcessingRequestIdentifier { 0 }; >+ Optional<uint64_t> m_pendingStatisticsProcessingRequestIdentifier; >+}; >+ >+} // namespace WebKit >diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..3dd781b8dc70819fbfe172060939a54714c8c219 >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp >@@ -0,0 +1,272 @@ >+/* >+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+#include "ResourceLoadStatisticsPersistentStorage.h" >+ >+#include "Logging.h" >+#include "PersistencyUtils.h" >+#include "ResourceLoadStatisticsMemoryStore.h" >+#include "WebResourceLoadStatisticsStore.h" >+#include <WebCore/FileMonitor.h> >+#include <WebCore/FileSystem.h> >+#include <WebCore/KeyedCoding.h> >+#include <WebCore/SharedBuffer.h> >+#include <wtf/RunLoop.h> >+#include <wtf/WorkQueue.h> >+ >+namespace WebKit { >+ >+constexpr Seconds minimumWriteInterval { 5_min }; >+ >+using namespace WebCore; >+ >+static bool hasFileChangedSince(const String& path, WallTime since) >+{ >+ ASSERT(!RunLoop::isMain()); >+ auto modificationTime = FileSystem::getFileModificationTime(path); >+ if (!modificationTime) >+ return true; >+ >+ return modificationTime.value() > since; >+} >+ >+ResourceLoadStatisticsPersistentStorage::ResourceLoadStatisticsPersistentStorage(ResourceLoadStatisticsMemoryStore& memoryStore, WorkQueue& workQueue, const String& storageDirectoryPath) >+ : m_memoryStore(memoryStore) >+ , m_workQueue(workQueue) >+ , m_storageDirectoryPath(storageDirectoryPath) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_memoryStore.setPersistentStorage(*this); >+ >+ populateMemoryStoreFromDisk(); >+ startMonitoringDisk(); >+} >+ >+ResourceLoadStatisticsPersistentStorage::~ResourceLoadStatisticsPersistentStorage() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ if (m_hasPendingWrite) >+ writeMemoryStoreToDisk(); >+} >+ >+String ResourceLoadStatisticsPersistentStorage::storageDirectoryPath() const >+{ >+ return m_storageDirectoryPath.isolatedCopy(); >+} >+ >+String ResourceLoadStatisticsPersistentStorage::resourceLogFilePath() const >+{ >+ String storagePath = storageDirectoryPath(); >+ if (storagePath.isEmpty()) >+ return emptyString(); >+ >+ return FileSystem::pathByAppendingComponent(storagePath, "full_browsing_session_resourceLog.plist"); >+} >+ >+void ResourceLoadStatisticsPersistentStorage::startMonitoringDisk() >+{ >+ ASSERT(!RunLoop::isMain()); >+ if (m_fileMonitor) >+ return; >+ >+ String resourceLogPath = resourceLogFilePath(); >+ if (resourceLogPath.isEmpty()) >+ return; >+ >+ m_fileMonitor = std::make_unique<FileMonitor>(resourceLogPath, m_workQueue.copyRef(), [this, weakThis = makeWeakPtr(*this)] (FileMonitor::FileChangeType type) { >+ ASSERT(!RunLoop::isMain()); >+ if (!weakThis) >+ return; >+ >+ switch (type) { >+ case FileMonitor::FileChangeType::Modification: >+ refreshMemoryStoreFromDisk(); >+ break; >+ case FileMonitor::FileChangeType::Removal: >+ m_memoryStore.clear([] { }); >+ m_fileMonitor = nullptr; >+ monitorDirectoryForNewStatistics(); >+ break; >+ } >+ }); >+} >+ >+void ResourceLoadStatisticsPersistentStorage::monitorDirectoryForNewStatistics() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ String storagePath = storageDirectoryPath(); >+ ASSERT(!storagePath.isEmpty()); >+ >+ if (!FileSystem::fileExists(storagePath)) { >+ if (!FileSystem::makeAllDirectories(storagePath)) { >+ RELEASE_LOG_ERROR(ResourceLoadStatistics, "ResourceLoadStatisticsPersistentStorage: Failed to create directory path %s", storagePath.utf8().data()); >+ return; >+ } >+ } >+ >+ m_fileMonitor = std::make_unique<FileMonitor>(storagePath, m_workQueue.copyRef(), [this] (FileMonitor::FileChangeType type) { >+ ASSERT(!RunLoop::isMain()); >+ if (type == FileMonitor::FileChangeType::Removal) { >+ // Directory was removed! >+ m_fileMonitor = nullptr; >+ return; >+ } >+ >+ String resourceLogPath = resourceLogFilePath(); >+ ASSERT(!resourceLogPath.isEmpty()); >+ >+ if (!FileSystem::fileExists(resourceLogPath)) >+ return; >+ >+ m_fileMonitor = nullptr; >+ >+ refreshMemoryStoreFromDisk(); >+ startMonitoringDisk(); >+ }); >+} >+ >+void ResourceLoadStatisticsPersistentStorage::stopMonitoringDisk() >+{ >+ ASSERT(!RunLoop::isMain()); >+ m_fileMonitor = nullptr; >+} >+ >+// This is called when the file changes on disk. >+void ResourceLoadStatisticsPersistentStorage::refreshMemoryStoreFromDisk() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ String filePath = resourceLogFilePath(); >+ if (filePath.isEmpty()) >+ return; >+ >+ // We sometimes see file changed events from before our load completed (we start >+ // reading at the first change event, but we might receive a series of events related >+ // to the same file operation). Catch this case to avoid reading overly often. >+ if (!hasFileChangedSince(filePath, m_lastStatisticsFileSyncTime)) >+ return; >+ >+ WallTime readTime = WallTime::now(); >+ >+ auto decoder = createForFile(filePath); >+ if (!decoder) >+ return; >+ >+ m_memoryStore.mergeWithDataFromDecoder(*decoder); >+ m_lastStatisticsFileSyncTime = readTime; >+} >+ >+void ResourceLoadStatisticsPersistentStorage::populateMemoryStoreFromDisk() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ String filePath = resourceLogFilePath(); >+ if (filePath.isEmpty() || !FileSystem::fileExists(filePath)) { >+ m_memoryStore.grandfatherExistingWebsiteData([]() { }); >+ monitorDirectoryForNewStatistics(); >+ return; >+ } >+ >+ if (!hasFileChangedSince(filePath, m_lastStatisticsFileSyncTime)) { >+ // No need to grandfather in this case. >+ return; >+ } >+ >+ WallTime readTime = WallTime::now(); >+ >+ auto decoder = createForFile(filePath); >+ if (!decoder) { >+ m_memoryStore.grandfatherExistingWebsiteData([]() { }); >+ return; >+ } >+ >+ // Debug mode has a prepoulated memory store. >+ ASSERT_WITH_MESSAGE(m_memoryStore.isEmpty() || m_memoryStore.isDebugModeEnabled(), "This is the initial import so the store should be empty"); >+ m_memoryStore.mergeWithDataFromDecoder(*decoder); >+ >+ m_lastStatisticsFileSyncTime = readTime; >+ >+ m_memoryStore.logTestingEvent("PopulatedWithoutGrandfathering"_s); >+} >+ >+void ResourceLoadStatisticsPersistentStorage::writeMemoryStoreToDisk() >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ m_hasPendingWrite = false; >+ stopMonitoringDisk(); >+ >+ writeToDisk(m_memoryStore.createEncoderFromData(), resourceLogFilePath()); >+ >+ m_lastStatisticsFileSyncTime = WallTime::now(); >+ m_lastStatisticsWriteTime = MonotonicTime::now(); >+ >+ startMonitoringDisk(); >+} >+ >+void ResourceLoadStatisticsPersistentStorage::scheduleOrWriteMemoryStore(ForceImmediateWrite forceImmediateWrite) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto timeSinceLastWrite = MonotonicTime::now() - m_lastStatisticsWriteTime; >+ if (forceImmediateWrite != ForceImmediateWrite::Yes && timeSinceLastWrite < minimumWriteInterval) { >+ if (!m_hasPendingWrite) { >+ m_hasPendingWrite = true; >+ Seconds delay = minimumWriteInterval - timeSinceLastWrite + 1_s; >+ m_workQueue->dispatchAfter(delay, [weakThis = makeWeakPtr(*this)] { >+ if (weakThis) >+ weakThis->writeMemoryStoreToDisk(); >+ }); >+ } >+ return; >+ } >+ >+ writeMemoryStoreToDisk(); >+} >+ >+void ResourceLoadStatisticsPersistentStorage::clear() >+{ >+ ASSERT(!RunLoop::isMain()); >+ String filePath = resourceLogFilePath(); >+ if (filePath.isEmpty()) >+ return; >+ >+ stopMonitoringDisk(); >+ >+ if (!FileSystem::deleteFile(filePath) && FileSystem::fileExists(filePath)) >+ RELEASE_LOG_ERROR(ResourceLoadStatistics, "ResourceLoadStatisticsPersistentStorage: Unable to delete statistics file: %s", filePath.utf8().data()); >+} >+ >+#if !PLATFORM(IOS_FAMILY) >+void ResourceLoadStatisticsPersistentStorage::excludeFromBackup() const >+{ >+} >+#endif >+ >+} // namespace WebKit >diff --git a/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.h b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.h >new file mode 100644 >index 0000000000000000000000000000000000000000..12aa9b8aec2fd5001f1d2cc6aa33dbb2728ee6be >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.h >@@ -0,0 +1,77 @@ >+/* >+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#include <wtf/Forward.h> >+#include <wtf/MonotonicTime.h> >+#include <wtf/RunLoop.h> >+#include <wtf/WallTime.h> >+#include <wtf/WeakPtr.h> >+#include <wtf/WorkQueue.h> >+#include <wtf/text/WTFString.h> >+ >+namespace WebCore { >+class FileMonitor; >+} >+ >+namespace WebKit { >+ >+class ResourceLoadStatisticsMemoryStore; >+ >+// Can only be constructed / destroyed / used from the WebResourceLoadStatisticsStore's statistic queue. >+class ResourceLoadStatisticsPersistentStorage : public CanMakeWeakPtr<ResourceLoadStatisticsPersistentStorage> { >+public: >+ ResourceLoadStatisticsPersistentStorage(ResourceLoadStatisticsMemoryStore&, WorkQueue&, const String& storageDirectoryPath); >+ ~ResourceLoadStatisticsPersistentStorage(); >+ >+ void clear(); >+ >+ enum class ForceImmediateWrite { No, Yes, }; >+ void scheduleOrWriteMemoryStore(ForceImmediateWrite); >+ >+private: >+ String storageDirectoryPath() const; >+ String resourceLogFilePath() const; >+ >+ void startMonitoringDisk(); >+ void stopMonitoringDisk(); >+ void monitorDirectoryForNewStatistics(); >+ >+ void writeMemoryStoreToDisk(); >+ void populateMemoryStoreFromDisk(); >+ void excludeFromBackup() const; >+ void refreshMemoryStoreFromDisk(); >+ >+ ResourceLoadStatisticsMemoryStore& m_memoryStore; >+ Ref<WorkQueue> m_workQueue; >+ const String m_storageDirectoryPath; >+ std::unique_ptr<WebCore::FileMonitor> m_fileMonitor; >+ WallTime m_lastStatisticsFileSyncTime; >+ MonotonicTime m_lastStatisticsWriteTime; >+ bool m_hasPendingWrite { false }; >+}; >+ >+} >diff --git a/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..451e7108d7810e41a56473408c2511dfa10fbbc3 >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >@@ -0,0 +1,902 @@ >+/* >+ * Copyright (C) 2016-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+#include "WebResourceLoadStatisticsStore.h" >+ >+#include "Logging.h" >+#include "ResourceLoadStatisticsMemoryStore.h" >+#include "ResourceLoadStatisticsPersistentStorage.h" >+#include "WebFrameProxy.h" >+#include "WebPageProxy.h" >+#include "WebProcessMessages.h" >+#include "WebProcessProxy.h" >+#include "WebResourceLoadStatisticsStoreMessages.h" >+#include "WebResourceLoadStatisticsTelemetry.h" >+#include "WebsiteDataFetchOption.h" >+#include "WebsiteDataStore.h" >+#include <WebCore/ResourceLoadStatistics.h> >+#include <wtf/CallbackAggregator.h> >+#include <wtf/CrossThreadCopier.h> >+#include <wtf/NeverDestroyed.h> >+#include <wtf/threads/BinarySemaphore.h> >+ >+namespace WebKit { >+using namespace WebCore; >+ >+template<typename T> static inline String isolatedPrimaryDomain(const T& value) >+{ >+ return ResourceLoadStatistics::primaryDomain(value).isolatedCopy(); >+} >+ >+const OptionSet<WebsiteDataType>& WebResourceLoadStatisticsStore::monitoredDataTypes() >+{ >+ static NeverDestroyed<OptionSet<WebsiteDataType>> dataTypes(std::initializer_list<WebsiteDataType>({ >+ WebsiteDataType::Cookies, >+ WebsiteDataType::DOMCache, >+ WebsiteDataType::IndexedDBDatabases, >+ WebsiteDataType::LocalStorage, >+ WebsiteDataType::MediaKeys, >+ WebsiteDataType::OfflineWebApplicationCache, >+#if ENABLE(NETSCAPE_PLUGIN_API) >+ WebsiteDataType::PlugInData, >+#endif >+ WebsiteDataType::SearchFieldRecentSearches, >+ WebsiteDataType::SessionStorage, >+#if ENABLE(SERVICE_WORKER) >+ WebsiteDataType::ServiceWorkerRegistrations, >+#endif >+ WebsiteDataType::WebSQLDatabases, >+ })); >+ >+ ASSERT(RunLoop::isMain()); >+ >+ return dataTypes; >+} >+ >+void WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bool value) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, value] { >+ if (m_memoryStore) >+ m_memoryStore->setNotifyPagesWhenDataRecordsWereScanned(value); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, value] { >+ if (m_memoryStore) >+ m_memoryStore->setShouldClassifyResourcesBeforeDataRecordsRemoval(value); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setShouldSubmitTelemetry(bool value) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, value] { >+ if (m_memoryStore) >+ m_memoryStore->setShouldSubmitTelemetry(value); >+ }); >+} >+ >+WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(WebsiteDataStore& websiteDataStore) >+ : m_websiteDataStore(makeWeakPtr(websiteDataStore)) >+ , m_statisticsQueue(WorkQueue::create("WebResourceLoadStatisticsStore Process Data Queue", WorkQueue::Type::Serial, WorkQueue::QOS::Utility)) >+ , m_dailyTasksTimer(RunLoop::main(), this, &WebResourceLoadStatisticsStore::performDailyTasks) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, resourceLoadStatisticsDirectory = websiteDataStore.resolvedResourceLoadStatisticsDirectory().isolatedCopy()] { >+ m_memoryStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue); >+ m_persistentStorage = std::make_unique<ResourceLoadStatisticsPersistentStorage>(*m_memoryStore, m_statisticsQueue, resourceLoadStatisticsDirectory); >+ }); >+ >+ m_dailyTasksTimer.startRepeating(24_h); >+} >+ >+WebResourceLoadStatisticsStore::~WebResourceLoadStatisticsStore() >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ flushAndDestroyPersistentStore(); >+} >+ >+inline void WebResourceLoadStatisticsStore::postTask(WTF::Function<void()>&& task) >+{ >+ ASSERT(RunLoop::isMain()); >+ m_statisticsQueue->dispatch([protectedThis = makeRef(*this), task = WTFMove(task)] { >+ task(); >+ }); >+} >+ >+inline void WebResourceLoadStatisticsStore::postTaskReply(WTF::Function<void()>&& reply) >+{ >+ ASSERT(!RunLoop::isMain()); >+ RunLoop::main().dispatch(WTFMove(reply)); >+} >+ >+void WebResourceLoadStatisticsStore::flushAndDestroyPersistentStore() >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (!m_persistentStorage && !m_memoryStore) >+ return; >+ >+ // Make sure we destroy the persistent store on the background queue and wait for it to die >+ // synchronously since it has a C++ reference to us. Blocking nature of this task allows us >+ // to not maintain a WebResourceLoadStatisticsStore reference for the duration of dispatch, >+ // avoiding double-deletion issues when this is invoked from the destructor. >+ BinarySemaphore semaphore; >+ m_statisticsQueue->dispatch([&semaphore, this] { >+ m_persistentStorage = nullptr; >+ m_memoryStore = nullptr; >+ semaphore.signal(); >+ }); >+ semaphore.wait(); >+} >+ >+void WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode(bool value, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, value, completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->setResourceLoadStatisticsDebugMode(value); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setPrevalentResourceForDebugMode(const URL& url, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->setPrevalentResourceForDebugMode(primaryDomain); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::scheduleStatisticsAndDataRecordsProcessing() >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this] { >+ if (m_memoryStore) >+ m_memoryStore->processStatisticsAndDataRecords(); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(Vector<WebCore::ResourceLoadStatistics>&& origins) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ // It is safe to move the origins to the background queue without isolated copy here because this is an r-value >+ // coming from IPC. ResourceLoadStatistics only contains strings which are safe to move to other threads as long >+ // as nobody on this thread holds a reference to those strings. >+ postTask([this, origins = WTFMove(origins)]() mutable { >+ if (!m_memoryStore) >+ return; >+ >+ m_memoryStore->mergeStatistics(WTFMove(origins)); >+ >+ // We can cancel any pending request to process statistics since we're doing it synchronously below. >+ m_memoryStore->cancelPendingStatisticsProcessingRequest(); >+ >+ // Fire before processing statistics to propagate user interaction as fast as possible to the network process. >+ m_memoryStore->updateCookieBlocking([]() { }); >+ m_memoryStore->processStatisticsAndDataRecords(); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::hasStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, CompletionHandler<void (bool)>&& completionHandler) >+{ >+ ASSERT(subFrameHost != topFrameHost); >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, subFramePrimaryDomain = isolatedPrimaryDomain(subFrameHost), topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHost), frameID, pageID, completionHandler = WTFMove(completionHandler)] () mutable { >+ if (!m_memoryStore) { >+ postTaskReply([completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(false); >+ }); >+ return; >+ } >+ m_memoryStore->hasStorageAccess(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, [completionHandler = WTFMove(completionHandler)](bool hasStorageAccess) mutable { >+ postTaskReply([completionHandler = WTFMove(completionHandler), hasStorageAccess] () mutable { >+ completionHandler(hasStorageAccess); >+ }); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::callHasStorageAccessForFrameHandler(const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool hasAccess)>&& callback) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+#if ENABLE(RESOURCE_LOAD_STATISTICS) >+ if (m_websiteDataStore) { >+ m_websiteDataStore->hasStorageAccessForFrameHandler(resourceDomain, firstPartyDomain, frameID, pageID, WTFMove(callback)); >+ return; >+ } >+#endif >+ callback(false); >+} >+ >+void WebResourceLoadStatisticsStore::requestStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&& completionHandler) >+{ >+ ASSERT(subFrameHost != topFrameHost); >+ ASSERT(RunLoop::isMain()); >+ >+ auto subFramePrimaryDomain = isolatedPrimaryDomain(subFrameHost); >+ auto topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHost); >+ if (subFramePrimaryDomain == topFramePrimaryDomain) { >+ completionHandler(StorageAccessStatus::HasAccess); >+ return; >+ } >+ >+ postTask([this, subFramePrimaryDomain = crossThreadCopy(subFramePrimaryDomain), topFramePrimaryDomain = crossThreadCopy(topFramePrimaryDomain), frameID, pageID, promptEnabled, completionHandler = WTFMove(completionHandler)] () mutable { >+ if (!m_memoryStore) { >+ postTaskReply([completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(StorageAccessStatus::CannotRequestAccess); >+ }); >+ return; >+ } >+ >+ m_memoryStore->requestStorageAccess(WTFMove(subFramePrimaryDomain), WTFMove(topFramePrimaryDomain), frameID, pageID, promptEnabled, [completionHandler = WTFMove(completionHandler)](StorageAccessStatus status) mutable { >+ postTaskReply([completionHandler = WTFMove(completionHandler), status] () mutable { >+ completionHandler(status); >+ }); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ // It is safe to move the strings to the background queue without isolated copy here because they are r-value references >+ // coming from IPC. Strings which are safe to move to other threads as long as nobody on this thread holds a reference >+ // to those strings. >+ postTask([this, primaryDomainInNeedOfStorageAccess = WTFMove(primaryDomainInNeedOfStorageAccess), openerPageID, openerPrimaryDomain = WTFMove(openerPrimaryDomain)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->requestStorageAccessUnderOpener(WTFMove(primaryDomainInNeedOfStorageAccess), openerPageID, WTFMove(openerPrimaryDomain)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ postTask([this, subFrameHost = crossThreadCopy(subFrameHost), topFrameHost = crossThreadCopy(topFrameHost), frameID, pageID, userWasPromptedNow, completionHandler = WTFMove(completionHandler)] () mutable { >+ if (!m_memoryStore) { >+ postTaskReply([completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(false); >+ }); >+ return; >+ } >+ >+ m_memoryStore->grantStorageAccess(WTFMove(subFrameHost), WTFMove(topFrameHost), frameID, pageID, userWasPromptedNow, [completionHandler = WTFMove(completionHandler)](bool wasGrantedAccess) mutable { >+ postTaskReply([completionHandler = WTFMove(completionHandler), wasGrantedAccess] () mutable { >+ completionHandler(wasGrantedAccess); >+ }); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::callGrantStorageAccessHandler(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, Optional<uint64_t> frameID, uint64_t pageID, CompletionHandler<void(bool)>&& callback) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+#if ENABLE(RESOURCE_LOAD_STATISTICS) >+ if (m_websiteDataStore) { >+ m_websiteDataStore->grantStorageAccessHandler(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, WTFMove(callback)); >+ return; >+ } >+#endif >+ callback(false); >+} >+ >+void WebResourceLoadStatisticsStore::didCreateNetworkProcess() >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this] { >+ if (!m_memoryStore) >+ return; >+ m_memoryStore->didCreateNetworkProcess(); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+#if ENABLE(RESOURCE_LOAD_STATISTICS) >+ if (m_websiteDataStore) >+ m_websiteDataStore->removeAllStorageAccessHandler(WTFMove(completionHandler)); >+ else >+ completionHandler(); >+#else >+ completionHandler(); >+#endif >+} >+ >+void WebResourceLoadStatisticsStore::applicationWillTerminate() >+{ >+ flushAndDestroyPersistentStore(); >+} >+ >+void WebResourceLoadStatisticsStore::performDailyTasks() >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this] { >+ if (!m_memoryStore) >+ return; >+ >+ m_memoryStore->includeTodayAsOperatingDateIfNecessary(); >+ m_memoryStore->calculateAndSubmitTelemetry(); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::submitTelemetry() >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this] { >+ if (m_memoryStore) >+ WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*m_memoryStore); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ auto sourceURL = redirectURL; >+ bool isRedirect = !redirectURL.isNull(); >+ if (!isRedirect) { >+ sourceURL = frame.url(); >+ if (sourceURL.isNull()) >+ sourceURL = pageURL; >+ } >+ >+ auto& targetURL = request.url(); >+ >+ if (!targetURL.isValid() || !pageURL.isValid()) >+ return; >+ >+ auto targetHost = targetURL.host(); >+ auto mainFrameHost = pageURL.host(); >+ >+ if (targetHost.isEmpty() || mainFrameHost.isEmpty() || targetHost == sourceURL.host()) >+ return; >+ >+ auto targetPrimaryDomain = ResourceLoadStatistics::primaryDomain(targetURL); >+ auto mainFramePrimaryDomain = ResourceLoadStatistics::primaryDomain(pageURL); >+ auto sourcePrimaryDomain = ResourceLoadStatistics::primaryDomain(sourceURL); >+ >+ postTask([this, targetPrimaryDomain = targetPrimaryDomain.isolatedCopy(), mainFramePrimaryDomain = mainFramePrimaryDomain.isolatedCopy(), sourcePrimaryDomain = sourcePrimaryDomain.isolatedCopy(), targetHost = targetHost.toString().isolatedCopy(), mainFrameHost = mainFrameHost.toString().isolatedCopy(), isRedirect, isMainFrame = frame.isMainFrame()] { >+ >+ if (m_memoryStore) >+ m_memoryStore->logFrameNavigation(targetPrimaryDomain, mainFramePrimaryDomain, sourcePrimaryDomain, targetHost, mainFrameHost, isRedirect, isMainFrame); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::logUserInteraction(const URL& url, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->logUserInteraction(primaryDomain); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::clearUserInteraction(const URL& url, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->clearUserInteraction(primaryDomain); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::hasHadUserInteraction(const URL& url, CompletionHandler<void (bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(false); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)] () mutable { >+ bool hadUserInteraction = m_memoryStore ? m_memoryStore->hasHadUserInteraction(primaryDomain) : false; >+ postTaskReply([hadUserInteraction, completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(hadUserInteraction); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setLastSeen(const URL& url, Seconds seconds, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), seconds, completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->setLastSeen(primaryDomain, seconds); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setPrevalentResource(const URL& url, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->setPrevalentResource(primaryDomain); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setVeryPrevalentResource(const URL& url, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->setVeryPrevalentResource(primaryDomain); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::dumpResourceLoadStatistics(CompletionHandler<void(const String&)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, completionHandler = WTFMove(completionHandler)] () mutable { >+ String result = m_memoryStore ? m_memoryStore->dumpResourceLoadStatistics() : emptyString(); >+ postTaskReply([result = result.isolatedCopy(), completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(result); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::isPrevalentResource(const URL& url, CompletionHandler<void (bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(false); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)] () mutable { >+ bool isPrevalentResource = m_memoryStore ? m_memoryStore->isPrevalentResource(primaryDomain) : false; >+ postTaskReply([isPrevalentResource, completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(isPrevalentResource); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::isVeryPrevalentResource(const URL& url, CompletionHandler<void(bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(false); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)] () mutable { >+ bool isVeryPrevalentResource = m_memoryStore ? m_memoryStore->isVeryPrevalentResource(primaryDomain) : false; >+ postTaskReply([isVeryPrevalentResource, completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(isVeryPrevalentResource); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::isRegisteredAsSubresourceUnder(const URL& subresource, const URL& topFrame, CompletionHandler<void(bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, subresourcePrimaryDomain = isolatedPrimaryDomain(subresource), topFramePrimaryDomain = isolatedPrimaryDomain(topFrame), completionHandler = WTFMove(completionHandler)] () mutable { >+ bool isRegisteredAsSubresourceUnder = m_memoryStore ? m_memoryStore->isRegisteredAsSubresourceUnder(subresourcePrimaryDomain, topFramePrimaryDomain) : false; >+ postTaskReply([isRegisteredAsSubresourceUnder, completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(isRegisteredAsSubresourceUnder); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::isRegisteredAsSubFrameUnder(const URL& subFrame, const URL& topFrame, CompletionHandler<void (bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, subFramePrimaryDomain = isolatedPrimaryDomain(subFrame), topFramePrimaryDomain = isolatedPrimaryDomain(topFrame), completionHandler = WTFMove(completionHandler)] () mutable { >+ bool isRegisteredAsSubFrameUnder = m_memoryStore ? m_memoryStore->isRegisteredAsSubFrameUnder(subFramePrimaryDomain, topFramePrimaryDomain) : false; >+ postTaskReply([isRegisteredAsSubFrameUnder, completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(isRegisteredAsSubFrameUnder); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::isRegisteredAsRedirectingTo(const URL& hostRedirectedFrom, const URL& hostRedirectedTo, CompletionHandler<void (bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, hostRedirectedFromPrimaryDomain = isolatedPrimaryDomain(hostRedirectedFrom), hostRedirectedToPrimaryDomain = isolatedPrimaryDomain(hostRedirectedTo), completionHandler = WTFMove(completionHandler)] () mutable { >+ bool isRegisteredAsRedirectingTo = m_memoryStore ? m_memoryStore->isRegisteredAsRedirectingTo(hostRedirectedFromPrimaryDomain, hostRedirectedToPrimaryDomain) : false; >+ postTaskReply([isRegisteredAsRedirectingTo, completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(isRegisteredAsRedirectingTo); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::clearPrevalentResource(const URL& url, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(); >+ return; >+ } >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->clearPrevalentResource(primaryDomain); >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setGrandfathered(const URL& url, bool value) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) >+ return; >+ >+ postTask([this, primaryDomain = isolatedPrimaryDomain(url), value] { >+ if (m_memoryStore) >+ m_memoryStore->setGrandfathered(primaryDomain, value); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::isGrandfathered(const URL& url, CompletionHandler<void (bool)>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (url.protocolIsAbout() || url.isEmpty()) { >+ completionHandler(false); >+ return; >+ } >+ >+ postTask([this, completionHandler = WTFMove(completionHandler), primaryDomain = isolatedPrimaryDomain(url)] () mutable { >+ bool isGrandFathered = m_memoryStore ? m_memoryStore->isGrandfathered(primaryDomain) : false; >+ postTaskReply([isGrandFathered, completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(isGrandFathered); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setSubframeUnderTopFrameOrigin(const URL& subframe, const URL& topFrame) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (subframe.protocolIsAbout() || subframe.isEmpty() || topFrame.protocolIsAbout() || topFrame.isEmpty()) >+ return; >+ >+ postTask([this, primaryTopFrameDomain = isolatedPrimaryDomain(topFrame), primarySubFrameDomain = isolatedPrimaryDomain(subframe)] { >+ if (m_memoryStore) >+ m_memoryStore->setSubframeUnderTopFrameOrigin(primarySubFrameDomain, primaryTopFrameDomain); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setSubresourceUnderTopFrameOrigin(const URL& subresource, const URL& topFrame) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (subresource.protocolIsAbout() || subresource.isEmpty() || topFrame.protocolIsAbout() || topFrame.isEmpty()) >+ return; >+ >+ postTask([this, primaryTopFrameDomain = isolatedPrimaryDomain(topFrame), primarySubresourceDomain = isolatedPrimaryDomain(subresource)] { >+ if (m_memoryStore) >+ m_memoryStore->setSubresourceUnderTopFrameOrigin(primarySubresourceDomain, primaryTopFrameDomain); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectTo(const URL& subresource, const URL& hostNameRedirectedTo) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (subresource.protocolIsAbout() || subresource.isEmpty() || hostNameRedirectedTo.protocolIsAbout() || hostNameRedirectedTo.isEmpty()) >+ return; >+ >+ postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedTo), primarySubresourceDomain = isolatedPrimaryDomain(subresource)] { >+ if (m_memoryStore) >+ m_memoryStore->setSubresourceUniqueRedirectTo(primarySubresourceDomain, primaryRedirectDomain); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectFrom(const URL& subresource, const URL& hostNameRedirectedFrom) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (subresource.protocolIsAbout() || subresource.isEmpty() || hostNameRedirectedFrom.protocolIsAbout() || hostNameRedirectedFrom.isEmpty()) >+ return; >+ >+ postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedFrom), primarySubresourceDomain = isolatedPrimaryDomain(subresource)] { >+ if (m_memoryStore) >+ m_memoryStore->setSubresourceUniqueRedirectFrom(primarySubresourceDomain, primaryRedirectDomain); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectTo(const URL& topFrameHostName, const URL& hostNameRedirectedTo) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (topFrameHostName.protocolIsAbout() || topFrameHostName.isEmpty() || hostNameRedirectedTo.protocolIsAbout() || hostNameRedirectedTo.isEmpty()) >+ return; >+ >+ postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedTo), topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHostName)] { >+ if (m_memoryStore) >+ m_memoryStore->setTopFrameUniqueRedirectTo(topFramePrimaryDomain, primaryRedirectDomain); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectFrom(const URL& topFrameHostName, const URL& hostNameRedirectedFrom) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (topFrameHostName.protocolIsAbout() || topFrameHostName.isEmpty() || hostNameRedirectedFrom.protocolIsAbout() || hostNameRedirectedFrom.isEmpty()) >+ return; >+ >+ postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedFrom), topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHostName)] { >+ if (m_memoryStore) >+ m_memoryStore->setTopFrameUniqueRedirectFrom(topFramePrimaryDomain, primaryRedirectDomain); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdate(CompletionHandler<void()>&& completionHandler) >+{ >+ // Helper function used by testing system. Should only be called from the main thread. >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, completionHandler = WTFMove(completionHandler)] () mutable { >+ if (!m_memoryStore) { >+ postTaskReply(WTFMove(completionHandler)); >+ return; >+ } >+ m_memoryStore->updateCookieBlocking([completionHandler = WTFMove(completionHandler)] () mutable { >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdateForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler) >+{ >+ // Helper function used by testing system. Should only be called from the main thread. >+ ASSERT(RunLoop::isMain()); >+ postTask([this, domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable { >+ if (!m_memoryStore) { >+ postTaskReply(WTFMove(completionHandler)); >+ return; >+ } >+ >+ m_memoryStore->updateCookieBlockingForDomains(domainsToBlock, [completionHandler = WTFMove(completionHandler)]() mutable { >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::scheduleClearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&& completionHandler) >+{ >+ // Helper function used by testing system. Should only be called from the main thread. >+ ASSERT(RunLoop::isMain()); >+ postTask([this, domains = crossThreadCopy(domains), completionHandler = WTFMove(completionHandler)] () mutable { >+ if (!m_memoryStore) { >+ postTaskReply(WTFMove(completionHandler)); >+ return; >+ } >+ >+ m_memoryStore->clearBlockingStateForDomains(domains, [completionHandler = WTFMove(completionHandler)]() mutable { >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent(ShouldGrandfather shouldGrandfather, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ postTask([this, protectedThis = makeRef(*this), shouldGrandfather, completionHandler = WTFMove(completionHandler)] () mutable { >+ if (m_persistentStorage) >+ m_persistentStorage->clear(); >+ >+ CompletionHandlerCallingScope completionHandlerCaller([completionHandler = WTFMove(completionHandler)]() mutable { >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+ >+ if (m_memoryStore) { >+ m_memoryStore->clear([this, protectedThis = protectedThis.copyRef(), shouldGrandfather, completionHandlerCaller = WTFMove(completionHandlerCaller)] () mutable { >+ if (shouldGrandfather == ShouldGrandfather::Yes) { >+ if (m_memoryStore) >+ m_memoryStore->grandfatherExistingWebsiteData(completionHandlerCaller.release()); >+ else >+ RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent After being cleared, m_memoryStore is null when trying to grandfather data."); >+ } >+ }); >+ } else { >+ if (shouldGrandfather == ShouldGrandfather::Yes) >+ RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent Before being cleared, m_memoryStore is null when trying to grandfather data."); >+ } >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent(WallTime modifiedSince, ShouldGrandfather shouldGrandfather, CompletionHandler<void()>&& callback) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ // For now, be conservative and clear everything regardless of modifiedSince. >+ UNUSED_PARAM(modifiedSince); >+ scheduleClearInMemoryAndPersistent(shouldGrandfather, WTFMove(callback)); >+} >+ >+void WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction(Seconds seconds) >+{ >+ ASSERT(RunLoop::isMain()); >+ postTask([this, seconds] { >+ if (m_memoryStore) >+ m_memoryStore->setTimeToLiveUserInteraction(seconds); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval(Seconds seconds) >+{ >+ ASSERT(RunLoop::isMain()); >+ postTask([this, seconds] { >+ if (m_memoryStore) >+ m_memoryStore->setMinimumTimeBetweenDataRecordsRemoval(seconds); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setGrandfatheringTime(Seconds seconds) >+{ >+ ASSERT(RunLoop::isMain()); >+ postTask([this, seconds] { >+ if (m_memoryStore) >+ m_memoryStore->setGrandfatheringTime(seconds); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setCacheMaxAgeCap(Seconds seconds, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ ASSERT(seconds >= 0_s); >+ >+#if ENABLE(RESOURCE_LOAD_STATISTICS) >+ if (m_websiteDataStore) { >+ m_websiteDataStore->setCacheMaxAgeCapForPrevalentResources(seconds, WTFMove(completionHandler)); >+ return; >+ } >+#endif >+ completionHandler(); >+} >+ >+void WebResourceLoadStatisticsStore::callUpdatePrevalentDomainsToBlockCookiesForHandler(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+#if ENABLE(RESOURCE_LOAD_STATISTICS) >+ if (m_websiteDataStore) { >+ m_websiteDataStore->updatePrevalentDomainsToBlockCookiesFor(domainsToBlock, WTFMove(completionHandler)); >+ return; >+ } >+#endif >+ completionHandler(); >+} >+ >+void WebResourceLoadStatisticsStore::callRemoveDomainsHandler(const Vector<String>& domains) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+#if ENABLE(RESOURCE_LOAD_STATISTICS) >+ if (m_websiteDataStore) >+ m_websiteDataStore->removePrevalentDomains(domains); >+#endif >+} >+ >+void WebResourceLoadStatisticsStore::setMaxStatisticsEntries(size_t maximumEntryCount) >+{ >+ ASSERT(RunLoop::isMain()); >+ postTask([this, maximumEntryCount] { >+ if (m_memoryStore) >+ m_memoryStore->setMaxStatisticsEntries(maximumEntryCount); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::setPruneEntriesDownTo(size_t pruneTargetCount) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, pruneTargetCount] { >+ if (m_memoryStore) >+ m_memoryStore->setPruneEntriesDownTo(pruneTargetCount); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::resetParametersToDefaultValues(CompletionHandler<void()>&& completionHandler) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ postTask([this, completionHandler = WTFMove(completionHandler)]() mutable { >+ if (m_memoryStore) >+ m_memoryStore->resetParametersToDefaultValues(); >+ >+ postTaskReply(WTFMove(completionHandler)); >+ }); >+} >+ >+void WebResourceLoadStatisticsStore::logTestingEvent(const String& event) >+{ >+ ASSERT(RunLoop::isMain()); >+ >+ if (m_statisticsTestingCallback) >+ m_statisticsTestingCallback(event); >+} >+ >+} // namespace WebKit >diff --git a/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h >new file mode 100644 >index 0000000000000000000000000000000000000000..581704a4ef299dc083d279f534c77281cfa76b4e >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h >@@ -0,0 +1,177 @@ >+/* >+ * Copyright (C) 2016-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#include "Connection.h" >+#include "WebsiteDataType.h" >+#include <wtf/CompletionHandler.h> >+#include <wtf/RunLoop.h> >+#include <wtf/ThreadSafeRefCounted.h> >+#include <wtf/Vector.h> >+#include <wtf/WallTime.h> >+#include <wtf/WeakPtr.h> >+#include <wtf/text/WTFString.h> >+ >+namespace WTF { >+class WorkQueue; >+} >+ >+namespace WebCore { >+class ResourceRequest; >+struct ResourceLoadStatistics; >+} >+ >+namespace WebKit { >+ >+class ResourceLoadStatisticsMemoryStore; >+class ResourceLoadStatisticsPersistentStorage; >+class WebFrameProxy; >+class WebProcessProxy; >+class WebsiteDataStore; >+ >+enum class StorageAccessStatus { >+ CannotRequestAccess, >+ RequiresUserPrompt, >+ HasAccess >+}; >+ >+class WebResourceLoadStatisticsStore final : public ThreadSafeRefCounted<WebResourceLoadStatisticsStore, WTF::DestructionThread::Main>, public IPC::MessageReceiver { >+public: >+ static Ref<WebResourceLoadStatisticsStore> create(WebsiteDataStore& websiteDataStore) >+ { >+ return adoptRef(*new WebResourceLoadStatisticsStore(websiteDataStore)); >+ } >+ >+ ~WebResourceLoadStatisticsStore(); >+ >+ static const OptionSet<WebsiteDataType>& monitoredDataTypes(); >+ >+ WorkQueue& statisticsQueue() { return m_statisticsQueue.get(); } >+ >+ void setNotifyPagesWhenDataRecordsWereScanned(bool); >+ void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool); >+ void setShouldSubmitTelemetry(bool); >+ >+ void hasStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&& callback); >+ void requestStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&); >+ void grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&); >+ void requestStorageAccessCallback(bool wasGranted, uint64_t contextId); >+ >+ void applicationWillTerminate(); >+ >+ void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL); >+ void logUserInteraction(const URL&, CompletionHandler<void()>&&); >+ void clearUserInteraction(const URL&, CompletionHandler<void()>&&); >+ void hasHadUserInteraction(const URL&, CompletionHandler<void(bool)>&&); >+ void setLastSeen(const URL&, Seconds, CompletionHandler<void()>&&); >+ void setPrevalentResource(const URL&, CompletionHandler<void()>&&); >+ void setVeryPrevalentResource(const URL&, CompletionHandler<void()>&&); >+ void dumpResourceLoadStatistics(CompletionHandler<void(const String&)>&&); >+ void isPrevalentResource(const URL&, CompletionHandler<void(bool)>&&); >+ void isVeryPrevalentResource(const URL&, CompletionHandler<void(bool)>&&); >+ void isRegisteredAsSubresourceUnder(const URL& subresource, const URL& topFrame, CompletionHandler<void(bool)>&&); >+ void isRegisteredAsSubFrameUnder(const URL& subFrame, const URL& topFrame, CompletionHandler<void(bool)>&&); >+ void isRegisteredAsRedirectingTo(const URL& hostRedirectedFrom, const URL& hostRedirectedTo, CompletionHandler<void(bool)>&&); >+ void clearPrevalentResource(const URL&, CompletionHandler<void()>&&); >+ void setGrandfathered(const URL&, bool); >+ void isGrandfathered(const URL&, CompletionHandler<void(bool)>&&); >+ void setSubframeUnderTopFrameOrigin(const URL& subframe, const URL& topFrame); >+ void setSubresourceUnderTopFrameOrigin(const URL& subresource, const URL& topFrame); >+ void setSubresourceUniqueRedirectTo(const URL& subresource, const URL& hostNameRedirectedTo); >+ void setSubresourceUniqueRedirectFrom(const URL& subresource, const URL& hostNameRedirectedFrom); >+ void setTopFrameUniqueRedirectTo(const URL& topFrameHostName, const URL& hostNameRedirectedTo); >+ void setTopFrameUniqueRedirectFrom(const URL& topFrameHostName, const URL& hostNameRedirectedFrom); >+ void scheduleCookieBlockingUpdate(CompletionHandler<void()>&&); >+ void scheduleCookieBlockingUpdateForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&&); >+ void scheduleClearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&&); >+ void scheduleStatisticsAndDataRecordsProcessing(); >+ void submitTelemetry(); >+ >+ enum class ShouldGrandfather { >+ No, >+ Yes, >+ }; >+ void scheduleClearInMemoryAndPersistent(ShouldGrandfather, CompletionHandler<void()>&&); >+ void scheduleClearInMemoryAndPersistent(WallTime modifiedSince, ShouldGrandfather, CompletionHandler<void()>&&); >+ >+ void setTimeToLiveUserInteraction(Seconds); >+ void setMinimumTimeBetweenDataRecordsRemoval(Seconds); >+ void setGrandfatheringTime(Seconds); >+ void setCacheMaxAgeCap(Seconds, CompletionHandler<void()>&&); >+ void setMaxStatisticsEntries(size_t); >+ void setPruneEntriesDownTo(size_t); >+ >+ void resetParametersToDefaultValues(CompletionHandler<void()>&&); >+ >+ void setResourceLoadStatisticsDebugMode(bool, CompletionHandler<void()>&&); >+ void setPrevalentResourceForDebugMode(const URL&, CompletionHandler<void()>&&); >+ >+ void setStatisticsTestingCallback(WTF::Function<void(const String&)>&& callback) { m_statisticsTestingCallback = WTFMove(callback); } >+ void logTestingEvent(const String&); >+ void callGrantStorageAccessHandler(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, Optional<uint64_t> frameID, uint64_t pageID, CompletionHandler<void(bool)>&&); >+ void removeAllStorageAccess(CompletionHandler<void()>&&); >+ void callUpdatePrevalentDomainsToBlockCookiesForHandler(const Vector<String>& domainsToBlock, CompletionHandler<void()>&&); >+ void callRemoveDomainsHandler(const Vector<String>& domains); >+ void callHasStorageAccessForFrameHandler(const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&&); >+ >+ void didCreateNetworkProcess(); >+ >+ WebsiteDataStore* websiteDataStore() { return m_websiteDataStore.get(); } >+ >+private: >+ explicit WebResourceLoadStatisticsStore(WebsiteDataStore&); >+ >+ void postTask(WTF::Function<void()>&&); >+ static void postTaskReply(WTF::Function<void()>&&); >+ >+ // IPC::MessageReceiver. >+ void didReceiveMessage(IPC::Connection&, IPC::Decoder&) override; >+ >+ // IPC message handlers. >+ void resourceLoadStatisticsUpdated(Vector<WebCore::ResourceLoadStatistics>&& origins); >+ void requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain); >+ >+ void performDailyTasks(); >+ >+ StorageAccessStatus storageAccessStatus(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain); >+ >+ void flushAndDestroyPersistentStore(); >+ >+ WeakPtr<WebsiteDataStore> m_websiteDataStore; >+ Ref<WorkQueue> m_statisticsQueue; >+ std::unique_ptr<ResourceLoadStatisticsMemoryStore> m_memoryStore; >+ std::unique_ptr<ResourceLoadStatisticsPersistentStorage> m_persistentStorage; >+ >+ RunLoop::Timer<WebResourceLoadStatisticsStore> m_dailyTasksTimer; >+ >+ bool m_hasScheduledProcessStats { false }; >+ >+ WTF::Function<void(const String&)> m_statisticsTestingCallback; >+ >+ bool m_firstNetworkProcessCreated { false }; >+}; >+ >+} // namespace WebKit >diff --git a/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..f9692e12912a6c1c098d3b9f60804ae2ff37df9a >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp >@@ -0,0 +1,296 @@ >+/* >+ * Copyright (C) 2017 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+#include "WebResourceLoadStatisticsTelemetry.h" >+ >+#include "ResourceLoadStatisticsMemoryStore.h" >+#include "WebProcessPool.h" >+#include "WebProcessProxy.h" >+#include <WebCore/DiagnosticLoggingKeys.h> >+#include <WebCore/ResourceLoadStatistics.h> >+#include <wtf/MainThread.h> >+#include <wtf/NeverDestroyed.h> >+#include <wtf/RunLoop.h> >+#include <wtf/text/StringBuilder.h> >+ >+namespace WebKit { >+using namespace WebCore; >+ >+const unsigned minimumPrevalentResourcesForTelemetry = 3; >+const unsigned significantFiguresForLoggedValues = 3; >+static bool notifyPagesWhenTelemetryWasCaptured = false; >+ >+struct PrevalentResourceTelemetry { >+ unsigned numberOfTimesDataRecordsRemoved; >+ bool hasHadUserInteraction; >+ unsigned daysSinceUserInteraction; >+ unsigned subframeUnderTopFrameOrigins; >+ unsigned subresourceUnderTopFrameOrigins; >+ unsigned subresourceUniqueRedirectsTo; >+ unsigned timesAccessedAsFirstPartyDueToUserInteraction; >+ unsigned timesAccessedAsFirstPartyDueToStorageAccessAPI; >+}; >+ >+static Vector<PrevalentResourceTelemetry> sortedPrevalentResourceTelemetry(const ResourceLoadStatisticsMemoryStore& store) >+{ >+ ASSERT(!RunLoop::isMain()); >+ Vector<PrevalentResourceTelemetry> sorted; >+ store.processStatistics([&sorted] (auto& statistic) { >+ if (!statistic.isPrevalentResource) >+ return; >+ >+ unsigned daysSinceUserInteraction = statistic.mostRecentUserInteractionTime <= WallTime() ? 0 : std::floor((WallTime::now() - statistic.mostRecentUserInteractionTime) / 24_h); >+ sorted.append(PrevalentResourceTelemetry { >+ statistic.dataRecordsRemoved, >+ statistic.hadUserInteraction, >+ daysSinceUserInteraction, >+ statistic.subframeUnderTopFrameOrigins.size(), >+ statistic.subresourceUnderTopFrameOrigins.size(), >+ statistic.subresourceUniqueRedirectsTo.size(), >+ statistic.timesAccessedAsFirstPartyDueToUserInteraction, >+ statistic.timesAccessedAsFirstPartyDueToStorageAccessAPI >+ }); >+ }); >+ >+ if (sorted.size() < minimumPrevalentResourcesForTelemetry) >+ return { }; >+ >+ std::sort(sorted.begin(), sorted.end(), [](const PrevalentResourceTelemetry& a, const PrevalentResourceTelemetry& b) { >+ return a.subframeUnderTopFrameOrigins + a.subresourceUnderTopFrameOrigins + a.subresourceUniqueRedirectsTo > >+ b.subframeUnderTopFrameOrigins + b.subresourceUnderTopFrameOrigins + b.subresourceUniqueRedirectsTo; >+ }); >+ >+ return sorted; >+} >+ >+static unsigned numberOfResourcesWithUserInteraction(const Vector<PrevalentResourceTelemetry>& resources, size_t begin, size_t end) >+{ >+ if (resources.isEmpty() || resources.size() < begin + 1 || resources.size() < end + 1) >+ return 0; >+ >+ unsigned result = 0; >+ for (size_t i = begin; i < end; ++i) { >+ if (resources[i].hasHadUserInteraction) >+ ++result; >+ } >+ >+ return result; >+} >+ >+static unsigned median(const Vector<unsigned>& v) >+{ >+ if (v.isEmpty()) >+ return 0; >+ if (v.size() == 1) >+ return v[0]; >+ >+ auto size = v.size(); >+ auto middle = size / 2; >+ if (size % 2) >+ return v[middle]; >+ return (v[middle - 1] + v[middle]) / 2; >+} >+ >+static unsigned median(const Vector<PrevalentResourceTelemetry>& v, unsigned begin, unsigned end, const WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)>& statisticGetter) >+{ >+ if (v.isEmpty() || v.size() < begin + 1 || v.size() < end + 1) >+ return 0; >+ >+ Vector<unsigned> part; >+ part.reserveInitialCapacity(end - begin + 1); >+ for (unsigned i = begin; i <= end; ++i) >+ part.uncheckedAppend(statisticGetter(v[i])); >+ >+ return median(part); >+} >+ >+static WebPageProxy* nonEphemeralWebPageProxy() >+{ >+ auto processPools = WebProcessPool::allProcessPools(); >+ if (processPools.isEmpty()) >+ return nullptr; >+ >+ auto processPool = processPools[0]; >+ if (!processPool) >+ return nullptr; >+ >+ for (auto& webProcess : processPool->processes()) { >+ for (auto& page : webProcess->pages()) { >+ if (page->sessionID().isEphemeral()) >+ continue; >+ return page; >+ } >+ } >+ return nullptr; >+} >+ >+static void submitTopList(unsigned numberOfResourcesFromTheTop, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, WebPageProxy& webPageProxy) >+{ >+ WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subframeUnderTopFrameOriginsGetter = [] (auto& t) { >+ return t.subframeUnderTopFrameOrigins; >+ }; >+ WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUnderTopFrameOriginsGetter = [] (auto& t) { >+ return t.subresourceUnderTopFrameOrigins; >+ }; >+ WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUniqueRedirectsToGetter = [] (auto& t) { >+ return t.subresourceUniqueRedirectsTo; >+ }; >+ WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesDataRecordsRemovedGetter = [] (auto& t) { >+ return t.numberOfTimesDataRecordsRemoved; >+ }; >+ WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesAccessedAsFirstPartyDueToUserInteractionGetter = [] (auto& t) { >+ return t.timesAccessedAsFirstPartyDueToUserInteraction; >+ }; >+ WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesAccessedAsFirstPartyDueToStorageAccessAPIGetter = [] (auto& t) { >+ return t.timesAccessedAsFirstPartyDueToStorageAccessAPI; >+ }; >+ >+ unsigned topPrevalentResourcesWithUserInteraction = numberOfResourcesWithUserInteraction(sortedPrevalentResources, 0, numberOfResourcesFromTheTop - 1); >+ unsigned topSubframeUnderTopFrameOrigins = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subframeUnderTopFrameOriginsGetter); >+ unsigned topSubresourceUnderTopFrameOrigins = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subresourceUnderTopFrameOriginsGetter); >+ unsigned topSubresourceUniqueRedirectsTo = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subresourceUniqueRedirectsToGetter); >+ unsigned topNumberOfTimesDataRecordsRemoved = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesDataRecordsRemovedGetter); >+ unsigned topNumberOfTimesAccessedAsFirstPartyDueToUserInteraction = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesAccessedAsFirstPartyDueToUserInteractionGetter); >+ unsigned topNumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesAccessedAsFirstPartyDueToStorageAccessAPIGetter); >+ >+ StringBuilder preambleBuilder; >+ preambleBuilder.appendLiteral("top"); >+ preambleBuilder.appendNumber(numberOfResourcesFromTheTop); >+ String descriptionPreamble = preambleBuilder.toString(); >+ >+ webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "PrevalentResourcesWithUserInteraction", >+ topPrevalentResourcesWithUserInteraction, significantFiguresForLoggedValues, ShouldSample::No); >+ webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "SubframeUnderTopFrameOrigins", >+ topSubframeUnderTopFrameOrigins, significantFiguresForLoggedValues, ShouldSample::No); >+ webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "SubresourceUnderTopFrameOrigins", >+ topSubresourceUnderTopFrameOrigins, significantFiguresForLoggedValues, ShouldSample::No); >+ webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "SubresourceUniqueRedirectsTo", >+ topSubresourceUniqueRedirectsTo, significantFiguresForLoggedValues, ShouldSample::No); >+ webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesDataRecordsRemoved", >+ topNumberOfTimesDataRecordsRemoved, significantFiguresForLoggedValues, ShouldSample::No); >+ webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesAccessedAsFirstPartyDueToUserInteraction", >+ topNumberOfTimesAccessedAsFirstPartyDueToUserInteraction, significantFiguresForLoggedValues, ShouldSample::No); >+ webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI", >+ topNumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI, significantFiguresForLoggedValues, ShouldSample::No); >+} >+ >+static void submitTopLists(const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, WebPageProxy& webPageProxy) >+{ >+ submitTopList(1, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >+ >+ if (sortedPrevalentResourcesWithoutUserInteraction.size() < 3) >+ return; >+ submitTopList(3, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >+ >+ if (sortedPrevalentResourcesWithoutUserInteraction.size() < 10) >+ return; >+ submitTopList(10, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >+ >+ if (sortedPrevalentResourcesWithoutUserInteraction.size() < 50) >+ return; >+ submitTopList(50, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >+ >+ if (sortedPrevalentResourcesWithoutUserInteraction.size() < 100) >+ return; >+ submitTopList(100, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >+} >+ >+// This function is for testing purposes. >+void static notifyPages(unsigned totalPrevalentResources, unsigned totalPrevalentResourcesWithUserInteraction, unsigned top3SubframeUnderTopFrameOrigins) >+{ >+ RunLoop::main().dispatch([totalPrevalentResources, totalPrevalentResourcesWithUserInteraction, top3SubframeUnderTopFrameOrigins] { >+ API::Dictionary::MapType messageBody; >+ messageBody.set("TotalPrevalentResources"_s, API::UInt64::create(totalPrevalentResources)); >+ messageBody.set("TotalPrevalentResourcesWithUserInteraction"_s, API::UInt64::create(totalPrevalentResourcesWithUserInteraction)); >+ messageBody.set("Top3SubframeUnderTopFrameOrigins"_s, API::UInt64::create(top3SubframeUnderTopFrameOrigins)); >+ WebProcessProxy::notifyPageStatisticsTelemetryFinished(API::Dictionary::create(messageBody).ptr()); >+ }); >+} >+ >+// This function is for testing purposes. >+void static notifyPages(const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, unsigned totalNumberOfPrevalentResourcesWithUserInteraction) >+{ >+ WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subframeUnderTopFrameOriginsGetter = [] (const PrevalentResourceTelemetry& t) { >+ return t.subframeUnderTopFrameOrigins; >+ }; >+ >+ notifyPages(sortedPrevalentResources.size(), totalNumberOfPrevalentResourcesWithUserInteraction, median(sortedPrevalentResourcesWithoutUserInteraction, 0, 2, subframeUnderTopFrameOriginsGetter)); >+} >+ >+void WebResourceLoadStatisticsTelemetry::calculateAndSubmit(const ResourceLoadStatisticsMemoryStore& resourceLoadStatisticsStore) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto sortedPrevalentResources = sortedPrevalentResourceTelemetry(resourceLoadStatisticsStore); >+ if (notifyPagesWhenTelemetryWasCaptured && sortedPrevalentResources.isEmpty()) { >+ notifyPages(0, 0, 0); >+ return; >+ } >+ >+ Vector<PrevalentResourceTelemetry> sortedPrevalentResourcesWithoutUserInteraction; >+ sortedPrevalentResourcesWithoutUserInteraction.reserveInitialCapacity(sortedPrevalentResources.size()); >+ Vector<unsigned> prevalentResourcesDaysSinceUserInteraction; >+ >+ for (auto& prevalentResource : sortedPrevalentResources) { >+ if (prevalentResource.hasHadUserInteraction) >+ prevalentResourcesDaysSinceUserInteraction.append(prevalentResource.daysSinceUserInteraction); >+ else >+ sortedPrevalentResourcesWithoutUserInteraction.uncheckedAppend(prevalentResource); >+ } >+ >+ // Dispatch on the main thread to make sure the WebPageProxy we're using doesn't go away. >+ RunLoop::main().dispatch([sortedPrevalentResources = WTFMove(sortedPrevalentResources), sortedPrevalentResourcesWithoutUserInteraction = WTFMove(sortedPrevalentResourcesWithoutUserInteraction), prevalentResourcesDaysSinceUserInteraction = WTFMove(prevalentResourcesDaysSinceUserInteraction)] () { >+ auto webPageProxy = nonEphemeralWebPageProxy(); >+ if (!webPageProxy) { >+ if (notifyPagesWhenTelemetryWasCaptured) >+ notifyPages(0, 0, 0); >+ return; >+ } >+ >+ if (notifyPagesWhenTelemetryWasCaptured) { >+ notifyPages(sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, prevalentResourcesDaysSinceUserInteraction.size()); >+ // The notify pages function is for testing so we don't need to do an actual submission. >+ return; >+ } >+ >+ webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "totalNumberOfPrevalentResources"_s, sortedPrevalentResources.size(), significantFiguresForLoggedValues, ShouldSample::No); >+ webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "totalNumberOfPrevalentResourcesWithUserInteraction"_s, prevalentResourcesDaysSinceUserInteraction.size(), significantFiguresForLoggedValues, ShouldSample::No); >+ >+ if (prevalentResourcesDaysSinceUserInteraction.size() > 0) >+ webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "topPrevalentResourceWithUserInteractionDaysSinceUserInteraction"_s, prevalentResourcesDaysSinceUserInteraction[0], significantFiguresForLoggedValues, ShouldSample::No); >+ if (prevalentResourcesDaysSinceUserInteraction.size() > 1) >+ webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "medianPrevalentResourcesWithUserInteractionDaysSinceUserInteraction"_s, median(prevalentResourcesDaysSinceUserInteraction), significantFiguresForLoggedValues, ShouldSample::No); >+ >+ submitTopLists(sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, *webPageProxy); >+ }); >+} >+ >+void WebResourceLoadStatisticsTelemetry::setNotifyPagesWhenTelemetryWasCaptured(bool always) >+{ >+ notifyPagesWhenTelemetryWasCaptured = always; >+} >+ >+} >diff --git a/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.h b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.h >new file mode 100644 >index 0000000000000000000000000000000000000000..f96e61a1af0ab7353b031db753f9ddb6f52c629b >--- /dev/null >+++ b/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.h >@@ -0,0 +1,40 @@ >+/* >+ * Copyright (C) 2017 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#include <wtf/text/WTFString.h> >+ >+namespace WebKit { >+ >+class ResourceLoadStatisticsMemoryStore; >+ >+namespace WebResourceLoadStatisticsTelemetry { >+ >+void calculateAndSubmit(const ResourceLoadStatisticsMemoryStore&); >+void setNotifyPagesWhenTelemetryWasCaptured(bool); >+ >+} >+} >diff --git a/Source/WebKit/NetworkProcess/Downloads/PendingDownload.cpp b/Source/WebKit/NetworkProcess/Downloads/PendingDownload.cpp >index 309a9319f6c3c864ebd57a6a9720cf3f5642a5b0..3ec983e1f689009868729aa1864d82ca1e519a3f 100644 >--- a/Source/WebKit/NetworkProcess/Downloads/PendingDownload.cpp >+++ b/Source/WebKit/NetworkProcess/Downloads/PendingDownload.cpp >@@ -27,6 +27,7 @@ > #include "PendingDownload.h" > > #include "DataReference.h" >+#include "Download.h" > #include "DownloadProxyMessages.h" > #include "NetworkLoad.h" > #include "NetworkProcess.h" >diff --git a/Source/WebKit/NetworkProcess/Downloads/PendingDownload.h b/Source/WebKit/NetworkProcess/Downloads/PendingDownload.h >index 06f5311f7980ec39d2d874b1c4e922e5da5e9235..4b92f1b0efdad6f644ce9f5dd6b2276de5e8ff73 100644 >--- a/Source/WebKit/NetworkProcess/Downloads/PendingDownload.h >+++ b/Source/WebKit/NetworkProcess/Downloads/PendingDownload.h >@@ -27,6 +27,7 @@ > > #include "MessageSender.h" > #include "NetworkLoadClient.h" >+#include "SandboxExtension.h" > > namespace IPC { > class Connection; >@@ -43,7 +44,7 @@ class DownloadID; > class NetworkLoad; > class NetworkLoadParameters; > class NetworkSession; >- >+ > class PendingDownload : public NetworkLoadClient, public IPC::MessageSender { > WTF_MAKE_FAST_ALLOCATED; > public: >diff --git a/Source/WebKit/PlatformMac.cmake b/Source/WebKit/PlatformMac.cmake >index d67e64cd974037288598697affd3c6e7320566ad..36871d3111af4fcadbd592564d297da8da794aba 100644 >--- a/Source/WebKit/PlatformMac.cmake >+++ b/Source/WebKit/PlatformMac.cmake >@@ -19,6 +19,9 @@ if (NOT AVFAUDIO_LIBRARY-NOTFOUND) > endif () > > list(APPEND WebKit_SOURCES >+ NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >+ NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp >+ > NetworkProcess/Cookies/mac/WebCookieManagerMac.mm > > NetworkProcess/CustomProtocols/LegacyCustomProtocolManager.cpp >@@ -158,8 +161,6 @@ list(APPEND WebKit_SOURCES > > UIProcess/HighPerformanceGraphicsUsageSampler.cpp > UIProcess/PerActivityStateCPUUsageSampler.cpp >- UIProcess/WebResourceLoadStatisticsStore.cpp >- UIProcess/WebResourceLoadStatisticsTelemetry.cpp > > UIProcess/Automation/WebAutomationSession.cpp > >diff --git a/Source/WebKit/PlatformWin.cmake b/Source/WebKit/PlatformWin.cmake >index 2aa35ad4835799641c0944a7af9c90794e8d901b..2003b0adf5adf4ef24b136b44d4db1929c0f1218 100644 >--- a/Source/WebKit/PlatformWin.cmake >+++ b/Source/WebKit/PlatformWin.cmake >@@ -8,6 +8,9 @@ file(MAKE_DIRECTORY ${DERIVED_SOURCES_WEBKIT_DIR}) > add_definitions(-DBUILDING_WEBKIT) > > list(APPEND WebKit_SOURCES >+ NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >+ NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp >+ > NetworkProcess/win/NetworkProcessMainWin.cpp > > Platform/IPC/win/AttachmentWin.cpp >@@ -36,8 +39,6 @@ list(APPEND WebKit_SOURCES > UIProcess/DrawingAreaProxyImpl.cpp > UIProcess/LegacySessionStateCodingNone.cpp > UIProcess/WebGrammarDetail.cpp >- UIProcess/WebResourceLoadStatisticsStore.cpp >- UIProcess/WebResourceLoadStatisticsTelemetry.cpp > UIProcess/WebViewportAttributes.cpp > > UIProcess/API/C/WKViewportAttributes.cpp >diff --git a/Source/WebKit/Shared/PersistencyUtils.cpp b/Source/WebKit/Shared/PersistencyUtils.cpp >new file mode 100644 >index 0000000000000000000000000000000000000000..5dba968bd7e0b38c3d22694d6d5d2e6b069d2a81 >--- /dev/null >+++ b/Source/WebKit/Shared/PersistencyUtils.cpp >@@ -0,0 +1,89 @@ >+/* >+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#include "config.h" >+#include "PersistencyUtils.h" >+ >+#include "Logging.h" >+#include <WebCore/FileSystem.h> >+#include <WebCore/SharedBuffer.h> >+#include <wtf/RunLoop.h> >+ >+namespace WebKit { >+ >+using namespace WebCore; >+ >+std::unique_ptr<KeyedDecoder> createForFile(const String& path) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto handle = FileSystem::openAndLockFile(path, FileSystem::FileOpenMode::Read); >+ if (handle == FileSystem::invalidPlatformFileHandle) >+ return nullptr; >+ >+ long long fileSize = 0; >+ if (!FileSystem::getFileSize(handle, fileSize) || !fileSize) { >+ FileSystem::unlockAndCloseFile(handle); >+ return nullptr; >+ } >+ >+ size_t bytesToRead; >+ if (!WTF::convertSafely(fileSize, bytesToRead)) { >+ FileSystem::unlockAndCloseFile(handle); >+ return nullptr; >+ } >+ >+ Vector<char> buffer(bytesToRead); >+ size_t totalBytesRead = FileSystem::readFromFile(handle, buffer.data(), buffer.size()); >+ >+ FileSystem::unlockAndCloseFile(handle); >+ >+ if (totalBytesRead != bytesToRead) >+ return nullptr; >+ >+ // FIXME: We should try to modify the constructor to pass &&. >+ return KeyedDecoder::decoder(reinterpret_cast<const uint8_t*>(buffer.data()), buffer.size()); >+} >+ >+void writeToDisk(std::unique_ptr<KeyedEncoder>&& encoder, String&& path) >+{ >+ ASSERT(!RunLoop::isMain()); >+ >+ auto rawData = encoder->finishEncoding(); >+ if (!rawData) >+ return; >+ >+ FileSystem::PlatformFileHandle handle = FileSystem::openAndLockFile(path, FileSystem::FileOpenMode::Write); >+ if (handle == FileSystem::invalidPlatformFileHandle) >+ return; >+ >+ auto writtenBytes = FileSystem::writeToFile(handle, rawData->data(), rawData->size()); >+ FileSystem::unlockAndCloseFile(handle); >+ >+ if (writtenBytes != static_cast<int64_t>(rawData->size())) >+ RELEASE_LOG_ERROR(DiskPersistency, "Disk persistency: We only wrote %d out of %zu bytes to disk", static_cast<unsigned>(writtenBytes), rawData->size()); >+} >+ >+} // namespace WebKit >diff --git a/Source/WebKit/Shared/PersistencyUtils.h b/Source/WebKit/Shared/PersistencyUtils.h >new file mode 100644 >index 0000000000000000000000000000000000000000..160270f19168f37f9d829aaed571b766752f9dbe >--- /dev/null >+++ b/Source/WebKit/Shared/PersistencyUtils.h >@@ -0,0 +1,35 @@ >+/* >+ * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#pragma once >+ >+#include <WebCore/KeyedCoding.h> >+ >+namespace WebKit { >+ >+std::unique_ptr<WebCore::KeyedDecoder> createForFile(const String& path); >+void writeToDisk(std::unique_ptr<WebCore::KeyedEncoder>&&, String&& path); >+ >+} // namespace WebKit >diff --git a/Source/WebKit/Sources.txt b/Source/WebKit/Sources.txt >index e0a3282e0092d97ce35d6646b4331b8ca11f48e0..496e88c83fbd9ec4cceb1c289f3d3f8a5aaf8b9f 100644 >--- a/Source/WebKit/Sources.txt >+++ b/Source/WebKit/Sources.txt >@@ -41,6 +41,9 @@ NetworkProcess/NetworkSocketStream.cpp > NetworkProcess/PingLoad.cpp > NetworkProcess/PreconnectTask.cpp > >+NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp >+NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp >+ > NetworkProcess/Cookies/WebCookieManager.cpp > > NetworkProcess/Downloads/Download.cpp >@@ -107,6 +110,7 @@ Shared/FrameInfoData.cpp > Shared/LayerTreeContext.cpp > Shared/LoadParameters.cpp > Shared/NavigationActionData.cpp >+Shared/PersistencyUtils.cpp > Shared/PlatformPopupMenuData.cpp > Shared/PrintInfo.cpp > Shared/RTCNetwork.cpp >@@ -225,12 +229,9 @@ UIProcess/GeolocationPermissionRequestManagerProxy.cpp > UIProcess/GeolocationPermissionRequestProxy.cpp > UIProcess/InspectorTargetProxy.cpp > UIProcess/PageLoadState.cpp >-UIProcess/PersistencyUtils.cpp > UIProcess/ProcessAssertion.cpp > UIProcess/ProcessThrottler.cpp > UIProcess/RemoteWebInspectorProxy.cpp >-UIProcess/ResourceLoadStatisticsMemoryStore.cpp >-UIProcess/ResourceLoadStatisticsPersistentStorage.cpp > UIProcess/ResponsivenessTimer.cpp > UIProcess/ServiceWorkerProcessProxy.cpp > UIProcess/StatisticsRequest.cpp >diff --git a/Source/WebKit/SourcesCocoa.txt b/Source/WebKit/SourcesCocoa.txt >index a4336a098094a5188057d2fca65241b599a943b2..308f66cbe6d23246eddafda31d2aa458582c94e2 100644 >--- a/Source/WebKit/SourcesCocoa.txt >+++ b/Source/WebKit/SourcesCocoa.txt >@@ -29,6 +29,9 @@ NetworkProcess/cocoa/NetworkDataTaskCocoa.mm > NetworkProcess/cocoa/NetworkProcessCocoa.mm > NetworkProcess/cocoa/NetworkSessionCocoa.mm > >+NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >+NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp >+ > NetworkProcess/Cookies/mac/WebCookieManagerMac.mm > > NetworkProcess/CustomProtocols/Cocoa/LegacyCustomProtocolManagerCocoa.mm >@@ -202,8 +205,6 @@ UIProcess/PerActivityStateCPUUsageSampler.cpp > UIProcess/WebDataListSuggestionsDropdown.cpp > UIProcess/WebMediaSessionFocusManager.cpp > UIProcess/WebMediaSessionFocusManagerClient.cpp >-UIProcess/WebResourceLoadStatisticsStore.cpp >-UIProcess/WebResourceLoadStatisticsTelemetry.cpp > UIProcess/WKImagePreviewViewController.mm > UIProcess/WKInspectorHighlightView.mm > >diff --git a/Source/WebKit/SourcesGTK.txt b/Source/WebKit/SourcesGTK.txt >index 265384b641887537895d07f3ad5832944c01d291..b08d723b4aee60b419e7b93e21d88644295d17f3 100644 >--- a/Source/WebKit/SourcesGTK.txt >+++ b/Source/WebKit/SourcesGTK.txt >@@ -25,6 +25,9 @@ > // linking PluginProcessGTK2. I don't know why this is necessary. These > // annotations should be revisited after removing PluginProcessGTK2. > >+NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >+NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp >+ > NetworkProcess/Cookies/soup/WebCookieManagerSoup.cpp > > NetworkProcess/CustomProtocols/LegacyCustomProtocolManager.cpp >@@ -109,8 +112,6 @@ UIProcess/DefaultUndoController.cpp > UIProcess/DrawingAreaProxyImpl.cpp > UIProcess/LegacySessionStateCodingNone.cpp > UIProcess/WebGrammarDetail.cpp >-UIProcess/WebResourceLoadStatisticsStore.cpp >-UIProcess/WebResourceLoadStatisticsTelemetry.cpp > UIProcess/WebTextChecker.cpp > UIProcess/WebTextCheckerClient.cpp > UIProcess/WebViewportAttributes.cpp >diff --git a/Source/WebKit/SourcesWPE.txt b/Source/WebKit/SourcesWPE.txt >index 8ee2fef42a6647859f35f8d34793af94a029393e..5119cae56bc06f22973f6733c08437ab2e3e3e12 100644 >--- a/Source/WebKit/SourcesWPE.txt >+++ b/Source/WebKit/SourcesWPE.txt >@@ -21,6 +21,9 @@ > // ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF > // THE POSSIBILITY OF SUCH DAMAGE. > >+NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >+NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp >+ > NetworkProcess/Cookies/soup/WebCookieManagerSoup.cpp > > NetworkProcess/CustomProtocols/LegacyCustomProtocolManager.cpp >@@ -98,8 +101,6 @@ UIProcess/AcceleratedDrawingAreaProxy.cpp > UIProcess/DefaultUndoController.cpp > UIProcess/LegacySessionStateCodingNone.cpp > UIProcess/WebGrammarDetail.cpp >-UIProcess/WebResourceLoadStatisticsStore.cpp >-UIProcess/WebResourceLoadStatisticsTelemetry.cpp > UIProcess/WebViewportAttributes.cpp > > UIProcess/API/C/WKGrammarDetail.cpp >diff --git a/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp b/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp >index a3e3468256bb67008295fc13f072df379b70012c..cf6caf689a6104da7777e8579938233ee340e8dc 100644 >--- a/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp >+++ b/Source/WebKit/UIProcess/API/C/WKWebsiteDataStoreRef.cpp >@@ -579,13 +579,13 @@ void WKWebsiteDataStoreSetCacheStoragePerOriginQuota(WKWebsiteDataStoreRef dataS > void WKWebsiteDataStoreSetWebAuthenticationMockConfiguration(WKWebsiteDataStoreRef dataStoreRef, WKDictionaryRef configurationRef) > { > #if ENABLE(WEB_AUTHN) >- MockWebAuthenticationConfiguration configuration; >+ WebKit::MockWebAuthenticationConfiguration configuration; > > if (auto silentFailureRef = static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(configurationRef, adoptWK(WKStringCreateWithUTF8CString("SilentFailure")).get()))) > configuration.silentFailure = WKBooleanGetValue(silentFailureRef); > > if (auto localRef = static_cast<WKDictionaryRef>(WKDictionaryGetItemForKey(configurationRef, adoptWK(WKStringCreateWithUTF8CString("Local")).get()))) { >- MockWebAuthenticationConfiguration::Local local; >+ WebKit::MockWebAuthenticationConfiguration::Local local; > local.acceptAuthentication = WKBooleanGetValue(static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(localRef, adoptWK(WKStringCreateWithUTF8CString("AcceptAuthentication")).get()))); > local.acceptAttestation = WKBooleanGetValue(static_cast<WKBooleanRef>(WKDictionaryGetItemForKey(localRef, adoptWK(WKStringCreateWithUTF8CString("AcceptAttestation")).get()))); > if (local.acceptAttestation) { >@@ -597,35 +597,35 @@ void WKWebsiteDataStoreSetWebAuthenticationMockConfiguration(WKWebsiteDataStoreR > } > > if (auto hidRef = static_cast<WKDictionaryRef>(WKDictionaryGetItemForKey(configurationRef, adoptWK(WKStringCreateWithUTF8CString("Hid")).get()))) { >- MockWebAuthenticationConfiguration::Hid hid; >+ WebKit::MockWebAuthenticationConfiguration::Hid hid; > > auto stage = WebKit::toImpl(static_cast<WKStringRef>(WKDictionaryGetItemForKey(hidRef, adoptWK(WKStringCreateWithUTF8CString("Stage")).get())))->string(); > if (stage == "info") >- hid.stage = MockWebAuthenticationConfiguration::Hid::Stage::Info; >+ hid.stage = WebKit::MockWebAuthenticationConfiguration::Hid::Stage::Info; > if (stage == "request") >- hid.stage = MockWebAuthenticationConfiguration::Hid::Stage::Request; >+ hid.stage = WebKit::MockWebAuthenticationConfiguration::Hid::Stage::Request; > > auto subStage = WebKit::toImpl(static_cast<WKStringRef>(WKDictionaryGetItemForKey(hidRef, adoptWK(WKStringCreateWithUTF8CString("SubStage")).get())))->string(); > if (subStage == "init") >- hid.subStage = MockWebAuthenticationConfiguration::Hid::SubStage::Init; >+ hid.subStage = WebKit::MockWebAuthenticationConfiguration::Hid::SubStage::Init; > if (subStage == "msg") >- hid.subStage = MockWebAuthenticationConfiguration::Hid::SubStage::Msg; >+ hid.subStage = WebKit::MockWebAuthenticationConfiguration::Hid::SubStage::Msg; > > auto error = WebKit::toImpl(static_cast<WKStringRef>(WKDictionaryGetItemForKey(hidRef, adoptWK(WKStringCreateWithUTF8CString("Error")).get())))->string(); > if (error == "success") >- hid.error = MockWebAuthenticationConfiguration::Hid::Error::Success; >+ hid.error = WebKit::MockWebAuthenticationConfiguration::Hid::Error::Success; > if (error == "data-not-sent") >- hid.error = MockWebAuthenticationConfiguration::Hid::Error::DataNotSent; >+ hid.error = WebKit::MockWebAuthenticationConfiguration::Hid::Error::DataNotSent; > if (error == "empty-report") >- hid.error = MockWebAuthenticationConfiguration::Hid::Error::EmptyReport; >+ hid.error = WebKit::MockWebAuthenticationConfiguration::Hid::Error::EmptyReport; > if (error == "wrong-channel-id") >- hid.error = MockWebAuthenticationConfiguration::Hid::Error::WrongChannelId; >+ hid.error = WebKit::MockWebAuthenticationConfiguration::Hid::Error::WrongChannelId; > if (error == "malicious-payload") >- hid.error = MockWebAuthenticationConfiguration::Hid::Error::MaliciousPayload; >+ hid.error = WebKit::MockWebAuthenticationConfiguration::Hid::Error::MaliciousPayload; > if (error == "unsupported-options") >- hid.error = MockWebAuthenticationConfiguration::Hid::Error::UnsupportedOptions; >+ hid.error = WebKit::MockWebAuthenticationConfiguration::Hid::Error::UnsupportedOptions; > if (error == "wrong-nonce") >- hid.error = MockWebAuthenticationConfiguration::Hid::Error::WrongNonce; >+ hid.error = WebKit::MockWebAuthenticationConfiguration::Hid::Error::WrongNonce; > > if (auto payloadBase64 = static_cast<WKStringRef>(WKDictionaryGetItemForKey(hidRef, adoptWK(WKStringCreateWithUTF8CString("PayloadBase64")).get()))) > hid.payloadBase64 = WebKit::toImpl(payloadBase64)->string(); >diff --git a/Source/WebKit/UIProcess/PersistencyUtils.cpp b/Source/WebKit/UIProcess/PersistencyUtils.cpp >deleted file mode 100644 >index 5dba968bd7e0b38c3d22694d6d5d2e6b069d2a81..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/PersistencyUtils.cpp >+++ /dev/null >@@ -1,89 +0,0 @@ >-/* >- * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#include "config.h" >-#include "PersistencyUtils.h" >- >-#include "Logging.h" >-#include <WebCore/FileSystem.h> >-#include <WebCore/SharedBuffer.h> >-#include <wtf/RunLoop.h> >- >-namespace WebKit { >- >-using namespace WebCore; >- >-std::unique_ptr<KeyedDecoder> createForFile(const String& path) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto handle = FileSystem::openAndLockFile(path, FileSystem::FileOpenMode::Read); >- if (handle == FileSystem::invalidPlatformFileHandle) >- return nullptr; >- >- long long fileSize = 0; >- if (!FileSystem::getFileSize(handle, fileSize) || !fileSize) { >- FileSystem::unlockAndCloseFile(handle); >- return nullptr; >- } >- >- size_t bytesToRead; >- if (!WTF::convertSafely(fileSize, bytesToRead)) { >- FileSystem::unlockAndCloseFile(handle); >- return nullptr; >- } >- >- Vector<char> buffer(bytesToRead); >- size_t totalBytesRead = FileSystem::readFromFile(handle, buffer.data(), buffer.size()); >- >- FileSystem::unlockAndCloseFile(handle); >- >- if (totalBytesRead != bytesToRead) >- return nullptr; >- >- // FIXME: We should try to modify the constructor to pass &&. >- return KeyedDecoder::decoder(reinterpret_cast<const uint8_t*>(buffer.data()), buffer.size()); >-} >- >-void writeToDisk(std::unique_ptr<KeyedEncoder>&& encoder, String&& path) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto rawData = encoder->finishEncoding(); >- if (!rawData) >- return; >- >- FileSystem::PlatformFileHandle handle = FileSystem::openAndLockFile(path, FileSystem::FileOpenMode::Write); >- if (handle == FileSystem::invalidPlatformFileHandle) >- return; >- >- auto writtenBytes = FileSystem::writeToFile(handle, rawData->data(), rawData->size()); >- FileSystem::unlockAndCloseFile(handle); >- >- if (writtenBytes != static_cast<int64_t>(rawData->size())) >- RELEASE_LOG_ERROR(DiskPersistency, "Disk persistency: We only wrote %d out of %zu bytes to disk", static_cast<unsigned>(writtenBytes), rawData->size()); >-} >- >-} // namespace WebKit >diff --git a/Source/WebKit/UIProcess/PersistencyUtils.h b/Source/WebKit/UIProcess/PersistencyUtils.h >deleted file mode 100644 >index 160270f19168f37f9d829aaed571b766752f9dbe..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/PersistencyUtils.h >+++ /dev/null >@@ -1,35 +0,0 @@ >-/* >- * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#pragma once >- >-#include <WebCore/KeyedCoding.h> >- >-namespace WebKit { >- >-std::unique_ptr<WebCore::KeyedDecoder> createForFile(const String& path); >-void writeToDisk(std::unique_ptr<WebCore::KeyedEncoder>&&, String&& path); >- >-} // namespace WebKit >diff --git a/Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.cpp b/Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.cpp >deleted file mode 100644 >index 7e718320324cc6b9b6444a7c6f6b241d8bafd010..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.cpp >+++ /dev/null >@@ -1,1297 +0,0 @@ >-/* >- * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#include "config.h" >-#include "ResourceLoadStatisticsMemoryStore.h" >- >-#include "Logging.h" >-#include "PluginProcessManager.h" >-#include "PluginProcessProxy.h" >-#include "ResourceLoadStatisticsPersistentStorage.h" >-#include "WebProcessProxy.h" >-#include "WebResourceLoadStatisticsTelemetry.h" >-#include "WebsiteDataStore.h" >-#include <WebCore/KeyedCoding.h> >-#include <WebCore/ResourceLoadStatistics.h> >-#include <wtf/CallbackAggregator.h> >-#include <wtf/DateMath.h> >-#include <wtf/MathExtras.h> >- >-#if !RELEASE_LOG_DISABLED >-#include <wtf/text/StringBuilder.h> >-#endif >- >-namespace WebKit { >-using namespace WebCore; >- >-constexpr unsigned statisticsModelVersion { 14 }; >-constexpr unsigned maxNumberOfRecursiveCallsInRedirectTraceBack { 50 }; >-constexpr Seconds minimumStatisticsProcessingInterval { 5_s }; >-constexpr unsigned operatingDatesWindow { 30 }; >-constexpr unsigned maxImportance { 3 }; >-static const char* debugStaticPrevalentResource { "3rdpartytestwebkit.org" }; >- >-#if !RELEASE_LOG_DISABLED >-static String domainsToString(const Vector<String>& domains) >-{ >- StringBuilder builder; >- for (auto& domain : domains) { >- if (!builder.isEmpty()) >- builder.appendLiteral(", "); >- builder.append(domain); >- } >- return builder.toString(); >-} >-#endif >- >-class OperatingDate { >-public: >- OperatingDate() = default; >- >- static OperatingDate fromWallTime(WallTime time) >- { >- double ms = time.secondsSinceEpoch().milliseconds(); >- int year = msToYear(ms); >- int yearDay = dayInYear(ms, year); >- int month = monthFromDayInYear(yearDay, isLeapYear(year)); >- int monthDay = dayInMonthFromDayInYear(yearDay, isLeapYear(year)); >- >- return OperatingDate { year, month, monthDay }; >- } >- >- static OperatingDate today() >- { >- return OperatingDate::fromWallTime(WallTime::now()); >- } >- >- Seconds secondsSinceEpoch() const >- { >- return Seconds { dateToDaysFrom1970(m_year, m_month, m_monthDay) * secondsPerDay }; >- } >- >- bool operator==(const OperatingDate& other) const >- { >- return m_monthDay == other.m_monthDay && m_month == other.m_month && m_year == other.m_year; >- } >- >- bool operator<(const OperatingDate& other) const >- { >- return secondsSinceEpoch() < other.secondsSinceEpoch(); >- } >- >- bool operator<=(const OperatingDate& other) const >- { >- return secondsSinceEpoch() <= other.secondsSinceEpoch(); >- } >- >-private: >- OperatingDate(int year, int month, int monthDay) >- : m_year(year) >- , m_month(month) >- , m_monthDay(monthDay) >- { } >- >- int m_year { 0 }; >- int m_month { 0 }; // [0, 11]. >- int m_monthDay { 0 }; // [1, 31]. >-}; >- >-static Vector<OperatingDate> mergeOperatingDates(const Vector<OperatingDate>& existingDates, Vector<OperatingDate>&& newDates) >-{ >- if (existingDates.isEmpty()) >- return WTFMove(newDates); >- >- Vector<OperatingDate> mergedDates(existingDates.size() + newDates.size()); >- >- // Merge the two sorted vectors of dates. >- std::merge(existingDates.begin(), existingDates.end(), newDates.begin(), newDates.end(), mergedDates.begin()); >- // Remove duplicate dates. >- removeRepeatedElements(mergedDates); >- >- // Drop old dates until the Vector size reaches operatingDatesWindow. >- while (mergedDates.size() > operatingDatesWindow) >- mergedDates.remove(0); >- >- return mergedDates; >-} >- >-struct StatisticsLastSeen { >- String topPrivatelyOwnedDomain; >- WallTime lastSeen; >-}; >- >-static void pruneResources(HashMap<String, WebCore::ResourceLoadStatistics>& statisticsMap, Vector<StatisticsLastSeen>& statisticsToPrune, size_t& numberOfEntriesToPrune) >-{ >- if (statisticsToPrune.size() > numberOfEntriesToPrune) { >- std::sort(statisticsToPrune.begin(), statisticsToPrune.end(), [](const StatisticsLastSeen& a, const StatisticsLastSeen& b) { >- return a.lastSeen < b.lastSeen; >- }); >- } >- >- for (size_t i = 0, end = std::min(numberOfEntriesToPrune, statisticsToPrune.size()); i != end; ++i, --numberOfEntriesToPrune) >- statisticsMap.remove(statisticsToPrune[i].topPrivatelyOwnedDomain); >-} >- >-static unsigned computeImportance(const ResourceLoadStatistics& resourceStatistic) >-{ >- unsigned importance = maxImportance; >- if (!resourceStatistic.isPrevalentResource) >- importance -= 1; >- if (!resourceStatistic.hadUserInteraction) >- importance -= 2; >- return importance; >-} >- >-ResourceLoadStatisticsMemoryStore::ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore& store, WorkQueue& workQueue) >- : m_store(store) >- , m_workQueue(workQueue) >-{ >- ASSERT(!RunLoop::isMain()); >- >-#if PLATFORM(COCOA) >- registerUserDefaultsIfNeeded(); >-#endif >- includeTodayAsOperatingDateIfNecessary(); >- >- m_workQueue->dispatchAfter(5_s, [weakThis = makeWeakPtr(*this)] { >- if (weakThis) >- weakThis->calculateAndSubmitTelemetry(); >- }); >-} >- >-ResourceLoadStatisticsMemoryStore::~ResourceLoadStatisticsMemoryStore() >-{ >- ASSERT(!RunLoop::isMain()); >-} >- >-void ResourceLoadStatisticsMemoryStore::setPersistentStorage(ResourceLoadStatisticsPersistentStorage& persistentStorage) >-{ >- m_persistentStorage = makeWeakPtr(persistentStorage); >-} >- >-void ResourceLoadStatisticsMemoryStore::calculateAndSubmitTelemetry() const >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (m_parameters.shouldSubmitTelemetry) >- WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*this); >-} >- >-void ResourceLoadStatisticsMemoryStore::setNotifyPagesWhenDataRecordsWereScanned(bool value) >-{ >- ASSERT(!RunLoop::isMain()); >- m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned = value; >-} >- >-void ResourceLoadStatisticsMemoryStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value) >-{ >- ASSERT(!RunLoop::isMain()); >- m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval = value; >-} >- >-void ResourceLoadStatisticsMemoryStore::setShouldSubmitTelemetry(bool value) >-{ >- ASSERT(!RunLoop::isMain()); >- m_parameters.shouldSubmitTelemetry = value; >-} >- >-void ResourceLoadStatisticsMemoryStore::removeDataRecords(CompletionHandler<void()>&& callback) >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (!shouldRemoveDataRecords()) { >- callback(); >- return; >- } >- >-#if ENABLE(NETSCAPE_PLUGIN_API) >- m_activePluginTokens.clear(); >- for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses()) >- m_activePluginTokens.add(plugin->pluginProcessToken()); >-#endif >- >- auto prevalentResourceDomains = topPrivatelyControlledDomainsToRemoveWebsiteDataFor(); >- if (prevalentResourceDomains.isEmpty()) { >- callback(); >- return; >- } >- >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "About to remove data records for %{public}s.", domainsToString(prevalentResourceDomains).utf8().data()); >-#endif >- >- setDataRecordsBeingRemoved(true); >- >- RunLoop::main().dispatch([prevalentResourceDomains = crossThreadCopy(prevalentResourceDomains), callback = WTFMove(callback), weakThis = makeWeakPtr(*this), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef()] () mutable { >- WebProcessProxy::deleteWebsiteDataForTopPrivatelyControlledDomainsInAllPersistentDataStores(WebResourceLoadStatisticsStore::monitoredDataTypes(), WTFMove(prevalentResourceDomains), shouldNotifyPagesWhenDataRecordsWereScanned, [callback = WTFMove(callback), weakThis = WTFMove(weakThis), workQueue = workQueue.copyRef()](const HashSet<String>& domainsWithDeletedWebsiteData) mutable { >- workQueue->dispatch([topDomains = crossThreadCopy(domainsWithDeletedWebsiteData), callback = WTFMove(callback), weakThis = WTFMove(weakThis)] () mutable { >- if (!weakThis) { >- callback(); >- return; >- } >- for (auto& prevalentResourceDomain : topDomains) { >- auto& statistic = weakThis->ensureResourceStatisticsForPrimaryDomain(prevalentResourceDomain); >- ++statistic.dataRecordsRemoved; >- } >- weakThis->setDataRecordsBeingRemoved(false); >- callback(); >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(weakThis->m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Done removing data records."); >-#endif >- }); >- }); >- }); >-} >- >-unsigned ResourceLoadStatisticsMemoryStore::recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(const WebCore::ResourceLoadStatistics& resourceStatistic, HashSet<String>& domainsThatHaveRedirectedTo, unsigned numberOfRecursiveCalls) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (numberOfRecursiveCalls >= maxNumberOfRecursiveCallsInRedirectTraceBack) { >- // Model version 14 invokes a deliberate re-classification of the whole set. >- if (statisticsModelVersion != 14) >- ASSERT_NOT_REACHED(); >- RELEASE_LOG(ResourceLoadStatistics, "Hit %u recursive calls in redirect backtrace. Returning early.", maxNumberOfRecursiveCallsInRedirectTraceBack); >- return numberOfRecursiveCalls; >- } >- >- numberOfRecursiveCalls++; >- >- for (auto& subresourceUniqueRedirectFromDomain : resourceStatistic.subresourceUniqueRedirectsFrom.values()) { >- auto mapEntry = m_resourceStatisticsMap.find(subresourceUniqueRedirectFromDomain); >- if (mapEntry == m_resourceStatisticsMap.end() || mapEntry->value.isPrevalentResource) >- continue; >- if (domainsThatHaveRedirectedTo.add(mapEntry->value.highLevelDomain).isNewEntry) >- numberOfRecursiveCalls = recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(mapEntry->value, domainsThatHaveRedirectedTo, numberOfRecursiveCalls); >- } >- for (auto& topFrameUniqueRedirectFromDomain : resourceStatistic.topFrameUniqueRedirectsFrom.values()) { >- auto mapEntry = m_resourceStatisticsMap.find(topFrameUniqueRedirectFromDomain); >- if (mapEntry == m_resourceStatisticsMap.end() || mapEntry->value.isPrevalentResource) >- continue; >- if (domainsThatHaveRedirectedTo.add(mapEntry->value.highLevelDomain).isNewEntry) >- numberOfRecursiveCalls = recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(mapEntry->value, domainsThatHaveRedirectedTo, numberOfRecursiveCalls); >- } >- >- return numberOfRecursiveCalls; >-} >- >-void ResourceLoadStatisticsMemoryStore::markAsPrevalentIfHasRedirectedToPrevalent(WebCore::ResourceLoadStatistics& resourceStatistic) >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (resourceStatistic.isPrevalentResource) >- return; >- >- for (auto& subresourceDomainRedirectedTo : resourceStatistic.subresourceUniqueRedirectsTo.values()) { >- auto mapEntry = m_resourceStatisticsMap.find(subresourceDomainRedirectedTo); >- if (mapEntry != m_resourceStatisticsMap.end() && mapEntry->value.isPrevalentResource) { >- setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >- return; >- } >- } >- >- for (auto& topFrameDomainRedirectedTo : resourceStatistic.topFrameUniqueRedirectsTo.values()) { >- auto mapEntry = m_resourceStatisticsMap.find(topFrameDomainRedirectedTo); >- if (mapEntry != m_resourceStatisticsMap.end() && mapEntry->value.isPrevalentResource) { >- setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >- return; >- } >- } >-} >- >-bool ResourceLoadStatisticsMemoryStore::isPrevalentDueToDebugMode(ResourceLoadStatistics& resourceStatistic) >-{ >- if (!m_debugModeEnabled) >- return false; >- >- return resourceStatistic.highLevelDomain == debugStaticPrevalentResource || resourceStatistic.highLevelDomain == m_debugManualPrevalentResource; >-} >- >-void ResourceLoadStatisticsMemoryStore::processStatisticsAndDataRecords() >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (m_parameters.shouldClassifyResourcesBeforeDataRecordsRemoval) { >- for (auto& resourceStatistic : m_resourceStatisticsMap.values()) { >- if (isPrevalentDueToDebugMode(resourceStatistic)) >- setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >- else if (!resourceStatistic.isVeryPrevalentResource) { >- markAsPrevalentIfHasRedirectedToPrevalent(resourceStatistic); >- auto currentPrevalence = resourceStatistic.isPrevalentResource ? ResourceLoadPrevalence::High : ResourceLoadPrevalence::Low; >- auto newPrevalence = m_resourceLoadStatisticsClassifier.calculateResourcePrevalence(resourceStatistic, currentPrevalence); >- if (newPrevalence != currentPrevalence) >- setPrevalentResource(resourceStatistic, newPrevalence); >- } >- } >- } >- >- removeDataRecords([this, weakThis = makeWeakPtr(*this)] { >- ASSERT(!RunLoop::isMain()); >- if (!weakThis) >- return; >- >- pruneStatisticsIfNeeded(); >- if (m_persistentStorage) >- m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::No); >- >- if (!m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned) >- return; >- >- RunLoop::main().dispatch([] { >- WebProcessProxy::notifyPageStatisticsAndDataRecordsProcessed(); >- }); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::hasStorageAccess(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >- if (shouldBlockAndPurgeCookies(subFrameStatistic)) { >- completionHandler(false); >- return; >- } >- >- if (!shouldBlockAndKeepCookies(subFrameStatistic)) { >- completionHandler(true); >- return; >- } >- >- RunLoop::main().dispatch([store = makeRef(m_store), subFramePrimaryDomain = subFramePrimaryDomain.isolatedCopy(), topFramePrimaryDomain = topFramePrimaryDomain.isolatedCopy(), frameID, pageID, completionHandler = WTFMove(completionHandler)]() mutable { >- store->callHasStorageAccessForFrameHandler(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)](bool result) mutable { >- store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler), result] () mutable { >- completionHandler(result); >- }); >- }); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::requestStorageAccess(String&& subFramePrimaryDomain, String&& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >- if (shouldBlockAndPurgeCookies(subFrameStatistic)) { >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Cannot grant storage access to %{public}s since its cookies are blocked in third-party contexts and it has not received user interaction as first-party.", subFramePrimaryDomain.utf8().data()); >-#endif >- completionHandler(StorageAccessStatus::CannotRequestAccess); >- return; >- } >- >- if (!shouldBlockAndKeepCookies(subFrameStatistic)) { >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "No need to grant storage access to %{public}s since its cookies are not blocked in third-party contexts.", subFramePrimaryDomain.utf8().data()); >-#endif >- completionHandler(StorageAccessStatus::HasAccess); >- return; >- } >- >- auto userWasPromptedEarlier = promptEnabled && hasUserGrantedStorageAccessThroughPrompt(subFrameStatistic, topFramePrimaryDomain); >- if (promptEnabled && !userWasPromptedEarlier) { >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "About to ask the user whether they want to grant storage access to %{public}s under %{public}s or not.", subFramePrimaryDomain.utf8().data(), topFramePrimaryDomain.utf8().data()); >-#endif >- completionHandler(StorageAccessStatus::RequiresUserPrompt); >- return; >- } else if (userWasPromptedEarlier) { >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Storage access was granted to %{public}s under %{public}s.", subFramePrimaryDomain.utf8().data(), topFramePrimaryDomain.utf8().data()); >-#endif >- } >- >- subFrameStatistic.timesAccessedAsFirstPartyDueToStorageAccessAPI++; >- >- grantStorageAccessInternal(WTFMove(subFramePrimaryDomain), WTFMove(topFramePrimaryDomain), frameID, pageID, userWasPromptedEarlier, [completionHandler = WTFMove(completionHandler)] (bool wasGrantedAccess) mutable { >- completionHandler(wasGrantedAccess ? StorageAccessStatus::HasAccess : StorageAccessStatus::CannotRequestAccess); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain) >-{ >- ASSERT(primaryDomainInNeedOfStorageAccess != openerPrimaryDomain); >- ASSERT(!RunLoop::isMain()); >- >- if (primaryDomainInNeedOfStorageAccess == openerPrimaryDomain) >- return; >- >- auto& domainInNeedOfStorageAccessStatistic = ensureResourceStatisticsForPrimaryDomain(primaryDomainInNeedOfStorageAccess); >- auto cookiesBlockedAndPurged = shouldBlockAndPurgeCookies(domainInNeedOfStorageAccessStatistic); >- >- // The domain already has access if its cookies are not blocked. >- if (!cookiesBlockedAndPurged && !shouldBlockAndKeepCookies(domainInNeedOfStorageAccessStatistic)) >- return; >- >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "[Temporary combatibility fix] Storage access was granted for %{public}s under opener page from %{public}s, with user interaction in the opened window.", primaryDomainInNeedOfStorageAccess.utf8().data(), openerPrimaryDomain.utf8().data()); >-#endif >- grantStorageAccessInternal(WTFMove(primaryDomainInNeedOfStorageAccess), WTFMove(openerPrimaryDomain), WTF::nullopt, openerPageID, false, [](bool) { }); >-} >- >-void ResourceLoadStatisticsMemoryStore::grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto subFramePrimaryDomain = ResourceLoadStatistics::primaryDomain(subFrameHost); >- auto topFramePrimaryDomain = ResourceLoadStatistics::primaryDomain(topFrameHost); >- if (userWasPromptedNow) { >- auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >- ASSERT(subFrameStatistic.hadUserInteraction); >- subFrameStatistic.storageAccessUnderTopFrameOrigins.add(topFramePrimaryDomain); >- } >- grantStorageAccessInternal(WTFMove(subFramePrimaryDomain), WTFMove(topFramePrimaryDomain), frameID, pageID, userWasPromptedNow, WTFMove(completionHandler)); >-} >- >-void ResourceLoadStatisticsMemoryStore::grantStorageAccessInternal(String&& subFramePrimaryDomain, String&& topFramePrimaryDomain, Optional<uint64_t> frameID, uint64_t pageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&& callback) >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (subFramePrimaryDomain == topFramePrimaryDomain) { >- callback(true); >- return; >- } >- >- // FIXME: Remove m_storageAccessPromptsEnabled check if prompting is no longer experimental. >- if (userWasPromptedNowOrEarlier && m_storageAccessPromptsEnabled) { >- auto& subFrameStatistic = ensureResourceStatisticsForPrimaryDomain(subFramePrimaryDomain); >- ASSERT(subFrameStatistic.hadUserInteraction); >- ASSERT(subFrameStatistic.storageAccessUnderTopFrameOrigins.contains(topFramePrimaryDomain)); >- subFrameStatistic.mostRecentUserInteractionTime = WallTime::now(); >- } >- >- RunLoop::main().dispatch([subFramePrimaryDomain = subFramePrimaryDomain.isolatedCopy(), topFramePrimaryDomain = topFramePrimaryDomain.isolatedCopy(), frameID, pageID, store = makeRef(m_store), callback = WTFMove(callback)]() mutable { >- store->callGrantStorageAccessHandler(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, [callback = WTFMove(callback), store = store.copyRef()](bool value) mutable { >- store->statisticsQueue().dispatch([callback = WTFMove(callback), value] () mutable { >- callback(value); >- }); >- }); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::grandfatherExistingWebsiteData(CompletionHandler<void()>&& callback) >-{ >- ASSERT(!RunLoop::isMain()); >- >- RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), callback = WTFMove(callback), shouldNotifyPagesWhenDataRecordsWereScanned = m_parameters.shouldNotifyPagesWhenDataRecordsWereScanned, workQueue = m_workQueue.copyRef()] () mutable { >- // FIXME: This method being a static call on WebProcessProxy is wrong. >- // It should be on the data store that this object belongs to. >- WebProcessProxy::topPrivatelyControlledDomainsWithWebsiteData(WebResourceLoadStatisticsStore::monitoredDataTypes(), shouldNotifyPagesWhenDataRecordsWereScanned, [weakThis = WTFMove(weakThis), callback = WTFMove(callback), workQueue = workQueue.copyRef()] (HashSet<String>&& topPrivatelyControlledDomainsWithWebsiteData) mutable { >- workQueue->dispatch([weakThis = WTFMove(weakThis), topDomains = crossThreadCopy(topPrivatelyControlledDomainsWithWebsiteData), callback = WTFMove(callback)] () mutable { >- if (!weakThis) { >- callback(); >- return; >- } >- >- for (auto& topPrivatelyControlledDomain : topDomains) { >- auto& statistic = weakThis->ensureResourceStatisticsForPrimaryDomain(topPrivatelyControlledDomain); >- statistic.grandfathered = true; >- } >- weakThis->m_endOfGrandfatheringTimestamp = WallTime::now() + weakThis->m_parameters.grandfatheringTime; >- if (weakThis->m_persistentStorage) >- weakThis->m_persistentStorage->scheduleOrWriteMemoryStore(ResourceLoadStatisticsPersistentStorage::ForceImmediateWrite::Yes); >- callback(); >- weakThis->logTestingEvent("Grandfathered"_s); >- }); >- }); >- }); >-} >- >-Vector<String> ResourceLoadStatisticsMemoryStore::ensurePrevalentResourcesForDebugMode() >-{ >- if (!m_debugModeEnabled) >- return { }; >- >- Vector<String> primaryDomainsToBlock; >- primaryDomainsToBlock.reserveInitialCapacity(2); >- >- auto& staticSesourceStatistic = ensureResourceStatisticsForPrimaryDomain(debugStaticPrevalentResource); >- setPrevalentResource(staticSesourceStatistic, ResourceLoadPrevalence::High); >- primaryDomainsToBlock.uncheckedAppend(debugStaticPrevalentResource); >- >- if (!m_debugManualPrevalentResource.isEmpty()) { >- auto& manualResourceStatistic = ensureResourceStatisticsForPrimaryDomain(m_debugManualPrevalentResource); >- setPrevalentResource(manualResourceStatistic, ResourceLoadPrevalence::High); >- primaryDomainsToBlock.uncheckedAppend(m_debugManualPrevalentResource); >- } >- >- return primaryDomainsToBlock; >-} >- >-void ResourceLoadStatisticsMemoryStore::setResourceLoadStatisticsDebugMode(bool enable) >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_debugModeEnabled = enable; >- m_debugLoggingEnabled = enable; >- >- ensurePrevalentResourcesForDebugMode(); >-} >- >-void ResourceLoadStatisticsMemoryStore::setPrevalentResourceForDebugMode(const String& domain) >-{ >- if (!m_debugModeEnabled) >- return; >- >- m_debugManualPrevalentResource = domain; >- auto& resourceStatistic = ensureResourceStatisticsForPrimaryDomain(domain); >- setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >- >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "Did set %{public}s as prevalent resource for the purposes of ITP Debug Mode.", domain.utf8().data()); >-#endif >-} >- >-void ResourceLoadStatisticsMemoryStore::scheduleStatisticsProcessingRequestIfNecessary() >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_pendingStatisticsProcessingRequestIdentifier = ++m_lastStatisticsProcessingRequestIdentifier; >- m_workQueue->dispatchAfter(minimumStatisticsProcessingInterval, [this, weakThis = makeWeakPtr(*this), statisticsProcessingRequestIdentifier = *m_pendingStatisticsProcessingRequestIdentifier] { >- if (!weakThis) >- return; >- >- if (!m_pendingStatisticsProcessingRequestIdentifier || *m_pendingStatisticsProcessingRequestIdentifier != statisticsProcessingRequestIdentifier) { >- // This request has been canceled. >- return; >- } >- >- updateCookieBlocking([]() { }); >- processStatisticsAndDataRecords(); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::cancelPendingStatisticsProcessingRequest() >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_pendingStatisticsProcessingRequestIdentifier = WTF::nullopt; >-} >- >-void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, const String& sourcePrimaryDomain, const String& targetHost, const String& mainFrameHost, bool isRedirect, bool isMainFrame) >-{ >- ASSERT(!RunLoop::isMain()); >- >- bool areTargetAndMainFrameDomainsAssociated = targetPrimaryDomain == mainFramePrimaryDomain; >- bool areTargetAndSourceDomainsAssociated = targetPrimaryDomain == sourcePrimaryDomain; >- >- bool statisticsWereUpdated = false; >- if (!isMainFrame && targetHost != mainFrameHost && !(areTargetAndMainFrameDomainsAssociated || areTargetAndSourceDomainsAssociated)) { >- auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain); >- targetStatistics.lastSeen = ResourceLoadStatistics::reduceTimeResolution(WallTime::now()); >- if (targetStatistics.subframeUnderTopFrameOrigins.add(mainFramePrimaryDomain).isNewEntry) >- statisticsWereUpdated = true; >- } >- >- if (isRedirect && !areTargetAndSourceDomainsAssociated) { >- if (isMainFrame) { >- auto& redirectingOriginStatistics = ensureResourceStatisticsForPrimaryDomain(sourcePrimaryDomain); >- if (redirectingOriginStatistics.topFrameUniqueRedirectsTo.add(targetPrimaryDomain).isNewEntry) >- statisticsWereUpdated = true; >- auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain); >- if (targetStatistics.topFrameUniqueRedirectsFrom.add(sourcePrimaryDomain).isNewEntry) >- statisticsWereUpdated = true; >- } else { >- auto& redirectingOriginStatistics = ensureResourceStatisticsForPrimaryDomain(sourcePrimaryDomain); >- if (redirectingOriginStatistics.subresourceUniqueRedirectsTo.add(targetPrimaryDomain).isNewEntry) >- statisticsWereUpdated = true; >- auto& targetStatistics = ensureResourceStatisticsForPrimaryDomain(targetPrimaryDomain); >- if (targetStatistics.subresourceUniqueRedirectsFrom.add(sourcePrimaryDomain).isNewEntry) >- statisticsWereUpdated = true; >- } >- } >- >- if (statisticsWereUpdated) >- scheduleStatisticsProcessingRequestIfNecessary(); >-} >- >-void ResourceLoadStatisticsMemoryStore::logUserInteraction(const String& primaryDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >- statistics.hadUserInteraction = true; >- statistics.mostRecentUserInteractionTime = WallTime::now(); >-} >- >-void ResourceLoadStatisticsMemoryStore::clearUserInteraction(const String& primaryDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >- statistics.hadUserInteraction = false; >- statistics.mostRecentUserInteractionTime = { }; >-} >- >-bool ResourceLoadStatisticsMemoryStore::hasHadUserInteraction(const String& primaryDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >- return mapEntry == m_resourceStatisticsMap.end() ? false: hasHadUnexpiredRecentUserInteraction(mapEntry->value); >-} >- >-void ResourceLoadStatisticsMemoryStore::setPrevalentResource(WebCore::ResourceLoadStatistics& resourceStatistic, ResourceLoadPrevalence newPrevalence) >-{ >- ASSERT(!RunLoop::isMain()); >- >- resourceStatistic.isPrevalentResource = true; >- resourceStatistic.isVeryPrevalentResource = newPrevalence == ResourceLoadPrevalence::VeryHigh; >- HashSet<String> domainsThatHaveRedirectedTo; >- recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(resourceStatistic, domainsThatHaveRedirectedTo, 0); >- for (auto& domain : domainsThatHaveRedirectedTo) { >- auto mapEntry = m_resourceStatisticsMap.find(domain); >- if (mapEntry == m_resourceStatisticsMap.end()) >- continue; >- ASSERT(!mapEntry->value.isPrevalentResource); >- mapEntry->value.isPrevalentResource = true; >- } >-} >- >-String ResourceLoadStatisticsMemoryStore::dumpResourceLoadStatistics() const >-{ >- ASSERT(!RunLoop::isMain()); >- >- StringBuilder result; >- result.appendLiteral("Resource load statistics:\n\n"); >- for (auto& mapEntry : m_resourceStatisticsMap.values()) >- result.append(mapEntry.toString()); >- return result.toString(); >-} >- >-bool ResourceLoadStatisticsMemoryStore::isPrevalentResource(const String& primaryDomain) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >- return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource; >-} >- >-bool ResourceLoadStatisticsMemoryStore::isVeryPrevalentResource(const String& primaryDomain) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >- return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.isPrevalentResource && mapEntry->value.isVeryPrevalentResource; >-} >- >-bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubresourceUnder(const String& subresourcePrimaryDomain, const String& topFramePrimaryDomain) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto mapEntry = m_resourceStatisticsMap.find(subresourcePrimaryDomain); >- return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subresourceUnderTopFrameOrigins.contains(topFramePrimaryDomain); >-} >- >-bool ResourceLoadStatisticsMemoryStore::isRegisteredAsSubFrameUnder(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto mapEntry = m_resourceStatisticsMap.find(subFramePrimaryDomain); >- return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subframeUnderTopFrameOrigins.contains(topFramePrimaryDomain); >-} >- >-bool ResourceLoadStatisticsMemoryStore::isRegisteredAsRedirectingTo(const String& hostRedirectedFromPrimaryDomain, const String& hostRedirectedToPrimaryDomain) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto mapEntry = m_resourceStatisticsMap.find(hostRedirectedFromPrimaryDomain); >- return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.subresourceUniqueRedirectsTo.contains(hostRedirectedToPrimaryDomain); >-} >- >-void ResourceLoadStatisticsMemoryStore::clearPrevalentResource(const String& primaryDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >- statistics.isPrevalentResource = false; >- statistics.isVeryPrevalentResource = false; >-} >- >-void ResourceLoadStatisticsMemoryStore::setGrandfathered(const String& primaryDomain, bool value) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >- statistics.grandfathered = value; >-} >- >-bool ResourceLoadStatisticsMemoryStore::isGrandfathered(const String& primaryDomain) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto mapEntry = m_resourceStatisticsMap.find(primaryDomain); >- return mapEntry == m_resourceStatisticsMap.end() ? false : mapEntry->value.grandfathered; >-} >- >-void ResourceLoadStatisticsMemoryStore::setSubframeUnderTopFrameOrigin(const String& primarySubFrameDomain, const String& primaryTopFrameDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubFrameDomain); >- statistics.subframeUnderTopFrameOrigins.add(primaryTopFrameDomain); >- // For consistency, make sure we also have a statistics entry for the top frame domain. >- ensureResourceStatisticsForPrimaryDomain(primaryTopFrameDomain); >-} >- >-void ResourceLoadStatisticsMemoryStore::setSubresourceUnderTopFrameOrigin(const String& primarySubresourceDomain, const String& primaryTopFrameDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomain); >- statistics.subresourceUnderTopFrameOrigins.add(primaryTopFrameDomain); >- // For consistency, make sure we also have a statistics entry for the top frame domain. >- ensureResourceStatisticsForPrimaryDomain(primaryTopFrameDomain); >-} >- >-void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectTo(const String& primarySubresourceDomain, const String& primaryRedirectDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomain); >- statistics.subresourceUniqueRedirectsTo.add(primaryRedirectDomain); >- // For consistency, make sure we also have a statistics entry for the redirect domain. >- ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >-} >- >-void ResourceLoadStatisticsMemoryStore::setSubresourceUniqueRedirectFrom(const String& primarySubresourceDomain, const String& primaryRedirectDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primarySubresourceDomain); >- statistics.subresourceUniqueRedirectsFrom.add(primaryRedirectDomain); >- // For consistency, make sure we also have a statistics entry for the redirect domain. >- ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >-} >- >-void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectTo(const String& topFramePrimaryDomain, const String& primaryRedirectDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(topFramePrimaryDomain); >- statistics.topFrameUniqueRedirectsTo.add(primaryRedirectDomain); >- // For consistency, make sure we also have a statistics entry for the redirect domain. >- ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >-} >- >-void ResourceLoadStatisticsMemoryStore::setTopFrameUniqueRedirectFrom(const String& topFramePrimaryDomain, const String& primaryRedirectDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(topFramePrimaryDomain); >- statistics.topFrameUniqueRedirectsFrom.add(primaryRedirectDomain); >- // For consistency, make sure we also have a statistics entry for the redirect domain. >- ensureResourceStatisticsForPrimaryDomain(primaryRedirectDomain); >-} >- >-void ResourceLoadStatisticsMemoryStore::setTimeToLiveUserInteraction(Seconds seconds) >-{ >- ASSERT(!RunLoop::isMain()); >- ASSERT(seconds >= 0_s); >- >- m_parameters.timeToLiveUserInteraction = seconds; >-} >- >-void ResourceLoadStatisticsMemoryStore::setMinimumTimeBetweenDataRecordsRemoval(Seconds seconds) >-{ >- ASSERT(!RunLoop::isMain()); >- ASSERT(seconds >= 0_s); >- >- m_parameters.minimumTimeBetweenDataRecordsRemoval = seconds; >-} >- >-void ResourceLoadStatisticsMemoryStore::setGrandfatheringTime(Seconds seconds) >-{ >- ASSERT(!RunLoop::isMain()); >- ASSERT(seconds >= 0_s); >- >- m_parameters.grandfatheringTime = seconds; >-} >- >-void ResourceLoadStatisticsMemoryStore::setCacheMaxAgeCap(Seconds seconds) >-{ >- ASSERT(!RunLoop::isMain()); >- ASSERT(seconds >= 0_s); >- >- m_parameters.cacheMaxAgeCapTime = seconds; >- updateCacheMaxAgeCap(); >-} >- >-void ResourceLoadStatisticsMemoryStore::updateCacheMaxAgeCap() >-{ >- ASSERT(!RunLoop::isMain()); >- >- RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.cacheMaxAgeCapTime] () { >- store->setCacheMaxAgeCap(seconds, [] { }); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::setAgeCapForClientSideCookies(Seconds seconds) >-{ >- ASSERT(!RunLoop::isMain()); >- ASSERT(seconds >= 0_s); >- >- m_parameters.clientSideCookiesAgeCapTime = seconds; >- updateClientSideCookiesAgeCap(); >-} >- >-void ResourceLoadStatisticsMemoryStore::updateClientSideCookiesAgeCap() >-{ >- ASSERT(!RunLoop::isMain()); >- >-#if ENABLE(RESOURCE_LOAD_STATISTICS) >- RunLoop::main().dispatch([store = makeRef(m_store), seconds = m_parameters.clientSideCookiesAgeCapTime] () { >- if (auto* websiteDataStore = store->websiteDataStore()) >- websiteDataStore->setAgeCapForClientSideCookies(seconds, [] { }); >- }); >-#endif >-} >- >-bool ResourceLoadStatisticsMemoryStore::shouldRemoveDataRecords() const >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (m_dataRecordsBeingRemoved) >- return false; >- >-#if ENABLE(NETSCAPE_PLUGIN_API) >- for (const auto& plugin : PluginProcessManager::singleton().pluginProcesses()) { >- if (!m_activePluginTokens.contains(plugin->pluginProcessToken())) >- return true; >- } >-#endif >- >- return !m_lastTimeDataRecordsWereRemoved || MonotonicTime::now() >= (m_lastTimeDataRecordsWereRemoved + m_parameters.minimumTimeBetweenDataRecordsRemoval); >-} >- >-void ResourceLoadStatisticsMemoryStore::setDataRecordsBeingRemoved(bool value) >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_dataRecordsBeingRemoved = value; >- if (m_dataRecordsBeingRemoved) >- m_lastTimeDataRecordsWereRemoved = MonotonicTime::now(); >-} >- >-ResourceLoadStatistics& ResourceLoadStatisticsMemoryStore::ensureResourceStatisticsForPrimaryDomain(const String& primaryDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- return m_resourceStatisticsMap.ensure(primaryDomain, [&primaryDomain] { >- return ResourceLoadStatistics(primaryDomain); >- }).iterator->value; >-} >- >-std::unique_ptr<KeyedEncoder> ResourceLoadStatisticsMemoryStore::createEncoderFromData() const >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto encoder = KeyedEncoder::encoder(); >- encoder->encodeUInt32("version", statisticsModelVersion); >- encoder->encodeDouble("endOfGrandfatheringTimestamp", m_endOfGrandfatheringTimestamp.secondsSinceEpoch().value()); >- >- encoder->encodeObjects("browsingStatistics", m_resourceStatisticsMap.begin(), m_resourceStatisticsMap.end(), [](KeyedEncoder& encoderInner, const auto& origin) { >- origin.value.encode(encoderInner); >- }); >- >- encoder->encodeObjects("operatingDates", m_operatingDates.begin(), m_operatingDates.end(), [](KeyedEncoder& encoderInner, OperatingDate date) { >- encoderInner.encodeDouble("date", date.secondsSinceEpoch().value()); >- }); >- >- return encoder; >-} >- >-void ResourceLoadStatisticsMemoryStore::mergeWithDataFromDecoder(KeyedDecoder& decoder) >-{ >- ASSERT(!RunLoop::isMain()); >- >- unsigned versionOnDisk; >- if (!decoder.decodeUInt32("version", versionOnDisk)) >- return; >- >- if (versionOnDisk > statisticsModelVersion) { >- WTFLogAlways("Found resource load statistics on disk with model version %u whereas the highest supported version is %u. Resetting.", versionOnDisk, statisticsModelVersion); >- return; >- } >- >- double endOfGrandfatheringTimestamp; >- if (decoder.decodeDouble("endOfGrandfatheringTimestamp", endOfGrandfatheringTimestamp)) >- m_endOfGrandfatheringTimestamp = WallTime::fromRawSeconds(endOfGrandfatheringTimestamp); >- else >- m_endOfGrandfatheringTimestamp = { }; >- >- Vector<ResourceLoadStatistics> loadedStatistics; >- bool succeeded = decoder.decodeObjects("browsingStatistics", loadedStatistics, [versionOnDisk](KeyedDecoder& decoderInner, ResourceLoadStatistics& statistics) { >- return statistics.decode(decoderInner, versionOnDisk); >- }); >- >- if (!succeeded) >- return; >- >- mergeStatistics(WTFMove(loadedStatistics)); >- updateCookieBlocking([]() { }); >- >- Vector<OperatingDate> operatingDates; >- succeeded = decoder.decodeObjects("operatingDates", operatingDates, [](KeyedDecoder& decoder, OperatingDate& date) { >- double value; >- if (!decoder.decodeDouble("date", value)) >- return false; >- >- date = OperatingDate::fromWallTime(WallTime::fromRawSeconds(value)); >- return true; >- }); >- >- if (!succeeded) >- return; >- >- m_operatingDates = mergeOperatingDates(m_operatingDates, WTFMove(operatingDates)); >-} >- >-void ResourceLoadStatisticsMemoryStore::clear(CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_resourceStatisticsMap.clear(); >- m_operatingDates.clear(); >- >- auto callbackAggregator = CallbackAggregator::create(WTFMove(completionHandler)); >- >- removeAllStorageAccess([callbackAggregator = callbackAggregator.copyRef()] { }); >- >- auto primaryDomainsToBlock = ensurePrevalentResourcesForDebugMode(); >- updateCookieBlockingForDomains(primaryDomainsToBlock, [callbackAggregator = callbackAggregator.copyRef()] { }); >-} >- >-bool ResourceLoadStatisticsMemoryStore::wasAccessedAsFirstPartyDueToUserInteraction(const ResourceLoadStatistics& current, const ResourceLoadStatistics& updated) const >-{ >- if (!current.hadUserInteraction && !updated.hadUserInteraction) >- return false; >- >- auto mostRecentUserInteractionTime = std::max(current.mostRecentUserInteractionTime, updated.mostRecentUserInteractionTime); >- >- return updated.lastSeen <= mostRecentUserInteractionTime + 24_h; >-} >- >-void ResourceLoadStatisticsMemoryStore::mergeStatistics(Vector<ResourceLoadStatistics>&& statistics) >-{ >- ASSERT(!RunLoop::isMain()); >- >- for (auto& statistic : statistics) { >- auto result = m_resourceStatisticsMap.ensure(statistic.highLevelDomain, [&statistic] { >- return WTFMove(statistic); >- }); >- if (!result.isNewEntry) { >- if (wasAccessedAsFirstPartyDueToUserInteraction(result.iterator->value, statistic)) >- result.iterator->value.timesAccessedAsFirstPartyDueToUserInteraction++; >- result.iterator->value.merge(statistic); >- } >- } >-} >- >-bool ResourceLoadStatisticsMemoryStore::shouldBlockAndKeepCookies(const ResourceLoadStatistics& statistic) >-{ >- return statistic.isPrevalentResource && statistic.hadUserInteraction; >-} >- >-bool ResourceLoadStatisticsMemoryStore::shouldBlockAndPurgeCookies(const ResourceLoadStatistics& statistic) >-{ >- return statistic.isPrevalentResource && !statistic.hadUserInteraction; >-} >- >-bool ResourceLoadStatisticsMemoryStore::hasUserGrantedStorageAccessThroughPrompt(const ResourceLoadStatistics& statistic, const String& firstPartyPrimaryDomain) >-{ >- return statistic.storageAccessUnderTopFrameOrigins.contains(firstPartyPrimaryDomain); >-} >- >-static void debugLogDomainsInBatches(const char* action, const Vector<String>& domains) >-{ >-#if !RELEASE_LOG_DISABLED >- static const auto maxNumberOfDomainsInOneLogStatement = 50; >- if (domains.isEmpty()) >- return; >- >- if (domains.size() <= maxNumberOfDomainsInOneLogStatement) { >- RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for: %{public}s.", action, domainsToString(domains).utf8().data()); >- return; >- } >- >- Vector<String> batch; >- batch.reserveInitialCapacity(maxNumberOfDomainsInOneLogStatement); >- auto batchNumber = 1; >- unsigned numberOfBatches = std::ceil(domains.size() / static_cast<float>(maxNumberOfDomainsInOneLogStatement)); >- >- for (auto& domain : domains) { >- if (batch.size() == maxNumberOfDomainsInOneLogStatement) { >- RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data()); >- batch.shrink(0); >- ++batchNumber; >- } >- batch.append(domain); >- } >- if (!batch.isEmpty()) >- RELEASE_LOG_INFO(ResourceLoadStatisticsDebug, "About to %{public}s cookies in third-party contexts for (%{public}d of %u): %{public}s.", action, batchNumber, numberOfBatches, domainsToString(batch).utf8().data()); >-#else >- UNUSED_PARAM(action); >- UNUSED_PARAM(domains); >-#endif >-} >- >-void ResourceLoadStatisticsMemoryStore::updateCookieBlocking(CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- >- Vector<String> domainsToBlock; >- for (auto& resourceStatistic : m_resourceStatisticsMap.values()) { >- if (resourceStatistic.isPrevalentResource) >- domainsToBlock.append(resourceStatistic.highLevelDomain); >- } >- >- if (domainsToBlock.isEmpty()) { >- completionHandler(); >- return; >- } >- >- if (m_debugLoggingEnabled && !domainsToBlock.isEmpty()) >- debugLogDomainsInBatches("block", domainsToBlock); >- >- RunLoop::main().dispatch([weakThis = makeWeakPtr(*this), store = makeRef(m_store), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable { >- store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [weakThis = WTFMove(weakThis), store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable { >- store->statisticsQueue().dispatch([weakThis = WTFMove(weakThis), completionHandler = WTFMove(completionHandler)]() mutable { >- completionHandler(); >- if (!weakThis) >- return; >-#if !RELEASE_LOG_DISABLED >- RELEASE_LOG_INFO_IF(weakThis->m_debugLoggingEnabled, ResourceLoadStatisticsDebug, "Done updating cookie blocking."); >-#endif >- }); >- }); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::updateCookieBlockingForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- >- RunLoop::main().dispatch([store = makeRef(m_store), domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable { >- store->callUpdatePrevalentDomainsToBlockCookiesForHandler(domainsToBlock, [store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable { >- store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable { >- completionHandler(); >- }); >- }); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::clearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (domains.isEmpty()) { >- completionHandler(); >- return; >- } >- >- RunLoop::main().dispatch([store = makeRef(m_store), domains = crossThreadCopy(domains)] { >- store->callRemoveDomainsHandler(domains); >- }); >- >- completionHandler(); >-} >- >-void ResourceLoadStatisticsMemoryStore::processStatistics(const Function<void(const ResourceLoadStatistics&)>& processFunction) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- for (auto& resourceStatistic : m_resourceStatisticsMap.values()) >- processFunction(resourceStatistic); >-} >- >-bool ResourceLoadStatisticsMemoryStore::hasHadUnexpiredRecentUserInteraction(ResourceLoadStatistics& resourceStatistic) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (resourceStatistic.hadUserInteraction && hasStatisticsExpired(resourceStatistic)) { >- // Drop privacy sensitive data because we no longer need it. >- // Set timestamp to 0 so that statistics merge will know >- // it has been reset as opposed to its default -1. >- resourceStatistic.mostRecentUserInteractionTime = { }; >- resourceStatistic.storageAccessUnderTopFrameOrigins.clear(); >- resourceStatistic.hadUserInteraction = false; >- } >- >- return resourceStatistic.hadUserInteraction; >-} >- >-Vector<String> ResourceLoadStatisticsMemoryStore::topPrivatelyControlledDomainsToRemoveWebsiteDataFor() >-{ >- ASSERT(!RunLoop::isMain()); >- >- bool shouldCheckForGrandfathering = m_endOfGrandfatheringTimestamp > WallTime::now(); >- bool shouldClearGrandfathering = !shouldCheckForGrandfathering && m_endOfGrandfatheringTimestamp; >- >- if (shouldClearGrandfathering) >- m_endOfGrandfatheringTimestamp = { }; >- >- Vector<String> prevalentResources; >- for (auto& statistic : m_resourceStatisticsMap.values()) { >- if (statistic.isPrevalentResource && !hasHadUnexpiredRecentUserInteraction(statistic) && (!shouldCheckForGrandfathering || !statistic.grandfathered)) >- prevalentResources.append(statistic.highLevelDomain); >- >- if (shouldClearGrandfathering && statistic.grandfathered) >- statistic.grandfathered = false; >- } >- >- return prevalentResources; >-} >- >-void ResourceLoadStatisticsMemoryStore::includeTodayAsOperatingDateIfNecessary() >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto today = OperatingDate::today(); >- if (!m_operatingDates.isEmpty() && today <= m_operatingDates.last()) >- return; >- >- while (m_operatingDates.size() >= operatingDatesWindow) >- m_operatingDates.remove(0); >- >- m_operatingDates.append(today); >-} >- >-bool ResourceLoadStatisticsMemoryStore::hasStatisticsExpired(const ResourceLoadStatistics& resourceStatistic) const >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (m_operatingDates.size() >= operatingDatesWindow) { >- if (OperatingDate::fromWallTime(resourceStatistic.mostRecentUserInteractionTime) < m_operatingDates.first()) >- return true; >- } >- >- // If we don't meet the real criteria for an expired statistic, check the user setting for a tighter restriction (mainly for testing). >- if (m_parameters.timeToLiveUserInteraction) { >- if (WallTime::now() > resourceStatistic.mostRecentUserInteractionTime + m_parameters.timeToLiveUserInteraction.value()) >- return true; >- } >- >- return false; >-} >- >-void ResourceLoadStatisticsMemoryStore::setMaxStatisticsEntries(size_t maximumEntryCount) >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_parameters.maxStatisticsEntries = maximumEntryCount; >-} >- >-void ResourceLoadStatisticsMemoryStore::setPruneEntriesDownTo(size_t pruneTargetCount) >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_parameters.pruneEntriesDownTo = pruneTargetCount; >-} >- >-void ResourceLoadStatisticsMemoryStore::pruneStatisticsIfNeeded() >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (m_resourceStatisticsMap.size() <= m_parameters.maxStatisticsEntries) >- return; >- >- ASSERT(m_parameters.pruneEntriesDownTo <= m_parameters.maxStatisticsEntries); >- >- size_t numberOfEntriesLeftToPrune = m_resourceStatisticsMap.size() - m_parameters.pruneEntriesDownTo; >- ASSERT(numberOfEntriesLeftToPrune); >- >- Vector<StatisticsLastSeen> resourcesToPrunePerImportance[maxImportance + 1]; >- for (auto& resourceStatistic : m_resourceStatisticsMap.values()) >- resourcesToPrunePerImportance[computeImportance(resourceStatistic)].append({ resourceStatistic.highLevelDomain, resourceStatistic.lastSeen }); >- >- for (unsigned importance = 0; numberOfEntriesLeftToPrune && importance <= maxImportance; ++importance) >- pruneResources(m_resourceStatisticsMap, resourcesToPrunePerImportance[importance], numberOfEntriesLeftToPrune); >- >- ASSERT(!numberOfEntriesLeftToPrune); >-} >- >-void ResourceLoadStatisticsMemoryStore::resetParametersToDefaultValues() >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_parameters = { }; >-} >- >-void ResourceLoadStatisticsMemoryStore::logTestingEvent(const String& event) >-{ >- ASSERT(!RunLoop::isMain()); >- >- RunLoop::main().dispatch([store = makeRef(m_store), event = event.isolatedCopy()] { >- store->logTestingEvent(event); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::setLastSeen(const String& primaryDomain, Seconds seconds) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& statistics = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >- statistics.lastSeen = WallTime::fromRawSeconds(seconds.seconds()); >-} >- >-void ResourceLoadStatisticsMemoryStore::setPrevalentResource(const String& primaryDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& resourceStatistic = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >- setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::High); >-} >- >-void ResourceLoadStatisticsMemoryStore::setVeryPrevalentResource(const String& primaryDomain) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto& resourceStatistic = ensureResourceStatisticsForPrimaryDomain(primaryDomain); >- setPrevalentResource(resourceStatistic, ResourceLoadPrevalence::VeryHigh); >-} >- >-void ResourceLoadStatisticsMemoryStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(!RunLoop::isMain()); >- RunLoop::main().dispatch([store = makeRef(m_store), completionHandler = WTFMove(completionHandler)]() mutable { >- store->removeAllStorageAccess([store = store.copyRef(), completionHandler = WTFMove(completionHandler)]() mutable { >- store->statisticsQueue().dispatch([completionHandler = WTFMove(completionHandler)]() mutable { >- completionHandler(); >- }); >- }); >- }); >-} >- >-void ResourceLoadStatisticsMemoryStore::didCreateNetworkProcess() >-{ >- ASSERT(!RunLoop::isMain()); >- >- updateCookieBlocking([]() { }); >- updateCacheMaxAgeCap(); >- updateClientSideCookiesAgeCap(); >-} >- >-} // namespace WebKit >diff --git a/Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.h b/Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.h >deleted file mode 100644 >index f3a9a7b96a00d85d0c1ea7b1caa61c72bd1db0c4..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/ResourceLoadStatisticsMemoryStore.h >+++ /dev/null >@@ -1,203 +0,0 @@ >-/* >- * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#pragma once >- >-#include "ResourceLoadStatisticsClassifier.h" >-#include "WebResourceLoadStatisticsStore.h" >-#include <wtf/CompletionHandler.h> >-#include <wtf/Vector.h> >-#include <wtf/WeakPtr.h> >-#include <wtf/WorkQueue.h> >- >-#if HAVE(CORE_PREDICTION) >-#include "ResourceLoadStatisticsClassifierCocoa.h" >-#endif >- >-namespace WebCore { >-class KeyedDecoder; >-class KeyedEncoder; >-struct ResourceLoadStatistics; >-} >- >-namespace WebKit { >- >-class OperatingDate; >-class ResourceLoadStatisticsPersistentStorage; >- >-// This is always constructed / used / destroyed on the WebResourceLoadStatisticsStore's statistics queue. >-class ResourceLoadStatisticsMemoryStore : public CanMakeWeakPtr<ResourceLoadStatisticsMemoryStore> { >-public: >- ResourceLoadStatisticsMemoryStore(WebResourceLoadStatisticsStore&, WorkQueue&); >- ~ResourceLoadStatisticsMemoryStore(); >- >- void setPersistentStorage(ResourceLoadStatisticsPersistentStorage&); >- >- void clear(CompletionHandler<void()>&&); >- bool isEmpty() const { return m_resourceStatisticsMap.isEmpty(); } >- >- std::unique_ptr<WebCore::KeyedEncoder> createEncoderFromData() const; >- void mergeWithDataFromDecoder(WebCore::KeyedDecoder&); >- >- void mergeStatistics(Vector<WebCore::ResourceLoadStatistics>&&); >- void processStatistics(const Function<void(const WebCore::ResourceLoadStatistics&)>&) const; >- >- void updateCookieBlocking(CompletionHandler<void()>&&); >- void updateCookieBlockingForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&&); >- void clearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&&); >- >- void includeTodayAsOperatingDateIfNecessary(); >- void processStatisticsAndDataRecords(); >- >- void requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain); >- void removeAllStorageAccess(CompletionHandler<void()>&&); >- >- void grandfatherExistingWebsiteData(CompletionHandler<void()>&&); >- void cancelPendingStatisticsProcessingRequest(); >- >- bool isRegisteredAsSubresourceUnder(const String& subresourcePrimaryDomain, const String& topFramePrimaryDomain) const; >- bool isRegisteredAsSubFrameUnder(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain) const; >- bool isRegisteredAsRedirectingTo(const String& hostRedirectedFromPrimaryDomain, const String& hostRedirectedToPrimaryDomain) const; >- >- void clearPrevalentResource(const String& primaryDomain); >- String dumpResourceLoadStatistics() const; >- bool isPrevalentResource(const String& primaryDomain) const; >- bool isVeryPrevalentResource(const String& primaryDomain) const; >- void setPrevalentResource(const String& primaryDomain); >- void setVeryPrevalentResource(const String& primaryDomain); >- >- void setGrandfathered(const String& primaryDomain, bool value); >- bool isGrandfathered(const String& primaryDomain) const; >- >- void setSubframeUnderTopFrameOrigin(const String& primarySubFrameDomain, const String& primaryTopFrameDomain); >- void setSubresourceUnderTopFrameOrigin(const String& primarySubresourceDomain, const String& primaryTopFrameDomain); >- void setSubresourceUniqueRedirectTo(const String& primarySubresourceDomain, const String& primaryRedirectDomain); >- void setSubresourceUniqueRedirectFrom(const String& primarySubresourceDomain, const String& primaryRedirectDomain); >- void setTopFrameUniqueRedirectTo(const String& topFramePrimaryDomain, const String& primaryRedirectDomain); >- void setTopFrameUniqueRedirectFrom(const String& topFramePrimaryDomain, const String& primaryRedirectDomain); >- >- void logTestingEvent(const String&); >- >- void setMaxStatisticsEntries(size_t maximumEntryCount); >- void setPruneEntriesDownTo(size_t pruneTargetCount); >- void resetParametersToDefaultValues(); >- >- void calculateAndSubmitTelemetry() const; >- >- void setNotifyPagesWhenDataRecordsWereScanned(bool); >- void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool); >- void setShouldSubmitTelemetry(bool); >- void setTimeToLiveUserInteraction(Seconds); >- void setMinimumTimeBetweenDataRecordsRemoval(Seconds); >- void setGrandfatheringTime(Seconds); >- void setResourceLoadStatisticsDebugMode(bool); >- bool isDebugModeEnabled() const { return m_debugModeEnabled; }; >- void setPrevalentResourceForDebugMode(const String& domain); >- >- void hasStorageAccess(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&&); >- void requestStorageAccess(String&& subFramePrimaryDomain, String&& topFramePrimaryDomain, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&); >- void grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&); >- >- void logFrameNavigation(const String& targetPrimaryDomain, const String& mainFramePrimaryDomain, const String& sourcePrimaryDomain, const String& targetHost, const String& mainFrameHost, bool isRedirect, bool isMainFrame); >- void logUserInteraction(const String& primaryDomain); >- >- void clearUserInteraction(const String& primaryDomain); >- bool hasHadUserInteraction(const String& primaryDomain); >- >- void setLastSeen(const String& primaryDomain, Seconds); >- >- void didCreateNetworkProcess(); >- >-private: >- static bool shouldBlockAndKeepCookies(const WebCore::ResourceLoadStatistics&); >- static bool shouldBlockAndPurgeCookies(const WebCore::ResourceLoadStatistics&); >- static bool hasUserGrantedStorageAccessThroughPrompt(const WebCore::ResourceLoadStatistics&, const String& firstPartyPrimaryDomain); >- bool hasHadUnexpiredRecentUserInteraction(WebCore::ResourceLoadStatistics&) const; >- bool hasStatisticsExpired(const WebCore::ResourceLoadStatistics&) const; >- bool wasAccessedAsFirstPartyDueToUserInteraction(const WebCore::ResourceLoadStatistics& current, const WebCore::ResourceLoadStatistics& updated) const; >- void setPrevalentResource(WebCore::ResourceLoadStatistics&, ResourceLoadPrevalence); >- unsigned recursivelyGetAllDomainsThatHaveRedirectedToThisDomain(const WebCore::ResourceLoadStatistics&, HashSet<String>& domainsThatHaveRedirectedTo, unsigned numberOfRecursiveCalls) const; >- void setStorageAccessPromptsEnabled(bool enabled) { m_storageAccessPromptsEnabled = enabled; } >- bool shouldRemoveDataRecords() const; >- void setDebugLogggingEnabled(bool enabled) { m_debugLoggingEnabled = enabled; } >- void setDataRecordsBeingRemoved(bool); >- void scheduleStatisticsProcessingRequestIfNecessary(); >- void grantStorageAccessInternal(String&& subFrameHost, String&& topFrameHost, Optional<uint64_t> frameID, uint64_t pageID, bool userWasPromptedNowOrEarlier, CompletionHandler<void(bool)>&&); >- void markAsPrevalentIfHasRedirectedToPrevalent(WebCore::ResourceLoadStatistics&); >- bool isPrevalentDueToDebugMode(WebCore::ResourceLoadStatistics&); >- Vector<String> ensurePrevalentResourcesForDebugMode(); >- void removeDataRecords(CompletionHandler<void()>&&); >- void pruneStatisticsIfNeeded(); >- WebCore::ResourceLoadStatistics& ensureResourceStatisticsForPrimaryDomain(const String&); >- Vector<String> topPrivatelyControlledDomainsToRemoveWebsiteDataFor(); >- void setCacheMaxAgeCap(Seconds); >- void updateCacheMaxAgeCap(); >- void setAgeCapForClientSideCookies(Seconds); >- void updateClientSideCookiesAgeCap(); >- >-#if PLATFORM(COCOA) >- void registerUserDefaultsIfNeeded(); >-#endif >- >- struct Parameters { >- size_t pruneEntriesDownTo { 800 }; >- size_t maxStatisticsEntries { 1000 }; >- Optional<Seconds> timeToLiveUserInteraction; >- Seconds minimumTimeBetweenDataRecordsRemoval { 1_h }; >- Seconds grandfatheringTime { 24_h * 7 }; >- Seconds cacheMaxAgeCapTime { 24_h * 7 }; >- Seconds clientSideCookiesAgeCapTime { 24_h * 7 }; >- bool shouldNotifyPagesWhenDataRecordsWereScanned { false }; >- bool shouldClassifyResourcesBeforeDataRecordsRemoval { true }; >- bool shouldSubmitTelemetry { true }; >- }; >- >- WebResourceLoadStatisticsStore& m_store; >- Ref<WorkQueue> m_workQueue; >- WeakPtr<ResourceLoadStatisticsPersistentStorage> m_persistentStorage; >- HashMap<String, WebCore::ResourceLoadStatistics> m_resourceStatisticsMap; >-#if HAVE(CORE_PREDICTION) >- ResourceLoadStatisticsClassifierCocoa m_resourceLoadStatisticsClassifier; >-#else >- ResourceLoadStatisticsClassifier m_resourceLoadStatisticsClassifier; >-#endif >-#if ENABLE(NETSCAPE_PLUGIN_API) >- HashSet<uint64_t> m_activePluginTokens; >-#endif >- Parameters m_parameters; >- Vector<OperatingDate> m_operatingDates; >- WallTime m_endOfGrandfatheringTimestamp; >- bool m_debugLoggingEnabled { false }; >- bool m_debugModeEnabled { false }; >- String m_debugManualPrevalentResource; >- bool m_storageAccessPromptsEnabled { false }; >- bool m_dataRecordsBeingRemoved { false }; >- MonotonicTime m_lastTimeDataRecordsWereRemoved; >- >- uint64_t m_lastStatisticsProcessingRequestIdentifier { 0 }; >- Optional<uint64_t> m_pendingStatisticsProcessingRequestIdentifier; >-}; >- >-} // namespace WebKit >diff --git a/Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.cpp b/Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.cpp >deleted file mode 100644 >index d08d80219e90f045e867597b67bd2b624b0f7b5f..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.cpp >+++ /dev/null >@@ -1,271 +0,0 @@ >-/* >- * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#include "config.h" >-#include "ResourceLoadStatisticsPersistentStorage.h" >- >-#include "Logging.h" >-#include "ResourceLoadStatisticsMemoryStore.h" >-#include "WebResourceLoadStatisticsStore.h" >-#include <WebCore/FileMonitor.h> >-#include <WebCore/FileSystem.h> >-#include <WebCore/KeyedCoding.h> >-#include <WebCore/SharedBuffer.h> >-#include <wtf/RunLoop.h> >-#include <wtf/WorkQueue.h> >- >-namespace WebKit { >- >-constexpr Seconds minimumWriteInterval { 5_min }; >- >-using namespace WebCore; >- >-static bool hasFileChangedSince(const String& path, WallTime since) >-{ >- ASSERT(!RunLoop::isMain()); >- auto modificationTime = FileSystem::getFileModificationTime(path); >- if (!modificationTime) >- return true; >- >- return modificationTime.value() > since; >-} >- >-ResourceLoadStatisticsPersistentStorage::ResourceLoadStatisticsPersistentStorage(ResourceLoadStatisticsMemoryStore& memoryStore, WorkQueue& workQueue, const String& storageDirectoryPath) >- : m_memoryStore(memoryStore) >- , m_workQueue(workQueue) >- , m_storageDirectoryPath(storageDirectoryPath) >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_memoryStore.setPersistentStorage(*this); >- >- populateMemoryStoreFromDisk(); >- startMonitoringDisk(); >-} >- >-ResourceLoadStatisticsPersistentStorage::~ResourceLoadStatisticsPersistentStorage() >-{ >- ASSERT(!RunLoop::isMain()); >- >- if (m_hasPendingWrite) >- writeMemoryStoreToDisk(); >-} >- >-String ResourceLoadStatisticsPersistentStorage::storageDirectoryPath() const >-{ >- return m_storageDirectoryPath.isolatedCopy(); >-} >- >-String ResourceLoadStatisticsPersistentStorage::resourceLogFilePath() const >-{ >- String storagePath = storageDirectoryPath(); >- if (storagePath.isEmpty()) >- return emptyString(); >- >- return FileSystem::pathByAppendingComponent(storagePath, "full_browsing_session_resourceLog.plist"); >-} >- >-void ResourceLoadStatisticsPersistentStorage::startMonitoringDisk() >-{ >- ASSERT(!RunLoop::isMain()); >- if (m_fileMonitor) >- return; >- >- String resourceLogPath = resourceLogFilePath(); >- if (resourceLogPath.isEmpty()) >- return; >- >- m_fileMonitor = std::make_unique<FileMonitor>(resourceLogPath, m_workQueue.copyRef(), [this, weakThis = makeWeakPtr(*this)] (FileMonitor::FileChangeType type) { >- ASSERT(!RunLoop::isMain()); >- if (!weakThis) >- return; >- >- switch (type) { >- case FileMonitor::FileChangeType::Modification: >- refreshMemoryStoreFromDisk(); >- break; >- case FileMonitor::FileChangeType::Removal: >- m_memoryStore.clear([] { }); >- m_fileMonitor = nullptr; >- monitorDirectoryForNewStatistics(); >- break; >- } >- }); >-} >- >-void ResourceLoadStatisticsPersistentStorage::monitorDirectoryForNewStatistics() >-{ >- ASSERT(!RunLoop::isMain()); >- >- String storagePath = storageDirectoryPath(); >- ASSERT(!storagePath.isEmpty()); >- >- if (!FileSystem::fileExists(storagePath)) { >- if (!FileSystem::makeAllDirectories(storagePath)) { >- RELEASE_LOG_ERROR(ResourceLoadStatistics, "ResourceLoadStatisticsPersistentStorage: Failed to create directory path %s", storagePath.utf8().data()); >- return; >- } >- } >- >- m_fileMonitor = std::make_unique<FileMonitor>(storagePath, m_workQueue.copyRef(), [this] (FileMonitor::FileChangeType type) { >- ASSERT(!RunLoop::isMain()); >- if (type == FileMonitor::FileChangeType::Removal) { >- // Directory was removed! >- m_fileMonitor = nullptr; >- return; >- } >- >- String resourceLogPath = resourceLogFilePath(); >- ASSERT(!resourceLogPath.isEmpty()); >- >- if (!FileSystem::fileExists(resourceLogPath)) >- return; >- >- m_fileMonitor = nullptr; >- >- refreshMemoryStoreFromDisk(); >- startMonitoringDisk(); >- }); >-} >- >-void ResourceLoadStatisticsPersistentStorage::stopMonitoringDisk() >-{ >- ASSERT(!RunLoop::isMain()); >- m_fileMonitor = nullptr; >-} >- >-// This is called when the file changes on disk. >-void ResourceLoadStatisticsPersistentStorage::refreshMemoryStoreFromDisk() >-{ >- ASSERT(!RunLoop::isMain()); >- >- String filePath = resourceLogFilePath(); >- if (filePath.isEmpty()) >- return; >- >- // We sometimes see file changed events from before our load completed (we start >- // reading at the first change event, but we might receive a series of events related >- // to the same file operation). Catch this case to avoid reading overly often. >- if (!hasFileChangedSince(filePath, m_lastStatisticsFileSyncTime)) >- return; >- >- WallTime readTime = WallTime::now(); >- >- auto decoder = createForFile(filePath); >- if (!decoder) >- return; >- >- m_memoryStore.mergeWithDataFromDecoder(*decoder); >- m_lastStatisticsFileSyncTime = readTime; >-} >- >-void ResourceLoadStatisticsPersistentStorage::populateMemoryStoreFromDisk() >-{ >- ASSERT(!RunLoop::isMain()); >- >- String filePath = resourceLogFilePath(); >- if (filePath.isEmpty() || !FileSystem::fileExists(filePath)) { >- m_memoryStore.grandfatherExistingWebsiteData([]() { }); >- monitorDirectoryForNewStatistics(); >- return; >- } >- >- if (!hasFileChangedSince(filePath, m_lastStatisticsFileSyncTime)) { >- // No need to grandfather in this case. >- return; >- } >- >- WallTime readTime = WallTime::now(); >- >- auto decoder = createForFile(filePath); >- if (!decoder) { >- m_memoryStore.grandfatherExistingWebsiteData([]() { }); >- return; >- } >- >- // Debug mode has a prepoulated memory store. >- ASSERT_WITH_MESSAGE(m_memoryStore.isEmpty() || m_memoryStore.isDebugModeEnabled(), "This is the initial import so the store should be empty"); >- m_memoryStore.mergeWithDataFromDecoder(*decoder); >- >- m_lastStatisticsFileSyncTime = readTime; >- >- m_memoryStore.logTestingEvent("PopulatedWithoutGrandfathering"_s); >-} >- >-void ResourceLoadStatisticsPersistentStorage::writeMemoryStoreToDisk() >-{ >- ASSERT(!RunLoop::isMain()); >- >- m_hasPendingWrite = false; >- stopMonitoringDisk(); >- >- writeToDisk(m_memoryStore.createEncoderFromData(), resourceLogFilePath()); >- >- m_lastStatisticsFileSyncTime = WallTime::now(); >- m_lastStatisticsWriteTime = MonotonicTime::now(); >- >- startMonitoringDisk(); >-} >- >-void ResourceLoadStatisticsPersistentStorage::scheduleOrWriteMemoryStore(ForceImmediateWrite forceImmediateWrite) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto timeSinceLastWrite = MonotonicTime::now() - m_lastStatisticsWriteTime; >- if (forceImmediateWrite != ForceImmediateWrite::Yes && timeSinceLastWrite < minimumWriteInterval) { >- if (!m_hasPendingWrite) { >- m_hasPendingWrite = true; >- Seconds delay = minimumWriteInterval - timeSinceLastWrite + 1_s; >- m_workQueue->dispatchAfter(delay, [weakThis = makeWeakPtr(*this)] { >- if (weakThis) >- weakThis->writeMemoryStoreToDisk(); >- }); >- } >- return; >- } >- >- writeMemoryStoreToDisk(); >-} >- >-void ResourceLoadStatisticsPersistentStorage::clear() >-{ >- ASSERT(!RunLoop::isMain()); >- String filePath = resourceLogFilePath(); >- if (filePath.isEmpty()) >- return; >- >- stopMonitoringDisk(); >- >- if (!FileSystem::deleteFile(filePath) && FileSystem::fileExists(filePath)) >- RELEASE_LOG_ERROR(ResourceLoadStatistics, "ResourceLoadStatisticsPersistentStorage: Unable to delete statistics file: %s", filePath.utf8().data()); >-} >- >-#if !PLATFORM(IOS_FAMILY) >-void ResourceLoadStatisticsPersistentStorage::excludeFromBackup() const >-{ >-} >-#endif >- >-} // namespace WebKit >diff --git a/Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.h b/Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.h >deleted file mode 100644 >index 12aa9b8aec2fd5001f1d2cc6aa33dbb2728ee6be..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/ResourceLoadStatisticsPersistentStorage.h >+++ /dev/null >@@ -1,77 +0,0 @@ >-/* >- * Copyright (C) 2017-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#pragma once >- >-#include <wtf/Forward.h> >-#include <wtf/MonotonicTime.h> >-#include <wtf/RunLoop.h> >-#include <wtf/WallTime.h> >-#include <wtf/WeakPtr.h> >-#include <wtf/WorkQueue.h> >-#include <wtf/text/WTFString.h> >- >-namespace WebCore { >-class FileMonitor; >-} >- >-namespace WebKit { >- >-class ResourceLoadStatisticsMemoryStore; >- >-// Can only be constructed / destroyed / used from the WebResourceLoadStatisticsStore's statistic queue. >-class ResourceLoadStatisticsPersistentStorage : public CanMakeWeakPtr<ResourceLoadStatisticsPersistentStorage> { >-public: >- ResourceLoadStatisticsPersistentStorage(ResourceLoadStatisticsMemoryStore&, WorkQueue&, const String& storageDirectoryPath); >- ~ResourceLoadStatisticsPersistentStorage(); >- >- void clear(); >- >- enum class ForceImmediateWrite { No, Yes, }; >- void scheduleOrWriteMemoryStore(ForceImmediateWrite); >- >-private: >- String storageDirectoryPath() const; >- String resourceLogFilePath() const; >- >- void startMonitoringDisk(); >- void stopMonitoringDisk(); >- void monitorDirectoryForNewStatistics(); >- >- void writeMemoryStoreToDisk(); >- void populateMemoryStoreFromDisk(); >- void excludeFromBackup() const; >- void refreshMemoryStoreFromDisk(); >- >- ResourceLoadStatisticsMemoryStore& m_memoryStore; >- Ref<WorkQueue> m_workQueue; >- const String m_storageDirectoryPath; >- std::unique_ptr<WebCore::FileMonitor> m_fileMonitor; >- WallTime m_lastStatisticsFileSyncTime; >- MonotonicTime m_lastStatisticsWriteTime; >- bool m_hasPendingWrite { false }; >-}; >- >-} >diff --git a/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp b/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp >index 4ef5dcc365aa39ae606d0bbbf38ef449fe4905d0..dee906f8b085dbcfd294c6bac0883c2622dac501 100644 >--- a/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp >+++ b/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp >@@ -29,7 +29,9 @@ > #include "WebPageMessages.h" > #include "WebPageProxy.h" > #include "WebProcess.h" >+#include "WebProcessPool.h" > #include "WebProcessProxy.h" >+#include "WebsiteDataStore.h" > #include <WebCore/MediaConstraints.h> > #include <WebCore/MockRealtimeMediaSourceCenter.h> > #include <WebCore/RealtimeMediaSource.h> >diff --git a/Source/WebKit/UIProcess/WebDataListSuggestionsDropdown.cpp b/Source/WebKit/UIProcess/WebDataListSuggestionsDropdown.cpp >index ee6ed8c98ee73629c1c50f77f37a3d34833e8a29..8242ceceb363801221eeddd00ebc17f5e617e64b 100644 >--- a/Source/WebKit/UIProcess/WebDataListSuggestionsDropdown.cpp >+++ b/Source/WebKit/UIProcess/WebDataListSuggestionsDropdown.cpp >@@ -26,6 +26,8 @@ > #include "config.h" > #include "WebDataListSuggestionsDropdown.h" > >+#include "WebPageProxy.h" >+ > #if ENABLE(DATALIST_ELEMENT) > > namespace WebKit { >diff --git a/Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp b/Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp >deleted file mode 100644 >index 451e7108d7810e41a56473408c2511dfa10fbbc3..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp >+++ /dev/null >@@ -1,902 +0,0 @@ >-/* >- * Copyright (C) 2016-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#include "config.h" >-#include "WebResourceLoadStatisticsStore.h" >- >-#include "Logging.h" >-#include "ResourceLoadStatisticsMemoryStore.h" >-#include "ResourceLoadStatisticsPersistentStorage.h" >-#include "WebFrameProxy.h" >-#include "WebPageProxy.h" >-#include "WebProcessMessages.h" >-#include "WebProcessProxy.h" >-#include "WebResourceLoadStatisticsStoreMessages.h" >-#include "WebResourceLoadStatisticsTelemetry.h" >-#include "WebsiteDataFetchOption.h" >-#include "WebsiteDataStore.h" >-#include <WebCore/ResourceLoadStatistics.h> >-#include <wtf/CallbackAggregator.h> >-#include <wtf/CrossThreadCopier.h> >-#include <wtf/NeverDestroyed.h> >-#include <wtf/threads/BinarySemaphore.h> >- >-namespace WebKit { >-using namespace WebCore; >- >-template<typename T> static inline String isolatedPrimaryDomain(const T& value) >-{ >- return ResourceLoadStatistics::primaryDomain(value).isolatedCopy(); >-} >- >-const OptionSet<WebsiteDataType>& WebResourceLoadStatisticsStore::monitoredDataTypes() >-{ >- static NeverDestroyed<OptionSet<WebsiteDataType>> dataTypes(std::initializer_list<WebsiteDataType>({ >- WebsiteDataType::Cookies, >- WebsiteDataType::DOMCache, >- WebsiteDataType::IndexedDBDatabases, >- WebsiteDataType::LocalStorage, >- WebsiteDataType::MediaKeys, >- WebsiteDataType::OfflineWebApplicationCache, >-#if ENABLE(NETSCAPE_PLUGIN_API) >- WebsiteDataType::PlugInData, >-#endif >- WebsiteDataType::SearchFieldRecentSearches, >- WebsiteDataType::SessionStorage, >-#if ENABLE(SERVICE_WORKER) >- WebsiteDataType::ServiceWorkerRegistrations, >-#endif >- WebsiteDataType::WebSQLDatabases, >- })); >- >- ASSERT(RunLoop::isMain()); >- >- return dataTypes; >-} >- >-void WebResourceLoadStatisticsStore::setNotifyPagesWhenDataRecordsWereScanned(bool value) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, value] { >- if (m_memoryStore) >- m_memoryStore->setNotifyPagesWhenDataRecordsWereScanned(value); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setShouldClassifyResourcesBeforeDataRecordsRemoval(bool value) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, value] { >- if (m_memoryStore) >- m_memoryStore->setShouldClassifyResourcesBeforeDataRecordsRemoval(value); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setShouldSubmitTelemetry(bool value) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, value] { >- if (m_memoryStore) >- m_memoryStore->setShouldSubmitTelemetry(value); >- }); >-} >- >-WebResourceLoadStatisticsStore::WebResourceLoadStatisticsStore(WebsiteDataStore& websiteDataStore) >- : m_websiteDataStore(makeWeakPtr(websiteDataStore)) >- , m_statisticsQueue(WorkQueue::create("WebResourceLoadStatisticsStore Process Data Queue", WorkQueue::Type::Serial, WorkQueue::QOS::Utility)) >- , m_dailyTasksTimer(RunLoop::main(), this, &WebResourceLoadStatisticsStore::performDailyTasks) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, resourceLoadStatisticsDirectory = websiteDataStore.resolvedResourceLoadStatisticsDirectory().isolatedCopy()] { >- m_memoryStore = std::make_unique<ResourceLoadStatisticsMemoryStore>(*this, m_statisticsQueue); >- m_persistentStorage = std::make_unique<ResourceLoadStatisticsPersistentStorage>(*m_memoryStore, m_statisticsQueue, resourceLoadStatisticsDirectory); >- }); >- >- m_dailyTasksTimer.startRepeating(24_h); >-} >- >-WebResourceLoadStatisticsStore::~WebResourceLoadStatisticsStore() >-{ >- ASSERT(RunLoop::isMain()); >- >- flushAndDestroyPersistentStore(); >-} >- >-inline void WebResourceLoadStatisticsStore::postTask(WTF::Function<void()>&& task) >-{ >- ASSERT(RunLoop::isMain()); >- m_statisticsQueue->dispatch([protectedThis = makeRef(*this), task = WTFMove(task)] { >- task(); >- }); >-} >- >-inline void WebResourceLoadStatisticsStore::postTaskReply(WTF::Function<void()>&& reply) >-{ >- ASSERT(!RunLoop::isMain()); >- RunLoop::main().dispatch(WTFMove(reply)); >-} >- >-void WebResourceLoadStatisticsStore::flushAndDestroyPersistentStore() >-{ >- ASSERT(RunLoop::isMain()); >- >- if (!m_persistentStorage && !m_memoryStore) >- return; >- >- // Make sure we destroy the persistent store on the background queue and wait for it to die >- // synchronously since it has a C++ reference to us. Blocking nature of this task allows us >- // to not maintain a WebResourceLoadStatisticsStore reference for the duration of dispatch, >- // avoiding double-deletion issues when this is invoked from the destructor. >- BinarySemaphore semaphore; >- m_statisticsQueue->dispatch([&semaphore, this] { >- m_persistentStorage = nullptr; >- m_memoryStore = nullptr; >- semaphore.signal(); >- }); >- semaphore.wait(); >-} >- >-void WebResourceLoadStatisticsStore::setResourceLoadStatisticsDebugMode(bool value, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, value, completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->setResourceLoadStatisticsDebugMode(value); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setPrevalentResourceForDebugMode(const URL& url, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->setPrevalentResourceForDebugMode(primaryDomain); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::scheduleStatisticsAndDataRecordsProcessing() >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this] { >- if (m_memoryStore) >- m_memoryStore->processStatisticsAndDataRecords(); >- }); >-} >- >-void WebResourceLoadStatisticsStore::resourceLoadStatisticsUpdated(Vector<WebCore::ResourceLoadStatistics>&& origins) >-{ >- ASSERT(RunLoop::isMain()); >- >- // It is safe to move the origins to the background queue without isolated copy here because this is an r-value >- // coming from IPC. ResourceLoadStatistics only contains strings which are safe to move to other threads as long >- // as nobody on this thread holds a reference to those strings. >- postTask([this, origins = WTFMove(origins)]() mutable { >- if (!m_memoryStore) >- return; >- >- m_memoryStore->mergeStatistics(WTFMove(origins)); >- >- // We can cancel any pending request to process statistics since we're doing it synchronously below. >- m_memoryStore->cancelPendingStatisticsProcessingRequest(); >- >- // Fire before processing statistics to propagate user interaction as fast as possible to the network process. >- m_memoryStore->updateCookieBlocking([]() { }); >- m_memoryStore->processStatisticsAndDataRecords(); >- }); >-} >- >-void WebResourceLoadStatisticsStore::hasStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, CompletionHandler<void (bool)>&& completionHandler) >-{ >- ASSERT(subFrameHost != topFrameHost); >- ASSERT(RunLoop::isMain()); >- >- postTask([this, subFramePrimaryDomain = isolatedPrimaryDomain(subFrameHost), topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHost), frameID, pageID, completionHandler = WTFMove(completionHandler)] () mutable { >- if (!m_memoryStore) { >- postTaskReply([completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(false); >- }); >- return; >- } >- m_memoryStore->hasStorageAccess(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, [completionHandler = WTFMove(completionHandler)](bool hasStorageAccess) mutable { >- postTaskReply([completionHandler = WTFMove(completionHandler), hasStorageAccess] () mutable { >- completionHandler(hasStorageAccess); >- }); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::callHasStorageAccessForFrameHandler(const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool hasAccess)>&& callback) >-{ >- ASSERT(RunLoop::isMain()); >- >-#if ENABLE(RESOURCE_LOAD_STATISTICS) >- if (m_websiteDataStore) { >- m_websiteDataStore->hasStorageAccessForFrameHandler(resourceDomain, firstPartyDomain, frameID, pageID, WTFMove(callback)); >- return; >- } >-#endif >- callback(false); >-} >- >-void WebResourceLoadStatisticsStore::requestStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&& completionHandler) >-{ >- ASSERT(subFrameHost != topFrameHost); >- ASSERT(RunLoop::isMain()); >- >- auto subFramePrimaryDomain = isolatedPrimaryDomain(subFrameHost); >- auto topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHost); >- if (subFramePrimaryDomain == topFramePrimaryDomain) { >- completionHandler(StorageAccessStatus::HasAccess); >- return; >- } >- >- postTask([this, subFramePrimaryDomain = crossThreadCopy(subFramePrimaryDomain), topFramePrimaryDomain = crossThreadCopy(topFramePrimaryDomain), frameID, pageID, promptEnabled, completionHandler = WTFMove(completionHandler)] () mutable { >- if (!m_memoryStore) { >- postTaskReply([completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(StorageAccessStatus::CannotRequestAccess); >- }); >- return; >- } >- >- m_memoryStore->requestStorageAccess(WTFMove(subFramePrimaryDomain), WTFMove(topFramePrimaryDomain), frameID, pageID, promptEnabled, [completionHandler = WTFMove(completionHandler)](StorageAccessStatus status) mutable { >- postTaskReply([completionHandler = WTFMove(completionHandler), status] () mutable { >- completionHandler(status); >- }); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain) >-{ >- ASSERT(RunLoop::isMain()); >- >- // It is safe to move the strings to the background queue without isolated copy here because they are r-value references >- // coming from IPC. Strings which are safe to move to other threads as long as nobody on this thread holds a reference >- // to those strings. >- postTask([this, primaryDomainInNeedOfStorageAccess = WTFMove(primaryDomainInNeedOfStorageAccess), openerPageID, openerPrimaryDomain = WTFMove(openerPrimaryDomain)]() mutable { >- if (m_memoryStore) >- m_memoryStore->requestStorageAccessUnderOpener(WTFMove(primaryDomainInNeedOfStorageAccess), openerPageID, WTFMove(openerPrimaryDomain)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- postTask([this, subFrameHost = crossThreadCopy(subFrameHost), topFrameHost = crossThreadCopy(topFrameHost), frameID, pageID, userWasPromptedNow, completionHandler = WTFMove(completionHandler)] () mutable { >- if (!m_memoryStore) { >- postTaskReply([completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(false); >- }); >- return; >- } >- >- m_memoryStore->grantStorageAccess(WTFMove(subFrameHost), WTFMove(topFrameHost), frameID, pageID, userWasPromptedNow, [completionHandler = WTFMove(completionHandler)](bool wasGrantedAccess) mutable { >- postTaskReply([completionHandler = WTFMove(completionHandler), wasGrantedAccess] () mutable { >- completionHandler(wasGrantedAccess); >- }); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::callGrantStorageAccessHandler(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, Optional<uint64_t> frameID, uint64_t pageID, CompletionHandler<void(bool)>&& callback) >-{ >- ASSERT(RunLoop::isMain()); >- >-#if ENABLE(RESOURCE_LOAD_STATISTICS) >- if (m_websiteDataStore) { >- m_websiteDataStore->grantStorageAccessHandler(subFramePrimaryDomain, topFramePrimaryDomain, frameID, pageID, WTFMove(callback)); >- return; >- } >-#endif >- callback(false); >-} >- >-void WebResourceLoadStatisticsStore::didCreateNetworkProcess() >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this] { >- if (!m_memoryStore) >- return; >- m_memoryStore->didCreateNetworkProcess(); >- }); >-} >- >-void WebResourceLoadStatisticsStore::removeAllStorageAccess(CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >-#if ENABLE(RESOURCE_LOAD_STATISTICS) >- if (m_websiteDataStore) >- m_websiteDataStore->removeAllStorageAccessHandler(WTFMove(completionHandler)); >- else >- completionHandler(); >-#else >- completionHandler(); >-#endif >-} >- >-void WebResourceLoadStatisticsStore::applicationWillTerminate() >-{ >- flushAndDestroyPersistentStore(); >-} >- >-void WebResourceLoadStatisticsStore::performDailyTasks() >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this] { >- if (!m_memoryStore) >- return; >- >- m_memoryStore->includeTodayAsOperatingDateIfNecessary(); >- m_memoryStore->calculateAndSubmitTelemetry(); >- }); >-} >- >-void WebResourceLoadStatisticsStore::submitTelemetry() >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this] { >- if (m_memoryStore) >- WebResourceLoadStatisticsTelemetry::calculateAndSubmit(*m_memoryStore); >- }); >-} >- >-void WebResourceLoadStatisticsStore::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL) >-{ >- ASSERT(RunLoop::isMain()); >- >- auto sourceURL = redirectURL; >- bool isRedirect = !redirectURL.isNull(); >- if (!isRedirect) { >- sourceURL = frame.url(); >- if (sourceURL.isNull()) >- sourceURL = pageURL; >- } >- >- auto& targetURL = request.url(); >- >- if (!targetURL.isValid() || !pageURL.isValid()) >- return; >- >- auto targetHost = targetURL.host(); >- auto mainFrameHost = pageURL.host(); >- >- if (targetHost.isEmpty() || mainFrameHost.isEmpty() || targetHost == sourceURL.host()) >- return; >- >- auto targetPrimaryDomain = ResourceLoadStatistics::primaryDomain(targetURL); >- auto mainFramePrimaryDomain = ResourceLoadStatistics::primaryDomain(pageURL); >- auto sourcePrimaryDomain = ResourceLoadStatistics::primaryDomain(sourceURL); >- >- postTask([this, targetPrimaryDomain = targetPrimaryDomain.isolatedCopy(), mainFramePrimaryDomain = mainFramePrimaryDomain.isolatedCopy(), sourcePrimaryDomain = sourcePrimaryDomain.isolatedCopy(), targetHost = targetHost.toString().isolatedCopy(), mainFrameHost = mainFrameHost.toString().isolatedCopy(), isRedirect, isMainFrame = frame.isMainFrame()] { >- >- if (m_memoryStore) >- m_memoryStore->logFrameNavigation(targetPrimaryDomain, mainFramePrimaryDomain, sourcePrimaryDomain, targetHost, mainFrameHost, isRedirect, isMainFrame); >- }); >-} >- >-void WebResourceLoadStatisticsStore::logUserInteraction(const URL& url, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->logUserInteraction(primaryDomain); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::clearUserInteraction(const URL& url, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->clearUserInteraction(primaryDomain); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::hasHadUserInteraction(const URL& url, CompletionHandler<void (bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(false); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)] () mutable { >- bool hadUserInteraction = m_memoryStore ? m_memoryStore->hasHadUserInteraction(primaryDomain) : false; >- postTaskReply([hadUserInteraction, completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(hadUserInteraction); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setLastSeen(const URL& url, Seconds seconds, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), seconds, completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->setLastSeen(primaryDomain, seconds); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setPrevalentResource(const URL& url, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->setPrevalentResource(primaryDomain); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setVeryPrevalentResource(const URL& url, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->setVeryPrevalentResource(primaryDomain); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::dumpResourceLoadStatistics(CompletionHandler<void(const String&)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, completionHandler = WTFMove(completionHandler)] () mutable { >- String result = m_memoryStore ? m_memoryStore->dumpResourceLoadStatistics() : emptyString(); >- postTaskReply([result = result.isolatedCopy(), completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(result); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::isPrevalentResource(const URL& url, CompletionHandler<void (bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(false); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)] () mutable { >- bool isPrevalentResource = m_memoryStore ? m_memoryStore->isPrevalentResource(primaryDomain) : false; >- postTaskReply([isPrevalentResource, completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(isPrevalentResource); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::isVeryPrevalentResource(const URL& url, CompletionHandler<void(bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(false); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)] () mutable { >- bool isVeryPrevalentResource = m_memoryStore ? m_memoryStore->isVeryPrevalentResource(primaryDomain) : false; >- postTaskReply([isVeryPrevalentResource, completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(isVeryPrevalentResource); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::isRegisteredAsSubresourceUnder(const URL& subresource, const URL& topFrame, CompletionHandler<void(bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, subresourcePrimaryDomain = isolatedPrimaryDomain(subresource), topFramePrimaryDomain = isolatedPrimaryDomain(topFrame), completionHandler = WTFMove(completionHandler)] () mutable { >- bool isRegisteredAsSubresourceUnder = m_memoryStore ? m_memoryStore->isRegisteredAsSubresourceUnder(subresourcePrimaryDomain, topFramePrimaryDomain) : false; >- postTaskReply([isRegisteredAsSubresourceUnder, completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(isRegisteredAsSubresourceUnder); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::isRegisteredAsSubFrameUnder(const URL& subFrame, const URL& topFrame, CompletionHandler<void (bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, subFramePrimaryDomain = isolatedPrimaryDomain(subFrame), topFramePrimaryDomain = isolatedPrimaryDomain(topFrame), completionHandler = WTFMove(completionHandler)] () mutable { >- bool isRegisteredAsSubFrameUnder = m_memoryStore ? m_memoryStore->isRegisteredAsSubFrameUnder(subFramePrimaryDomain, topFramePrimaryDomain) : false; >- postTaskReply([isRegisteredAsSubFrameUnder, completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(isRegisteredAsSubFrameUnder); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::isRegisteredAsRedirectingTo(const URL& hostRedirectedFrom, const URL& hostRedirectedTo, CompletionHandler<void (bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, hostRedirectedFromPrimaryDomain = isolatedPrimaryDomain(hostRedirectedFrom), hostRedirectedToPrimaryDomain = isolatedPrimaryDomain(hostRedirectedTo), completionHandler = WTFMove(completionHandler)] () mutable { >- bool isRegisteredAsRedirectingTo = m_memoryStore ? m_memoryStore->isRegisteredAsRedirectingTo(hostRedirectedFromPrimaryDomain, hostRedirectedToPrimaryDomain) : false; >- postTaskReply([isRegisteredAsRedirectingTo, completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(isRegisteredAsRedirectingTo); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::clearPrevalentResource(const URL& url, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(); >- return; >- } >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->clearPrevalentResource(primaryDomain); >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setGrandfathered(const URL& url, bool value) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) >- return; >- >- postTask([this, primaryDomain = isolatedPrimaryDomain(url), value] { >- if (m_memoryStore) >- m_memoryStore->setGrandfathered(primaryDomain, value); >- }); >-} >- >-void WebResourceLoadStatisticsStore::isGrandfathered(const URL& url, CompletionHandler<void (bool)>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (url.protocolIsAbout() || url.isEmpty()) { >- completionHandler(false); >- return; >- } >- >- postTask([this, completionHandler = WTFMove(completionHandler), primaryDomain = isolatedPrimaryDomain(url)] () mutable { >- bool isGrandFathered = m_memoryStore ? m_memoryStore->isGrandfathered(primaryDomain) : false; >- postTaskReply([isGrandFathered, completionHandler = WTFMove(completionHandler)] () mutable { >- completionHandler(isGrandFathered); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setSubframeUnderTopFrameOrigin(const URL& subframe, const URL& topFrame) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (subframe.protocolIsAbout() || subframe.isEmpty() || topFrame.protocolIsAbout() || topFrame.isEmpty()) >- return; >- >- postTask([this, primaryTopFrameDomain = isolatedPrimaryDomain(topFrame), primarySubFrameDomain = isolatedPrimaryDomain(subframe)] { >- if (m_memoryStore) >- m_memoryStore->setSubframeUnderTopFrameOrigin(primarySubFrameDomain, primaryTopFrameDomain); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setSubresourceUnderTopFrameOrigin(const URL& subresource, const URL& topFrame) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (subresource.protocolIsAbout() || subresource.isEmpty() || topFrame.protocolIsAbout() || topFrame.isEmpty()) >- return; >- >- postTask([this, primaryTopFrameDomain = isolatedPrimaryDomain(topFrame), primarySubresourceDomain = isolatedPrimaryDomain(subresource)] { >- if (m_memoryStore) >- m_memoryStore->setSubresourceUnderTopFrameOrigin(primarySubresourceDomain, primaryTopFrameDomain); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectTo(const URL& subresource, const URL& hostNameRedirectedTo) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (subresource.protocolIsAbout() || subresource.isEmpty() || hostNameRedirectedTo.protocolIsAbout() || hostNameRedirectedTo.isEmpty()) >- return; >- >- postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedTo), primarySubresourceDomain = isolatedPrimaryDomain(subresource)] { >- if (m_memoryStore) >- m_memoryStore->setSubresourceUniqueRedirectTo(primarySubresourceDomain, primaryRedirectDomain); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setSubresourceUniqueRedirectFrom(const URL& subresource, const URL& hostNameRedirectedFrom) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (subresource.protocolIsAbout() || subresource.isEmpty() || hostNameRedirectedFrom.protocolIsAbout() || hostNameRedirectedFrom.isEmpty()) >- return; >- >- postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedFrom), primarySubresourceDomain = isolatedPrimaryDomain(subresource)] { >- if (m_memoryStore) >- m_memoryStore->setSubresourceUniqueRedirectFrom(primarySubresourceDomain, primaryRedirectDomain); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectTo(const URL& topFrameHostName, const URL& hostNameRedirectedTo) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (topFrameHostName.protocolIsAbout() || topFrameHostName.isEmpty() || hostNameRedirectedTo.protocolIsAbout() || hostNameRedirectedTo.isEmpty()) >- return; >- >- postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedTo), topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHostName)] { >- if (m_memoryStore) >- m_memoryStore->setTopFrameUniqueRedirectTo(topFramePrimaryDomain, primaryRedirectDomain); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setTopFrameUniqueRedirectFrom(const URL& topFrameHostName, const URL& hostNameRedirectedFrom) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (topFrameHostName.protocolIsAbout() || topFrameHostName.isEmpty() || hostNameRedirectedFrom.protocolIsAbout() || hostNameRedirectedFrom.isEmpty()) >- return; >- >- postTask([this, primaryRedirectDomain = isolatedPrimaryDomain(hostNameRedirectedFrom), topFramePrimaryDomain = isolatedPrimaryDomain(topFrameHostName)] { >- if (m_memoryStore) >- m_memoryStore->setTopFrameUniqueRedirectFrom(topFramePrimaryDomain, primaryRedirectDomain); >- }); >-} >- >-void WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdate(CompletionHandler<void()>&& completionHandler) >-{ >- // Helper function used by testing system. Should only be called from the main thread. >- ASSERT(RunLoop::isMain()); >- >- postTask([this, completionHandler = WTFMove(completionHandler)] () mutable { >- if (!m_memoryStore) { >- postTaskReply(WTFMove(completionHandler)); >- return; >- } >- m_memoryStore->updateCookieBlocking([completionHandler = WTFMove(completionHandler)] () mutable { >- postTaskReply(WTFMove(completionHandler)); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::scheduleCookieBlockingUpdateForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler) >-{ >- // Helper function used by testing system. Should only be called from the main thread. >- ASSERT(RunLoop::isMain()); >- postTask([this, domainsToBlock = crossThreadCopy(domainsToBlock), completionHandler = WTFMove(completionHandler)] () mutable { >- if (!m_memoryStore) { >- postTaskReply(WTFMove(completionHandler)); >- return; >- } >- >- m_memoryStore->updateCookieBlockingForDomains(domainsToBlock, [completionHandler = WTFMove(completionHandler)]() mutable { >- postTaskReply(WTFMove(completionHandler)); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::scheduleClearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&& completionHandler) >-{ >- // Helper function used by testing system. Should only be called from the main thread. >- ASSERT(RunLoop::isMain()); >- postTask([this, domains = crossThreadCopy(domains), completionHandler = WTFMove(completionHandler)] () mutable { >- if (!m_memoryStore) { >- postTaskReply(WTFMove(completionHandler)); >- return; >- } >- >- m_memoryStore->clearBlockingStateForDomains(domains, [completionHandler = WTFMove(completionHandler)]() mutable { >- postTaskReply(WTFMove(completionHandler)); >- }); >- }); >-} >- >-void WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent(ShouldGrandfather shouldGrandfather, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- postTask([this, protectedThis = makeRef(*this), shouldGrandfather, completionHandler = WTFMove(completionHandler)] () mutable { >- if (m_persistentStorage) >- m_persistentStorage->clear(); >- >- CompletionHandlerCallingScope completionHandlerCaller([completionHandler = WTFMove(completionHandler)]() mutable { >- postTaskReply(WTFMove(completionHandler)); >- }); >- >- if (m_memoryStore) { >- m_memoryStore->clear([this, protectedThis = protectedThis.copyRef(), shouldGrandfather, completionHandlerCaller = WTFMove(completionHandlerCaller)] () mutable { >- if (shouldGrandfather == ShouldGrandfather::Yes) { >- if (m_memoryStore) >- m_memoryStore->grandfatherExistingWebsiteData(completionHandlerCaller.release()); >- else >- RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent After being cleared, m_memoryStore is null when trying to grandfather data."); >- } >- }); >- } else { >- if (shouldGrandfather == ShouldGrandfather::Yes) >- RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent Before being cleared, m_memoryStore is null when trying to grandfather data."); >- } >- }); >-} >- >-void WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent(WallTime modifiedSince, ShouldGrandfather shouldGrandfather, CompletionHandler<void()>&& callback) >-{ >- ASSERT(RunLoop::isMain()); >- >- // For now, be conservative and clear everything regardless of modifiedSince. >- UNUSED_PARAM(modifiedSince); >- scheduleClearInMemoryAndPersistent(shouldGrandfather, WTFMove(callback)); >-} >- >-void WebResourceLoadStatisticsStore::setTimeToLiveUserInteraction(Seconds seconds) >-{ >- ASSERT(RunLoop::isMain()); >- postTask([this, seconds] { >- if (m_memoryStore) >- m_memoryStore->setTimeToLiveUserInteraction(seconds); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setMinimumTimeBetweenDataRecordsRemoval(Seconds seconds) >-{ >- ASSERT(RunLoop::isMain()); >- postTask([this, seconds] { >- if (m_memoryStore) >- m_memoryStore->setMinimumTimeBetweenDataRecordsRemoval(seconds); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setGrandfatheringTime(Seconds seconds) >-{ >- ASSERT(RunLoop::isMain()); >- postTask([this, seconds] { >- if (m_memoryStore) >- m_memoryStore->setGrandfatheringTime(seconds); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setCacheMaxAgeCap(Seconds seconds, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- ASSERT(seconds >= 0_s); >- >-#if ENABLE(RESOURCE_LOAD_STATISTICS) >- if (m_websiteDataStore) { >- m_websiteDataStore->setCacheMaxAgeCapForPrevalentResources(seconds, WTFMove(completionHandler)); >- return; >- } >-#endif >- completionHandler(); >-} >- >-void WebResourceLoadStatisticsStore::callUpdatePrevalentDomainsToBlockCookiesForHandler(const Vector<String>& domainsToBlock, CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >-#if ENABLE(RESOURCE_LOAD_STATISTICS) >- if (m_websiteDataStore) { >- m_websiteDataStore->updatePrevalentDomainsToBlockCookiesFor(domainsToBlock, WTFMove(completionHandler)); >- return; >- } >-#endif >- completionHandler(); >-} >- >-void WebResourceLoadStatisticsStore::callRemoveDomainsHandler(const Vector<String>& domains) >-{ >- ASSERT(RunLoop::isMain()); >- >-#if ENABLE(RESOURCE_LOAD_STATISTICS) >- if (m_websiteDataStore) >- m_websiteDataStore->removePrevalentDomains(domains); >-#endif >-} >- >-void WebResourceLoadStatisticsStore::setMaxStatisticsEntries(size_t maximumEntryCount) >-{ >- ASSERT(RunLoop::isMain()); >- postTask([this, maximumEntryCount] { >- if (m_memoryStore) >- m_memoryStore->setMaxStatisticsEntries(maximumEntryCount); >- }); >-} >- >-void WebResourceLoadStatisticsStore::setPruneEntriesDownTo(size_t pruneTargetCount) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, pruneTargetCount] { >- if (m_memoryStore) >- m_memoryStore->setPruneEntriesDownTo(pruneTargetCount); >- }); >-} >- >-void WebResourceLoadStatisticsStore::resetParametersToDefaultValues(CompletionHandler<void()>&& completionHandler) >-{ >- ASSERT(RunLoop::isMain()); >- >- postTask([this, completionHandler = WTFMove(completionHandler)]() mutable { >- if (m_memoryStore) >- m_memoryStore->resetParametersToDefaultValues(); >- >- postTaskReply(WTFMove(completionHandler)); >- }); >-} >- >-void WebResourceLoadStatisticsStore::logTestingEvent(const String& event) >-{ >- ASSERT(RunLoop::isMain()); >- >- if (m_statisticsTestingCallback) >- m_statisticsTestingCallback(event); >-} >- >-} // namespace WebKit >diff --git a/Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.h b/Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.h >deleted file mode 100644 >index 581704a4ef299dc083d279f534c77281cfa76b4e..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.h >+++ /dev/null >@@ -1,177 +0,0 @@ >-/* >- * Copyright (C) 2016-2018 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#pragma once >- >-#include "Connection.h" >-#include "WebsiteDataType.h" >-#include <wtf/CompletionHandler.h> >-#include <wtf/RunLoop.h> >-#include <wtf/ThreadSafeRefCounted.h> >-#include <wtf/Vector.h> >-#include <wtf/WallTime.h> >-#include <wtf/WeakPtr.h> >-#include <wtf/text/WTFString.h> >- >-namespace WTF { >-class WorkQueue; >-} >- >-namespace WebCore { >-class ResourceRequest; >-struct ResourceLoadStatistics; >-} >- >-namespace WebKit { >- >-class ResourceLoadStatisticsMemoryStore; >-class ResourceLoadStatisticsPersistentStorage; >-class WebFrameProxy; >-class WebProcessProxy; >-class WebsiteDataStore; >- >-enum class StorageAccessStatus { >- CannotRequestAccess, >- RequiresUserPrompt, >- HasAccess >-}; >- >-class WebResourceLoadStatisticsStore final : public ThreadSafeRefCounted<WebResourceLoadStatisticsStore, WTF::DestructionThread::Main>, public IPC::MessageReceiver { >-public: >- static Ref<WebResourceLoadStatisticsStore> create(WebsiteDataStore& websiteDataStore) >- { >- return adoptRef(*new WebResourceLoadStatisticsStore(websiteDataStore)); >- } >- >- ~WebResourceLoadStatisticsStore(); >- >- static const OptionSet<WebsiteDataType>& monitoredDataTypes(); >- >- WorkQueue& statisticsQueue() { return m_statisticsQueue.get(); } >- >- void setNotifyPagesWhenDataRecordsWereScanned(bool); >- void setShouldClassifyResourcesBeforeDataRecordsRemoval(bool); >- void setShouldSubmitTelemetry(bool); >- >- void hasStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&& callback); >- void requestStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool promptEnabled, CompletionHandler<void(StorageAccessStatus)>&&); >- void grantStorageAccess(String&& subFrameHost, String&& topFrameHost, uint64_t frameID, uint64_t pageID, bool userWasPromptedNow, CompletionHandler<void(bool)>&&); >- void requestStorageAccessCallback(bool wasGranted, uint64_t contextId); >- >- void applicationWillTerminate(); >- >- void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL); >- void logUserInteraction(const URL&, CompletionHandler<void()>&&); >- void clearUserInteraction(const URL&, CompletionHandler<void()>&&); >- void hasHadUserInteraction(const URL&, CompletionHandler<void(bool)>&&); >- void setLastSeen(const URL&, Seconds, CompletionHandler<void()>&&); >- void setPrevalentResource(const URL&, CompletionHandler<void()>&&); >- void setVeryPrevalentResource(const URL&, CompletionHandler<void()>&&); >- void dumpResourceLoadStatistics(CompletionHandler<void(const String&)>&&); >- void isPrevalentResource(const URL&, CompletionHandler<void(bool)>&&); >- void isVeryPrevalentResource(const URL&, CompletionHandler<void(bool)>&&); >- void isRegisteredAsSubresourceUnder(const URL& subresource, const URL& topFrame, CompletionHandler<void(bool)>&&); >- void isRegisteredAsSubFrameUnder(const URL& subFrame, const URL& topFrame, CompletionHandler<void(bool)>&&); >- void isRegisteredAsRedirectingTo(const URL& hostRedirectedFrom, const URL& hostRedirectedTo, CompletionHandler<void(bool)>&&); >- void clearPrevalentResource(const URL&, CompletionHandler<void()>&&); >- void setGrandfathered(const URL&, bool); >- void isGrandfathered(const URL&, CompletionHandler<void(bool)>&&); >- void setSubframeUnderTopFrameOrigin(const URL& subframe, const URL& topFrame); >- void setSubresourceUnderTopFrameOrigin(const URL& subresource, const URL& topFrame); >- void setSubresourceUniqueRedirectTo(const URL& subresource, const URL& hostNameRedirectedTo); >- void setSubresourceUniqueRedirectFrom(const URL& subresource, const URL& hostNameRedirectedFrom); >- void setTopFrameUniqueRedirectTo(const URL& topFrameHostName, const URL& hostNameRedirectedTo); >- void setTopFrameUniqueRedirectFrom(const URL& topFrameHostName, const URL& hostNameRedirectedFrom); >- void scheduleCookieBlockingUpdate(CompletionHandler<void()>&&); >- void scheduleCookieBlockingUpdateForDomains(const Vector<String>& domainsToBlock, CompletionHandler<void()>&&); >- void scheduleClearBlockingStateForDomains(const Vector<String>& domains, CompletionHandler<void()>&&); >- void scheduleStatisticsAndDataRecordsProcessing(); >- void submitTelemetry(); >- >- enum class ShouldGrandfather { >- No, >- Yes, >- }; >- void scheduleClearInMemoryAndPersistent(ShouldGrandfather, CompletionHandler<void()>&&); >- void scheduleClearInMemoryAndPersistent(WallTime modifiedSince, ShouldGrandfather, CompletionHandler<void()>&&); >- >- void setTimeToLiveUserInteraction(Seconds); >- void setMinimumTimeBetweenDataRecordsRemoval(Seconds); >- void setGrandfatheringTime(Seconds); >- void setCacheMaxAgeCap(Seconds, CompletionHandler<void()>&&); >- void setMaxStatisticsEntries(size_t); >- void setPruneEntriesDownTo(size_t); >- >- void resetParametersToDefaultValues(CompletionHandler<void()>&&); >- >- void setResourceLoadStatisticsDebugMode(bool, CompletionHandler<void()>&&); >- void setPrevalentResourceForDebugMode(const URL&, CompletionHandler<void()>&&); >- >- void setStatisticsTestingCallback(WTF::Function<void(const String&)>&& callback) { m_statisticsTestingCallback = WTFMove(callback); } >- void logTestingEvent(const String&); >- void callGrantStorageAccessHandler(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain, Optional<uint64_t> frameID, uint64_t pageID, CompletionHandler<void(bool)>&&); >- void removeAllStorageAccess(CompletionHandler<void()>&&); >- void callUpdatePrevalentDomainsToBlockCookiesForHandler(const Vector<String>& domainsToBlock, CompletionHandler<void()>&&); >- void callRemoveDomainsHandler(const Vector<String>& domains); >- void callHasStorageAccessForFrameHandler(const String& resourceDomain, const String& firstPartyDomain, uint64_t frameID, uint64_t pageID, CompletionHandler<void(bool)>&&); >- >- void didCreateNetworkProcess(); >- >- WebsiteDataStore* websiteDataStore() { return m_websiteDataStore.get(); } >- >-private: >- explicit WebResourceLoadStatisticsStore(WebsiteDataStore&); >- >- void postTask(WTF::Function<void()>&&); >- static void postTaskReply(WTF::Function<void()>&&); >- >- // IPC::MessageReceiver. >- void didReceiveMessage(IPC::Connection&, IPC::Decoder&) override; >- >- // IPC message handlers. >- void resourceLoadStatisticsUpdated(Vector<WebCore::ResourceLoadStatistics>&& origins); >- void requestStorageAccessUnderOpener(String&& primaryDomainInNeedOfStorageAccess, uint64_t openerPageID, String&& openerPrimaryDomain); >- >- void performDailyTasks(); >- >- StorageAccessStatus storageAccessStatus(const String& subFramePrimaryDomain, const String& topFramePrimaryDomain); >- >- void flushAndDestroyPersistentStore(); >- >- WeakPtr<WebsiteDataStore> m_websiteDataStore; >- Ref<WorkQueue> m_statisticsQueue; >- std::unique_ptr<ResourceLoadStatisticsMemoryStore> m_memoryStore; >- std::unique_ptr<ResourceLoadStatisticsPersistentStorage> m_persistentStorage; >- >- RunLoop::Timer<WebResourceLoadStatisticsStore> m_dailyTasksTimer; >- >- bool m_hasScheduledProcessStats { false }; >- >- WTF::Function<void(const String&)> m_statisticsTestingCallback; >- >- bool m_firstNetworkProcessCreated { false }; >-}; >- >-} // namespace WebKit >diff --git a/Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.cpp b/Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.cpp >deleted file mode 100644 >index f9692e12912a6c1c098d3b9f60804ae2ff37df9a..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.cpp >+++ /dev/null >@@ -1,296 +0,0 @@ >-/* >- * Copyright (C) 2017 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#include "config.h" >-#include "WebResourceLoadStatisticsTelemetry.h" >- >-#include "ResourceLoadStatisticsMemoryStore.h" >-#include "WebProcessPool.h" >-#include "WebProcessProxy.h" >-#include <WebCore/DiagnosticLoggingKeys.h> >-#include <WebCore/ResourceLoadStatistics.h> >-#include <wtf/MainThread.h> >-#include <wtf/NeverDestroyed.h> >-#include <wtf/RunLoop.h> >-#include <wtf/text/StringBuilder.h> >- >-namespace WebKit { >-using namespace WebCore; >- >-const unsigned minimumPrevalentResourcesForTelemetry = 3; >-const unsigned significantFiguresForLoggedValues = 3; >-static bool notifyPagesWhenTelemetryWasCaptured = false; >- >-struct PrevalentResourceTelemetry { >- unsigned numberOfTimesDataRecordsRemoved; >- bool hasHadUserInteraction; >- unsigned daysSinceUserInteraction; >- unsigned subframeUnderTopFrameOrigins; >- unsigned subresourceUnderTopFrameOrigins; >- unsigned subresourceUniqueRedirectsTo; >- unsigned timesAccessedAsFirstPartyDueToUserInteraction; >- unsigned timesAccessedAsFirstPartyDueToStorageAccessAPI; >-}; >- >-static Vector<PrevalentResourceTelemetry> sortedPrevalentResourceTelemetry(const ResourceLoadStatisticsMemoryStore& store) >-{ >- ASSERT(!RunLoop::isMain()); >- Vector<PrevalentResourceTelemetry> sorted; >- store.processStatistics([&sorted] (auto& statistic) { >- if (!statistic.isPrevalentResource) >- return; >- >- unsigned daysSinceUserInteraction = statistic.mostRecentUserInteractionTime <= WallTime() ? 0 : std::floor((WallTime::now() - statistic.mostRecentUserInteractionTime) / 24_h); >- sorted.append(PrevalentResourceTelemetry { >- statistic.dataRecordsRemoved, >- statistic.hadUserInteraction, >- daysSinceUserInteraction, >- statistic.subframeUnderTopFrameOrigins.size(), >- statistic.subresourceUnderTopFrameOrigins.size(), >- statistic.subresourceUniqueRedirectsTo.size(), >- statistic.timesAccessedAsFirstPartyDueToUserInteraction, >- statistic.timesAccessedAsFirstPartyDueToStorageAccessAPI >- }); >- }); >- >- if (sorted.size() < minimumPrevalentResourcesForTelemetry) >- return { }; >- >- std::sort(sorted.begin(), sorted.end(), [](const PrevalentResourceTelemetry& a, const PrevalentResourceTelemetry& b) { >- return a.subframeUnderTopFrameOrigins + a.subresourceUnderTopFrameOrigins + a.subresourceUniqueRedirectsTo > >- b.subframeUnderTopFrameOrigins + b.subresourceUnderTopFrameOrigins + b.subresourceUniqueRedirectsTo; >- }); >- >- return sorted; >-} >- >-static unsigned numberOfResourcesWithUserInteraction(const Vector<PrevalentResourceTelemetry>& resources, size_t begin, size_t end) >-{ >- if (resources.isEmpty() || resources.size() < begin + 1 || resources.size() < end + 1) >- return 0; >- >- unsigned result = 0; >- for (size_t i = begin; i < end; ++i) { >- if (resources[i].hasHadUserInteraction) >- ++result; >- } >- >- return result; >-} >- >-static unsigned median(const Vector<unsigned>& v) >-{ >- if (v.isEmpty()) >- return 0; >- if (v.size() == 1) >- return v[0]; >- >- auto size = v.size(); >- auto middle = size / 2; >- if (size % 2) >- return v[middle]; >- return (v[middle - 1] + v[middle]) / 2; >-} >- >-static unsigned median(const Vector<PrevalentResourceTelemetry>& v, unsigned begin, unsigned end, const WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)>& statisticGetter) >-{ >- if (v.isEmpty() || v.size() < begin + 1 || v.size() < end + 1) >- return 0; >- >- Vector<unsigned> part; >- part.reserveInitialCapacity(end - begin + 1); >- for (unsigned i = begin; i <= end; ++i) >- part.uncheckedAppend(statisticGetter(v[i])); >- >- return median(part); >-} >- >-static WebPageProxy* nonEphemeralWebPageProxy() >-{ >- auto processPools = WebProcessPool::allProcessPools(); >- if (processPools.isEmpty()) >- return nullptr; >- >- auto processPool = processPools[0]; >- if (!processPool) >- return nullptr; >- >- for (auto& webProcess : processPool->processes()) { >- for (auto& page : webProcess->pages()) { >- if (page->sessionID().isEphemeral()) >- continue; >- return page; >- } >- } >- return nullptr; >-} >- >-static void submitTopList(unsigned numberOfResourcesFromTheTop, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, WebPageProxy& webPageProxy) >-{ >- WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subframeUnderTopFrameOriginsGetter = [] (auto& t) { >- return t.subframeUnderTopFrameOrigins; >- }; >- WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUnderTopFrameOriginsGetter = [] (auto& t) { >- return t.subresourceUnderTopFrameOrigins; >- }; >- WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subresourceUniqueRedirectsToGetter = [] (auto& t) { >- return t.subresourceUniqueRedirectsTo; >- }; >- WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesDataRecordsRemovedGetter = [] (auto& t) { >- return t.numberOfTimesDataRecordsRemoved; >- }; >- WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesAccessedAsFirstPartyDueToUserInteractionGetter = [] (auto& t) { >- return t.timesAccessedAsFirstPartyDueToUserInteraction; >- }; >- WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> numberOfTimesAccessedAsFirstPartyDueToStorageAccessAPIGetter = [] (auto& t) { >- return t.timesAccessedAsFirstPartyDueToStorageAccessAPI; >- }; >- >- unsigned topPrevalentResourcesWithUserInteraction = numberOfResourcesWithUserInteraction(sortedPrevalentResources, 0, numberOfResourcesFromTheTop - 1); >- unsigned topSubframeUnderTopFrameOrigins = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subframeUnderTopFrameOriginsGetter); >- unsigned topSubresourceUnderTopFrameOrigins = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subresourceUnderTopFrameOriginsGetter); >- unsigned topSubresourceUniqueRedirectsTo = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, subresourceUniqueRedirectsToGetter); >- unsigned topNumberOfTimesDataRecordsRemoved = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesDataRecordsRemovedGetter); >- unsigned topNumberOfTimesAccessedAsFirstPartyDueToUserInteraction = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesAccessedAsFirstPartyDueToUserInteractionGetter); >- unsigned topNumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI = median(sortedPrevalentResourcesWithoutUserInteraction, 0, numberOfResourcesFromTheTop - 1, numberOfTimesAccessedAsFirstPartyDueToStorageAccessAPIGetter); >- >- StringBuilder preambleBuilder; >- preambleBuilder.appendLiteral("top"); >- preambleBuilder.appendNumber(numberOfResourcesFromTheTop); >- String descriptionPreamble = preambleBuilder.toString(); >- >- webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "PrevalentResourcesWithUserInteraction", >- topPrevalentResourcesWithUserInteraction, significantFiguresForLoggedValues, ShouldSample::No); >- webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "SubframeUnderTopFrameOrigins", >- topSubframeUnderTopFrameOrigins, significantFiguresForLoggedValues, ShouldSample::No); >- webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "SubresourceUnderTopFrameOrigins", >- topSubresourceUnderTopFrameOrigins, significantFiguresForLoggedValues, ShouldSample::No); >- webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "SubresourceUniqueRedirectsTo", >- topSubresourceUniqueRedirectsTo, significantFiguresForLoggedValues, ShouldSample::No); >- webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesDataRecordsRemoved", >- topNumberOfTimesDataRecordsRemoved, significantFiguresForLoggedValues, ShouldSample::No); >- webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesAccessedAsFirstPartyDueToUserInteraction", >- topNumberOfTimesAccessedAsFirstPartyDueToUserInteraction, significantFiguresForLoggedValues, ShouldSample::No); >- webPageProxy.logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), descriptionPreamble + "NumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI", >- topNumberOfTimesAccessedAsFirstPartyDueToStorageAccessAPI, significantFiguresForLoggedValues, ShouldSample::No); >-} >- >-static void submitTopLists(const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, WebPageProxy& webPageProxy) >-{ >- submitTopList(1, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >- >- if (sortedPrevalentResourcesWithoutUserInteraction.size() < 3) >- return; >- submitTopList(3, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >- >- if (sortedPrevalentResourcesWithoutUserInteraction.size() < 10) >- return; >- submitTopList(10, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >- >- if (sortedPrevalentResourcesWithoutUserInteraction.size() < 50) >- return; >- submitTopList(50, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >- >- if (sortedPrevalentResourcesWithoutUserInteraction.size() < 100) >- return; >- submitTopList(100, sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, webPageProxy); >-} >- >-// This function is for testing purposes. >-void static notifyPages(unsigned totalPrevalentResources, unsigned totalPrevalentResourcesWithUserInteraction, unsigned top3SubframeUnderTopFrameOrigins) >-{ >- RunLoop::main().dispatch([totalPrevalentResources, totalPrevalentResourcesWithUserInteraction, top3SubframeUnderTopFrameOrigins] { >- API::Dictionary::MapType messageBody; >- messageBody.set("TotalPrevalentResources"_s, API::UInt64::create(totalPrevalentResources)); >- messageBody.set("TotalPrevalentResourcesWithUserInteraction"_s, API::UInt64::create(totalPrevalentResourcesWithUserInteraction)); >- messageBody.set("Top3SubframeUnderTopFrameOrigins"_s, API::UInt64::create(top3SubframeUnderTopFrameOrigins)); >- WebProcessProxy::notifyPageStatisticsTelemetryFinished(API::Dictionary::create(messageBody).ptr()); >- }); >-} >- >-// This function is for testing purposes. >-void static notifyPages(const Vector<PrevalentResourceTelemetry>& sortedPrevalentResources, const Vector<PrevalentResourceTelemetry>& sortedPrevalentResourcesWithoutUserInteraction, unsigned totalNumberOfPrevalentResourcesWithUserInteraction) >-{ >- WTF::Function<unsigned(const PrevalentResourceTelemetry& telemetry)> subframeUnderTopFrameOriginsGetter = [] (const PrevalentResourceTelemetry& t) { >- return t.subframeUnderTopFrameOrigins; >- }; >- >- notifyPages(sortedPrevalentResources.size(), totalNumberOfPrevalentResourcesWithUserInteraction, median(sortedPrevalentResourcesWithoutUserInteraction, 0, 2, subframeUnderTopFrameOriginsGetter)); >-} >- >-void WebResourceLoadStatisticsTelemetry::calculateAndSubmit(const ResourceLoadStatisticsMemoryStore& resourceLoadStatisticsStore) >-{ >- ASSERT(!RunLoop::isMain()); >- >- auto sortedPrevalentResources = sortedPrevalentResourceTelemetry(resourceLoadStatisticsStore); >- if (notifyPagesWhenTelemetryWasCaptured && sortedPrevalentResources.isEmpty()) { >- notifyPages(0, 0, 0); >- return; >- } >- >- Vector<PrevalentResourceTelemetry> sortedPrevalentResourcesWithoutUserInteraction; >- sortedPrevalentResourcesWithoutUserInteraction.reserveInitialCapacity(sortedPrevalentResources.size()); >- Vector<unsigned> prevalentResourcesDaysSinceUserInteraction; >- >- for (auto& prevalentResource : sortedPrevalentResources) { >- if (prevalentResource.hasHadUserInteraction) >- prevalentResourcesDaysSinceUserInteraction.append(prevalentResource.daysSinceUserInteraction); >- else >- sortedPrevalentResourcesWithoutUserInteraction.uncheckedAppend(prevalentResource); >- } >- >- // Dispatch on the main thread to make sure the WebPageProxy we're using doesn't go away. >- RunLoop::main().dispatch([sortedPrevalentResources = WTFMove(sortedPrevalentResources), sortedPrevalentResourcesWithoutUserInteraction = WTFMove(sortedPrevalentResourcesWithoutUserInteraction), prevalentResourcesDaysSinceUserInteraction = WTFMove(prevalentResourcesDaysSinceUserInteraction)] () { >- auto webPageProxy = nonEphemeralWebPageProxy(); >- if (!webPageProxy) { >- if (notifyPagesWhenTelemetryWasCaptured) >- notifyPages(0, 0, 0); >- return; >- } >- >- if (notifyPagesWhenTelemetryWasCaptured) { >- notifyPages(sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, prevalentResourcesDaysSinceUserInteraction.size()); >- // The notify pages function is for testing so we don't need to do an actual submission. >- return; >- } >- >- webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "totalNumberOfPrevalentResources"_s, sortedPrevalentResources.size(), significantFiguresForLoggedValues, ShouldSample::No); >- webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "totalNumberOfPrevalentResourcesWithUserInteraction"_s, prevalentResourcesDaysSinceUserInteraction.size(), significantFiguresForLoggedValues, ShouldSample::No); >- >- if (prevalentResourcesDaysSinceUserInteraction.size() > 0) >- webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "topPrevalentResourceWithUserInteractionDaysSinceUserInteraction"_s, prevalentResourcesDaysSinceUserInteraction[0], significantFiguresForLoggedValues, ShouldSample::No); >- if (prevalentResourcesDaysSinceUserInteraction.size() > 1) >- webPageProxy->logDiagnosticMessageWithValue(DiagnosticLoggingKeys::resourceLoadStatisticsTelemetryKey(), "medianPrevalentResourcesWithUserInteractionDaysSinceUserInteraction"_s, median(prevalentResourcesDaysSinceUserInteraction), significantFiguresForLoggedValues, ShouldSample::No); >- >- submitTopLists(sortedPrevalentResources, sortedPrevalentResourcesWithoutUserInteraction, *webPageProxy); >- }); >-} >- >-void WebResourceLoadStatisticsTelemetry::setNotifyPagesWhenTelemetryWasCaptured(bool always) >-{ >- notifyPagesWhenTelemetryWasCaptured = always; >-} >- >-} >diff --git a/Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.h b/Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.h >deleted file mode 100644 >index f96e61a1af0ab7353b031db753f9ddb6f52c629b..0000000000000000000000000000000000000000 >--- a/Source/WebKit/UIProcess/WebResourceLoadStatisticsTelemetry.h >+++ /dev/null >@@ -1,40 +0,0 @@ >-/* >- * Copyright (C) 2017 Apple Inc. All rights reserved. >- * >- * Redistribution and use in source and binary forms, with or without >- * modification, are permitted provided that the following conditions >- * are met: >- * 1. Redistributions of source code must retain the above copyright >- * notice, this list of conditions and the following disclaimer. >- * 2. Redistributions in binary form must reproduce the above copyright >- * notice, this list of conditions and the following disclaimer in the >- * documentation and/or other materials provided with the distribution. >- * >- * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >- * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >- * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >- * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >- * THE POSSIBILITY OF SUCH DAMAGE. >- */ >- >-#pragma once >- >-#include <wtf/text/WTFString.h> >- >-namespace WebKit { >- >-class ResourceLoadStatisticsMemoryStore; >- >-namespace WebResourceLoadStatisticsTelemetry { >- >-void calculateAndSubmit(const ResourceLoadStatisticsMemoryStore&); >-void setNotifyPagesWhenTelemetryWasCaptured(bool); >- >-} >-} >diff --git a/Source/WebKit/UIProcess/mac/DisplayLink.cpp b/Source/WebKit/UIProcess/mac/DisplayLink.cpp >index f313a48fd7d78cace83301a4bc4fc5a07dd98be6..bfb032426d98cff388b5104934843811b110ef89 100644 >--- a/Source/WebKit/UIProcess/mac/DisplayLink.cpp >+++ b/Source/WebKit/UIProcess/mac/DisplayLink.cpp >@@ -29,6 +29,7 @@ > #if ENABLE(WEBPROCESS_WINDOWSERVER_BLOCKING) > > #include "DrawingAreaMessages.h" >+#include "WebProcessMessages.h" > #include "WebProcessProxy.h" > #include <wtf/ProcessPrivilege.h> > >diff --git a/Source/WebKit/UnifiedSources-input.xcfilelist b/Source/WebKit/UnifiedSources-input.xcfilelist >index dba6b37f42fe664cbeb55cd2a4d9ca6ac34c17ca..e6c7a0c7a2660e18a82e3c1525c892d706886f57 100644 >--- a/Source/WebKit/UnifiedSources-input.xcfilelist >+++ b/Source/WebKit/UnifiedSources-input.xcfilelist >@@ -1,4 +1,8 @@ > $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/EditableImageControllerMessageReceiver.cpp >+$(SRCROOT)/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp >+$(SRCROOT)/NetworkProcess/Classifier/ResourceLoadStatisticsPersistentStorage.cpp >+$(SRCROOT)/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp >+$(SRCROOT)/NetworkProcess/Classifier/WebResourceLoadStatisticsTelemetry.cpp > $(SRCROOT)/NetworkProcess/Cookies/WebCookieManager.cpp > $(SRCROOT)/NetworkProcess/Cookies/mac/WebCookieManagerMac.mm > $(SRCROOT)/NetworkProcess/CustomProtocols/Cocoa/LegacyCustomProtocolManagerCocoa.mm >@@ -550,8 +554,6 @@ $(SRCROOT)/UIProcess/RemoteLayerTree/ios/RemoteScrollingCoordinatorProxyIOS.mm > $(SRCROOT)/UIProcess/RemoteLayerTree/ios/ScrollingTreeOverflowScrollingNodeIOS.mm > $(SRCROOT)/UIProcess/RemoteLayerTree/ios/ScrollingTreeScrollingNodeDelegateIOS.mm > $(SRCROOT)/UIProcess/RemoteWebInspectorProxy.cpp >-$(SRCROOT)/UIProcess/ResourceLoadStatisticsMemoryStore.cpp >-$(SRCROOT)/UIProcess/ResourceLoadStatisticsPersistentStorage.cpp > $(SRCROOT)/UIProcess/ResponsivenessTimer.cpp > $(SRCROOT)/UIProcess/ServiceWorkerProcessProxy.cpp > $(SRCROOT)/UIProcess/StatisticsRequest.cpp >@@ -620,8 +622,6 @@ $(SRCROOT)/UIProcess/WebProcessLifetimeObserver.cpp > $(SRCROOT)/UIProcess/WebProcessLifetimeTracker.cpp > $(SRCROOT)/UIProcess/WebProcessPool.cpp > $(SRCROOT)/UIProcess/WebProcessProxy.cpp >-$(SRCROOT)/UIProcess/WebResourceLoadStatisticsStore.cpp >-$(SRCROOT)/UIProcess/WebResourceLoadStatisticsTelemetry.cpp > $(SRCROOT)/UIProcess/WebStorage/LocalStorageDatabase.cpp > $(SRCROOT)/UIProcess/WebStorage/LocalStorageDatabaseTracker.cpp > $(SRCROOT)/UIProcess/WebStorage/StorageManager.cpp >diff --git a/Source/WebKit/WebKit.xcodeproj/project.pbxproj b/Source/WebKit/WebKit.xcodeproj/project.pbxproj >index 00a1c5f6499a9bda2609a200bb30147f7cd6fb34..e7e4772a036fe9e2ca1fa36f710e224734a7c6ae 100644 >--- a/Source/WebKit/WebKit.xcodeproj/project.pbxproj >+++ b/Source/WebKit/WebKit.xcodeproj/project.pbxproj >@@ -75,7 +75,6 @@ > 005D158F18E4C4EB00734619 /* _WKFindDelegate.h in Headers */ = {isa = PBXBuildFile; fileRef = 005D158E18E4C4EB00734619 /* _WKFindDelegate.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 00B9661618E24CBA00CE1F88 /* APIFindClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 00B9661518E24CBA00CE1F88 /* APIFindClient.h */; }; > 00B9661A18E25AE100CE1F88 /* FindClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 00B9661818E25AE100CE1F88 /* FindClient.h */; }; >- 07297F9F1C17AA1A015F0735 /* PersistencyUtils.h in Headers */ = {isa = PBXBuildFile; fileRef = 07321F9D1C17BBEA223F0735 /* PersistencyUtils.h */; }; > 07297F9F1C17BBEA003F0735 /* UserMediaPermissionCheckProxy.h in Headers */ = {isa = PBXBuildFile; fileRef = 07297F9D1C17BBEA003F0735 /* UserMediaPermissionCheckProxy.h */; }; > 07297F9F1C17BBEA015F0735 /* DeviceIdHashSaltStorage.h in Headers */ = {isa = PBXBuildFile; fileRef = 07297F9D1C17BBEA223F0735 /* DeviceIdHashSaltStorage.h */; }; > 07297FA31C186ADB003F0735 /* WKUserMediaPermissionCheck.h in Headers */ = {isa = PBXBuildFile; fileRef = 07297FA11C186ADB003F0735 /* WKUserMediaPermissionCheck.h */; settings = {ATTRIBUTES = (Private, ); }; }; >@@ -987,7 +986,6 @@ > 51DD9F2816367DA2001578E9 /* NetworkConnectionToWebProcessMessageReceiver.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 51DD9F2616367DA2001578E9 /* NetworkConnectionToWebProcessMessageReceiver.cpp */; }; > 51DD9F2916367DA2001578E9 /* NetworkConnectionToWebProcessMessages.h in Headers */ = {isa = PBXBuildFile; fileRef = 51DD9F2716367DA2001578E9 /* NetworkConnectionToWebProcessMessages.h */; }; > 51E351CB180F2CCC00E53BE9 /* IDBUtilities.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E351C9180F2CCC00E53BE9 /* IDBUtilities.h */; }; >- 51E6C1641F2935DD00FD3437 /* ResourceLoadStatisticsPersistentStorage.h in Headers */ = {isa = PBXBuildFile; fileRef = 51E6C1611F2935CD00FD3437 /* ResourceLoadStatisticsPersistentStorage.h */; }; > 51EFC1CF1524E62500C9A938 /* WKBundleDOMWindowExtension.h in Headers */ = {isa = PBXBuildFile; fileRef = 51FA2D541521118600C1BA0B /* WKBundleDOMWindowExtension.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 51F060E01654317F00F3281B /* WebResourceLoaderMessages.h in Headers */ = {isa = PBXBuildFile; fileRef = 51F060DE1654317500F3281B /* WebResourceLoaderMessages.h */; }; > 51F060E11654318500F3281B /* WebResourceLoaderMessageReceiver.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 51F060DD1654317500F3281B /* WebResourceLoaderMessageReceiver.cpp */; }; >@@ -1086,7 +1084,6 @@ > 63C32C281E98119000699BD0 /* _WKGeolocationPositionInternal.h in Headers */ = {isa = PBXBuildFile; fileRef = 63C32C271E98119000699BD0 /* _WKGeolocationPositionInternal.h */; }; > 65B86F1E12F11DE300B7DD8A /* WKBundleInspector.h in Headers */ = {isa = PBXBuildFile; fileRef = 65B86F1812F11D7B00B7DD8A /* WKBundleInspector.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 6A5080BF1F0EDAAA00E617C5 /* WKWindowFeaturesPrivate.h in Headers */ = {isa = PBXBuildFile; fileRef = 6A5080BE1F0EDAAA00E617C5 /* WKWindowFeaturesPrivate.h */; settings = {ATTRIBUTES = (Private, ); }; }; >- 6B821DDC1EEF05DD00D7AF4A /* WebResourceLoadStatisticsTelemetry.h in Headers */ = {isa = PBXBuildFile; fileRef = 6B821DDA1EEF05DD00D7AF4A /* WebResourceLoadStatisticsTelemetry.h */; }; > 6BE969C11E54D452008B7483 /* corePrediction_model in Resources */ = {isa = PBXBuildFile; fileRef = 6BE969C01E54D452008B7483 /* corePrediction_model */; }; > 6BE969CB1E54D4CF008B7483 /* ResourceLoadStatisticsClassifierCocoa.h in Headers */ = {isa = PBXBuildFile; fileRef = 6BE969C91E54D4CF008B7483 /* ResourceLoadStatisticsClassifierCocoa.h */; }; > 6BE969CD1E54E054008B7483 /* ResourceLoadStatisticsClassifier.h in Headers */ = {isa = PBXBuildFile; fileRef = 6BE969CC1E54E054008B7483 /* ResourceLoadStatisticsClassifier.h */; }; >@@ -1109,6 +1106,10 @@ > 7AB6EA451EEAAE3800037B2B /* APIIconDatabaseClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 7AB6EA441EEAAE2300037B2B /* APIIconDatabaseClient.h */; }; > 7AB6EA471EEAB6B800037B2B /* APIGeolocationProvider.h in Headers */ = {isa = PBXBuildFile; fileRef = 7AB6EA461EEAB6B000037B2B /* APIGeolocationProvider.h */; }; > 7AF236211E79A40800438A05 /* WebErrors.h in Headers */ = {isa = PBXBuildFile; fileRef = 7AF2361F1E79A3D800438A05 /* WebErrors.h */; }; >+ 7AFBD36321E50F39005DBACB /* WebResourceLoadStatisticsStore.h in Headers */ = {isa = PBXBuildFile; fileRef = 7AFBD36221E50F39005DBACB /* WebResourceLoadStatisticsStore.h */; }; >+ 7AFBD36721E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.h in Headers */ = {isa = PBXBuildFile; fileRef = 7AFBD36521E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.h */; }; >+ 7AFBD36A21E542F1005DBACB /* ResourceLoadStatisticsPersistentStorage.h in Headers */ = {isa = PBXBuildFile; fileRef = 7AFBD36821E542E8005DBACB /* ResourceLoadStatisticsPersistentStorage.h */; }; >+ 7AFBD36F21E546F8005DBACB /* PersistencyUtils.h in Headers */ = {isa = PBXBuildFile; fileRef = 7AFBD36D21E546E3005DBACB /* PersistencyUtils.h */; }; > 7C065F2C1C8CD95F00C2D950 /* WebUserContentControllerDataTypes.h in Headers */ = {isa = PBXBuildFile; fileRef = 7C065F2A1C8CD95F00C2D950 /* WebUserContentControllerDataTypes.h */; }; > 7C135AA9173B0BCA00586AE2 /* WKPluginInformation.h in Headers */ = {isa = PBXBuildFile; fileRef = 7C135AA7173B0BCA00586AE2 /* WKPluginInformation.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 7C1BA33E1A4A0E600043E249 /* APIDictionary.h in Headers */ = {isa = PBXBuildFile; fileRef = 7C1BA33C1A4A0E600043E249 /* APIDictionary.h */; }; >@@ -1194,7 +1195,6 @@ > 839902031BE9A02B000F3653 /* NetworkLoad.h in Headers */ = {isa = PBXBuildFile; fileRef = 839901FE1BE9A01B000F3653 /* NetworkLoad.h */; }; > 839A2F321E2067450039057E /* HighPerformanceGraphicsUsageSampler.h in Headers */ = {isa = PBXBuildFile; fileRef = 839A2F301E2067390039057E /* HighPerformanceGraphicsUsageSampler.h */; }; > 83A0ED351F747CCF003299EB /* PreconnectTask.h in Headers */ = {isa = PBXBuildFile; fileRef = 83A0ED321F747CC6003299EB /* PreconnectTask.h */; }; >- 83B3CB1020E2AB0100441E9B /* ResourceLoadStatisticsMemoryStore.h in Headers */ = {isa = PBXBuildFile; fileRef = 83681DCA20E2904A00C249CD /* ResourceLoadStatisticsMemoryStore.h */; }; > 83D454D71BE9D3C4006C93BD /* NetworkLoadClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 83D454D61BE9D3C4006C93BD /* NetworkLoadClient.h */; }; > 83EE575C1DB7D61100C74C50 /* WebValidationMessageClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 83EE575A1DB7D60600C74C50 /* WebValidationMessageClient.h */; }; > 83F1A07A1F96E7790045B94E /* WebSWOriginTable.h in Headers */ = {isa = PBXBuildFile; fileRef = 83F1A0781F96E7710045B94E /* WebSWOriginTable.h */; }; >@@ -1854,14 +1854,12 @@ > 00B9661518E24CBA00CE1F88 /* APIFindClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIFindClient.h; sourceTree = "<group>"; }; > 00B9661718E25AE100CE1F88 /* FindClient.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = FindClient.mm; sourceTree = "<group>"; }; > 00B9661818E25AE100CE1F88 /* FindClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = FindClient.h; sourceTree = "<group>"; }; >- 0729455C1C1711EA003F0735 /* PersistencyUtils.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PersistencyUtils.cpp; sourceTree = "<group>"; }; > 07297F9C1C1711EA003F0735 /* DeviceIdHashSaltStorage.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = DeviceIdHashSaltStorage.cpp; sourceTree = "<group>"; }; > 07297F9C1C17BBEA003F0735 /* UserMediaPermissionCheckProxy.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = UserMediaPermissionCheckProxy.cpp; sourceTree = "<group>"; }; > 07297F9D1C17BBEA003F0735 /* UserMediaPermissionCheckProxy.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = UserMediaPermissionCheckProxy.h; sourceTree = "<group>"; }; > 07297F9D1C17BBEA223F0735 /* DeviceIdHashSaltStorage.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DeviceIdHashSaltStorage.h; sourceTree = "<group>"; }; > 07297FA01C186ADB003F0735 /* WKUserMediaPermissionCheck.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WKUserMediaPermissionCheck.cpp; sourceTree = "<group>"; }; > 07297FA11C186ADB003F0735 /* WKUserMediaPermissionCheck.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKUserMediaPermissionCheck.h; sourceTree = "<group>"; }; >- 07321F9D1C17BBEA223F0735 /* PersistencyUtils.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = PersistencyUtils.h; sourceTree = "<group>"; }; > 074E75FB1DF1FD1300D318EC /* UserMediaProcessManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = UserMediaProcessManager.h; sourceTree = "<group>"; }; > 074E75FC1DF2002400D318EC /* UserMediaProcessManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = UserMediaProcessManager.cpp; sourceTree = "<group>"; }; > 074E76001DF7075D00D318EC /* MediaDeviceSandboxExtensions.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MediaDeviceSandboxExtensions.cpp; sourceTree = "<group>"; }; >@@ -3312,8 +3310,6 @@ > 51E351C9180F2CCC00E53BE9 /* IDBUtilities.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = IDBUtilities.h; sourceTree = "<group>"; }; > 51E399051D6F54C5009C8831 /* UIGamepadProviderCocoa.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = UIGamepadProviderCocoa.mm; sourceTree = "<group>"; }; > 51E6C15F1F29355100FD3437 /* ResourceLoadStatisticsPersistentStorageIOS.mm */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.objcpp; name = ResourceLoadStatisticsPersistentStorageIOS.mm; path = ios/ResourceLoadStatisticsPersistentStorageIOS.mm; sourceTree = "<group>"; }; >- 51E6C1611F2935CD00FD3437 /* ResourceLoadStatisticsPersistentStorage.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = ResourceLoadStatisticsPersistentStorage.h; sourceTree = "<group>"; }; >- 51E6C1621F2935CD00FD3437 /* ResourceLoadStatisticsPersistentStorage.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = ResourceLoadStatisticsPersistentStorage.cpp; sourceTree = "<group>"; }; > 51E8B68D1E712873001B7132 /* WebURLSchemeTask.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebURLSchemeTask.cpp; sourceTree = "<group>"; }; > 51E949961D761CC700EC9EB9 /* UIGamepadProviderIOS.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = UIGamepadProviderIOS.mm; sourceTree = "<group>"; }; > 51F060DD1654317500F3281B /* WebResourceLoaderMessageReceiver.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = WebResourceLoaderMessageReceiver.cpp; path = DerivedSources/WebKit2/WebResourceLoaderMessageReceiver.cpp; sourceTree = BUILT_PRODUCTS_DIR; }; >@@ -3508,8 +3504,6 @@ > 65B86F1712F11D7B00B7DD8A /* WKBundleInspector.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WKBundleInspector.cpp; sourceTree = "<group>"; }; > 65B86F1812F11D7B00B7DD8A /* WKBundleInspector.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKBundleInspector.h; sourceTree = "<group>"; }; > 6A5080BE1F0EDAAA00E617C5 /* WKWindowFeaturesPrivate.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WKWindowFeaturesPrivate.h; sourceTree = "<group>"; }; >- 6B821DDA1EEF05DD00D7AF4A /* WebResourceLoadStatisticsTelemetry.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WebResourceLoadStatisticsTelemetry.h; sourceTree = "<group>"; }; >- 6B821DDB1EEF05DD00D7AF4A /* WebResourceLoadStatisticsTelemetry.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = WebResourceLoadStatisticsTelemetry.cpp; sourceTree = "<group>"; }; > 6BE969C01E54D452008B7483 /* corePrediction_model */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = corePrediction_model; sourceTree = "<group>"; }; > 6BE969C61E54D4B6008B7483 /* ResourceLoadStatisticsClassifier.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ResourceLoadStatisticsClassifier.cpp; sourceTree = "<group>"; }; > 6BE969C81E54D4CF008B7483 /* ResourceLoadStatisticsClassifierCocoa.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ResourceLoadStatisticsClassifierCocoa.cpp; sourceTree = "<group>"; }; >@@ -3535,11 +3529,10 @@ > 7A821F4B1E2F664800604577 /* LegacyCustomProtocolManagerClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = LegacyCustomProtocolManagerClient.h; sourceTree = "<group>"; }; > 7A821F4D1E2F679E00604577 /* LegacyCustomProtocolManagerClient.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = LegacyCustomProtocolManagerClient.mm; sourceTree = "<group>"; }; > 7A821F4F1E2F7A5C00604577 /* APICustomProtocolManagerClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APICustomProtocolManagerClient.h; sourceTree = "<group>"; }; >+ 7A843A1A21E41FB200DEF663 /* WebResourceLoadStatisticsStore.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = WebResourceLoadStatisticsStore.cpp; path = Classifier/WebResourceLoadStatisticsStore.cpp; sourceTree = "<group>"; }; > 7A8A9D571EF119AA009801AE /* APIInjectedBundleClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIInjectedBundleClient.h; sourceTree = "<group>"; }; > 7A8A9D591EF13020009801AE /* APIInjectedBundleBundleClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIInjectedBundleBundleClient.h; sourceTree = "<group>"; }; > 7A8A9D5B1EF1458E009801AE /* APIInjectedBundlePageResourceLoadClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIInjectedBundlePageResourceLoadClient.h; sourceTree = "<group>"; }; >- 7A9CD8C01C77984900D9F6C7 /* WebResourceLoadStatisticsStore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebResourceLoadStatisticsStore.cpp; sourceTree = "<group>"; }; >- 7A9CD8C11C77984900D9F6C7 /* WebResourceLoadStatisticsStore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebResourceLoadStatisticsStore.h; sourceTree = "<group>"; }; > 7A9CD8C21C779AD600D9F6C7 /* WebResourceLoadStatisticsStore.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = WebResourceLoadStatisticsStore.messages.in; sourceTree = "<group>"; }; > 7AB6EA441EEAAE2300037B2B /* APIIconDatabaseClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIIconDatabaseClient.h; sourceTree = "<group>"; }; > 7AB6EA461EEAB6B000037B2B /* APIGeolocationProvider.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIGeolocationProvider.h; sourceTree = "<group>"; }; >@@ -3548,6 +3541,15 @@ > 7AF2361F1E79A3D800438A05 /* WebErrors.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebErrors.h; sourceTree = "<group>"; }; > 7AF236221E79A43100438A05 /* WebErrorsCocoa.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = WebErrorsCocoa.mm; sourceTree = "<group>"; }; > 7AF66E1120C07CB6007828EA /* WebContent-OSX.entitlements */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.plist.entitlements; path = "WebContent-OSX.entitlements"; sourceTree = "<group>"; }; >+ 7AFBD36221E50F39005DBACB /* WebResourceLoadStatisticsStore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = WebResourceLoadStatisticsStore.h; path = Classifier/WebResourceLoadStatisticsStore.h; sourceTree = "<group>"; }; >+ 7AFBD36421E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ResourceLoadStatisticsMemoryStore.cpp; path = Classifier/ResourceLoadStatisticsMemoryStore.cpp; sourceTree = "<group>"; }; >+ 7AFBD36521E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = ResourceLoadStatisticsMemoryStore.h; path = Classifier/ResourceLoadStatisticsMemoryStore.h; sourceTree = "<group>"; }; >+ 7AFBD36821E542E8005DBACB /* ResourceLoadStatisticsPersistentStorage.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = ResourceLoadStatisticsPersistentStorage.h; path = Classifier/ResourceLoadStatisticsPersistentStorage.h; sourceTree = "<group>"; }; >+ 7AFBD36921E542E8005DBACB /* ResourceLoadStatisticsPersistentStorage.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = ResourceLoadStatisticsPersistentStorage.cpp; path = Classifier/ResourceLoadStatisticsPersistentStorage.cpp; sourceTree = "<group>"; }; >+ 7AFBD36B21E54544005DBACB /* WebResourceLoadStatisticsTelemetry.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = WebResourceLoadStatisticsTelemetry.h; path = Classifier/WebResourceLoadStatisticsTelemetry.h; sourceTree = "<group>"; }; >+ 7AFBD36C21E54544005DBACB /* WebResourceLoadStatisticsTelemetry.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = WebResourceLoadStatisticsTelemetry.cpp; path = Classifier/WebResourceLoadStatisticsTelemetry.cpp; sourceTree = "<group>"; }; >+ 7AFBD36D21E546E3005DBACB /* PersistencyUtils.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = PersistencyUtils.h; sourceTree = "<group>"; }; >+ 7AFBD36E21E546E3005DBACB /* PersistencyUtils.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = PersistencyUtils.cpp; sourceTree = "<group>"; }; > 7C065F291C8CD95F00C2D950 /* WebUserContentControllerDataTypes.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebUserContentControllerDataTypes.cpp; sourceTree = "<group>"; }; > 7C065F2A1C8CD95F00C2D950 /* WebUserContentControllerDataTypes.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebUserContentControllerDataTypes.h; sourceTree = "<group>"; }; > 7C0BB9A818DCDE890006C086 /* WebContent-iOS.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.xml; path = "WebContent-iOS.entitlements"; sourceTree = "<group>"; }; >@@ -3680,8 +3682,6 @@ > 834B25101A842C8700CFB150 /* NetworkCacheStatistics.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkCacheStatistics.h; sourceTree = "<group>"; }; > 8360349D1ACB34D600626549 /* WebSQLiteDatabaseTracker.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebSQLiteDatabaseTracker.cpp; sourceTree = "<group>"; }; > 8360349E1ACB34D600626549 /* WebSQLiteDatabaseTracker.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebSQLiteDatabaseTracker.h; sourceTree = "<group>"; }; >- 83681DCA20E2904A00C249CD /* ResourceLoadStatisticsMemoryStore.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ResourceLoadStatisticsMemoryStore.h; sourceTree = "<group>"; }; >- 83681DCB20E2904B00C249CD /* ResourceLoadStatisticsMemoryStore.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ResourceLoadStatisticsMemoryStore.cpp; sourceTree = "<group>"; }; > 8372DB241A674C8F00C697C5 /* WKPageDiagnosticLoggingClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WKPageDiagnosticLoggingClient.h; sourceTree = "<group>"; }; > 8372DB261A67562800C697C5 /* WebPageDiagnosticLoggingClient.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = WebPageDiagnosticLoggingClient.cpp; sourceTree = "<group>"; }; > 8372DB271A67562800C697C5 /* WebPageDiagnosticLoggingClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebPageDiagnosticLoggingClient.h; sourceTree = "<group>"; }; >@@ -5051,6 +5051,8 @@ > 1ADCB868189831B30022EE5A /* NavigationActionData.cpp */, > 1ADCB869189831B30022EE5A /* NavigationActionData.h */, > 9BC59D6D1EFCDC6D001E8D09 /* OptionalCallbackID.h */, >+ 7AFBD36E21E546E3005DBACB /* PersistencyUtils.cpp */, >+ 7AFBD36D21E546E3005DBACB /* PersistencyUtils.h */, > BCC43AB8127B95DC00317F16 /* PlatformPopupMenuData.cpp */, > BCC43AB9127B95DC00317F16 /* PlatformPopupMenuData.h */, > E18C92F312DB9E7100CF2AEB /* PrintInfo.cpp */, >@@ -5515,8 +5517,8 @@ > 2D1551A91F5A9B420006E3FE /* RemoteLayerTree */ = { > isa = PBXGroup; > children = ( >- E404906F21DE65D70037F0DB /* mac */, > 2D1551AA1F5A9BA70006E3FE /* ios */, >+ E404906F21DE65D70037F0DB /* mac */, > 1AB16AE01648656D00290D62 /* RemoteLayerTreeDrawingAreaProxy.h */, > 0FF24A2F1879E4FE003ABF0C /* RemoteLayerTreeDrawingAreaProxy.messages.in */, > 1AB16ADF1648656D00290D62 /* RemoteLayerTreeDrawingAreaProxy.mm */, >@@ -6412,6 +6414,7 @@ > isa = PBXGroup; > children = ( > E489D2821A0A2BE80078C06A /* cache */, >+ 7A843A1D21E41FD900DEF663 /* Classifier */, > 7EC4F0F818E4A922008056AF /* cocoa */, > 3309344B1315B93A0097A7BC /* Cookies */, > 5C1426F21C23F82D00D41183 /* CustomProtocols */, >@@ -6893,6 +6896,21 @@ > name = mac; > sourceTree = "<group>"; > }; >+ 7A843A1D21E41FD900DEF663 /* Classifier */ = { >+ isa = PBXGroup; >+ children = ( >+ 7AFBD36421E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.cpp */, >+ 7AFBD36521E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.h */, >+ 7AFBD36921E542E8005DBACB /* ResourceLoadStatisticsPersistentStorage.cpp */, >+ 7AFBD36821E542E8005DBACB /* ResourceLoadStatisticsPersistentStorage.h */, >+ 7A843A1A21E41FB200DEF663 /* WebResourceLoadStatisticsStore.cpp */, >+ 7AFBD36221E50F39005DBACB /* WebResourceLoadStatisticsStore.h */, >+ 7AFBD36C21E54544005DBACB /* WebResourceLoadStatisticsTelemetry.cpp */, >+ 7AFBD36B21E54544005DBACB /* WebResourceLoadStatisticsTelemetry.h */, >+ ); >+ name = Classifier; >+ sourceTree = "<group>"; >+ }; > 7C6E70F818B2D47E00F24E2E /* cocoa */ = { > isa = PBXGroup; > children = ( >@@ -7327,8 +7345,6 @@ > 1AC7537A183A9FDB0072CB15 /* PageLoadState.h */, > 832ED1891E2FE13B006BA64A /* PerActivityStateCPUUsageSampler.cpp */, > 832ED18A1E2FE13B006BA64A /* PerActivityStateCPUUsageSampler.h */, >- 0729455C1C1711EA003F0735 /* PersistencyUtils.cpp */, >- 07321F9D1C17BBEA223F0735 /* PersistencyUtils.h */, > 37716A59195B910500EE8B1B /* ProcessAssertion.cpp */, > 86F9536018FF4FD4001DB2EF /* ProcessAssertion.h */, > 86E67A22190F411800004AB7 /* ProcessThrottler.cpp */, >@@ -7337,10 +7353,6 @@ > A55BA81B1BA25B1E007CD33D /* RemoteWebInspectorProxy.cpp */, > A55BA8191BA25B1E007CD33D /* RemoteWebInspectorProxy.h */, > A55BA81A1BA25B1E007CD33D /* RemoteWebInspectorProxy.messages.in */, >- 83681DCB20E2904B00C249CD /* ResourceLoadStatisticsMemoryStore.cpp */, >- 83681DCA20E2904A00C249CD /* ResourceLoadStatisticsMemoryStore.h */, >- 51E6C1621F2935CD00FD3437 /* ResourceLoadStatisticsPersistentStorage.cpp */, >- 51E6C1611F2935CD00FD3437 /* ResourceLoadStatisticsPersistentStorage.h */, > BC111B08112F5E3C00337BAB /* ResponsivenessTimer.cpp */, > 1A30066C1110F4F70031937C /* ResponsivenessTimer.h */, > 5CA98549210BEB5A0057EB6B /* SafeBrowsingWarning.h */, >@@ -7456,11 +7468,7 @@ > BC111B0D112F5E4F00337BAB /* WebProcessProxy.cpp */, > BC032DCF10F4389F0058C15A /* WebProcessProxy.h */, > BCEE7AB312817095009827DA /* WebProcessProxy.messages.in */, >- 7A9CD8C01C77984900D9F6C7 /* WebResourceLoadStatisticsStore.cpp */, >- 7A9CD8C11C77984900D9F6C7 /* WebResourceLoadStatisticsStore.h */, > 7A9CD8C21C779AD600D9F6C7 /* WebResourceLoadStatisticsStore.messages.in */, >- 6B821DDB1EEF05DD00D7AF4A /* WebResourceLoadStatisticsTelemetry.cpp */, >- 6B821DDA1EEF05DD00D7AF4A /* WebResourceLoadStatisticsTelemetry.h */, > 51D124241E6D3CC3002B2820 /* WebURLSchemeHandler.cpp */, > 51D124251E6D3CC3002B2820 /* WebURLSchemeHandler.h */, > 51E8B68D1E712873001B7132 /* WebURLSchemeTask.cpp */, >@@ -9190,7 +9198,7 @@ > 2DC18FB0218912640025A88D /* PencilKitSPI.h in Headers */, > 5C298DA01C3DF02100470AFE /* PendingDownload.h in Headers */, > 832ED18C1E2FE157006BA64A /* PerActivityStateCPUUsageSampler.h in Headers */, >- 07297F9F1C17AA1A015F0735 /* PersistencyUtils.h in Headers */, >+ 7AFBD36F21E546F8005DBACB /* PersistencyUtils.h in Headers */, > 5CE85B201C88E64B0070BFCE /* PingLoad.h in Headers */, > 0F5E200418E77051003EC3E5 /* PlatformCAAnimationRemote.h in Headers */, > 2DA049B4180CCCD300AAFA9E /* PlatformCALayerRemote.h in Headers */, >@@ -9263,8 +9271,8 @@ > A55BA8171BA23E12007CD33D /* RemoteWebInspectorUI.h in Headers */, > 6BE969CD1E54E054008B7483 /* ResourceLoadStatisticsClassifier.h in Headers */, > 6BE969CB1E54D4CF008B7483 /* ResourceLoadStatisticsClassifierCocoa.h in Headers */, >- 83B3CB1020E2AB0100441E9B /* ResourceLoadStatisticsMemoryStore.h in Headers */, >- 51E6C1641F2935DD00FD3437 /* ResourceLoadStatisticsPersistentStorage.h in Headers */, >+ 7AFBD36721E51BAB005DBACB /* ResourceLoadStatisticsMemoryStore.h in Headers */, >+ 7AFBD36A21E542F1005DBACB /* ResourceLoadStatisticsPersistentStorage.h in Headers */, > 1A30066E1110F4F70031937C /* ResponsivenessTimer.h in Headers */, > 410482CE1DDD324F00F006D0 /* RTCNetwork.h in Headers */, > 0E97D74D200E900400BF6643 /* SafeBrowsingSPI.h in Headers */, >@@ -9518,8 +9526,8 @@ > 3760881F150413E900FC82C7 /* WebRenderObject.h in Headers */, > 510AFFBA16542048001BA05E /* WebResourceLoader.h in Headers */, > 51F060E01654317F00F3281B /* WebResourceLoaderMessages.h in Headers */, >+ 7AFBD36321E50F39005DBACB /* WebResourceLoadStatisticsStore.h in Headers */, > 7A791EFB1C7CFD0100C4C52B /* WebResourceLoadStatisticsStoreMessages.h in Headers */, >- 6B821DDC1EEF05DD00D7AF4A /* WebResourceLoadStatisticsTelemetry.h in Headers */, > 413075B01DE85F580039EC69 /* WebRTCMonitor.h in Headers */, > 41FAF5F51E3C0649001AE678 /* WebRTCResolver.h in Headers */, > 41DC459F1E3DBDA500B11F51 /* WebRTCSocket.h in Headers */,
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 193252
:
358627
|
358629
|
358632