WebKit Bugzilla
Attachment 370646 Details for
Bug 198243
: [iOS] Dropped text, attachments, and images should animate into place
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Patch
bug-198243-20190526011625.patch (text/plain), 58.51 KB, created by
Wenson Hsieh
on 2019-05-26 01:16:25 PDT
(
hide
)
Description:
Patch
Filename:
MIME Type:
Creator:
Wenson Hsieh
Created:
2019-05-26 01:16:25 PDT
Size:
58.51 KB
patch
obsolete
>Subversion Revision: 245777 >diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog >index 2e4050f12c74535c53373175ecfb84a01845ece8..9bbda225fb017ce3f13e96760bf7eb1ffeb4e84e 100644 >--- a/Source/WebCore/ChangeLog >+++ b/Source/WebCore/ChangeLog >@@ -1,3 +1,24 @@ >+2019-05-25 Wenson Hsieh <wenson_hsieh@apple.com> >+ >+ [iOS] Dropped text, attachments, and images should animate into place >+ https://bugs.webkit.org/show_bug.cgi?id=198243 >+ <rdar://problem/35205373> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Add some hooks to notify the chrome client when an HTMLImageElement's image is finished loading. See WebKit >+ changelog for more detail. >+ >+ Test: DragAndDropTests.DropPreviewForImageInEditableArea >+ >+ * loader/EmptyClients.h: >+ * page/ChromeClient.h: >+ * page/Page.cpp: >+ (WebCore::Page::didFinishLoadingImageForElement): >+ * page/Page.h: >+ * rendering/RenderImage.cpp: >+ (WebCore::RenderImage::notifyFinished): >+ > 2019-05-25 Zalan Bujtas <zalan@apple.com> > > [LFC][IFC] Introduce DisplayRun to display tree >diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog >index 800bddcef13920e4ba73fff38bd7e6477c3ec15d..30ea6407425164936decdabdbfc43ee1d42bebf9 100644 >--- a/Source/WebKit/ChangeLog >+++ b/Source/WebKit/ChangeLog >@@ -1,3 +1,144 @@ >+2019-05-25 Wenson Hsieh <wenson_hsieh@apple.com> >+ >+ [iOS] Dropped text, attachments, and images should animate into place >+ https://bugs.webkit.org/show_bug.cgi?id=198243 >+ <rdar://problem/35205373> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Adds support for targeted drop animations on iOS in modern WebKit by adopting UIKit SPI introduced in >+ <rdar://problem/31075005> to allow updating the drop preview mid-flight. To get the animation right, we refactor >+ and augment existing logic for taking snapshots after performing a drop in an editable content. >+ >+ Currently, upon dropping in editable content, we first snapshot the web view and temporarily cover the real web >+ view with this snapshot. When the TextIndicator data arrives that contains (1) a snapshot of the visible part of >+ the web view ignoring the selection, and (2) a snapshot of just the selected contents after drop, we crossfade >+ from the web view snapshot to the snapshot in (1) using a hard-coded time delay (~500ms), and target the drop >+ preview to the drag caret rect. During this process, snapshot (2) is completely ignored. >+ >+ This was effectively a halfway implemention of the desired effect of animating the dropped content into place >+ and crossfading to the final content; before UIKit implemented updateable drag previews, the full implementation >+ was not possible in modern WebKit (without using synchronous IPC). >+ >+ Now that we're able to update the drag preview in the middle of the drop animation, we can now utilize snapshot >+ (2) above and clean up some parts of the drop animation in editable content. See below for more details. >+ >+ * UIProcess/API/Cocoa/WKWebView.mm: >+ (-[WKWebView _doAfterReceivingEditDragSnapshotForTesting:]): >+ >+ Add a testing hook to perform the given block after any pending edit drag snapshot has been received. See >+ TestWebKitAPI changes for more detail. >+ >+ * UIProcess/API/Cocoa/WKWebViewPrivate.h: >+ * UIProcess/PageClient.h: >+ * UIProcess/WebPageProxy.h: >+ * UIProcess/WebPageProxy.messages.in: >+ >+ Split up the existing DidConcludeEditDrag IPC message into two messages: WillReceiveEditDragSnapshot and >+ DidReceiveEditDragSnapshot. This allows us to defer cleaning up the drag session state during an edit drop, >+ until after the final edit drag snapshot has been received. >+ >+ * UIProcess/ios/DragDropInteractionState.h: >+ >+ Add some new methods to help manage the lifecycle of drop preview provider blocks. >+ >+ * UIProcess/ios/DragDropInteractionState.mm: >+ (WebKit::DragDropInteractionState::prepareForDelayedDropPreview): >+ >+ Stores a given drop preview provider, given to us by UIKit. >+ >+ (WebKit::DragDropInteractionState::deliverDelayedDropPreview): >+ >+ Invokes the stored drop preview providers with given text indicator data. This is invoked after snapshots are >+ taken following an edit drag (this is additionally after all images in the inserted fragment have finished >+ loading). >+ >+ (WebKit::DragDropInteractionState::clearAllDelayedItemPreviewProviders): >+ >+ Invokes all stored drop preview providers with a nil preview. This is invoked in any case where drag session >+ cleanup occurs earlier than normal (e.g., if the web process crashes during drop), and ensures that the handlers >+ are always invoked when cleaning up the drag session. >+ >+ (WebKit::DragDropInteractionState::dragAndDropSessionsDidEnd): >+ >+ Call clearAllDelayedItemPreviewProviders. >+ >+ * UIProcess/ios/PageClientImplIOS.h: >+ * UIProcess/ios/PageClientImplIOS.mm: >+ (WebKit::PageClientImpl::willReceiveEditDragSnapshot): >+ (WebKit::PageClientImpl::didReceiveEditDragSnapshot): >+ (WebKit::PageClientImpl::didConcludeEditDrag): Deleted. >+ >+ More plumbing (see changes to DidConcludeEditDrag above). >+ >+ * UIProcess/ios/WKContentViewInteraction.h: >+ * UIProcess/ios/WKContentViewInteraction.mm: >+ (-[WKContentView cleanupInteraction]): >+ (-[WKContentView cleanUpDragSourceSessionState]): >+ (-[WKContentView _willReceiveEditDragSnapshot]): >+ (-[WKContentView _didReceiveEditDragSnapshot:]): >+ >+ Set _waitingForEditDragSnapshot to YES in the gap between when -_willReceiveEditDragSnapshot is invoked, and >+ when -_didReceiveEditDragSnapshot is invoked. If _waitingForEditDragSnapshot is YES, we bail out of >+ -cleanUpDragSourceSessionState, and instead clean up drag session state after the edit drag snapshot is >+ received. >+ >+ (-[WKContentView _deliverDelayedDropPreviewIfPossible:]): >+ (-[WKContentView _didPerformDragOperation:]): >+ (-[WKContentView textEffectsWindow]): >+ >+ Drive-by fix to remove a workaround for a deprecation warning. >+ >+ (-[WKContentView dropInteraction:item:willAnimateDropWithAnimator:]): >+ (-[WKContentView dropInteraction:concludeDrop:]): >+ >+ Implement this hook to ensure that the unselected content snapshot view and visible content snapshot view are >+ guaranteed to be removed from the view after a drop in editable content, even if the drag edit snapshot arrives >+ after the drop is concluded. >+ >+ (-[WKContentView dropInteraction:previewForDroppingItem:withDefault:]): >+ (-[WKContentView _dropInteraction:delayedPreviewProviderForDroppingItem:previewProvider:]): >+ >+ Implement the new UIKit SPI here. UIKit hands us a preview provider here, which we can invoke at a later time >+ to update the drop preview. We do this in _didReceiveEditDragSnapshot. >+ >+ (-[WKContentView _doAfterReceivingEditDragSnapshotForTesting:]): >+ (-[WKContentView _didConcludeEditDrag:]): Deleted. >+ * UIProcess/ios/WebPageProxyIOS.mm: >+ (WebKit::WebPageProxy::willReceiveEditDragSnapshot): >+ (WebKit::WebPageProxy::didReceiveEditDragSnapshot): >+ (WebKit::WebPageProxy::didConcludeDrop): >+ (WebKit::WebPageProxy::didConcludeEditDrag): Deleted. >+ * WebProcess/WebCoreSupport/WebChromeClient.cpp: >+ (WebKit::WebChromeClient::didFinishLoadingImageForElement): >+ * WebProcess/WebCoreSupport/WebChromeClient.h: >+ * WebProcess/WebPage/WebPage.cpp: >+ (WebKit::WebPage::didFinishLoadingImageForElement): >+ * WebProcess/WebPage/WebPage.h: >+ * WebProcess/WebPage/WebPage.messages.in: >+ * WebProcess/WebPage/ios/WebPageIOS.mm: >+ (WebKit::WebPage::didConcludeDrop): >+ >+ If an edit drag has been concluded, there's no need to hang on to pending dropped image elements anymore; clear >+ out the set here. >+ >+ (WebKit::WebPage::didConcludeEditDrag): >+ >+ After concluding an edit drag, we try to deliver two web content snapshots to the UI process, so that the UI >+ process can assemble a targeted drop preview for UIKit. One snapshot is of the visible content area, not >+ including any selected content. The other snapshot is of the selected content only. However, when dropping >+ images (or a text selection containing images), these images may not yet have been loaded. If that is the case, >+ these images will appear to be missing from these snapshots. >+ >+ To ensure that we don't take this snapshot too early, defer it until all image elements in the dropped content >+ range have finished loading. We can tell that all dropped images have finished loading by using a new client >+ hook that is invoked when an image has finished loading. >+ >+ (WebKit::WebPage::didFinishLoadingImageForElement): >+ (WebKit::WebPage::computeAndSendEditDragSnapshot): >+ >+ Snapshot the selected content and send it to the UI process. >+ > 2019-05-24 Youenn Fablet <youenn@apple.com> > > REGRESSION (r245715?) [WK2] Layout Test http/wpt/cache-storage/cache-storage-networkprocess-crash.html is a flaky failure >diff --git a/Source/WebKitLegacy/mac/ChangeLog b/Source/WebKitLegacy/mac/ChangeLog >index f9700f31fcdf87a1b12f01c1e2a74f5067871fab..c2a02fa405e4a64f4bfd14b7fb79b0ed77f8812e 100644 >--- a/Source/WebKitLegacy/mac/ChangeLog >+++ b/Source/WebKitLegacy/mac/ChangeLog >@@ -1,3 +1,17 @@ >+2019-05-25 Wenson Hsieh <wenson_hsieh@apple.com> >+ >+ [iOS] Dropped text, attachments, and images should animate into place >+ https://bugs.webkit.org/show_bug.cgi?id=198243 >+ <rdar://problem/35205373> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Add a new chrome client method. See other changelogs for more detail. >+ >+ * WebCoreSupport/WebChromeClient.h: >+ * WebCoreSupport/WebChromeClient.mm: >+ (WebChromeClient::didFinishLoadingImageForElement): >+ > 2019-05-21 Alex Christensen <achristensen@webkit.org> > > Fix IOSMAC build >diff --git a/Source/WebCore/loader/EmptyClients.h b/Source/WebCore/loader/EmptyClients.h >index d45228eed20eb995f5df2a462fb4e34b73b98f35..b69b9306f19b43aac4d250fa730be845d9dfee49 100644 >--- a/Source/WebCore/loader/EmptyClients.h >+++ b/Source/WebCore/loader/EmptyClients.h >@@ -40,6 +40,7 @@ namespace WebCore { > > class DiagnosticLoggingClient; > class EditorClient; >+class HTMLImageElement; > class PageConfiguration; > > class EmptyChromeClient : public ChromeClient { >@@ -111,6 +112,8 @@ class EmptyChromeClient : public ChromeClient { > IntPoint accessibilityScreenToRootView(const IntPoint& p) const final { return p; }; > IntRect rootViewToAccessibilityScreen(const IntRect& r) const final { return r; }; > >+ void didFinishLoadingImageForElement(HTMLImageElement&) final { } >+ > PlatformPageClient platformPageClient() const final { return 0; } > void contentsSizeChanged(Frame&, const IntSize&) const final { } > void intrinsicContentsSizeChanged(const IntSize&) const final { } >diff --git a/Source/WebCore/page/ChromeClient.h b/Source/WebCore/page/ChromeClient.h >index f35390c8c3ae279f9d3695fa6de2e7f79769c7ef..16d3f9c1aed5a75a0a49f845fe4410302db4e17f 100644 >--- a/Source/WebCore/page/ChromeClient.h >+++ b/Source/WebCore/page/ChromeClient.h >@@ -87,6 +87,7 @@ class FrameLoadRequest; > class Geolocation; > class GraphicsLayer; > class GraphicsLayerFactory; >+class HTMLImageElement; > class HTMLInputElement; > class HTMLMediaElement; > class HTMLVideoElement; >@@ -182,6 +183,8 @@ public: > virtual IntPoint accessibilityScreenToRootView(const IntPoint&) const = 0; > virtual IntRect rootViewToAccessibilityScreen(const IntRect&) const = 0; > >+ virtual void didFinishLoadingImageForElement(HTMLImageElement&) = 0; >+ > virtual PlatformPageClient platformPageClient() const = 0; > > #if ENABLE(CURSOR_SUPPORT) >diff --git a/Source/WebCore/page/Page.cpp b/Source/WebCore/page/Page.cpp >index aa013bf7633ece036de90bfe745ba437df25805d..29932ba936714f279b0061369f4e50374161249f 100644 >--- a/Source/WebCore/page/Page.cpp >+++ b/Source/WebCore/page/Page.cpp >@@ -2996,4 +2996,9 @@ void Page::configureLoggingChannel(const String& channelName, WTFLogChannelState > #endif > } > >+void Page::didFinishLoadingImageForElement(HTMLImageElement& element) >+{ >+ chrome().client().didFinishLoadingImageForElement(element); >+} >+ > } // namespace WebCore >diff --git a/Source/WebCore/page/Page.h b/Source/WebCore/page/Page.h >index ec3f53c8288265bf8e6fcf9d1b1ce9911dc6dd45..e7a1eae62ff7cd2551fa40f690940bfafa27aa44 100644 >--- a/Source/WebCore/page/Page.h >+++ b/Source/WebCore/page/Page.h >@@ -527,6 +527,8 @@ public: > WEBCORE_EXPORT VisibilityState visibilityState() const; > WEBCORE_EXPORT void resumeAnimatingImages(); > >+ void didFinishLoadingImageForElement(HTMLImageElement&); >+ > WEBCORE_EXPORT void addLayoutMilestones(OptionSet<LayoutMilestone>); > WEBCORE_EXPORT void removeLayoutMilestones(OptionSet<LayoutMilestone>); > OptionSet<LayoutMilestone> requestedLayoutMilestones() const { return m_requestedLayoutMilestones; } >diff --git a/Source/WebCore/rendering/RenderImage.cpp b/Source/WebCore/rendering/RenderImage.cpp >index 8e8400235ae172ec82f40abd4e094519e4f86d1d..4d6aa757eaa381ce0cacf8fe5fd13b8b0844d2f0 100644 >--- a/Source/WebCore/rendering/RenderImage.cpp >+++ b/Source/WebCore/rendering/RenderImage.cpp >@@ -378,6 +378,9 @@ void RenderImage::notifyFinished(CachedResource& newImage) > // that the image is done and they can reference it directly. > contentChanged(ImageChanged); > } >+ >+ if (is<HTMLImageElement>(element())) >+ page().didFinishLoadingImageForElement(downcast<HTMLImageElement>(*element())); > } > > bool RenderImage::isShowingMissingOrImageError() const >diff --git a/Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm b/Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm >index f6a7865db73b11cd763849ecf3ba747bca058413..d873d1481a70bad53222e6ee98f9e8476924740e 100644 >--- a/Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm >+++ b/Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm >@@ -6889,6 +6889,11 @@ - (NSDictionary *)_propertiesOfLayerWithID:(unsigned long long)layerID > }; > } > >+- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action >+{ >+ [_contentView _doAfterReceivingEditDragSnapshotForTesting:action]; >+} >+ > #endif // PLATFORM(IOS_FAMILY) > > #if PLATFORM(MAC) >diff --git a/Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h b/Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h >index 0f178c0983c7ef0f5b4bed1c4b7d7181581670b3..c675bfff266b5748a566d2740bd6d731ede64909 100644 >--- a/Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h >+++ b/Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h >@@ -489,6 +489,8 @@ typedef NS_OPTIONS(NSUInteger, _WKRectEdge) { > > @property (nonatomic, readonly) CGRect _dragCaretRect WK_API_AVAILABLE(ios(11.0)); > >+- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action WK_API_AVAILABLE(ios(WK_IOS_TBA)); >+ > - (void)_requestActivatedElementAtPosition:(CGPoint)position completionBlock:(void (^)(_WKActivatedElementInfo *))block WK_API_AVAILABLE(ios(11.0)); > - (void)_accessibilityRetrieveRectsAtSelectionOffset:(NSInteger)offset withText:(NSString *)text completionHandler:(void (^)(NSArray<NSValue *> *rects))completionHandler WK_API_AVAILABLE(ios(11.3)); > - (void)_accessibilityStoreSelection WK_API_AVAILABLE(ios(11.3)); >diff --git a/Source/WebKit/UIProcess/PageClient.h b/Source/WebKit/UIProcess/PageClient.h >index afc94ce902dfd913e84eb7d9658d0d753ab09bd1..669c1da6dbb98f6695ccaded22b747ad0efcd9a1 100644 >--- a/Source/WebKit/UIProcess/PageClient.h >+++ b/Source/WebKit/UIProcess/PageClient.h >@@ -479,7 +479,8 @@ public: > #if ENABLE(DATA_INTERACTION) > virtual void didHandleDragStartRequest(bool started) = 0; > virtual void didHandleAdditionalDragItemsRequest(bool added) = 0; >- virtual void didConcludeEditDrag(Optional<WebCore::TextIndicatorData>) = 0; >+ virtual void willReceiveEditDragSnapshot() = 0; >+ virtual void didReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData>) = 0; > virtual void didChangeDragCaretRect(const WebCore::IntRect& previousCaretRect, const WebCore::IntRect& caretRect) = 0; > #endif > >diff --git a/Source/WebKit/UIProcess/WebPageProxy.h b/Source/WebKit/UIProcess/WebPageProxy.h >index 6c5ad625d1204c7c0d839699e7e2f65a5ba64e03..631a8fe540973a7befa32c7f0b2f9df7c8f4ff03 100644 >--- a/Source/WebKit/UIProcess/WebPageProxy.h >+++ b/Source/WebKit/UIProcess/WebPageProxy.h >@@ -726,12 +726,14 @@ public: > void updateSelectionWithDelta(int64_t locationDelta, int64_t lengthDelta, CompletionHandler<void()>&&); > void requestDocumentEditingContext(WebKit::DocumentEditingContextRequest, CompletionHandler<void(WebKit::DocumentEditingContext)>&&); > void generateSyntheticEditingCommand(SyntheticEditingCommandType); >-#if ENABLE(DATA_INTERACTION) >+#if ENABLE(DRAG_SUPPORT) > void didHandleDragStartRequest(bool started); > void didHandleAdditionalDragItemsRequest(bool added); > void requestDragStart(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, WebCore::DragSourceAction allowedActions); > void requestAdditionalItemsForDragSession(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, WebCore::DragSourceAction allowedActions); >- void didConcludeEditDrag(Optional<WebCore::TextIndicatorData>); >+ void willReceiveEditDragSnapshot(); >+ void didReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData>); >+ void didConcludeDrop(); > #endif > #endif // PLATFORM(IOS_FAMILY) > #if ENABLE(DATA_DETECTION) >diff --git a/Source/WebKit/UIProcess/WebPageProxy.messages.in b/Source/WebKit/UIProcess/WebPageProxy.messages.in >index 6fcecdc6b65d88c9c98585395ab59e7d8126ba06..4046e2e2b2462293e29fa4580f0714a90c399833 100644 >--- a/Source/WebKit/UIProcess/WebPageProxy.messages.in >+++ b/Source/WebKit/UIProcess/WebPageProxy.messages.in >@@ -333,7 +333,8 @@ messages -> WebPageProxy { > #if ENABLE(DATA_INTERACTION) > DidHandleDragStartRequest(bool started) > DidHandleAdditionalDragItemsRequest(bool added) >- DidConcludeEditDrag(Optional<WebCore::TextIndicatorData> textIndicator) >+ WillReceiveEditDragSnapshot() >+ DidReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData> textIndicator) > #endif > > #if PLATFORM(COCOA) >diff --git a/Source/WebKit/UIProcess/ios/DragDropInteractionState.h b/Source/WebKit/UIProcess/ios/DragDropInteractionState.h >index f1849a252cd9972c6e02b78bfd91cc79a380e17f..f57db9f09e15dbc4e3bdc0b94095b50817e81489 100644 >--- a/Source/WebKit/UIProcess/ios/DragDropInteractionState.h >+++ b/Source/WebKit/UIProcess/ios/DragDropInteractionState.h >@@ -40,6 +40,7 @@ > > namespace WebCore { > struct DragItem; >+struct TextIndicatorData; > } > > namespace WebKit { >@@ -58,6 +59,11 @@ struct DragSourceState { > NSInteger itemIdentifier { 0 }; > }; > >+struct ItemAndPreviewProvider { >+ RetainPtr<UIDragItem> item; >+ BlockPtr<void(UITargetedDragPreview *)> provider; >+}; >+ > class DragDropInteractionState { > public: > bool anyActiveDragSourceIs(WebCore::DragSourceAction) const; >@@ -92,6 +98,10 @@ public: > BlockPtr<void()> takeDragCancelSetDownBlock() { return WTFMove(m_dragCancelSetDownBlock); } > BlockPtr<void(NSArray<UIDragItem *> *)> takeAddDragItemCompletionBlock() { return WTFMove(m_addDragItemCompletionBlock); } > >+ void prepareForDelayedDropPreview(UIDragItem *, void(^provider)(UITargetedDragPreview *preview)); >+ void deliverDelayedDropPreview(UIView *contentView, UIView *previewContainer, const WebCore::TextIndicatorData&); >+ void clearAllDelayedItemPreviewProviders(); >+ > private: > void updatePreviewsForActiveDragSources(); > Optional<DragSourceState> activeDragSourceForItem(UIDragItem *) const; >@@ -108,6 +118,7 @@ private: > > Optional<DragSourceState> m_stagedDragSource; > Vector<DragSourceState> m_activeDragSources; >+ Vector<ItemAndPreviewProvider> m_delayedItemPreviewProviders; > }; > > } // namespace WebKit >diff --git a/Source/WebKit/UIProcess/ios/DragDropInteractionState.mm b/Source/WebKit/UIProcess/ios/DragDropInteractionState.mm >index 0670c459c6a851c407bc9b276811ba8ddec68156..0d3bd0cbba6151d58ba639f4679f9f9aed899f55 100644 >--- a/Source/WebKit/UIProcess/ios/DragDropInteractionState.mm >+++ b/Source/WebKit/UIProcess/ios/DragDropInteractionState.mm >@@ -186,6 +186,30 @@ void DragDropInteractionState::dragSessionWillBegin() > updatePreviewsForActiveDragSources(); > } > >+void DragDropInteractionState::prepareForDelayedDropPreview(UIDragItem *item, void(^provider)(UITargetedDragPreview *preview)) >+{ >+ m_delayedItemPreviewProviders.append({ item, provider }); >+} >+ >+void DragDropInteractionState::deliverDelayedDropPreview(UIView *contentView, UIView *previewContainer, const WebCore::TextIndicatorData& indicator) >+{ >+ if (m_delayedItemPreviewProviders.isEmpty()) >+ return; >+ >+ auto textIndicatorImage = uiImageForImage(indicator.contentImage.get()); >+ UITargetedDragPreview *preview = createTargetedDragPreview(textIndicatorImage.get(), contentView, previewContainer, indicator.textBoundingRectInRootViewCoordinates, indicator.textRectsInBoundingRectCoordinates, [UIColor colorWithCGColor:cachedCGColor(indicator.estimatedBackgroundColor)], nil); >+ for (auto& itemAndPreviewProvider : m_delayedItemPreviewProviders) >+ itemAndPreviewProvider.provider(preview); >+ m_delayedItemPreviewProviders.clear(); >+} >+ >+void DragDropInteractionState::clearAllDelayedItemPreviewProviders() >+{ >+ for (auto& itemAndPreviewProvider : m_delayedItemPreviewProviders) >+ itemAndPreviewProvider.provider(nil); >+ m_delayedItemPreviewProviders.clear(); >+} >+ > UITargetedDragPreview *DragDropInteractionState::previewForDragItem(UIDragItem *item, UIView *contentView, UIView *previewContainer) const > { > auto foundSource = activeDragSourceForItem(item); >@@ -266,6 +290,8 @@ void DragDropInteractionState::clearStagedDragSource(DidBecomeActive didBecomeAc > > void DragDropInteractionState::dragAndDropSessionsDidEnd() > { >+ clearAllDelayedItemPreviewProviders(); >+ > // If any of UIKit's completion blocks are still in-flight when the drag interaction ends, we need to ensure that they are still invoked > // to prevent UIKit from getting into an inconsistent state. > if (auto completionBlock = takeDragCancelSetDownBlock()) >diff --git a/Source/WebKit/UIProcess/ios/PageClientImplIOS.h b/Source/WebKit/UIProcess/ios/PageClientImplIOS.h >index 876ac910808d2e97d018ec62eed79d3c7050791b..413c93b3ac05e6c5f6537d49f5128235ef5c4a0f 100644 >--- a/Source/WebKit/UIProcess/ios/PageClientImplIOS.h >+++ b/Source/WebKit/UIProcess/ios/PageClientImplIOS.h >@@ -237,7 +237,8 @@ private: > void didHandleDragStartRequest(bool started) override; > void didHandleAdditionalDragItemsRequest(bool added) override; > void startDrag(const WebCore::DragItem&, const ShareableBitmap::Handle& image) override; >- void didConcludeEditDrag(Optional<WebCore::TextIndicatorData>) override; >+ void willReceiveEditDragSnapshot() override; >+ void didReceiveEditDragSnapshot(Optional<WebCore::TextIndicatorData>) override; > void didChangeDragCaretRect(const WebCore::IntRect& previousCaretRect, const WebCore::IntRect& caretRect) override; > #endif > >diff --git a/Source/WebKit/UIProcess/ios/PageClientImplIOS.mm b/Source/WebKit/UIProcess/ios/PageClientImplIOS.mm >index 09532ff27e99f4e9c893adc818df90d24fa1621a..267bd43bdd101cda946f8488a175333346cdd110 100644 >--- a/Source/WebKit/UIProcess/ios/PageClientImplIOS.mm >+++ b/Source/WebKit/UIProcess/ios/PageClientImplIOS.mm >@@ -820,9 +820,14 @@ void PageClientImpl::startDrag(const DragItem& item, const ShareableBitmap::Hand > [m_contentView _startDrag:ShareableBitmap::create(image)->makeCGImageCopy() item:item]; > } > >-void PageClientImpl::didConcludeEditDrag(Optional<TextIndicatorData> data) >+void PageClientImpl::willReceiveEditDragSnapshot() > { >- [m_contentView _didConcludeEditDrag:data]; >+ [m_contentView _willReceiveEditDragSnapshot]; >+} >+ >+void PageClientImpl::didReceiveEditDragSnapshot(Optional<TextIndicatorData> data) >+{ >+ [m_contentView _didReceiveEditDragSnapshot:data]; > } > > void PageClientImpl::didChangeDragCaretRect(const IntRect& previousCaretRect, const IntRect& caretRect) >diff --git a/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h b/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h >index c92b3f9d2208720acb31ab9894cd742f9bdf95ef..08b64d2b1fc505c9d41c02f39372d40c28b603a7 100644 >--- a/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h >+++ b/Source/WebKit/UIProcess/ios/WKContentViewInteraction.h >@@ -334,6 +334,7 @@ struct WKAutoCorrectionData { > BOOL _isBlurringFocusedElement; > > BOOL _focusRequiresStrongPasswordAssistance; >+ BOOL _waitingForEditDragSnapshot; > > BOOL _hasSetUpInteractions; > NSUInteger _ignoreSelectionCommandFadeCount; >@@ -345,9 +346,10 @@ struct WKAutoCorrectionData { > RetainPtr<UIDragInteraction> _dragInteraction; > RetainPtr<UIDropInteraction> _dropInteraction; > BOOL _shouldRestoreCalloutBarAfterDrop; >- BOOL _isAnimatingConcludeEditDrag; > RetainPtr<UIView> _visibleContentViewSnapshot; >+ RetainPtr<UIView> _unselectedContentSnapshot; > RetainPtr<_UITextDragCaretView> _editDropCaretView; >+ BlockPtr<void()> _actionToPerformAfterReceivingEditDragSnapshot; > #endif > > #if PLATFORM(WATCHOS) >@@ -490,7 +492,8 @@ FOR_EACH_PRIVATE_WKCONTENTVIEW_ACTION(DECLARE_WKCONTENTVIEW_ACTION_FOR_WEB_VIEW) > - (void)_didHandleDragStartRequest:(BOOL)started; > - (void)_didHandleAdditionalDragItemsRequest:(BOOL)added; > - (void)_startDrag:(RetainPtr<CGImageRef>)image item:(const WebCore::DragItem&)item; >-- (void)_didConcludeEditDrag:(Optional<WebCore::TextIndicatorData>)data; >+- (void)_willReceiveEditDragSnapshot; >+- (void)_didReceiveEditDragSnapshot:(Optional<WebCore::TextIndicatorData>)data; > - (void)_didChangeDragCaretRect:(CGRect)previousRect currentRect:(CGRect)rect; > #endif > >@@ -517,6 +520,7 @@ FOR_EACH_PRIVATE_WKCONTENTVIEW_ACTION(DECLARE_WKCONTENTVIEW_ACTION_FOR_WEB_VIEW) > - (void)selectFormAccessoryPickerRow:(NSInteger)rowIndex; > - (void)setTimePickerValueToHour:(NSInteger)hour minute:(NSInteger)minute; > - (NSDictionary *)_contentsOfUserInterfaceItem:(NSString *)userInterfaceItem; >+- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action; > > @property (nonatomic, readonly) NSString *textContentTypeForTesting; > @property (nonatomic, readonly) NSString *selectFormPopoverTitle; >diff --git a/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm b/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm >index 24077daf9387e1eedd3c37e4620b2bf9ab86bf34..36367a3f8df922ed47ec1bbbb71bebca5ca0f245 100644 >--- a/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm >+++ b/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm >@@ -841,6 +841,7 @@ - (void)cleanupInteraction > _outstandingPositionInformationRequest = WTF::nullopt; > > _focusRequiresStrongPasswordAssistance = NO; >+ _waitingForEditDragSnapshot = NO; > > #if USE(UIKIT_KEYBOARD_ADDITIONS) > _candidateViewNeedsUpdate = NO; >@@ -6269,6 +6270,9 @@ - (WebCore::DragData)dragDataForDropSession:(id <UIDropSession>)session dragDest > > - (void)cleanUpDragSourceSessionState > { >+ if (_waitingForEditDragSnapshot) >+ return; >+ > if (_dragDropInteractionState.dragSession() || _dragDropInteractionState.isPerformingDrop()) > RELEASE_LOG(DragAndDrop, "Cleaning up dragging state (has pending operation: %d)", [[WebItemProviderPasteboard sharedInstance] hasPendingOperation]); > >@@ -6281,11 +6285,9 @@ - (void)cleanUpDragSourceSessionState > [[WebItemProviderPasteboard sharedInstance] stageRegistrationList:nil]; > [self _restoreCalloutBarIfNeeded]; > >- [_visibleContentViewSnapshot removeFromSuperview]; >- _visibleContentViewSnapshot = nil; >+ [std::exchange(_visibleContentViewSnapshot, nil) removeFromSuperview]; > [_editDropCaretView remove]; > _editDropCaretView = nil; >- _isAnimatingConcludeEditDrag = NO; > _shouldRestoreCalloutBarAfterDrop = NO; > > _dragDropInteractionState.dragAndDropSessionsDidEnd(); >@@ -6307,11 +6309,33 @@ static NSArray<NSItemProvider *> *extractItemProvidersFromDropSession(id <UIDrop > return extractItemProvidersFromDragItems(session.items); > } > >-- (void)_didConcludeEditDrag:(Optional<WebCore::TextIndicatorData>)data >+- (void)_willReceiveEditDragSnapshot >+{ >+ _waitingForEditDragSnapshot = YES; >+} >+ >+- (void)_didReceiveEditDragSnapshot:(Optional<WebCore::TextIndicatorData>)data >+{ >+ _waitingForEditDragSnapshot = NO; >+ >+ [self _deliverDelayedDropPreviewIfPossible:data]; >+ [self cleanUpDragSourceSessionState]; >+ >+ if (auto action = WTFMove(_actionToPerformAfterReceivingEditDragSnapshot)) >+ action(); >+} >+ >+- (void)_deliverDelayedDropPreviewIfPossible:(Optional<WebCore::TextIndicatorData>)data > { >+ if (!_visibleContentViewSnapshot) >+ return; >+ > if (!data) > return; > >+ if (!data->contentImage) >+ return; >+ > auto snapshotWithoutSelection = data->contentImageWithoutSelection; > if (!snapshotWithoutSelection) > return; >@@ -6321,25 +6345,11 @@ - (void)_didConcludeEditDrag:(Optional<WebCore::TextIndicatorData>)data > return; > > auto unselectedContentImageForEditDrag = adoptNS([[UIImage alloc] initWithCGImage:unselectedSnapshotImage.get() scale:_page->deviceScaleFactor() orientation:UIImageOrientationUp]); >- auto unselectedContentSnapshot = adoptNS([[UIImageView alloc] initWithImage:unselectedContentImageForEditDrag.get()]); >- [unselectedContentSnapshot setFrame:data->contentImageWithoutSelectionRectInRootViewCoordinates]; >- >- auto protectedSelf = retainPtr(self); >- auto visibleContentViewSnapshot = adoptNS(_visibleContentViewSnapshot.leakRef()); >- >- _isAnimatingConcludeEditDrag = YES; >- [self insertSubview:unselectedContentSnapshot.get() belowSubview:visibleContentViewSnapshot.get()]; >- [UIView animateWithDuration:0.25 animations:^() { >- [visibleContentViewSnapshot setAlpha:0]; >- } completion:^(BOOL completed) { >- [visibleContentViewSnapshot removeFromSuperview]; >- [UIView animateWithDuration:0.25 animations:^() { >- [protectedSelf _stopSuppressingSelectionAssistantForReason:WebKit::DropAnimationIsRunning]; >- [unselectedContentSnapshot setAlpha:0]; >- } completion:^(BOOL completed) { >- [unselectedContentSnapshot removeFromSuperview]; >- }]; >- }]; >+ _unselectedContentSnapshot = adoptNS([[UIImageView alloc] initWithImage:unselectedContentImageForEditDrag.get()]); >+ [_unselectedContentSnapshot setFrame:data->contentImageWithoutSelectionRectInRootViewCoordinates]; >+ >+ [self insertSubview:_unselectedContentSnapshot.get() belowSubview:_visibleContentViewSnapshot.get()]; >+ _dragDropInteractionState.deliverDelayedDropPreview(self, self.unscaledView, data.value()); > } > > - (void)_didPerformDragOperation:(BOOL)handled >@@ -6350,9 +6360,6 @@ - (void)_didPerformDragOperation:(BOOL)handled > if ([self.webViewUIDelegate respondsToSelector:@selector(_webView:dataInteractionOperationWasHandled:forSession:itemProviders:)]) > [self.webViewUIDelegate _webView:_webView dataInteractionOperationWasHandled:handled forSession:dropSession itemProviders:[WebItemProviderPasteboard sharedInstance].itemProviders]; > >- if (!_isAnimatingConcludeEditDrag) >- [self _stopSuppressingSelectionAssistantForReason:WebKit::DropAnimationIsRunning]; >- > CGPoint global; > CGPoint client; > [self computeClientAndGlobalPointsForDropSession:dropSession outClientPoint:&client outGlobalPoint:&global]; >@@ -6509,6 +6516,15 @@ - (NSArray<UIDragItem *> *)_itemsForBeginningOrAddingToSessionWithRegistrationLi > return dragItems; > } > >+- (UIView *)textEffectsWindow >+{ >+#if HAVE(UISCENE) >+ return [UITextEffectsWindow sharedTextEffectsWindowForWindowScene:self.window.windowScene]; >+#else >+ return [UITextEffectsWindow sharedTextEffectsWindow]; >+#endif >+} >+ > - (NSDictionary *)_autofillContext > { > BOOL provideStrongPasswordAssistance = _focusRequiresStrongPasswordAssistance && _focusedElementInformation.elementType == WebKit::InputType::Password; >@@ -6890,23 +6906,42 @@ - (void)dropInteraction:(UIDropInteraction *)interaction performDrop:(id <UIDrop > }]; > } > >+- (void)dropInteraction:(UIDropInteraction *)interaction item:(UIDragItem *)item willAnimateDropWithAnimator:(id <UIDragAnimating>)animator >+{ >+ [animator addCompletion:[strongSelf = retainPtr(self)] (UIViewAnimatingPosition) { >+ [std::exchange(strongSelf->_unselectedContentSnapshot, nil) removeFromSuperview]; >+ }]; >+} >+ >+- (void)dropInteraction:(UIDropInteraction *)interaction concludeDrop:(id <UIDropSession>)session >+{ >+ [self _stopSuppressingSelectionAssistantForReason:WebKit::DropAnimationIsRunning]; >+ [std::exchange(_visibleContentViewSnapshot, nil) removeFromSuperview]; >+ [std::exchange(_unselectedContentSnapshot, nil) removeFromSuperview]; >+ _dragDropInteractionState.clearAllDelayedItemPreviewProviders(); >+ _page->didConcludeDrop(); >+} >+ > - (UITargetedDragPreview *)dropInteraction:(UIDropInteraction *)interaction previewForDroppingItem:(UIDragItem *)item withDefault:(UITargetedDragPreview *)defaultPreview > { > CGRect caretRect = _page->currentDragCaretRect(); > if (CGRectIsEmpty(caretRect)) > return nil; > >-ALLOW_DEPRECATED_DECLARATIONS_BEGIN >- // FIXME: <rdar://problem/31074376> [WK2] Performing an edit drag should transition from the initial drag preview to the final drop preview >- // This is blocked on UIKit support, since we aren't able to update the text clipping rects of a UITargetedDragPreview mid-flight. For now, >- // just zoom to the center of the caret rect while shrinking the drop preview. >- auto caretRectInWindowCoordinates = [self convertRect:caretRect toView:[UITextEffectsWindow sharedTextEffectsWindow]]; >+ UIView *textEffectsWindow = self.textEffectsWindow; >+ auto caretRectInWindowCoordinates = [self convertRect:caretRect toView:textEffectsWindow]; > auto caretCenterInWindowCoordinates = CGPointMake(CGRectGetMidX(caretRectInWindowCoordinates), CGRectGetMidY(caretRectInWindowCoordinates)); >- auto target = adoptNS([[UIDragPreviewTarget alloc] initWithContainer:[UITextEffectsWindow sharedTextEffectsWindow] center:caretCenterInWindowCoordinates transform:CGAffineTransformMakeScale(0, 0)]); >-ALLOW_DEPRECATED_DECLARATIONS_END >+ auto targetPreviewCenterInWindowCoordinates = CGPointMake(caretCenterInWindowCoordinates.x + defaultPreview.size.width / 2, caretCenterInWindowCoordinates.y + defaultPreview.size.height / 2); >+ auto target = adoptNS([[UIDragPreviewTarget alloc] initWithContainer:textEffectsWindow center:targetPreviewCenterInWindowCoordinates transform:CGAffineTransformIdentity]); > return [defaultPreview retargetedPreviewWithTarget:target.get()]; > } > >+- (void)_dropInteraction:(UIDropInteraction *)interaction delayedPreviewProviderForDroppingItem:(UIDragItem *)item previewProvider:(void(^)(UITargetedDragPreview *preview))previewProvider >+{ >+ // FIXME: This doesn't currently handle multiple items in a drop session. >+ _dragDropInteractionState.prepareForDelayedDropPreview(item, previewProvider); >+} >+ > - (void)dropInteraction:(UIDropInteraction *)interaction sessionDidEnd:(id <UIDropSession>)session > { > RELEASE_LOG(DragAndDrop, "Drop session ended: %p (performing operation: %d, began dragging: %d)", session, _dragDropInteractionState.isPerformingDrop(), _dragDropInteractionState.didBeginDragging()); >@@ -7166,6 +7201,15 @@ @end > > @implementation WKContentView (WKTesting) > >+- (void)_doAfterReceivingEditDragSnapshotForTesting:(dispatch_block_t)action >+{ >+ ASSERT(!_actionToPerformAfterReceivingEditDragSnapshot); >+ if (_waitingForEditDragSnapshot) >+ _actionToPerformAfterReceivingEditDragSnapshot = action; >+ else >+ action(); >+} >+ > - (WKFormInputControl *)formInputControl > { > if ([_inputPeripheral isKindOfClass:WKFormInputControl.class]) >diff --git a/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm b/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm >index 03584b5503873ea02e15db377e54a6f8e13de6d4..8e9bed5effd43f77e8ae85ed858a9015dd184560 100644 >--- a/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm >+++ b/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm >@@ -1219,9 +1219,19 @@ void WebPageProxy::requestAdditionalItemsForDragSession(const IntPoint& clientPo > m_process->send(Messages::WebPage::RequestAdditionalItemsForDragSession(clientPosition, globalPosition, allowedActions), m_pageID); > } > >-void WebPageProxy::didConcludeEditDrag(Optional<TextIndicatorData> data) >+void WebPageProxy::willReceiveEditDragSnapshot() > { >- pageClient().didConcludeEditDrag(data); >+ pageClient().willReceiveEditDragSnapshot(); >+} >+ >+void WebPageProxy::didReceiveEditDragSnapshot(Optional<TextIndicatorData> data) >+{ >+ pageClient().didReceiveEditDragSnapshot(data); >+} >+ >+void WebPageProxy::didConcludeDrop() >+{ >+ m_process->send(Messages::WebPage::DidConcludeDrop(), m_pageID); > } > > #endif >diff --git a/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp b/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp >index 496296f6d881052392451601305646bb6d6b1cca..2726c23d2a4f484510ab018322e7d90279b14f67 100644 >--- a/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp >+++ b/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.cpp >@@ -587,6 +587,11 @@ IntRect WebChromeClient::rootViewToAccessibilityScreen(const IntRect& rect) cons > return m_page.rootViewToAccessibilityScreen(rect); > } > >+void WebChromeClient::didFinishLoadingImageForElement(HTMLImageElement& element) >+{ >+ m_page.didFinishLoadingImageForElement(element); >+} >+ > PlatformPageClient WebChromeClient::platformPageClient() const > { > notImplemented(); >diff --git a/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h b/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h >index dd3b225435d9263441599c805c36373cf26261ce..b289fb23fe41abbbca1ce172ce8a4f25accd5ec0 100644 >--- a/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h >+++ b/Source/WebKit/WebProcess/WebCoreSupport/WebChromeClient.h >@@ -29,6 +29,7 @@ > #include <WebCore/ChromeClient.h> > > namespace WebCore { >+class HTMLImageElement; > class RegistrableDomain; > enum class StorageAccessPromptWasShown : bool; > enum class StorageAccessWasGranted : bool; >@@ -121,6 +122,8 @@ private: > WebCore::IntPoint accessibilityScreenToRootView(const WebCore::IntPoint&) const final; > WebCore::IntRect rootViewToAccessibilityScreen(const WebCore::IntRect&) const final; > >+ void didFinishLoadingImageForElement(WebCore::HTMLImageElement&) final; >+ > PlatformPageClient platformPageClient() const final; > void contentsSizeChanged(WebCore::Frame&, const WebCore::IntSize&) const final; > void intrinsicContentsSizeChanged(const WebCore::IntSize&) const final; >diff --git a/Source/WebKit/WebProcess/WebPage/WebPage.cpp b/Source/WebKit/WebProcess/WebPage/WebPage.cpp >index 3baafd1854cbc995ca4e89803e3b441823eb6165..26682ff247bbd49fac438ac8f66b6547fdf6574d 100644 >--- a/Source/WebKit/WebProcess/WebPage/WebPage.cpp >+++ b/Source/WebKit/WebProcess/WebPage/WebPage.cpp >@@ -6740,6 +6740,14 @@ void WebPage::updateMockAccessibilityElementAfterCommittingLoad() > } > #endif > >+#if !PLATFORM(IOS_FAMILY) || !ENABLE(DRAG_SUPPORT) >+ >+void WebPage::didFinishLoadingImageForElement(WebCore::HTMLImageElement&) >+{ >+} >+ >+#endif >+ > } // namespace WebKit > > #undef RELEASE_LOG_IF_ALLOWED >diff --git a/Source/WebKit/WebProcess/WebPage/WebPage.h b/Source/WebKit/WebProcess/WebPage/WebPage.h >index 868ae92f2fb3293510110baee8ccb678231c33ad..2a8146b745077302e23e43e6a764b126c9e41bbe 100644 >--- a/Source/WebKit/WebProcess/WebPage/WebPage.h >+++ b/Source/WebKit/WebProcess/WebPage/WebPage.h >@@ -154,6 +154,7 @@ class Frame; > class FrameSelection; > class FrameView; > class GraphicsContext; >+class HTMLImageElement; > class HTMLMenuElement; > class HTMLMenuItemElement; > class HTMLPlugInElement; >@@ -1113,10 +1114,13 @@ public: > void didGetLoadDecisionForIcon(bool decision, CallbackID loadIdentifier, OptionalCallbackID); > void setUseIconLoadingClient(bool); > >-#if ENABLE(DATA_INTERACTION) >+#if PLATFORM(IOS_FAMILY) && ENABLE(DRAG_SUPPORT) > void didConcludeEditDrag(); >+ void didConcludeDrop(); > #endif > >+ void didFinishLoadingImageForElement(WebCore::HTMLImageElement&); >+ > WebURLSchemeHandlerProxy* urlSchemeHandlerForScheme(const String&); > void stopAllURLSchemeTasks(); > >@@ -1258,6 +1262,7 @@ private: > #if PLATFORM(IOS_FAMILY) && ENABLE(DATA_INTERACTION) > void requestDragStart(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, uint64_t allowedActions); > void requestAdditionalItemsForDragSession(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, uint64_t allowedActions); >+ void computeAndSendEditDragSnapshot(); > #endif > > #if !PLATFORM(COCOA) && !PLATFORM(WPE) >@@ -1770,6 +1775,10 @@ private: > WebCore::DragSourceAction m_allowedDragSourceActions { WebCore::DragSourceActionAny }; > #endif > >+#if ENABLE(DRAG_SUPPORT) && PLATFORM(IOS_FAMILY) >+ HashSet<RefPtr<WebCore::HTMLImageElement>> m_pendingImageElementsForDropSnapshot; >+#endif >+ > bool m_cachedMainFrameIsPinnedToLeftSide { true }; > bool m_cachedMainFrameIsPinnedToRightSide { true }; > bool m_cachedMainFrameIsPinnedToTopSide { true }; >diff --git a/Source/WebKit/WebProcess/WebPage/WebPage.messages.in b/Source/WebKit/WebProcess/WebPage/WebPage.messages.in >index 18f06f884e19774ffca1d781b68809ffee170d08..f7feb3a5a33dc8ca97b067bfb4fff9a5afa4aaca 100644 >--- a/Source/WebKit/WebProcess/WebPage/WebPage.messages.in >+++ b/Source/WebKit/WebProcess/WebPage/WebPage.messages.in >@@ -303,9 +303,10 @@ GenerateSyntheticEditingCommand(enum:uint8_t WebKit::SyntheticEditingCommandType > DragCancelled() > #endif > >-#if ENABLE(DATA_INTERACTION) >+#if PLATFORM(IOS_FAMILY) && ENABLE(DRAG_SUPPORT) > RequestDragStart(WebCore::IntPoint clientPosition, WebCore::IntPoint globalPosition, uint64_t allowedActions) > RequestAdditionalItemsForDragSession(WebCore::IntPoint clientPosition, WebCore::IntPoint globalPosition, uint64_t allowedActions) >+ DidConcludeDrop() > #endif > > # Popup menu. >diff --git a/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm b/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm >index c119d4f1247ef65c6b88dacedccb7e81e71e2fb0..9bae7d45a42e9ae1fe0df70737cfbd4abd263e67 100644 >--- a/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm >+++ b/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm >@@ -825,19 +825,63 @@ void WebPage::requestAdditionalItemsForDragSession(const IntPoint& clientPositio > send(Messages::WebPageProxy::DidHandleAdditionalDragItemsRequest(didHandleDrag)); > } > >+void WebPage::didConcludeDrop() >+{ >+ m_pendingImageElementsForDropSnapshot.clear(); >+} >+ > void WebPage::didConcludeEditDrag() > { >- Optional<TextIndicatorData> textIndicatorData; >+ send(Messages::WebPageProxy::WillReceiveEditDragSnapshot()); >+ >+ layoutIfNeeded(); >+ >+ m_pendingImageElementsForDropSnapshot.clear(); > >+ bool waitingForAnyImageToLoad = false; >+ auto& frame = m_page->focusController().focusedOrMainFrame(); >+ if (auto range = frame.selection().selection().toNormalizedRange()) { >+ for (TextIterator iterator(range.get()); !iterator.atEnd(); iterator.advance()) { >+ auto* node = iterator.node(); >+ if (!is<HTMLImageElement>(node)) >+ continue; >+ >+ auto& imageElement = downcast<HTMLImageElement>(*node); >+ auto* cachedImage = imageElement.cachedImage(); >+ if (cachedImage && cachedImage->image() && cachedImage->image()->isNull()) { >+ m_pendingImageElementsForDropSnapshot.add(&imageElement); >+ waitingForAnyImageToLoad = true; >+ } >+ } >+ } >+ >+ if (!waitingForAnyImageToLoad) >+ computeAndSendEditDragSnapshot(); >+} >+ >+void WebPage::didFinishLoadingImageForElement(WebCore::HTMLImageElement& element) >+{ >+ if (m_pendingImageElementsForDropSnapshot.isEmpty()) >+ return; >+ >+ m_pendingImageElementsForDropSnapshot.remove(&element); >+ >+ if (m_pendingImageElementsForDropSnapshot.isEmpty()) >+ computeAndSendEditDragSnapshot(); >+} >+ >+void WebPage::computeAndSendEditDragSnapshot() >+{ >+ Optional<TextIndicatorData> textIndicatorData; > static auto defaultTextIndicatorOptionsForEditDrag = TextIndicatorOptionIncludeSnapshotOfAllVisibleContentWithoutSelection | TextIndicatorOptionExpandClipBeyondVisibleRect | TextIndicatorOptionPaintAllContent | TextIndicatorOptionIncludeMarginIfRangeMatchesSelection | TextIndicatorOptionPaintBackgrounds | TextIndicatorOptionComputeEstimatedBackgroundColor| TextIndicatorOptionUseSelectionRectForSizing | TextIndicatorOptionIncludeSnapshotWithSelectionHighlight; > auto& frame = m_page->focusController().focusedOrMainFrame(); > if (auto range = frame.selection().selection().toNormalizedRange()) { >- if (auto textIndicator = TextIndicator::createWithRange(*range, defaultTextIndicatorOptionsForEditDrag, TextIndicatorPresentationTransition::None, FloatSize())) >+ if (auto textIndicator = TextIndicator::createWithRange(*range, defaultTextIndicatorOptionsForEditDrag, TextIndicatorPresentationTransition::None, { })) > textIndicatorData = textIndicator->data(); > } >- >- send(Messages::WebPageProxy::DidConcludeEditDrag(WTFMove(textIndicatorData))); >+ send(Messages::WebPageProxy::DidReceiveEditDragSnapshot(WTFMove(textIndicatorData))); > } >+ > #endif > > void WebPage::sendTapHighlightForNodeIfNecessary(uint64_t requestID, Node* node) >diff --git a/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h b/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h >index 640fe58fd104794b3c583e659ab61acdf17c2e20..7e0a6258ac02a87ca9c10ce4a9f85cf432759996 100644 >--- a/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h >+++ b/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.h >@@ -31,6 +31,10 @@ > #import <WebCore/FocusDirection.h> > #import <wtf/Forward.h> > >+namespace WebCore { >+class HTMLImageElement; >+} >+ > @class WebView; > > // FIXME: This class is used as a concrete class on Mac, but on iOS this is an abstract >@@ -102,6 +106,8 @@ private: > WebCore::IntPoint accessibilityScreenToRootView(const WebCore::IntPoint&) const final; > WebCore::IntRect rootViewToAccessibilityScreen(const WebCore::IntRect&) const final; > >+ void didFinishLoadingImageForElement(WebCore::HTMLImageElement&) final; >+ > PlatformPageClient platformPageClient() const final; > void contentsSizeChanged(WebCore::Frame&, const WebCore::IntSize&) const final; > void intrinsicContentsSizeChanged(const WebCore::IntSize&) const final { } >diff --git a/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm b/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm >index aac8715664f2cbe7ce70df27463ec4f175d82c79..abe2fb3298d16de414cbf93c8bed8bec18b2107e 100644 >--- a/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm >+++ b/Source/WebKitLegacy/mac/WebCoreSupport/WebChromeClient.mm >@@ -610,6 +610,10 @@ IntRect WebChromeClient::rootViewToAccessibilityScreen(const IntRect& r) const > return rootViewToScreen(r); > } > >+void WebChromeClient::didFinishLoadingImageForElement(HTMLImageElement&) >+{ >+} >+ > PlatformPageClient WebChromeClient::platformPageClient() const > { > return 0; >diff --git a/Tools/ChangeLog b/Tools/ChangeLog >index e8242607bdb2d1f4a8dd0387aed8b4f4f1e99441..19348b4a5a86fcc25f80c49a56b027a7cb1bf590 100644 >--- a/Tools/ChangeLog >+++ b/Tools/ChangeLog >@@ -1,3 +1,41 @@ >+2019-05-25 Wenson Hsieh <wenson_hsieh@apple.com> >+ >+ [iOS] Dropped text, attachments, and images should animate into place >+ https://bugs.webkit.org/show_bug.cgi?id=198243 >+ <rdar://problem/35205373> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ Adjusts the iOS dragging simulator, and adds a new API test. See below for more detail. >+ >+ * TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm: >+ (TestWebKitAPI::isCompletelyWhite): >+ (TestWebKitAPI::TEST): >+ >+ Add a test that drags and drops an image into a contenteditable element, and then observes the resulting >+ UITargetedDragPreviews upon drop. >+ >+ * TestWebKitAPI/cocoa/DragAndDropSimulator.h: >+ * TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm: >+ (-[DragAndDropSimulator _resetSimulatedState]): >+ (-[DragAndDropSimulator runFrom:to:additionalItemRequestLocations:]): >+ (-[DragAndDropSimulator _concludeDropAndPerformOperationIfNecessary]): >+ >+ Teach the iOS version of DragAndDropSimulator to invoke -dropInteraction:concludeDrop: on the drop interaction >+ delegate when the drop finishes. This additionally uses _doAfterReceivingEditDragSnapshotForTesting: to defer >+ the end of the simulated drag and drop until after drag previews have been received during an edit drag. >+ >+ (-[DragAndDropSimulator dropPreviews]): >+ (-[DragAndDropSimulator delayedDropPreviews]): >+ >+ Have the drag and drop simulator remember which previews were returned by the delegate on drop, as well as which >+ previews were provided asynchronously. >+ >+ (-[DragAndDropSimulator _webView:dataInteractionOperationWasHandled:forSession:itemProviders:]): >+ * TestWebKitAPI/ios/UIKitSPI.h: >+ >+ Stage the new private drop interacton delegate method. >+ > 2019-05-25 Simon Fraser <simon.fraser@apple.com> > > MobileMiniBrowser: Add iPad support, and allow insecure loads >diff --git a/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm b/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm >index 21936bd63015daafd83f017be94c09422d02beaf..f1cc240e539c817f388c31afd1e981846b3e5373 100644 >--- a/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm >+++ b/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm >@@ -2107,6 +2107,44 @@ TEST(DragAndDropTests, DataTransferSanitizeHTML) > TestWebKitAPI::Util::run(&done); > } > >+static BOOL isCompletelyWhite(UIImage *image) >+{ >+ auto data = adoptCF(CGDataProviderCopyData(CGImageGetDataProvider(image.CGImage))); >+ auto* dataPtr = CFDataGetBytePtr(data.get()); >+ int imageWidth = image.size.width; >+ for (int row = 0; row < image.size.height; ++row) { >+ for (int column = 0; column < imageWidth; ++column) { >+ int pixelOffset = ((imageWidth * row) + column) * 4; >+ if (dataPtr[pixelOffset] != 0xFF || dataPtr[pixelOffset + 1] != 0xFF || dataPtr[pixelOffset + 2] != 0xFF) >+ return NO; >+ } >+ } >+ return YES; >+} >+ >+TEST(DragAndDropTests, DropPreviewForImageInEditableArea) >+{ >+ auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]); >+ [webView synchronouslyLoadTestPageNamed:@"image-and-contenteditable"]; >+ >+ // Ensure that the resulting snapshot on drop contains only the dragged image. >+ [webView stringByEvaluatingJavaScript:@"editor.style.border = 'none'; editor.style.outline = 'none'"]; >+ >+ auto simulator = adoptNS([[DragAndDropSimulator alloc] initWithWebView:webView.get()]); >+ [simulator runFrom:CGPointMake(100, 50) to:CGPointMake(100, 300)]; >+ >+ NSArray *dropPreviews = [simulator dropPreviews]; >+ NSArray *delayedDropPreviews = [simulator delayedDropPreviews]; >+ EXPECT_EQ(1U, dropPreviews.count); >+ EXPECT_EQ(1U, delayedDropPreviews.count); >+ EXPECT_EQ(UITargetedDragPreview.class, [dropPreviews.firstObject class]); >+ EXPECT_EQ(UITargetedDragPreview.class, [delayedDropPreviews.firstObject class]); >+ >+ UITargetedDragPreview *finalPreview = (UITargetedDragPreview *)delayedDropPreviews.firstObject; >+ EXPECT_EQ(UIImageView.class, finalPreview.view.class); >+ EXPECT_FALSE(isCompletelyWhite([(UIImageView *)finalPreview.view image])); >+} >+ > } // namespace TestWebKitAPI > > #endif // ENABLE(DRAG_SUPPORT) && PLATFORM(IOS_FAMILY) >diff --git a/Tools/TestWebKitAPI/cocoa/DragAndDropSimulator.h b/Tools/TestWebKitAPI/cocoa/DragAndDropSimulator.h >index 847c776adcfe68a70453cdf487657b8fa3641ff7..c5bc0de792f8f3639f3ceb1c1e0d8ae00f1d5609 100644 >--- a/Tools/TestWebKitAPI/cocoa/DragAndDropSimulator.h >+++ b/Tools/TestWebKitAPI/cocoa/DragAndDropSimulator.h >@@ -111,6 +111,8 @@ typedef NSDictionary<NSNumber *, NSValue *> *ProgressToCGPointValueMap; > @property (nonatomic, readonly) NSArray *finalSelectionRects; > @property (nonatomic, readonly) CGRect lastKnownDragCaretRect; > @property (nonatomic, readonly) NSArray<UITargetedDragPreview *> *liftPreviews; >+@property (nonatomic, readonly) NSArray *dropPreviews; >+@property (nonatomic, readonly) NSArray *delayedDropPreviews; > @property (nonatomic, readonly) BOOL suppressedSelectionCommandsDuringDrop; > > #endif // PLATFORM(IOS_FAMILY) >diff --git a/Tools/TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm b/Tools/TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm >index ff0187dc4e6c05f51fc90a515afe5d78c700293b..8edae8ce8a9484358532727e6685f01a750cfa20 100644 >--- a/Tools/TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm >+++ b/Tools/TestWebKitAPI/ios/DragAndDropSimulatorIOS.mm >@@ -34,6 +34,7 @@ > > #import <UIKit/UIDragInteraction.h> > #import <UIKit/UIDragItem.h> >+#import <UIKit/UIDropInteraction.h> > #import <UIKit/UIInteraction.h> > #import <WebKit/WKWebViewPrivate.h> > #import <WebKit/_WKFocusedElementInfo.h> >@@ -310,6 +311,8 @@ @implementation DragAndDropSimulator { > RetainPtr<NSMutableDictionary<NSNumber *, NSValue *>>_remainingAdditionalItemRequestLocationsByProgress; > RetainPtr<NSMutableArray<NSValue *>>_queuedAdditionalItemRequestLocations; > RetainPtr<NSMutableArray<UITargetedDragPreview *>> _liftPreviews; >+ RetainPtr<NSMutableArray> _dropPreviews; >+ RetainPtr<NSMutableArray> _delayedDropPreviews; > > RetainPtr<NSMutableArray<_WKAttachment *>> _insertedAttachments; > RetainPtr<NSMutableArray<_WKAttachment *>> _removedAttachments; >@@ -317,6 +320,7 @@ @implementation DragAndDropSimulator { > bool _hasStartedInputSession; > double _currentProgress; > bool _isDoneWithCurrentRun; >+ bool _isDoneWaitingForDelayedDropPreviews; > DragAndDropPhase _phase; > > BOOL _suppressedSelectionCommandsDuringDrop; >@@ -373,6 +377,7 @@ - (void)_resetSimulatedState > _phase = DragAndDropPhaseBeginning; > _currentProgress = 0; > _isDoneWithCurrentRun = false; >+ _isDoneWaitingForDelayedDropPreviews = true; > _observedEventNames = adoptNS([[NSMutableArray alloc] init]); > _insertedAttachments = adoptNS([[NSMutableArray alloc] init]); > _removedAttachments = adoptNS([[NSMutableArray alloc] init]); >@@ -384,6 +389,8 @@ - (void)_resetSimulatedState > _remainingAdditionalItemRequestLocationsByProgress = nil; > _queuedAdditionalItemRequestLocations = adoptNS([[NSMutableArray alloc] init]); > _liftPreviews = adoptNS([[NSMutableArray alloc] init]); >+ _dropPreviews = adoptNS([[NSMutableArray alloc] init]); >+ _delayedDropPreviews = adoptNS([[NSMutableArray alloc] init]); > _hasStartedInputSession = false; > } > >@@ -454,6 +461,7 @@ - (void)runFrom:(CGPoint)startLocation to:(CGPoint)endLocation additionalItemReq > } > > Util::run(&_isDoneWithCurrentRun); >+ Util::run(&_isDoneWaitingForDelayedDropPreviews); > [_webView clearMessageHandlers:dragAndDropEventNames()]; > _finalSelectionRects = [_webView selectionRectsAfterPresentationUpdate]; > >@@ -470,11 +478,30 @@ - (void)_concludeDropAndPerformOperationIfNecessary > _lastKnownDragCaretRect = [_webView _dragCaretRect]; > auto operation = [_lastKnownDropProposal operation]; > if (operation != UIDropOperationCancel && operation != UIDropOperationForbidden) { >+ NSInteger dropPreviewIndex = 0; >+ __block NSUInteger numberOfPendingPreviews = [_dropSession items].count; >+ _isDoneWaitingForDelayedDropPreviews = !numberOfPendingPreviews; >+ for (UIDragItem *item in [_dropSession items]) { >+ auto defaultPreview = adoptNS([[UITargetedDragPreview alloc] initWithView:_webView.get()]); >+ id <UIDropInteractionDelegate_Staging_31075005> delegate = (id <UIDropInteractionDelegate_Staging_31075005>)[_webView dropInteractionDelegate]; >+ UIDropInteraction *interaction = [_webView dropInteraction]; >+ [_dropPreviews addObject:[delegate dropInteraction:interaction previewForDroppingItem:item withDefault:defaultPreview.get()] ?: NSNull.null]; >+ [_delayedDropPreviews addObject:NSNull.null]; >+ [delegate _dropInteraction:interaction delayedPreviewProviderForDroppingItem:item previewProvider:^(UITargetedDragPreview *preview) { >+ if (preview) >+ [_delayedDropPreviews setObject:preview atIndexedSubscript:dropPreviewIndex]; >+ >+ if (!--numberOfPendingPreviews) >+ _isDoneWaitingForDelayedDropPreviews = true; >+ }]; >+ ++dropPreviewIndex; >+ } > [[_webView dropInteractionDelegate] dropInteraction:[_webView dropInteraction] performDrop:_dropSession.get()]; > _phase = DragAndDropPhasePerformingDrop; > } else { >- _isDoneWithCurrentRun = YES; >+ _isDoneWithCurrentRun = true; > _phase = DragAndDropPhaseCancelled; >+ [[_webView dropInteractionDelegate] dropInteraction:[_webView dropInteraction] concludeDrop:_dropSession.get()]; > } > > [[_webView dropInteractionDelegate] dropInteraction:[_webView dropInteraction] sessionDidEnd:_dropSession.get()]; >@@ -644,6 +671,16 @@ - (NSArray<UITargetedDragPreview *> *)liftPreviews > return _liftPreviews.get(); > } > >+- (NSArray<UITargetedDragPreview *> *)dropPreviews >+{ >+ return _dropPreviews.get(); >+} >+ >+- (NSArray<UITargetedDragPreview *> *)delayedDropPreviews >+{ >+ return _delayedDropPreviews.get(); >+} >+ > - (CGRect)lastKnownDragCaretRect > { > return _lastKnownDragCaretRect; >@@ -729,10 +766,14 @@ - (void(^)(BOOL, NSArray *))dropCompletionBlock > - (void)_webView:(WKWebView *)webView dataInteractionOperationWasHandled:(BOOL)handled forSession:(id)session itemProviders:(NSArray<NSItemProvider *> *)itemProviders > { > _suppressedSelectionCommandsDuringDrop = [_webView textInputContentView]._shouldSuppressSelectionCommands; >- _isDoneWithCurrentRun = true; > > if (self.dropCompletionBlock) > self.dropCompletionBlock(handled, itemProviders); >+ >+ [_webView _doAfterReceivingEditDragSnapshotForTesting:^{ >+ [[_webView dropInteractionDelegate] dropInteraction:[_webView dropInteraction] concludeDrop:_dropSession.get()]; >+ _isDoneWithCurrentRun = true; >+ }]; > } > > - (UIDropProposal *)_webView:(WKWebView *)webView willUpdateDropProposalToProposal:(UIDropProposal *)proposal forSession:(id <UIDropSession>)session >diff --git a/Tools/TestWebKitAPI/ios/UIKitSPI.h b/Tools/TestWebKitAPI/ios/UIKitSPI.h >index 3fa668437d2d62dbd73b987b0147b917b6e03cb6..72ec9fd8ca39915c6ed93ea216f77d26084b371a 100644 >--- a/Tools/TestWebKitAPI/ios/UIKitSPI.h >+++ b/Tools/TestWebKitAPI/ios/UIKitSPI.h >@@ -183,4 +183,8 @@ typedef NS_OPTIONS(NSInteger, UIWKDocumentRequestFlags) { > - (void)setNextPreviousItemsVisible:(BOOL)visible; > @end > >+@protocol UIDropInteractionDelegate_Staging_31075005 <UIDropInteractionDelegate> >+- (void)_dropInteraction:(UIDropInteraction *)interaction delayedPreviewProviderForDroppingItem:(UIDragItem *)item previewProvider:(void(^)(UITargetedDragPreview *preview))previewProvider; >+@end >+ > #endif // PLATFORM(IOS_FAMILY)
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 198243
:
370645
|
370646
|
370647
|
370648