WebKit Bugzilla
Attachment 360194 Details for
Bug 193852
: Web Automation: add support for simulating single touches to Automation.performInteractionSequence
Home
|
New
|
Browse
|
Search
|
[?]
|
Reports
|
Requests
|
Help
|
New Account
|
Log In
Remember
[x]
|
Forgot Password
Login:
[x]
[patch]
Patch
bug-193852-20190125170635.patch (text/plain), 67.19 KB, created by
BJ Burg
on 2019-01-25 17:06:35 PST
(
hide
)
Description:
Patch
Filename:
MIME Type:
Creator:
BJ Burg
Created:
2019-01-25 17:06:35 PST
Size:
67.19 KB
patch
obsolete
>Subversion Revision: 240525 >diff --git a/Source/WebKit/ChangeLog b/Source/WebKit/ChangeLog >index e007ba6bb8d119130128c479232bfcfd27609f60..2bf8b71284dd404bf7b0527d2d2d00b6f2243359 100644 >--- a/Source/WebKit/ChangeLog >+++ b/Source/WebKit/ChangeLog >@@ -1,3 +1,108 @@ >+2019-01-25 Brian Burg <bburg@apple.com> >+ >+ Web Automation: add support for simulating single touches to Automation.performInteractionSequence >+ https://bugs.webkit.org/show_bug.cgi?id=193852 >+ <rdar://problem/28360781> >+ >+ Reviewed by NOBODY (OOPS!). >+ >+ This patch makes it possible to simulate touches via the Actions API. The approach is to >+ use a stripped down version of HIDEventGenerator to send HID events to the UIWindow. >+ The initial version supports a single touch point; this may be expanded if needed, but >+ so far it hasn't been an impediment when running desktop-oriented WebDriver test suites. >+ >+ As part of implementing this support, I went through and added [somewhat obnoxious] ENABLE() >+ guards so that we don't mistakenly compile mouse support on iOS and touch on Mac, >+ and vice versa. Each of the interaction methods---touch, mouse, keyboard---can be independently >+ compiled out. If none is supported (i.e., Windows), then we don't even try to compile >+ platformSimulate*Interaction methods or SimulatedInputDispatcher. This allows WebAutomationSession >+ to not include members and code that are never going to be used on a particular platform. >+ >+ This functionality is covered by existing WebDriver test suites that use Element Click command. >+ Additional tests that explicitly include 'touch' pointerType will be submitted to WPT later. >+ >+ * UIProcess/Automation/Automation.json: Update comment. >+ >+ * UIProcess/Automation/SimulatedInputDispatcher.h: >+ * UIProcess/Automation/SimulatedInputDispatcher.cpp: >+ (WebKit::SimulatedInputDispatcher::transitionInputSourceToState): >+ - Add appropriate guards for code specific to each interaction type. >+ - Handle SimulatedInputSourceType::Touch and call out to dispatch events. >+ >+ * UIProcess/Automation/WebAutomationSession.h: >+ * UIProcess/Automation/WebAutomationSession.cpp: >+ (WebKit::WebAutomationSession::WebAutomationSession): >+ (WebKit::WebAutomationSession::terminate): >+ (WebKit::WebAutomationSession::willShowJavaScriptDialog): >+ (WebKit::WebAutomationSession::mouseEventsFlushedForPage): >+ (WebKit::WebAutomationSession::keyboardEventsFlushedForPage): >+ (WebKit::WebAutomationSession::willClosePage): >+ Add appropriate guards for code specific to each interaction type. >+ >+ (WebKit::WebAutomationSession::isSimulatingUserInteraction const): >+ Add new hook so that we can detect when a touch simulation is in progress. >+ We don't rely on checking WebKit's event queue, so use a flag. >+ >+ (WebKit::WebAutomationSession::simulateTouchInteraction): >+ (WebKit::WebAutomationSession::performMouseInteraction): >+ (WebKit::WebAutomationSession::performKeyboardInteractions): >+ (WebKit::WebAutomationSession::cancelInteractionSequence): >+ (WebKit::WebAutomationSession::performInteractionSequence): >+ - Add appropriate guards for code specific to each interaction type. >+ - Bail out immediately if a requested interaction type is not supported. >+ - Shim Touch input type to Mouse if mouse is not supported but touches are. >+ Nearly all WebDriver tests in the wild will be requesting a 'mouse' pointerType, >+ so this is necessary for compatibility. It's easier to shim at this level than try >+ to implement platformSimulateMouseInteraction for iOS because >+ platformSimulateMouseinteraction does not use a completion handler but the iOS >+ implementation would require that. >+ >+ (WebKit::WebAutomationSession::platformSimulateMouseInteraction): Deleted. >+ (WebKit::WebAutomationSession::platformSimulateKeyboardInteraction): Deleted. >+ - Remove these stubs. Platform methods of this class are no longer being filled >+ with stubs on unsupported platforms because we expect the command to fail earlier. >+ >+ * UIProcess/Automation/ios/WebAutomationSessionIOS.mm: >+ (WebKit::WebAutomationSession::platformSimulateTouchInteraction): >+ - Add appropriate guards for code specific to each interaction type. >+ - Implement new platform method to simulate touches. This uses _WKTouchEventGenerator. >+ >+ * UIProcess/Automation/mac/WebAutomationSessionMac.mm: >+ Add appropriate guards for code specific to each interaction type. >+ >+ * UIProcess/_WKTouchEventGenerator.h: Added. >+ * UIProcess/_WKTouchEventGenerator.mm: Added. >+ (simpleCurveInterpolation): >+ (calculateNextCurveLocation): >+ (delayBetweenMove): >+ (+[_WKTouchEventGenerator sharedTouchEventGenerator]): >+ (+[_WKTouchEventGenerator nextEventCallbackID]): >+ (-[_WKTouchEventGenerator init]): >+ (-[_WKTouchEventGenerator dealloc]): >+ (-[_WKTouchEventGenerator _createIOHIDEventType:]): >+ (-[_WKTouchEventGenerator _sendHIDEvent:]): >+ (-[_WKTouchEventGenerator _sendMarkerHIDEventWithCompletionBlock:]): >+ (-[_WKTouchEventGenerator _updateTouchPoints:count:]): >+ (-[_WKTouchEventGenerator touchDownAtPoints:touchCount:]): >+ (-[_WKTouchEventGenerator touchDown:touchCount:]): >+ (-[_WKTouchEventGenerator touchDown:]): >+ (-[_WKTouchEventGenerator liftUpAtPoints:touchCount:]): >+ (-[_WKTouchEventGenerator liftUp:touchCount:]): >+ (-[_WKTouchEventGenerator liftUp:]): >+ (-[_WKTouchEventGenerator moveToPoints:touchCount:duration:]): >+ (-[_WKTouchEventGenerator touchDown:completionBlock:]): >+ (-[_WKTouchEventGenerator liftUp:completionBlock:]): >+ (-[_WKTouchEventGenerator moveToPoint:duration:completionBlock:]): >+ (-[_WKTouchEventGenerator receivedHIDEvent:]): >+ Copied and simplified from HIDEventGenerator in WebKitTestRunner. >+ >+ * WebKit.xcodeproj/project.pbxproj: >+ Add _WKTouchEventGenerator.{h,mm} and make it a private header. The client needs to >+ route received HID events to -receivedHIDEvent: in order to detect when the marker >+ HID event has been processed (and thus the interaction is completed). >+ >+ * config.h: Add ENABLE(WEBDRIVER_*_INTERACTIONS) macros here. >+ > 2019-01-25 Keith Rollin <krollin@apple.com> > > Update Xcode projects with "Check .xcfilelists" build phase >diff --git a/Source/WebKit/UIProcess/Automation/Automation.json b/Source/WebKit/UIProcess/Automation/Automation.json >index 7f74172d3e6bb97ff1107f8e7ee042471a5ad516..d47574466462b7415c023500953349eab4dbd8e4 100644 >--- a/Source/WebKit/UIProcess/Automation/Automation.json >+++ b/Source/WebKit/UIProcess/Automation/Automation.json >@@ -299,7 +299,7 @@ > { "name": "sourceId", "type": "string", "description": "The input source whose state is described by this object." }, > { "name": "pressedCharKey", "type": "string", "optional": true, "description": "For 'keyboard' input sources, specifies a character key that has 'pressed' state. Unmentioned character keys are assumed to have a 'released' state." }, > { "name": "pressedVirtualKeys", "type": "array", "items": { "$ref": "VirtualKey" }, "optional": true, "description": "For 'keyboard' input sources, specifies virtual keys that have a 'pressed' state. Unmentioned virtual keys are assumed to have a 'released' state." }, >- { "name": "pressedButton", "$ref": "MouseButton", "optional": true, "description": "For 'mouse' input sources, specifies which mouse button has a 'pressed' state. Unmentioned mouse buttons are assumed to have a 'released' state." }, >+ { "name": "pressedButton", "$ref": "MouseButton", "optional": true, "description": "For 'mouse' input sources, specifies which mouse button has a 'pressed' state. Unmentioned mouse buttons are assumed to have a 'released' state. For 'touch' input sources, passing MouseButton::Left denotes a touch-down state." }, > { "name": "origin", "$ref": "MouseMoveOrigin", "optional": true, "description": "For 'mouse' input sources, specifies the origin type of a mouse move transition. Defaults to 'Viewport' if omitted."}, > { "name": "nodeHandle", "$ref": "NodeHandle", "optional": true, "description": "The handle of the element to use as origin when origin type is 'Element'."}, > { "name": "location", "$ref": "Point", "optional": true, "description": "For 'mouse' or 'touch' input sources, specifies a location in view coordinates to which the input source should transition. Transitioning to this state may interpolate intemediate input source states to better simulate real user movements and gestures." }, >diff --git a/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.cpp b/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.cpp >index 0590c57b297df639d8465b64d5e872f04b3446b6..35eb71954fdcc7ade2c06539252494a03e9b5e9a 100644 >--- a/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.cpp >+++ b/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.cpp >@@ -1,5 +1,5 @@ > /* >- * Copyright (C) 2018 Apple Inc. All rights reserved. >+ * Copyright (C) 2018, 2019 Apple Inc. All rights reserved. > * > * Redistribution and use in source and binary forms, with or without > * modification, are permitted provided that the following conditions >@@ -26,6 +26,8 @@ > #include "config.h" > #include "SimulatedInputDispatcher.h" > >+#if ENABLE(WEBDRIVER_ACTIONS_API) >+ > #include "AutomationProtocolObjects.h" > #include "Logging.h" > #include "WebAutomationSession.h" >@@ -259,6 +261,9 @@ void SimulatedInputDispatcher::transitionInputSourceToState(SimulatedInputSource > eventDispatchFinished(WTF::nullopt); > break; > case SimulatedInputSourceType::Mouse: { >+#if !ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) >+ RELEASE_ASSERT_NOT_REACHED(); >+#else > resolveLocation(a.location.valueOr(WebCore::IntPoint()), b.location, b.origin.valueOr(MouseMoveOrigin::Viewport), b.nodeHandle, [this, &a, &b, eventDispatchFinished = WTFMove(eventDispatchFinished)](Optional<WebCore::IntPoint> location, Optional<AutomationCommandError> error) mutable { > if (error) { > eventDispatchFinished(error); >@@ -270,25 +275,56 @@ void SimulatedInputDispatcher::transitionInputSourceToState(SimulatedInputSource > if (!a.pressedMouseButton && b.pressedMouseButton) { > #if !LOG_DISABLED > String mouseButtonName = Inspector::Protocol::AutomationHelpers::getEnumConstantValue(b.pressedMouseButton.value()); >- LOG(Automation, "SimulatedInputDispatcher[%p]: simulating MouseDown[button=%s] for transition to %d.%d", this, mouseButtonName.utf8().data(), m_keyframeIndex, m_inputSourceStateIndex); >+ LOG(Automation, "SimulatedInputDispatcher[%p]: simulating MouseDown[button=%s] @ (%d, %d) for transition to %d.%d", this, mouseButtonName.utf8().data(), b.location.value().x(), b.location.value().y(), m_keyframeIndex, m_inputSourceStateIndex); > #endif > m_client.simulateMouseInteraction(m_page, MouseInteraction::Down, b.pressedMouseButton.value(), b.location.value(), WTFMove(eventDispatchFinished)); > } else if (a.pressedMouseButton && !b.pressedMouseButton) { > #if !LOG_DISABLED > String mouseButtonName = Inspector::Protocol::AutomationHelpers::getEnumConstantValue(a.pressedMouseButton.value()); >- LOG(Automation, "SimulatedInputDispatcher[%p]: simulating MouseUp[button=%s] for transition to %d.%d", this, mouseButtonName.utf8().data(), m_keyframeIndex, m_inputSourceStateIndex); >+ LOG(Automation, "SimulatedInputDispatcher[%p]: simulating MouseUp[button=%s] @ (%d, %d) for transition to %d.%d", this, mouseButtonName.utf8().data(), a.location.value().x(), a.location.value().y(), m_keyframeIndex, m_inputSourceStateIndex); > #endif > m_client.simulateMouseInteraction(m_page, MouseInteraction::Up, a.pressedMouseButton.value(), b.location.value(), WTFMove(eventDispatchFinished)); > } else if (a.location != b.location) { >- LOG(Automation, "SimulatedInputDispatcher[%p]: simulating MouseMove for transition to %d.%d", this, m_keyframeIndex, m_inputSourceStateIndex); >+ LOG(Automation, "SimulatedInputDispatcher[%p]: simulating MouseMove from (%d, %d) to (%d, %d) for transition to %d.%d", this, a.location.value().x(), a.location.value().y(), b.location.value().x(), b.location.value().y(), m_keyframeIndex, m_inputSourceStateIndex); > // FIXME: This does not interpolate mousemoves per the "perform a pointer move" algorithm (§17.4 Dispatching Actions). > m_client.simulateMouseInteraction(m_page, MouseInteraction::Move, b.pressedMouseButton.valueOr(MouseButton::NoButton), b.location.value(), WTFMove(eventDispatchFinished)); > } else > eventDispatchFinished(WTF::nullopt); > }); >+#endif // ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) >+ break; >+ } >+ case SimulatedInputSourceType::Touch: { >+#if !ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ RELEASE_ASSERT_NOT_REACHED(); >+#else >+ resolveLocation(a.location.valueOr(WebCore::IntPoint()), b.location, b.origin.valueOr(MouseMoveOrigin::Viewport), b.nodeHandle, [this, &a, &b, eventDispatchFinished = WTFMove(eventDispatchFinished)](Optional<WebCore::IntPoint> location, Optional<AutomationCommandError> error) mutable { >+ if (error) { >+ eventDispatchFinished(error); >+ return; >+ } >+ RELEASE_ASSERT(location); >+ b.location = location; >+ // The "dispatch a pointer{Down,Up,Move} action" algorithms (§17.4 Dispatching Actions). >+ if (!a.pressedMouseButton && b.pressedMouseButton) { >+ LOG(Automation, "SimulatedInputDispatcher[%p]: simulating TouchDown @ (%d, %d) for transition to %d.%d", this, b.location.value().x(), b.location.value().y(), m_keyframeIndex, m_inputSourceStateIndex); >+ m_client.simulateTouchInteraction(m_page, TouchInteraction::TouchDown, b.location.value(), WTF::nullopt, WTFMove(eventDispatchFinished)); >+ } else if (a.pressedMouseButton && !b.pressedMouseButton) { >+ LOG(Automation, "SimulatedInputDispatcher[%p]: simulating LiftUp @ (%d, %d) for transition to %d.%d", this, a.location.value().x(), a.location.value().y(), m_keyframeIndex, m_inputSourceStateIndex); >+ m_client.simulateTouchInteraction(m_page, TouchInteraction::LiftUp, b.location.value(), WTF::nullopt, WTFMove(eventDispatchFinished)); >+ } else if (a.location != b.location) { >+ LOG(Automation, "SimulatedInputDispatcher[%p]: simulating MoveTo from (%d, %d) to (%d, %d) for transition to %d.%d", this, a.location.value().x(), a.location.value().y(), b.location.value().x(), b.location.value().y(), m_keyframeIndex, m_inputSourceStateIndex); >+ m_client.simulateTouchInteraction(m_page, TouchInteraction::MoveTo, b.location.value(), a.duration.valueOr(0_s), WTFMove(eventDispatchFinished)); >+ } else >+ eventDispatchFinished(WTF::nullopt); >+ }); >+#endif // !ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) > break; > } > case SimulatedInputSourceType::Keyboard: >+#if !ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) >+ RELEASE_ASSERT_NOT_REACHED(); >+#else > // The "dispatch a key{Down,Up} action" algorithms (§17.4 Dispatching Actions). > if (!a.pressedCharKey && b.pressedCharKey) { > LOG(Automation, "SimulatedInputDispatcher[%p]: simulating KeyPress[key=%c] for transition to %d.%d", this, b.pressedCharKey.value(), m_keyframeIndex, m_inputSourceStateIndex); >@@ -327,11 +363,7 @@ void SimulatedInputDispatcher::transitionInputSourceToState(SimulatedInputSource > } > } else > eventDispatchFinished(WTF::nullopt); >- break; >- case SimulatedInputSourceType::Touch: >- // Not supported yet. >- ASSERT_NOT_REACHED(); >- eventDispatchFinished(AUTOMATION_COMMAND_ERROR_WITH_NAME(NotImplemented)); >+#endif // !ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > break; > } > } >@@ -387,3 +419,5 @@ void SimulatedInputDispatcher::finishDispatching(Optional<AutomationCommandError > } > > } // namespace Webkit >+ >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) >diff --git a/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.h b/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.h >index 47733ac744103280070e41ffe241fb0ddda6735a..7f4141ce0a39e56073c7ec8a96560cbc348adfa6 100644 >--- a/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.h >+++ b/Source/WebKit/UIProcess/Automation/SimulatedInputDispatcher.h >@@ -1,5 +1,5 @@ > /* >- * Copyright (C) 2018 Apple Inc. All rights reserved. >+ * Copyright (C) 2018, 2019 Apple Inc. All rights reserved. > * > * Redistribution and use in source and binary forms, with or without > * modification, are permitted provided that the following conditions >@@ -25,6 +25,8 @@ > > #pragma once > >+#if ENABLE(WEBDRIVER_ACTIONS_API) >+ > #include "WebEvent.h" > #include <wtf/CompletionHandler.h> > #include <wtf/HashSet.h> >@@ -65,6 +67,12 @@ enum class SimulatedInputSourceType { > Touch, > }; > >+enum class TouchInteraction { >+ TouchDown, >+ MoveTo, >+ LiftUp, >+}; >+ > struct SimulatedInputSourceState { > Optional<CharKey> pressedCharKey; > VirtualKeySet pressedVirtualKeys; >@@ -115,8 +123,15 @@ public: > class Client { > public: > virtual ~Client() { } >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > virtual void simulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint& locationInView, AutomationCompletionHandler&&) = 0; >+#endif >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ virtual void simulateTouchInteraction(WebPageProxy&, TouchInteraction, const WebCore::IntPoint& locationInView, Optional<Seconds> duration, AutomationCompletionHandler&&) = 0; >+#endif >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > virtual void simulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, WTF::Variant<VirtualKey, CharKey>&&, AutomationCompletionHandler&&) = 0; >+#endif > virtual void viewportInViewCenterPointOfElement(WebPageProxy&, uint64_t frameID, const String& nodeHandle, Function<void (Optional<WebCore::IntPoint>, Optional<AutomationCommandError>)>&&) = 0; > }; > >@@ -167,3 +182,5 @@ private: > }; > > } // namespace WebKit >+ >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) >diff --git a/Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp b/Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp >index f8844afe6d22e4cd5ee208148b4f492f7cc6310d..20ca3e53d208fc7a4303a9a677786147b8ef7cb2 100644 >--- a/Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp >+++ b/Source/WebKit/UIProcess/Automation/WebAutomationSession.cpp >@@ -51,6 +51,7 @@ > #include <wtf/UUID.h> > #include <wtf/text/StringConcatenate.h> > >+ > namespace WebKit { > > using namespace Inspector; >@@ -78,10 +79,19 @@ WebAutomationSession::WebAutomationSession() > , m_domainNotifier(std::make_unique<AutomationFrontendDispatcher>(m_frontendRouter)) > , m_loadTimer(RunLoop::main(), this, &WebAutomationSession::loadTimerFired) > { >+#if ENABLE(WEBDRIVER_ACTIONS_API) > // Set up canonical input sources to be used for 'performInteractionSequence' and 'cancelInteractionSequence'. >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ m_inputSources.add(SimulatedInputSource::create(SimulatedInputSourceType::Touch)); >+#endif >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > m_inputSources.add(SimulatedInputSource::create(SimulatedInputSourceType::Mouse)); >+#endif >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > m_inputSources.add(SimulatedInputSource::create(SimulatedInputSourceType::Keyboard)); >+#endif > m_inputSources.add(SimulatedInputSource::create(SimulatedInputSourceType::Null)); >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) > } > > WebAutomationSession::~WebAutomationSession() >@@ -139,15 +149,19 @@ void WebAutomationSession::disconnect(Inspector::FrontendChannel& channel) > > void WebAutomationSession::terminate() > { >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > for (auto& identifier : copyToVector(m_pendingKeyboardEventsFlushedCallbacksPerPage.keys())) { > auto callback = m_pendingKeyboardEventsFlushedCallbacksPerPage.take(identifier); > callback(AUTOMATION_COMMAND_ERROR_WITH_NAME(InternalError)); > } >+#endif // ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > for (auto& identifier : copyToVector(m_pendingMouseEventsFlushedCallbacksPerPage.keys())) { > auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(identifier); > callback(AUTOMATION_COMMAND_ERROR_WITH_NAME(InternalError)); > } >+#endif // ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > > #if ENABLE(REMOTE_INSPECTOR) > if (Inspector::FrontendChannel* channel = m_remoteChannel) { >@@ -645,20 +659,24 @@ void WebAutomationSession::willShowJavaScriptDialog(WebPageProxy& page) > callback->sendFailure(unexpectedAlertOpenError); > } > } >- >+ >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > if (!m_pendingMouseEventsFlushedCallbacksPerPage.isEmpty()) { > for (auto key : copyToVector(m_pendingMouseEventsFlushedCallbacksPerPage.keys())) { > auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(key); > callback(WTF::nullopt); > } > } >+#endif // ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > if (!m_pendingKeyboardEventsFlushedCallbacksPerPage.isEmpty()) { > for (auto key : copyToVector(m_pendingKeyboardEventsFlushedCallbacksPerPage.keys())) { > auto callback = m_pendingKeyboardEventsFlushedCallbacksPerPage.take(key); > callback(WTF::nullopt); > } > } >+#endif // ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > }); > } > >@@ -776,14 +794,22 @@ void WebAutomationSession::inspectorFrontendLoaded(const WebPageProxy& page) > > void WebAutomationSession::mouseEventsFlushedForPage(const WebPageProxy& page) > { >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > if (auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(page.pageID())) > callback(WTF::nullopt); >+#else >+ UNUSED_PARAM(page); >+#endif > } > > void WebAutomationSession::keyboardEventsFlushedForPage(const WebPageProxy& page) > { >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > if (auto callback = m_pendingKeyboardEventsFlushedCallbacksPerPage.take(page.pageID())) > callback(WTF::nullopt); >+#else >+ UNUSED_PARAM(page); >+#endif > } > > void WebAutomationSession::willClosePage(const WebPageProxy& page) >@@ -793,15 +819,21 @@ void WebAutomationSession::willClosePage(const WebPageProxy& page) > > // Cancel pending interactions on this page. By providing an error, this will cause subsequent > // actions to be aborted and the SimulatedInputDispatcher::run() call will unwind and fail. >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > if (auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(page.pageID())) > callback(AUTOMATION_COMMAND_ERROR_WITH_NAME(WindowNotFound)); >- if (auto callback = m_pendingMouseEventsFlushedCallbacksPerPage.take(page.pageID())) >+#endif >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) >+ if (auto callback = m_pendingKeyboardEventsFlushedCallbacksPerPage.take(page.pageID())) > callback(AUTOMATION_COMMAND_ERROR_WITH_NAME(WindowNotFound)); >+#endif > >+#if ENABLE(WEBDRIVER_ACTIONS_API) > // Then tell the input dispatcher to cancel so timers are stopped, and let it go out of scope. > Optional<Ref<SimulatedInputDispatcher>> inputDispatcher = m_inputDispatchersByPage.take(page.pageID()); > if (inputDispatcher.hasValue()) > inputDispatcher.value()->cancel(); >+#endif > } > > static bool fileCanBeAcceptedForUpload(const String& filename, const HashSet<String>& allowedMIMETypes, const HashSet<String>& allowedFileExtensions) >@@ -1434,14 +1466,22 @@ bool WebAutomationSession::shouldAllowGetUserMediaForPage(const WebPageProxy&) c > > bool WebAutomationSession::isSimulatingUserInteraction() const > { >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > if (!m_pendingMouseEventsFlushedCallbacksPerPage.isEmpty()) > return true; >+#endif >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > if (!m_pendingKeyboardEventsFlushedCallbacksPerPage.isEmpty()) > return true; >- >+#endif >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ if (m_simulatingTouchInteraction) >+ return true; >+#endif > return false; > } > >+#if ENABLE(WEBDRIVER_ACTIONS_API) > SimulatedInputDispatcher& WebAutomationSession::inputDispatcherForPage(WebPageProxy& page) > { > return m_inputDispatchersByPage.ensure(page.pageID(), [&] { >@@ -1460,7 +1500,7 @@ SimulatedInputSource* WebAutomationSession::inputSourceForType(SimulatedInputSou > return nullptr; > } > >-// SimulatedInputDispatcher::Client API >+// MARK: SimulatedInputDispatcher::Client API > void WebAutomationSession::viewportInViewCenterPointOfElement(WebPageProxy& page, uint64_t frameID, const String& nodeHandle, Function<void (Optional<WebCore::IntPoint>, Optional<AutomationCommandError>)>&& completionHandler) > { > // Start at 3 and use only odd numbers to not conflict with m_nextComputeElementLayoutCallbackID. >@@ -1470,6 +1510,7 @@ void WebAutomationSession::viewportInViewCenterPointOfElement(WebPageProxy& page > page.process().send(Messages::WebAutomationSessionProxy::ComputeElementLayout(page.pageID(), frameID, nodeHandle, false, CoordinateSystem::LayoutViewport, callbackID), 0); > } > >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > void WebAutomationSession::simulateMouseInteraction(WebPageProxy& page, MouseInteraction interaction, WebMouseEvent::Button mouseButton, const WebCore::IntPoint& locationInViewport, CompletionHandler<void(Optional<AutomationCommandError>)>&& completionHandler) > { > WebCore::IntPoint locationInView = WebCore::IntPoint(locationInViewport.x(), locationInViewport.y() + page.topContentInset()); >@@ -1502,7 +1543,27 @@ void WebAutomationSession::simulateMouseInteraction(WebPageProxy& page, MouseInt > // Otherwise, wait for mouseEventsFlushedCallback to run when all events are handled. > }); > } >+#endif // ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+void WebAutomationSession::simulateTouchInteraction(WebPageProxy& page, TouchInteraction interaction, const WebCore::IntPoint& locationInViewport, Optional<Seconds> duration, CompletionHandler<void(Optional<AutomationCommandError>)>&& completionHandler) >+{ >+#if PLATFORM(IOS_FAMILY) >+ if (!page.unobscuredContentRect().contains(locationInViewport)) { >+ completionHandler(AUTOMATION_COMMAND_ERROR_WITH_NAME(TargetOutOfBounds)); >+ return; >+ } >+#endif >+ >+ m_simulatingTouchInteraction = true; >+ platformSimulateTouchInteraction(page, interaction, locationInViewport, duration, [this, protectedThis = makeRef(*this), completionHandler = WTFMove(completionHandler)](Optional<AutomationCommandError> error) mutable { >+ m_simulatingTouchInteraction = false; >+ completionHandler(error); >+ }); >+} >+#endif // ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > void WebAutomationSession::simulateKeyboardInteraction(WebPageProxy& page, KeyboardInteraction interaction, WTF::Variant<VirtualKey, CharKey>&& key, CompletionHandler<void(Optional<AutomationCommandError>)>&& completionHandler) > { > // Bridge the flushed callback to our command's completion handler. >@@ -1526,8 +1587,11 @@ void WebAutomationSession::simulateKeyboardInteraction(WebPageProxy& page, Keybo > > // Otherwise, wait for keyboardEventsFlushedCallback to run when all events are handled. > } >+#endif // ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > >-#if USE(APPKIT) || PLATFORM(GTK) || PLATFORM(WPE) >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) >+ >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > static WebEvent::Modifiers protocolModifierToWebEventModifier(Inspector::Protocol::Automation::KeyModifier modifier) > { > switch (modifier) { >@@ -1545,7 +1609,9 @@ static WebEvent::Modifiers protocolModifierToWebEventModifier(Inspector::Protoco > > RELEASE_ASSERT_NOT_REACHED(); > } >+#endif // ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > >+#if ENABLE(WEBDRIVER_ACTIONS_API) > static WebMouseEvent::Button protocolMouseButtonToWebMouseEventButton(Inspector::Protocol::Automation::MouseButton button) > { > switch (button) { >@@ -1561,11 +1627,11 @@ static WebMouseEvent::Button protocolMouseButtonToWebMouseEventButton(Inspector: > > RELEASE_ASSERT_NOT_REACHED(); > } >-#endif // USE(APPKIT) || PLATFORM(GTK) || PLATFORM(WPE) >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) > > void WebAutomationSession::performMouseInteraction(const String& handle, const JSON::Object& requestedPositionObject, const String& mouseButtonString, const String& mouseInteractionString, const JSON::Array& keyModifierStrings, Ref<PerformMouseInteractionCallback>&& callback) > { >-#if !USE(APPKIT) && !PLATFORM(GTK) && !PLATFORM(WPE) >+#if !ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > ASYNC_FAIL_WITH_PREDEFINED_ERROR(NotImplemented); > #else > WebPageProxy* page = webPageProxyForHandle(handle); >@@ -1634,12 +1700,12 @@ void WebAutomationSession::performMouseInteraction(const String& handle, const J > callbackToCancel(WTF::nullopt); > } > }); >-#endif // USE(APPKIT) || PLATFORM(GTK) || PLATFORM(WPE) >+#endif // ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > } > > void WebAutomationSession::performKeyboardInteractions(const String& handle, const JSON::Array& interactions, Ref<PerformKeyboardInteractionsCallback>&& callback) > { >-#if !PLATFORM(COCOA) && !PLATFORM(GTK) && !PLATFORM(WPE) >+#if !ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > ASYNC_FAIL_WITH_PREDEFINED_ERROR(NotImplemented); > #else > WebPageProxy* page = webPageProxyForHandle(handle); >@@ -1716,10 +1782,10 @@ void WebAutomationSession::performKeyboardInteractions(const String& handle, con > > for (auto& action : actionsToPerform) > action(); >-#endif // PLATFORM(COCOA) || PLATFORM(GTK) || PLATFORM(WPE) >+#endif // ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > } > >-#if USE(APPKIT) || PLATFORM(GTK) || PLATFORM(WPE) >+#if ENABLE(WEBDRIVER_ACTIONS_API) > static SimulatedInputSourceType simulatedInputSourceTypeFromProtocolSourceType(Inspector::Protocol::Automation::InputSourceType protocolType) > { > switch (protocolType) { >@@ -1735,13 +1801,13 @@ static SimulatedInputSourceType simulatedInputSourceTypeFromProtocolSourceType(I > > RELEASE_ASSERT_NOT_REACHED(); > } >-#endif // USE(APPKIT) || PLATFORM(GTK) || PLATFORM(WPE) >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) > > void WebAutomationSession::performInteractionSequence(const String& handle, const String* optionalFrameHandle, const JSON::Array& inputSources, const JSON::Array& steps, Ref<WebAutomationSession::PerformInteractionSequenceCallback>&& callback) > { > // This command implements WebKit support for §17.5 Perform Actions. > >-#if !USE(APPKIT) && !PLATFORM(GTK) && !PLATFORM(WPE) >+#if !ENABLE(WEBDRIVER_ACTIONS_API) > ASYNC_FAIL_WITH_PREDEFINED_ERROR(NotImplemented); > #else > WebPageProxy* page = webPageProxyForHandle(handle); >@@ -1778,9 +1844,26 @@ void WebAutomationSession::performInteractionSequence(const String& handle, cons > ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "An input source in the 'inputSources' parameter has an invalid 'sourceType'."); > > SimulatedInputSourceType inputSourceType = simulatedInputSourceTypeFromProtocolSourceType(*parsedInputSourceType); >+ >+ // Note: iOS does not support mouse input sources, and other platforms do not support touch input sources. >+ // If a mismatch happens, alias to the supported input source. This works because both Mouse and Touch input sources >+ // use a MouseButton to indicate the result of interacting (down/up/move), which can be interpreted for touch or mouse. >+#if !ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) && ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ if (inputSourceType == SimulatedInputSourceType::Mouse) >+ inputSourceType = SimulatedInputSourceType::Touch; >+#endif >+#if !ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) >+ if (inputSourceType == SimulatedInputSourceType::Mouse) >+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(NotImplemented, "Mouse input sources are not yet supported."); >+#endif >+#if !ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) > if (inputSourceType == SimulatedInputSourceType::Touch) > ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(NotImplemented, "Touch input sources are not yet supported."); >- >+#endif >+#if !ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) >+ if (inputSourceType == SimulatedInputSourceType::Keyboard) >+ ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(NotImplemented, "Keyboard input sources are not yet supported."); >+#endif > if (typeToSourceIdMap.contains(inputSourceType)) > ASYNC_FAIL_WITH_PREDEFINED_ERROR_AND_DETAILS(InvalidParameter, "Two input sources with the same type were specified."); > if (sourceIdToInputSourceMap.contains(sourceId)) >@@ -1892,14 +1975,14 @@ void WebAutomationSession::performInteractionSequence(const String& handle, cons > else > callback->sendSuccess(); > }); >-#endif // PLATFORM(COCOA) || PLATFORM(GTK) || PLATFORM(WPE) >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) > } > > void WebAutomationSession::cancelInteractionSequence(const String& handle, const String* optionalFrameHandle, Ref<CancelInteractionSequenceCallback>&& callback) > { > // This command implements WebKit support for §17.6 Release Actions. > >-#if !USE(APPKIT) && !PLATFORM(GTK) && !PLATFORM(WPE) >+#if !ENABLE(WEBDRIVER_ACTIONS_API) > ASYNC_FAIL_WITH_PREDEFINED_ERROR(NotImplemented); > #else > WebPageProxy* page = webPageProxyForHandle(handle); >@@ -1920,7 +2003,7 @@ void WebAutomationSession::cancelInteractionSequence(const String& handle, const > else > callback->sendSuccess(); > }); >-#endif // PLATFORM(COCOA) || PLATFORM(GTK) || PLATFORM(WPE) >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) > } > > void WebAutomationSession::takeScreenshot(const String& handle, const String* optionalFrameHandle, const String* optionalNodeHandle, const bool* optionalScrollIntoViewIfNeeded, const bool* optionalClipToViewport, Ref<TakeScreenshotCallback>&& callback) >@@ -1961,22 +2044,6 @@ void WebAutomationSession::didTakeScreenshot(uint64_t callbackID, const Shareabl > callback->sendSuccess(base64EncodedData.value()); > } > >-// Platform-dependent Implementation Stubs. >- >-#if !PLATFORM(MAC) && !PLATFORM(GTK) && !PLATFORM(WPE) >-void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint&, WebEvent::Modifiers) >-{ >-} >-#endif // !PLATFORM(MAC) && !PLATFORM(GTK) && !PLATFORM(WPE) >- >-#if !PLATFORM(COCOA) && !PLATFORM(GTK) && !PLATFORM(WPE) >- >- >-void WebAutomationSession::platformSimulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, WTF::Variant<VirtualKey, CharKey>&&) >-{ >-} >-#endif // !PLATFORM(COCOA) && !PLATFORM(GTK) && !PLATFORM(WPE) >- > #if !PLATFORM(COCOA) && !USE(CAIRO) > Optional<String> WebAutomationSession::platformGetBase64EncodedPNGData(const ShareableBitmap::Handle&) > { >diff --git a/Source/WebKit/UIProcess/Automation/WebAutomationSession.h b/Source/WebKit/UIProcess/Automation/WebAutomationSession.h >index be0afec67f578f6109eae535a3d83f8683f7201c..bad07029f547d674e623447f75f30a737c816db0 100644 >--- a/Source/WebKit/UIProcess/Automation/WebAutomationSession.h >+++ b/Source/WebKit/UIProcess/Automation/WebAutomationSession.h >@@ -99,7 +99,9 @@ class WebAutomationSession final : public API::ObjectImpl<API::Object::Type::Aut > , public Inspector::RemoteAutomationTarget > #endif > , public Inspector::AutomationBackendDispatcherHandler >+#if ENABLE(WEBDRIVER_ACTIONS_API) > , public SimulatedInputDispatcher::Client >+#endif > { > public: > WebAutomationSession(); >@@ -135,11 +137,22 @@ public: > #endif > void terminate(); > >+#if ENABLE(WEBDRIVER_ACTIONS_API) >+ > // SimulatedInputDispatcher::Client API >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > void simulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint& locationInView, AutomationCompletionHandler&&) final; >+#endif >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ void simulateTouchInteraction(WebPageProxy&, TouchInteraction, const WebCore::IntPoint& locationInView, Optional<Seconds> duration, AutomationCompletionHandler&&) final; >+#endif >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > void simulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, WTF::Variant<VirtualKey, CharKey>&&, AutomationCompletionHandler&&) final; >+#endif > void viewportInViewCenterPointOfElement(WebPageProxy&, uint64_t frameID, const String& nodeHandle, Function<void (Optional<WebCore::IntPoint>, Optional<AutomationCommandError>)>&&) final; > >+#endif // ENABLE(WEBDRIVER_ACTIONS_API) >+ > // Inspector::AutomationBackendDispatcherHandler API > // NOTE: the set of declarations included in this interface depend on the "platform" property in Automation.json > // and the --platform argument passed to the protocol bindings generator. >@@ -188,8 +201,10 @@ public: > > // Event Simulation Support. > bool isSimulatingUserInteraction() const; >+#if ENABLE(WEBDRIVER_ACTIONS_API) > SimulatedInputDispatcher& inputDispatcherForPage(WebPageProxy&); > SimulatedInputSource* inputSourceForType(SimulatedInputSourceType) const; >+#endif > > #if PLATFORM(MAC) > bool wasEventSynthesizedForAutomation(NSEvent *); >@@ -231,11 +246,20 @@ private: > void didDeleteCookie(uint64_t callbackID, const String& errorType); > > // Platform-dependent implementations. >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > void platformSimulateMouseInteraction(WebPageProxy&, MouseInteraction, WebMouseEvent::Button, const WebCore::IntPoint& locationInView, WebEvent::Modifiers keyModifiers); >+#endif >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ // Simulates a single touch point being pressed, moved, and released. >+ void platformSimulateTouchInteraction(WebPageProxy&, TouchInteraction, const WebCore::IntPoint& locationInViewport, Optional<Seconds> duration, AutomationCompletionHandler&&); >+#endif >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > // Simulates a single virtual or char key being pressed/released, such as 'a', Control, F-keys, Numpad keys, etc. as allowed by the protocol. > void platformSimulateKeyboardInteraction(WebPageProxy&, KeyboardInteraction, WTF::Variant<VirtualKey, CharKey>&&); > // Simulates key presses to produce the codepoints in a string. One or more code points are delivered atomically at grapheme cluster boundaries. > void platformSimulateKeySequence(WebPageProxy&, const String&); >+#endif // ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) >+ > // Get base64 encoded PNG data from a bitmap. > Optional<String> platformGetBase64EncodedPNGData(const ShareableBitmap::Handle&); > >@@ -267,8 +291,12 @@ private: > HashMap<uint64_t, RefPtr<Inspector::BackendDispatcher::CallbackBase>> m_pendingNormalNavigationInBrowsingContextCallbacksPerFrame; > HashMap<uint64_t, RefPtr<Inspector::BackendDispatcher::CallbackBase>> m_pendingEagerNavigationInBrowsingContextCallbacksPerFrame; > HashMap<uint64_t, RefPtr<Inspector::BackendDispatcher::CallbackBase>> m_pendingInspectorCallbacksPerPage; >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > HashMap<uint64_t, Function<void(Optional<AutomationCommandError>)>> m_pendingKeyboardEventsFlushedCallbacksPerPage; >+#endif >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > HashMap<uint64_t, Function<void(Optional<AutomationCommandError>)>> m_pendingMouseEventsFlushedCallbacksPerPage; >+#endif > > uint64_t m_nextEvaluateJavaScriptCallbackID { 1 }; > HashMap<uint64_t, RefPtr<Inspector::AutomationBackendDispatcherHandler::EvaluateJavaScriptFunctionCallback>> m_evaluateJavaScriptFunctionCallbacks; >@@ -310,14 +338,20 @@ private: > > bool m_permissionForGetUserMedia { true }; > >- // Keep track of currently active modifiers across multiple keystrokes. >- // Most platforms do not track current modifiers from synthesized events. >- unsigned m_currentModifiers { 0 }; >- >+#if ENABLE(WEBDRIVER_ACTIONS_API) > // SimulatedInputDispatcher APIs take a set of input sources. We also intern these > // so that previous input source state is used as initial state for later commands. > HashSet<Ref<SimulatedInputSource>> m_inputSources; > HashMap<uint64_t, Ref<SimulatedInputDispatcher>> m_inputDispatchersByPage; >+#endif >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) >+ // Keep track of currently active modifiers across multiple keystrokes. >+ // Most platforms do not track current modifiers from synthesized events. >+ unsigned m_currentModifiers { 0 }; >+#endif >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+ bool m_simulatingTouchInteraction { false }; >+#endif > > #if ENABLE(REMOTE_INSPECTOR) > Inspector::FrontendChannel* m_remoteChannel { nullptr }; >diff --git a/Source/WebKit/UIProcess/Automation/ios/WebAutomationSessionIOS.mm b/Source/WebKit/UIProcess/Automation/ios/WebAutomationSessionIOS.mm >index c87f5674346f5abe2e6a850ddb27a7f4a34c4770..48ba280e3a84c1d3a87c4358438b9cbe099bcc99 100644 >--- a/Source/WebKit/UIProcess/Automation/ios/WebAutomationSessionIOS.mm >+++ b/Source/WebKit/UIProcess/Automation/ios/WebAutomationSessionIOS.mm >@@ -1,5 +1,5 @@ > /* >- * Copyright (C) 2017 Apple Inc. All rights reserved. >+ * Copyright (C) 2017, 2019 Apple Inc. All rights reserved. > * > * Redistribution and use in source and binary forms, with or without > * modification, are permitted provided that the following conditions >@@ -31,9 +31,11 @@ > #import "NativeWebKeyboardEvent.h" > #import "WebAutomationSessionMacros.h" > #import "WebPageProxy.h" >+#import "_WKTouchEventGenerator.h" > #import <WebCore/KeyEventCodesIOS.h> > #import <WebCore/NotImplemented.h> > #import <WebCore/WebEvent.h> >+#import <wtf/BlockPtr.h> > > namespace WebKit { > using namespace WebCore; >@@ -64,6 +66,7 @@ void WebAutomationSession::sendSynthesizedEventsToPage(WebPageProxy& page, NSArr > > #pragma mark Commands for Platform: 'iOS' > >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > void WebAutomationSession::platformSimulateKeyboardInteraction(WebPageProxy& page, KeyboardInteraction interaction, WTF::Variant<VirtualKey, CharKey>&& key) > { > // The modifiers changed by the virtual key when it is pressed or released. >@@ -169,6 +172,31 @@ void WebAutomationSession::platformSimulateKeySequence(WebPageProxy& page, const > > sendSynthesizedEventsToPage(page, eventsToBeSent.get()); > } >+#endif // ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) >+ >+#if ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) >+void WebAutomationSession::platformSimulateTouchInteraction(WebPageProxy& page, TouchInteraction interaction, const WebCore::IntPoint& locationInViewport, Optional<Seconds> duration, AutomationCompletionHandler&& completionHandler) >+{ >+ WebCore::IntPoint locationOnScreen = page.syncRootViewToScreen(IntRect(locationInViewport, IntSize())).location(); >+ _WKTouchEventGenerator *generator = [_WKTouchEventGenerator sharedTouchEventGenerator]; >+ >+ auto interactionFinished = makeBlockPtr([completionHandler = WTFMove(completionHandler)] () mutable { >+ completionHandler(WTF::nullopt); >+ }); >+ >+ switch (interaction) { >+ case TouchInteraction::TouchDown: >+ [generator touchDown:locationOnScreen completionBlock:interactionFinished.get()]; >+ break; >+ case TouchInteraction::LiftUp: >+ [generator liftUp:locationOnScreen completionBlock:interactionFinished.get()]; >+ break; >+ case TouchInteraction::MoveTo: >+ [generator moveToPoint:locationOnScreen duration:duration.valueOr(0_s).seconds() completionBlock:interactionFinished.get()]; >+ break; >+ } >+} >+#endif // ENABLE(WEBDRIVER_TOUCH_INTERACTIONS) > > } // namespace WebKit > >diff --git a/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm b/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm >index dfc909316e250d511c6537e6d994147985808af7..d7f65ae43976f4be795e5efe2d1d6ed55f6adee0 100644 >--- a/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm >+++ b/Source/WebKit/UIProcess/Automation/mac/WebAutomationSessionMac.mm >@@ -120,6 +120,7 @@ bool WebAutomationSession::wasEventSynthesizedForAutomation(NSEvent *event) > > #pragma mark Platform-dependent Implementations > >+#if ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) > void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy& page, MouseInteraction interaction, WebMouseEvent::Button button, const WebCore::IntPoint& locationInView, WebEvent::Modifiers keyModifiers) > { > IntRect windowRect; >@@ -210,6 +211,9 @@ void WebAutomationSession::platformSimulateMouseInteraction(WebPageProxy& page, > > sendSynthesizedEventsToPage(page, eventsToBeSent.get()); > } >+#endif // ENABLE(WEBDRIVER_MOUSE_INTERACTIONS) >+ >+#if ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) > > static bool virtualKeyHasStickyModifier(VirtualKey key) > { >@@ -719,6 +723,8 @@ void WebAutomationSession::platformSimulateKeySequence(WebPageProxy& page, const > sendSynthesizedEventsToPage(page, eventsToBeSent.get()); > } > >+#endif // ENABLE(WEBDRIVER_KEYBOARD_INTERACTIONS) >+ > } // namespace WebKit > > #endif // PLATFORM(MAC) >diff --git a/Source/WebKit/UIProcess/_WKTouchEventGenerator.h b/Source/WebKit/UIProcess/_WKTouchEventGenerator.h >new file mode 100644 >index 0000000000000000000000000000000000000000..12829d0cf0067ebf3d474c69e32a7d4bc35ecea6 >--- /dev/null >+++ b/Source/WebKit/UIProcess/_WKTouchEventGenerator.h >@@ -0,0 +1,48 @@ >+/* >+ * Copyright (C) 2015, 2019 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#if TARGET_OS_IPHONE >+ >+#import <CoreGraphics/CGGeometry.h> >+#import <WebKit/WKFoundation.h> >+ >+NS_ASSUME_NONNULL_BEGIN >+ >+typedef struct __IOHIDEvent * IOHIDEventRef; >+ >+WK_CLASS_AVAILABLE(ios(WK_IOS_TBA)) >+@interface _WKTouchEventGenerator : NSObject >++ (_WKTouchEventGenerator *)sharedTouchEventGenerator; >+ >+- (void)touchDown:(CGPoint)location completionBlock:(void (^)(void))completionBlock; >+- (void)liftUp:(CGPoint)location completionBlock:(void (^)(void))completionBlock; >+- (void)moveToPoint:(CGPoint)location duration:(NSTimeInterval)seconds completionBlock:(void (^)(void))completionBlock; >+ >+- (void)receivedHIDEvent:(IOHIDEventRef)event; >+@end >+ >+NS_ASSUME_NONNULL_END >+ >+#endif // TARGET_OS_IPHONE >diff --git a/Source/WebKit/UIProcess/_WKTouchEventGenerator.mm b/Source/WebKit/UIProcess/_WKTouchEventGenerator.mm >new file mode 100644 >index 0000000000000000000000000000000000000000..4f4a1f07f31dd2f1faf569ccf35327853ab5e653 >--- /dev/null >+++ b/Source/WebKit/UIProcess/_WKTouchEventGenerator.mm >@@ -0,0 +1,407 @@ >+/* >+ * Copyright (C) 2015, 2019 Apple Inc. All rights reserved. >+ * >+ * Redistribution and use in source and binary forms, with or without >+ * modification, are permitted provided that the following conditions >+ * are met: >+ * 1. Redistributions of source code must retain the above copyright >+ * notice, this list of conditions and the following disclaimer. >+ * 2. Redistributions in binary form must reproduce the above copyright >+ * notice, this list of conditions and the following disclaimer in the >+ * documentation and/or other materials provided with the distribution. >+ * >+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' >+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, >+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR >+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS >+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR >+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF >+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS >+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN >+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) >+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF >+ * THE POSSIBILITY OF SUCH DAMAGE. >+ */ >+ >+#import "config.h" >+#import "_WKTouchEventGenerator.h" >+ >+#if PLATFORM(IOS_FAMILY) >+ >+#import "UIKitSPI.h" >+#import <mach/mach_time.h> >+#import <pal/spi/cocoa/IOKitSPI.h> >+#import <wtf/Assertions.h> >+#import <wtf/Optional.h> >+#import <wtf/RetainPtr.h> >+#import <wtf/SoftLinking.h> >+ >+SOFT_LINK_PRIVATE_FRAMEWORK(BackBoardServices) >+SOFT_LINK(BackBoardServices, BKSHIDEventSetDigitizerInfo, void, (IOHIDEventRef digitizerEvent, uint32_t contextID, uint8_t systemGestureisPossible, uint8_t isSystemGestureStateChangeEvent, CFStringRef displayUUID, CFTimeInterval initialTouchTimestamp, float maxForce), (digitizerEvent, contextID, systemGestureisPossible, isSystemGestureStateChangeEvent, displayUUID, initialTouchTimestamp, maxForce)); >+ >+static const NSTimeInterval fingerMoveInterval = 0.016; >+static const IOHIDFloat defaultMajorRadius = 5; >+static const IOHIDFloat defaultPathPressure = 0; >+static const long nanosecondsPerSecond = 1e9; >+ >+NSUInteger const HIDMaxTouchCount = 5; >+ >+static int fingerIdentifiers[HIDMaxTouchCount] = { 2, 3, 4, 5, 1 }; >+ >+typedef enum { >+ HandEventNull, >+ HandEventTouched, >+ HandEventMoved, >+ HandEventChordChanged, >+ HandEventLifted, >+ HandEventCanceled, >+} HandEventType; >+ >+typedef struct { >+ int identifier; >+ CGPoint point; >+ IOHIDFloat pathMajorRadius; >+ IOHIDFloat pathPressure; >+ UInt8 pathProximity; >+} SyntheticEventDigitizerInfo; >+ >+static CFTimeInterval secondsSinceAbsoluteTime(CFAbsoluteTime startTime) >+{ >+ return (CFAbsoluteTimeGetCurrent() - startTime); >+} >+ >+static double simpleCurveInterpolation(double a, double b, double t) >+{ >+ return (a + (b - a) * sin(sin(t * M_PI / 2) * t * M_PI / 2)); >+} >+ >+static CGPoint calculateNextCurveLocation(CGPoint a, CGPoint b, CFTimeInterval t) >+{ >+ return CGPointMake(simpleCurveInterpolation(a.x, b.x, t), simpleCurveInterpolation(a.y, b.y, t)); >+} >+ >+typedef double(*pressureInterpolationFunction)(double, double, CFTimeInterval); >+ >+static void delayBetweenMove(int eventIndex, double elapsed) >+{ >+ // Delay next event until expected elapsed time. >+ double delay = (eventIndex * fingerMoveInterval) - elapsed; >+ if (delay > 0) { >+ struct timespec moveDelay = { 0, static_cast<long>(delay * nanosecondsPerSecond) }; >+ nanosleep(&moveDelay, NULL); >+ } >+} >+ >+// NOTE: this event synthesizer is derived from WebKitTestRunner code. >+// Compared to that version, this lacks support for stylus event simulation, >+// event stream, and only single touches are exposed via the touch/lift/move method calls. >+@interface _WKTouchEventGenerator () >+@property (nonatomic, strong) NSMutableDictionary *eventCallbacks; >+@end >+ >+@implementation _WKTouchEventGenerator { >+ IOHIDEventSystemClientRef _ioSystemClient; >+ SyntheticEventDigitizerInfo _activePoints[HIDMaxTouchCount]; >+ NSUInteger _activePointCount; >+} >+ >++ (_WKTouchEventGenerator *)sharedTouchEventGenerator >+{ >+ static _WKTouchEventGenerator *eventGenerator = nil; >+ if (!eventGenerator) >+ eventGenerator = [[_WKTouchEventGenerator alloc] init]; >+ >+ return eventGenerator; >+} >+ >++ (CFIndex)nextEventCallbackID >+{ >+ static CFIndex callbackID = 0; >+ return ++callbackID; >+} >+ >+- (instancetype)init >+{ >+ self = [super init]; >+ if (!self) >+ return nil; >+ >+ for (NSUInteger i = 0; i < HIDMaxTouchCount; ++i) >+ _activePoints[i].identifier = fingerIdentifiers[i]; >+ >+ _eventCallbacks = [[NSMutableDictionary alloc] init]; >+ >+ return self; >+} >+ >+- (void)dealloc >+{ >+ [_eventCallbacks release]; >+ [super dealloc]; >+} >+ >+- (IOHIDEventRef)_createIOHIDEventType:(HandEventType)eventType >+{ >+ BOOL isTouching = (eventType == HandEventTouched || eventType == HandEventMoved || eventType == HandEventChordChanged); >+ >+ IOHIDDigitizerEventMask eventMask = kIOHIDDigitizerEventTouch; >+ if (eventType == HandEventMoved) { >+ eventMask &= ~kIOHIDDigitizerEventTouch; >+ eventMask |= kIOHIDDigitizerEventPosition; >+ eventMask |= kIOHIDDigitizerEventAttribute; >+ } else if (eventType == HandEventChordChanged) { >+ eventMask |= kIOHIDDigitizerEventPosition; >+ eventMask |= kIOHIDDigitizerEventAttribute; >+ } else if (eventType == HandEventTouched || eventType == HandEventCanceled || eventType == HandEventLifted) >+ eventMask |= kIOHIDDigitizerEventIdentity; >+ >+ uint64_t machTime = mach_absolute_time(); >+ RetainPtr<IOHIDEventRef> eventRef = adoptCF(IOHIDEventCreateDigitizerEvent(kCFAllocatorDefault, machTime, >+ kIOHIDDigitizerTransducerTypeHand, >+ 0, >+ 0, >+ eventMask, >+ 0, >+ 0, 0, 0, >+ 0, >+ 0, >+ 0, >+ isTouching, >+ kIOHIDEventOptionNone)); >+ >+ IOHIDEventSetIntegerValue(eventRef.get(), kIOHIDEventFieldDigitizerIsDisplayIntegrated, 1); >+ >+ for (NSUInteger i = 0; i < _activePointCount; ++i) { >+ SyntheticEventDigitizerInfo* pointInfo = &_activePoints[i]; >+ if (eventType == HandEventTouched) { >+ if (!pointInfo->pathMajorRadius) >+ pointInfo->pathMajorRadius = defaultMajorRadius; >+ if (!pointInfo->pathPressure) >+ pointInfo->pathPressure = defaultPathPressure; >+ if (!pointInfo->pathProximity) >+ pointInfo->pathProximity = kGSEventPathInfoInTouch | kGSEventPathInfoInRange; >+ } else if (eventType == HandEventLifted || eventType == HandEventCanceled) { >+ pointInfo->pathMajorRadius = 0; >+ pointInfo->pathPressure = 0; >+ pointInfo->pathProximity = 0; >+ } >+ >+ CGPoint point = pointInfo->point; >+ point = CGPointMake(roundf(point.x), roundf(point.y)); >+ >+ RetainPtr<IOHIDEventRef> subEvent = adoptCF(IOHIDEventCreateDigitizerFingerEvent(kCFAllocatorDefault, machTime, >+ pointInfo->identifier, >+ pointInfo->identifier, >+ eventMask, >+ point.x, point.y, 0, >+ pointInfo->pathPressure, >+ 0, >+ pointInfo->pathProximity & kGSEventPathInfoInRange, >+ pointInfo->pathProximity & kGSEventPathInfoInTouch, >+ kIOHIDEventOptionNone)); >+ >+ IOHIDEventSetFloatValue(subEvent.get(), kIOHIDEventFieldDigitizerMajorRadius, pointInfo->pathMajorRadius); >+ IOHIDEventSetFloatValue(subEvent.get(), kIOHIDEventFieldDigitizerMinorRadius, pointInfo->pathMajorRadius); >+ >+ IOHIDEventAppendEvent(eventRef.get(), subEvent.get(), 0); >+ } >+ >+ return eventRef.leakRef(); >+} >+ >+- (BOOL)_sendHIDEvent:(IOHIDEventRef)eventRef >+{ >+ if (!_ioSystemClient) >+ _ioSystemClient = IOHIDEventSystemClientCreate(kCFAllocatorDefault); >+ >+ if (eventRef) { >+ RetainPtr<IOHIDEventRef> strongEvent = eventRef; >+ dispatch_async(dispatch_get_main_queue(), ^{ >+ uint32_t contextID = [UIApplication sharedApplication].keyWindow._contextId; >+ ASSERT(contextID); >+ BKSHIDEventSetDigitizerInfo(strongEvent.get(), contextID, false, false, NULL, 0, 0); >+ [[UIApplication sharedApplication] _enqueueHIDEvent:strongEvent.get()]; >+ }); >+ } >+ return YES; >+} >+ >+- (BOOL)_sendMarkerHIDEventWithCompletionBlock:(void (^)(void))completionBlock >+{ >+ auto callbackID = [_WKTouchEventGenerator nextEventCallbackID]; >+ [_eventCallbacks setObject:Block_copy(completionBlock) forKey:@(callbackID)]; >+ >+ auto markerEvent = adoptCF(IOHIDEventCreateVendorDefinedEvent(kCFAllocatorDefault, >+ mach_absolute_time(), >+ kHIDPage_VendorDefinedStart + 100, >+ 0, >+ 1, >+ (uint8_t*)&callbackID, >+ sizeof(CFIndex), >+ kIOHIDEventOptionNone)); >+ >+ if (markerEvent) { >+ dispatch_async(dispatch_get_main_queue(), [markerEvent = WTFMove(markerEvent)] { >+ auto contextID = [UIApplication sharedApplication].keyWindow._contextId; >+ ASSERT(contextID); >+ BKSHIDEventSetDigitizerInfo(markerEvent.get(), contextID, false, false, NULL, 0, 0); >+ [[UIApplication sharedApplication] _enqueueHIDEvent:markerEvent.get()]; >+ }); >+ } >+ return YES; >+} >+ >+- (void)_updateTouchPoints:(CGPoint*)points count:(NSUInteger)count >+{ >+ HandEventType handEventType; >+ >+ // The hand event type is based on previous state. >+ if (!_activePointCount) >+ handEventType = HandEventTouched; >+ else if (!count) >+ handEventType = HandEventLifted; >+ else if (count == _activePointCount) >+ handEventType = HandEventMoved; >+ else >+ handEventType = HandEventChordChanged; >+ >+ // Update previous count for next event. >+ _activePointCount = count; >+ >+ // Update point locations. >+ for (NSUInteger i = 0; i < count; ++i) >+ _activePoints[i].point = points[i]; >+ >+ RetainPtr<IOHIDEventRef> eventRef = adoptCF([self _createIOHIDEventType:handEventType]); >+ [self _sendHIDEvent:eventRef.get()]; >+} >+ >+- (void)touchDownAtPoints:(CGPoint*)locations touchCount:(NSUInteger)touchCount >+{ >+ touchCount = std::min(touchCount, HIDMaxTouchCount); >+ >+ _activePointCount = touchCount; >+ >+ for (NSUInteger index = 0; index < touchCount; ++index) >+ _activePoints[index].point = locations[index]; >+ >+ RetainPtr<IOHIDEventRef> eventRef = adoptCF([self _createIOHIDEventType:HandEventTouched]); >+ [self _sendHIDEvent:eventRef.get()]; >+} >+ >+- (void)touchDown:(CGPoint)location touchCount:(NSUInteger)touchCount >+{ >+ touchCount = std::min(touchCount, HIDMaxTouchCount); >+ >+ CGPoint locations[touchCount]; >+ >+ for (NSUInteger index = 0; index < touchCount; ++index) >+ locations[index] = location; >+ >+ [self touchDownAtPoints:locations touchCount:touchCount]; >+} >+ >+- (void)touchDown:(CGPoint)location >+{ >+ [self touchDownAtPoints:&location touchCount:1]; >+} >+ >+- (void)liftUpAtPoints:(CGPoint*)locations touchCount:(NSUInteger)touchCount >+{ >+ touchCount = std::min(touchCount, HIDMaxTouchCount); >+ touchCount = std::min(touchCount, _activePointCount); >+ >+ NSUInteger newPointCount = _activePointCount - touchCount; >+ >+ for (NSUInteger index = 0; index < touchCount; ++index) >+ _activePoints[newPointCount + index].point = locations[index]; >+ >+ RetainPtr<IOHIDEventRef> eventRef = adoptCF([self _createIOHIDEventType:HandEventLifted]); >+ [self _sendHIDEvent:eventRef.get()]; >+ >+ _activePointCount = newPointCount; >+} >+ >+- (void)liftUp:(CGPoint)location touchCount:(NSUInteger)touchCount >+{ >+ touchCount = std::min(touchCount, HIDMaxTouchCount); >+ >+ CGPoint locations[touchCount]; >+ >+ for (NSUInteger index = 0; index < touchCount; ++index) >+ locations[index] = location; >+ >+ [self liftUpAtPoints:locations touchCount:touchCount]; >+} >+ >+- (void)liftUp:(CGPoint)location >+{ >+ [self liftUp:location touchCount:1]; >+} >+ >+- (void)moveToPoints:(CGPoint*)newLocations touchCount:(NSUInteger)touchCount duration:(NSTimeInterval)seconds >+{ >+ touchCount = std::min(touchCount, HIDMaxTouchCount); >+ >+ CGPoint startLocations[touchCount]; >+ CGPoint nextLocations[touchCount]; >+ >+ CFAbsoluteTime startTime = CFAbsoluteTimeGetCurrent(); >+ CFTimeInterval elapsed = 0; >+ >+ int eventIndex = 0; >+ while (elapsed < (seconds - fingerMoveInterval)) { >+ elapsed = secondsSinceAbsoluteTime(startTime); >+ CFTimeInterval interval = elapsed / seconds; >+ >+ for (NSUInteger i = 0; i < touchCount; ++i) { >+ if (!eventIndex) >+ startLocations[i] = _activePoints[i].point; >+ >+ nextLocations[i] = calculateNextCurveLocation(startLocations[i], newLocations[i], interval); >+ } >+ [self _updateTouchPoints:nextLocations count:touchCount]; >+ >+ delayBetweenMove(eventIndex++, elapsed); >+ } >+ >+ [self _updateTouchPoints:newLocations count:touchCount]; >+} >+ >+- (void)touchDown:(CGPoint)location completionBlock:(void (^)(void))completionBlock >+{ >+ [self touchDown:location touchCount:1]; >+ [self _sendMarkerHIDEventWithCompletionBlock:completionBlock]; >+} >+ >+- (void)liftUp:(CGPoint)location completionBlock:(void (^)(void))completionBlock >+{ >+ [self liftUp:location touchCount:1]; >+ [self _sendMarkerHIDEventWithCompletionBlock:completionBlock]; >+} >+ >+- (void)moveToPoint:(CGPoint)location duration:(NSTimeInterval)seconds completionBlock:(void (^)(void))completionBlock >+{ >+ CGPoint locations[1]; >+ locations[0] = location; >+ [self moveToPoints:locations touchCount:1 duration:seconds]; >+ [self _sendMarkerHIDEventWithCompletionBlock:completionBlock]; >+} >+ >+- (void)receivedHIDEvent:(IOHIDEventRef)event >+{ >+ if (IOHIDEventGetType(event) != kIOHIDEventTypeVendorDefined) >+ return; >+ >+ CFIndex callbackID = IOHIDEventGetIntegerValue(event, kIOHIDEventFieldVendorDefinedData); >+ void (^completionBlock)() = [_eventCallbacks objectForKey:@(callbackID)]; >+ if (completionBlock) { >+ [_eventCallbacks removeObjectForKey:@(callbackID)]; >+ completionBlock(); >+ Block_release(completionBlock); >+ } >+} >+ >+@end >+ >+#endif // PLATFORM(IOS_FAMILY) >diff --git a/Source/WebKit/WebKit.xcodeproj/project.pbxproj b/Source/WebKit/WebKit.xcodeproj/project.pbxproj >index 46eab2b89f639b5220efd4c7add12f27e0827ffd..dc8744ec3f7cb467eb050b3f26564f6475063424 100644 >--- a/Source/WebKit/WebKit.xcodeproj/project.pbxproj >+++ b/Source/WebKit/WebKit.xcodeproj/project.pbxproj >@@ -1261,6 +1261,8 @@ > 99C81D5A1C20E7E2005C4C82 /* AutomationClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 99C81D551C20DFBE005C4C82 /* AutomationClient.h */; }; > 99C81D5D1C21F38B005C4C82 /* APIAutomationClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 99C81D5B1C20E817005C4C82 /* APIAutomationClient.h */; }; > 99E714C51C124A0400665B3A /* _WKAutomationDelegate.h in Headers */ = {isa = PBXBuildFile; fileRef = 99E714C11C1249E600665B3A /* _WKAutomationDelegate.h */; settings = {ATTRIBUTES = (Private, ); }; }; >+ 99E7189A21F79D9E0055E975 /* _WKTouchEventGenerator.mm in Sources */ = {isa = PBXBuildFile; fileRef = 99E7189621F79D9D0055E975 /* _WKTouchEventGenerator.mm */; }; >+ 99E7189C21F79D9E0055E975 /* _WKTouchEventGenerator.h in Headers */ = {isa = PBXBuildFile; fileRef = 99E7189821F79D9E0055E975 /* _WKTouchEventGenerator.h */; settings = {ATTRIBUTES = (Private, ); }; }; > 9FB5F395169E6A80002C25BF /* WKContextPrivateMac.h in Headers */ = {isa = PBXBuildFile; fileRef = 9FB5F393169E6A80002C25BF /* WKContextPrivateMac.h */; settings = {ATTRIBUTES = (Private, ); }; }; > A102A7081EC0EEE900D81D82 /* com.macromedia.Flash Player ESR.plugin.sb in Copy Plug-in Sandbox Profiles */ = {isa = PBXBuildFile; fileRef = 7A5E39491D5BD8A700B4B7CE /* com.macromedia.Flash Player ESR.plugin.sb */; }; > A1046EA12079263100F0C5D8 /* WKPDFView.h in Headers */ = {isa = PBXBuildFile; fileRef = A1046E9F2079263100F0C5D8 /* WKPDFView.h */; }; >@@ -3803,6 +3805,8 @@ > 99C81D5B1C20E817005C4C82 /* APIAutomationClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = APIAutomationClient.h; sourceTree = "<group>"; }; > 99CA66C8203668220074F35E /* EnterFullscreen.js */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.javascript; path = EnterFullscreen.js; sourceTree = "<group>"; }; > 99E714C11C1249E600665B3A /* _WKAutomationDelegate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = _WKAutomationDelegate.h; sourceTree = "<group>"; }; >+ 99E7189621F79D9D0055E975 /* _WKTouchEventGenerator.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = _WKTouchEventGenerator.mm; sourceTree = "<group>"; }; >+ 99E7189821F79D9E0055E975 /* _WKTouchEventGenerator.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = _WKTouchEventGenerator.h; sourceTree = "<group>"; }; > 99F642D21FABE378009621E9 /* CoordinateSystem.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CoordinateSystem.h; sourceTree = "<group>"; }; > 9BC59D6C1EFCCCB6001E8D09 /* CallbackID.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = CallbackID.h; sourceTree = "<group>"; }; > 9BC59D6D1EFCDC6D001E8D09 /* OptionalCallbackID.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = OptionalCallbackID.h; sourceTree = "<group>"; }; >@@ -5803,6 +5807,8 @@ > children = ( > C54256AE18BEC16100DE4179 /* forms */, > CDC2831A201BD75600E6E745 /* fullscreen */, >+ 99E7189821F79D9E0055E975 /* _WKTouchEventGenerator.h */, >+ 99E7189621F79D9D0055E975 /* _WKTouchEventGenerator.mm */, > A115DC6E191D82AB00DA8072 /* _WKWebViewPrintFormatter.h */, > A115DC6D191D82AB00DA8072 /* _WKWebViewPrintFormatter.mm */, > A19DD3BF1D07D16800AC823B /* _WKWebViewPrintFormatterInternal.h */, >@@ -9259,6 +9265,7 @@ > 2DDF731518E95060004F5A66 /* RemoteLayerBackingStoreCollection.h in Headers */, > 1AB16AEA164B3A8800290D62 /* RemoteLayerTreeContext.h in Headers */, > 2D29ECD0192F2C2E00984B78 /* RemoteLayerTreeDisplayRefreshMonitor.h in Headers */, >+ 99E7189C21F79D9E0055E975 /* _WKTouchEventGenerator.h in Headers */, > 1AB16ADE1648598400290D62 /* RemoteLayerTreeDrawingArea.h in Headers */, > 1AB16AE21648656D00290D62 /* RemoteLayerTreeDrawingAreaProxy.h in Headers */, > 0FF24A2E1879E4BC003ABF0C /* RemoteLayerTreeDrawingAreaProxyMessages.h in Headers */, >@@ -10825,6 +10832,7 @@ > 2D92A795212B6AD400F493FD /* PluginProcessCreationParameters.cpp in Sources */, > 1AA4792312A59FD9008236C3 /* PluginProcessMac.mm in Sources */, > 1A043CEB124FE38F00FFBFB5 /* PluginProcessMessageReceiver.cpp in Sources */, >+ 99E7189A21F79D9E0055E975 /* _WKTouchEventGenerator.mm in Sources */, > 1A043B5D124D5E9D00FFBFB5 /* PluginProcessProxyMessageReceiver.cpp in Sources */, > 2D91344C212CF9F000128AFD /* PluginProxy.cpp in Sources */, > 1A8EFA701252B84100F7067F /* PluginProxyMessageReceiver.cpp in Sources */, >diff --git a/Source/WebKit/config.h b/Source/WebKit/config.h >index d9e8d69951aa3e71a60d6851e655d700e71e8d71..65dcb3b0ff5ba7c3f8f3482ac0f93e7dc071486a 100644 >--- a/Source/WebKit/config.h >+++ b/Source/WebKit/config.h >@@ -79,3 +79,7 @@ > #endif > #endif > >+#define ENABLE_WEBDRIVER_TOUCH_INTERACTIONS PLATFORM(IOS_FAMILY) >+#define ENABLE_WEBDRIVER_MOUSE_INTERACTIONS PLATFORM(MAC) || PLATFORM(GTK) || PLATFORM(WPE) >+#define ENABLE_WEBDRIVER_KEYBOARD_INTERACTIONS PLATFORM(COCOA) || PLATFORM(GTK) || PLATFORM(WPE) >+#define ENABLE_WEBDRIVER_ACTIONS_API ENABLE_WEBDRIVER_TOUCH_INTERACTIONS || ENABLE_WEBDRIVER_MOUSE_INTERACTIONS || ENABLE_WEBDRIVER_KEYBOARD_INTERACTIONS
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Diff
View Attachment As Raw
Flags:
joepeck
:
review+
Actions:
View
|
Formatted Diff
|
Diff
Attachments on
bug 193852
: 360194