198040 2019-05-20 07:46:54 -0700 getDisplayMedia fails when called from a promise created during a user gesture 2019-05-24 15:00:55 -0700 1 1 1 Unclassified WebKit WebRTC Safari Technology Preview Unspecified Unspecified NEW InRadar P2 Normal --- 1 philipp.weissensteiner webkit-unassigned eric.carlson ggaren webkit-bug-importer youennf oldest_to_newest 1537572 0 370251 philipp.weissensteiner 2019-05-20 07:46:54 -0700 Created attachment 370251 getDisplayMedia example Hi, Since Safari Tech Preview 82 > getDisplayMedia must be called from a user gesture handler. If said handler has an async call to say getUserMedia or enumerateDevices, before invoking getDisplayMedia, the call will always fail with the above message. ```javascript // my user gesture handler is called after a button is clicked async function myUserGestureHandler() { // both will cause a failure // const camera = await navigator.mediaDevices.getUserMedia({ video: true }); const devices = await navigator.mediaDevices.enumerateDevices(); // calling getUserMedia here works const screen = await navigator.mediaDevices.getDisplayMedia({ video: true }); } ``` - Invoking getUserMedia works. - Recent Firefox and Chrome versions work too. - Worked in Safari Tech Preview 81. I've attached an example page demonstrating the issue. 1537611 1 webkit-bug-importer 2019-05-20 10:31:45 -0700 <rdar://problem/50950989> 1537626 2 youennf 2019-05-20 10:52:38 -0700 Hi Philipp, I guess a workaround is to do: const gumPromise = navigator.mediaDevices.getUserMedia({ video: true }); const screenPromise = await anavigator.mediaDevices.getDisplayMedia({ video: true }); const camera = await gumPromise; Would that work for you? Or do you want to NOT request getDisplayMedia access if getUserMedia is rejected? I am also not sure why you want to do enumerateDevices there. FWIW, it will, in most cases, trigger two prompts to the user in a very short amount of time. I wonder whether this is a good application flow. 1537836 3 philipp.weissensteiner 2019-05-21 02:41:22 -0700 Hi! Thanks for the quick response. Let's say we want to build a screen sharing app where we mix the camera and the screen stream. We'd have a button "share screen" and invoke the two .getMedia functions in a handler. Is that handler function not a "user gesture handler"? My main concern is that "user gesture handler" is ambiguous. Can you clarify what a user gesture handler is and why the example code I posted, doesn't qualify as such? If I invoke a different Promise instead of getUserMedia or enumarteDevices, e.g: const examplePromise = () => { return new Promise(res => window.setTimeout(() => res(), 500)); }; A subsequent call to getDisplayMedia works just fine, but increasing the delay to 1s causes the error message too. The work-around is viable, but I'd it's still consider it a "work-around". It's not clear why you'd have to invoke gDM before gUM, right? I'd probably have to add a comment "order is important here!" so I won't forget it in 6 months :). (As far as enumarteDevices goes, that was just an example). Hope that clarifies my issue. 1538464 4 youennf 2019-05-22 16:41:23 -0700 > A subsequent call to getDisplayMedia works just fine, but increasing > the delay to 1s causes the error message too. We currently bound the user gesture duration to 1s. > The work-around is viable, but I'd it's still consider it a "work-around". > It's not clear why you'd have to invoke gDM before gUM, right? My example is invoking gum first, then gdm. It is not waiting for gum response to call gdm response. The questions might be: 1. Can your application live with just screen, just camera or require both? 2. Do you want to ask the user for gdm in case gum is rejected? The current design does not allow you to not ask gdm if gum is rejected without another user gesture. Note though that two prompts that are happening shortly one after the other might be confusing to the user. A more user friendly flow might be to enter a room and at that time trigger gum or gdm. Then have the user interact with the UI to enable the other one. 1538627 5 philipp.weissensteiner 2019-05-23 03:10:28 -0700 > We currently bound the user gesture duration to 1s. Thanks for the clarification. > My example is invoking gum first, then gdm. > It is not waiting for gum response to call gdm response. My bad. The point is that the order is important, because of the user gesture restrictions. > 1. Can your application live with just screen, just camera or require both? For this application I'd require both. > 2. Do you want to ask the user for gdm in case gum is rejected? No. You're absolutely right, the current flow could be improved in that way. I think a clarification for the behaviour might be good. Maybe something like: > getDisplayMedia must be called from a user gesture handler. Learn more at: > https://bugs.webkit.org/show_bug.cgi?id=198040 or whatever the link might be. 370251 2019-05-20 07:46:54 -0700 2019-05-20 07:46:54 -0700 getDisplayMedia example index.html text/html 911 philipp.weissensteiner PCFET0NUWVBFIGh0bWw+CjxodG1sPgogIDxoZWFkPgogICAgPG1ldGEgY2hhcnNldD0iVVRGLTgi IC8+CiAgICA8dGl0bGU+Z2V0RGlzcGxheU1lZGlhPC90aXRsZT4KICA8L2hlYWQ+CiAgPGJvZHk+ CiAgICA8YnV0dG9uIG9uY2xpY2s9ImdldE1lZGlhKCkiPmdldCBtZWRpYTwvYnV0dG9uPgogICAg PHZpZGVvIGlkPSJ2aWRlby0xIiB3aWR0aD0iNDAwIiBoZWlnaHQ9IjMwMCI+PC92aWRlbz4KICAg IDx2aWRlbyBpZD0idmlkZW8tMiIgd2lkdGg9IjQwMCIgaGVpZ2h0PSIzMDAiPjwvdmlkZW8+CiAg PC9ib2R5PgogIDxzY3JpcHQ+CiAgICBmdW5jdGlvbiBhc3NpZ25WaWRlbyhpZCwgc3RyZWFtKSB7 CiAgICAgIGNvbnN0IHZpZGVvID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoaWQpOwogICAgICB2 aWRlby5zcmNPYmplY3QgPSBzdHJlYW07CiAgICAgIHZpZGVvLnBsYXkoKTsKICAgIH0KCiAgICBh c3luYyBmdW5jdGlvbiBnZXRNZWRpYSgpIHsKICAgICAgLy8gY29uc3QgY2FtZXJhID0gYXdhaXQg bmF2aWdhdG9yLm1lZGlhRGV2aWNlcy5nZXRVc2VyTWVkaWEoeyB2aWRlbzogdHJ1ZSB9KTsKICAg ICAgLy8gYXNzaWduVmlkZW8oJ3ZpZGVvLTEnLCBjYW1lcmEpOwogICAgICBjb25zdCBkZXZpY2Vz ID0gYXdhaXQgbmF2aWdhdG9yLm1lZGlhRGV2aWNlcy5lbnVtZXJhdGVEZXZpY2VzKCk7CiAgICAg IGNvbnNvbGUubG9nKGRldmljZXMpOwoKICAgICAgLy8gY2FsbGluZyBnZXRVc2VyTWVkaWEgaGVy ZSB3b3JrcwogICAgICBjb25zdCBzY3JlZW4gPSBhd2FpdCBuYXZpZ2F0b3IubWVkaWFEZXZpY2Vz LmdldERpc3BsYXlNZWRpYSh7CiAgICAgICAgdmlkZW86IHRydWUKICAgICAgfSk7CiAgICAgIGFz c2lnblZpZGVvKCd2aWRlby0yJywgc2NyZWVuKTsKICAgIH0KICA8L3NjcmlwdD4KPC9odG1sPgo=