NEW303336
[GTK] Watchdog crash after 10 seconds due to deadlock when destroying two different GLContexts on separate threads 
https://bugs.webkit.org/show_bug.cgi?id=303336
Summary [GTK] Watchdog crash after 10 seconds due to deadlock when destroying two dif...
Michael Catanzaro
Reported 2025-12-01 12:53:32 PST
Created attachment 477575 [details] Full backtrace We have yet another problem with destroying the EGL display in an exit handler. Thread 1 crashes in WebKit::crashAfter10Seconds. At the time, thread 17 is waiting on a mutex while trying to destroy GLContext 0x7f32393f00c0. Meanwhile, thread 25 is doing the same while trying to destroy GLContext 0x7f323931c2c0. Since they are different GLContexts, I would expect this to be safe, but apparently not. Could it be a bug in mesa? We've been plagued by this class of bug for at least a decade now. It is impossible on Apple platforms because WebKit does not allow exit-time destructors in cross-platform code or on Apple platforms. But we still allow it in the Linux-specific code. Using NeverDestroyed would eliminate this entire class of bug. Thread 17 (Thread 0x7f31ad7fa6c0 (LWP 35)): #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x00007f31cd9bef0a in futex_wait () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #2 0x00007f31cd9cc170 in do_futex_fence_wait () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #3 0x00007f31cd9cd34c in util_queue_finish () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #4 0x00007f31cde3d4ea in si_set_debug_callback () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #5 0x00007f31cde3dd58 in si_destroy_context () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #6 0x00007f31cdc08031 in tc_destroy () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #7 0x00007f31cd590220 in st_destroy_context_priv () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #8 0x00007f31cd591d0d in st_destroy_context () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #9 0x00007f31cd4f2ffb in dri_destroy_context () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libgallium-25.2.6.so #10 0x00007f31de72a329 in dri2_destroy_context () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libEGL_mesa.so.0 #11 0x00007f31de71d04d in eglDestroyContext () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libEGL_mesa.so.0 #12 0x00007f3251346b91 in WebCore::GLContext::~GLContext (this=0x7f32393f00c0) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/egl/GLContext.cpp:335 #13 0x00007f32513fc337 in std::default_delete<WebCore::GLContext>::operator() (this=0x7f32393c80c8, __ptr=0x7f32393f00c0) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/15.2.0/../../../../include/c++/15.2.0/bits/unique_ptr.h:93 #14 std::unique_ptr<WebCore::GLContext, std::default_delete<WebCore::GLContext> >::~unique_ptr (this=0x7f32393c80c8) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/15.2.0/../../../../include/c++/15.2.0/bits/unique_ptr.h:399 #15 WebCore::SkiaGLContext::~SkiaGLContext (this=0x7f32393c80c0) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/skia/PlatformDisplaySkia.cpp:238 #16 WTF::ThreadSafeWeakPtrControlBlock::strongDeref<WebCore::SkiaGLContext, (WTF::DestructionThread)0>() const::{lambda()#1}::operator()() const (this=<optimized out>) at WTF/Headers/wtf/ThreadSafeWeakPtr.h:93 #17 WTF::ThreadSafeWeakPtrControlBlock::strongDeref<WebCore::SkiaGLContext, (WTF::DestructionThread)0> (this=<optimized out>) at WTF/Headers/wtf/ThreadSafeWeakPtr.h:108 #18 0x00007f324e04585f in __GI___call_tls_dtors () at cxa_thread_atexit_impl.c:156 #19 0x00007f324e09b40a in start_thread (arg=<optimized out>) at pthread_create.c:453 #20 0x00007f324e11ee54 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100 Thread 25 (Thread 0x7f3243ffde40 (LWP 2)): #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x00007f31de73787a in futex_wait () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libEGL_mesa.so.0 #2 0x00007f31de71b8ef in _eglLockDisplay () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libEGL_mesa.so.0 #3 0x00007f31de71d1c4 in eglMakeCurrent () at /usr/lib/x86_64-linux-gnu/GL/default/lib/libEGL_mesa.so.0 #4 0x00007f3247eac6ad in InternalLoseCurrent () at ../src/EGL/libegl.c:551 #5 0x00007f3251346b81 in WebCore::GLContext::~GLContext (this=0x7f323931c2c0) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/egl/GLContext.cpp:334 #6 0x00007f32513fc337 in std::default_delete<WebCore::GLContext>::operator() (this=0x7f323939c4c8, __ptr=0x7f323931c2c0) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/15.2.0/../../../../include/c++/15.2.0/bits/unique_ptr.h:93 #7 std::unique_ptr<WebCore::GLContext, std::default_delete<WebCore::GLContext> >::~unique_ptr (this=0x7f323939c4c8) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/15.2.0/../../../../include/c++/15.2.0/bits/unique_ptr.h:399 #8 WebCore::SkiaGLContext::~SkiaGLContext (this=0x7f323939c4c0) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/skia/PlatformDisplaySkia.cpp:238 #9 WTF::ThreadSafeWeakPtrControlBlock::strongDeref<WebCore::SkiaGLContext, (WTF::DestructionThread)0>() const::{lambda()#1}::operator()() const (this=<optimized out>) at WTF/Headers/wtf/ThreadSafeWeakPtr.h:93 #10 WTF::ThreadSafeWeakPtrControlBlock::strongDeref<WebCore::SkiaGLContext, (WTF::DestructionThread)0> (this=<optimized out>) at WTF/Headers/wtf/ThreadSafeWeakPtr.h:108 #11 0x00007f3251336723 in WebCore::PlatformDisplay::clearGLContexts (this=0x55f63fc52c98) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/PlatformDisplay.cpp:130 #12 0x00007f324f469484 in WebKit::WebProcess::stopRunLoop (this=0x7f3238030110) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/WebProcess/glib/WebProcessGLib.cpp:124 #13 0x00007f324eee78f2 in WebKit::AuxiliaryProcess::terminate (this=0x7f3238030110) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/Shared/AuxiliaryProcess.cpp:242 #14 0x00007f324f19a6c0 in WebKit::WebProcess::removeWebPage (this=0x7f3238030110, pageID=...) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/WebProcess/WebProcess.cpp:1056 #15 0x00007f324f4169b3 in WebKit::WebPage::close (this=0x7f32380e4170) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/WebProcess/WebPage/WebPage.cpp:2048 #16 0x00007f324f469429 in WebKit::WebProcess::stopRunLoop (this=0x7f3238030110) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/WebProcess/glib/WebProcessGLib.cpp:121 #17 0x00007f324eeda120 in IPC::Connection::dispatchDidCloseAndInvalidate()::$_0::operator()() const (this=0x7f3238038118) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/Platform/IPC/Connection.cpp:1311 #18 WTF::Detail::CallableWrapper<IPC::Connection::dispatchDidCloseAndInvalidate()::$_0, void>::call (this=0x7f3238038110) at WTF/Headers/wtf/Function.h:59 #19 0x00007f324c61ee55 in WTF::Function<void()>::operator() (this=0x7ffe00cb4810) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/Function.h:103 #20 WTF::RunLoop::performWork (this=0x7f3238014180) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/RunLoop.cpp:148 #21 0x00007f324c6e566d in WTF::RunLoop::RunLoop()::$_0::operator()(void*) const (userData=0x55f63fc52c98, userData@entry=0x7f3238014180, this=<optimized out>) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:80 #22 WTF::RunLoop::RunLoop()::$_0::__invoke(void*) (userData=0x55f63fc52c98) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:79 #23 0x00007f324c6e3d11 in WTF::RunLoop::$_1::operator() (source=0x55f63fa1b460, callback=0x7f324c6e5660 <WTF::RunLoop::RunLoop()::$_0::__invoke(void*)>, userData=0x7f3238014180, this=<optimized out>) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:57 #24 WTF::RunLoop::$_1::__invoke (source=0x55f63fa1b460, callback=0x7f324c6e5660 <WTF::RunLoop::RunLoop()::$_0::__invoke(void*)>, userData=0x7f3238014180) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:49 #25 0x00007f324ce7d7cb in g_main_dispatch (context=context@entry=0x55f63fa1b2c0) at ../glib/gmain.c:3565 #26 0x00007f324ce81380 in g_main_context_dispatch_unlocked (context=0x55f63fa1b2c0) at ../glib/gmain.c:4425 #27 g_main_context_dispatch (context=0x55f63fa1b2c0) at ../glib/gmain.c:4413 #28 0x00007f324c6e42f3 in WTF::RunLoop::runGLibMainLoopIteration (this=0x7f3238014180, mayBlock=WTF::RunLoop::MayBlock::Yes) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:118 #29 0x00007f324c6e468a in WTF::RunLoop::runGLibMainLoop (this=0x7f3238014180) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:127 #30 WTF::RunLoop::run () at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:140 #31 0x00007f324f469d94 in WebKit::AuxiliaryProcessMainBase<WebKit::WebProcess, true>::run (this=0x7ffe00cb49f0, argc=<optimized out>, argv=<optimized out>) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/Shared/AuxiliaryProcessMain.h:77 #32 WebKit::AuxiliaryProcessMain<WebKit::WebProcessMainGtk> (argc=3, argv=<optimized out>) at /usr/lib/debug/source/sdk/webkitgtk-6.0.bst/Source/WebKit/Shared/AuxiliaryProcessMain.h:103 #33 0x00007f324e02c975 in __libc_start_call_main (main=main@entry=0x55f628e04150 <main(int, char**)>, argc=argc@entry=3, argv=argv@entry=0x7ffe00cb4b88) at ../sysdeps/nptl/libc_start_call_main.h:58 #34 0x00007f324e02ca28 in __libc_start_main_impl (main=0x55f628e04150 <main(int, char**)>, argc=3, argv=0x7ffe00cb4b88, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe00cb4b78) at ../csu/libc-start.c:360 #35 0x000055f628e04085 in _start () at ../sysdeps/x86_64/start.S:115 Full backtrace attached.
Attachments
Full backtrace (157.72 KB, text/plain)
2025-12-01 12:53 PST, Michael Catanzaro 		no flags
Details
Key-treads (3.81 KB, text/markdown)
2026-07-21 09:40 PDT, chetrf2@gmail.com 		no flags
Details
Full symbolized 'thread apply all bt' — coredump 1 (85.25 KB, text/plain)
2026-07-21 09:41 PDT, chetrf2@gmail.com 		no flags
Details
Full symbolized 'thread apply all bt' — coredump 2 (83.04 KB, text/plain)
2026-07-21 09:42 PDT, chetrf2@gmail.com 		no flags
Details
systemd-coredump metadata + module list — coredump 1 (39.44 KB, text/plain)
2026-07-21 09:42 PDT, chetrf2@gmail.com 		no flags
Details
systemd-coredump metadata + module list — coredump 2 (38.63 KB, text/plain)
2026-07-21 09:43 PDT, chetrf2@gmail.com 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Carlos Garcia Campos
Comment 1 2025-12-02 00:22:21 PST
Where is the exit handler? I don't find it in the bt. And where is the egl display being destroyed? I only see threads destroying their GL contexts.
Michael Catanzaro
Comment 2 2025-12-02 09:30:24 PST
Thread 17 is in an exit handler (__GI___call_tls_dtors), but you're right: the EGL display is not being destroyed. I just assumed it was.
Michael Catanzaro
Comment 3 2025-12-02 09:32:22 PST
Well, I suppose thread-local storage destruction is not the same as an exit handler.
chetrf2@gmail.com
Comment 4 2026-07-21 09:35:14 PDT
We are hitting what looks like the same defect as #303336 on a newer WebKitGTK, but it surfaces as heap corruption + SIGABRT at process exit rather than as the 10-second watchdog deadlock. Two coredumps captured, both with the identical signature. Full backtraces attached. ## Version / build - **WebKitGTK 2.52.4**, API/ABI 6.0 (`libwebkitgtk-6.0.so.4`, `libjavascriptcoregtk-6.0.so.1`). Skia rendering backend. - Build source: **NixOS 26.05** (nixpkgs), stock webkitgtk derivation. This build already contains the 2.52.1 heap-corruption fix (RedHat bug 2450634), so this is not that issue. - GTK 4.22.4, GLib 2.88.1, glibc 2.42. - **Mesa 26.1.4**, `libEGL_mesa` / `libgallium-26.1.4`. - GPU: AMD integrated (amdgpu, PCI 1002:1900). Real hardware EGL on Wayland. - Kernel Linux 6.18.38 x86_64. - Session: KDE Plasma 6.6.6 / KWin 6.6.6 **Wayland**. ## Host application The WebKitGTK view is embedded in a first-party GTK4 / WebKitGTK-6.0 application (built with Wails v3: a Go backend hosting a `WebKitWebView` in a GTK4 window). It runs natively on the live Plasma Wayland session. The app processes only local mock/test data. ## Steps to reproduce 1. Run a GTK4 application embedding a WebKitGTK 6.0 `WebKitWebView`, on a real Wayland session with hardware EGL (here: Plasma/KWin Wayland, Mesa, amdgpu). 2. Close the top-level window normally (KWin delivers `xdg_toplevel` close); the app shuts down and the main process exits. 3. Repeat. In roughly **2 of 20** window-close teardowns (~6-10%), after the main process has already exited cleanly (status 0), the **WebKitWebProcess** renderer child aborts during its own exit. This is a non-deterministic teardown race, so it takes repeated runs. ## Actual result `WebKitWebProcess` prints `free(): corrupted unsorted chunks` to stderr and aborts (SIGABRT). The core shows two teardown activities racing: - A secondary thread is destroying a `SkiaGLContext` from a **libc TLS destructor** at exit, on the wrong/secondary thread: ``` #0 syscall #1 futex_wait (libEGL_mesa.so.0) #2 _eglLockDisplay (libEGL_mesa.so.0) #3 eglDestroyContext (libEGL_mesa.so.0) #4 WebCore::GLContext::~GLContext() (libwebkitgtk-6.0.so.4) #5 WTF::ThreadSafeWeakPtrControlBlock::strongDeref<WebCore::SkiaGLContext, (WTF::DestructionThread)0>() const #6 __call_tls_dtors (libc.so.6) #7 start_thread #8 clone ``` This is the same `eglDestroyContext` -> `_eglLockDisplay` + `SkiaGLContext strongDeref<DestructionThread(0)>` + `__call_tls_dtors` pattern described in #303336, blocked on Mesa's EGL display lock. - Meanwhile another exiting worker thread aborts while releasing its per-thread malloc arena — glibc's checker finds a corrupted chunk: ``` #0 abort #1 __libc_message_impl.cold #2 malloc_printerr #3 _int_free_create_chunk #4 _int_free_merge_chunk #5 _int_free_chunk #6 __malloc_arena_thread_freeres #7 start_thread #8 clone ``` - The main thread is in normal `exit()`, running Mesa's atexit handler (`atexit_handler` -> `util_queue_kill_threads` -> `thrd_join`). So at `exit()`, GL contexts are being torn down on secondary threads via TLS destructors while other threads free their arenas; the heap is left corrupted and the corruption is tripped during arena cleanup. **Note on MALLOC_CHECK_:** the process ran with `MALLOC_CHECK_=3` (and `MALLOC_PERTURB_=170`) set. That does not cause the corruption — it makes the pre-existing heap damage surface **deterministically as an immediate abort** instead of a silent/late failure. This is likely why we see a clean SIGABRT where #303336 saw a watchdog deadlock: the same unsynchronized teardown, made visible earlier by glibc malloc checking. ## Expected result The WebKitWebProcess should exit cleanly after the parent closes its window; GL/Skia contexts should be torn down without cross-thread races or heap corruption. ## Environment specificity - Reproduces only on the real EGL/Wayland GPU path (Mesa, amdgpu). - **Never** observed under X11/Xvfb software GL across 30+ historical runs. - The parent app is always unaffected (exit 0; captured data intact); only the renderer child aborts, after the parent is gone. ## What we have NOT done (honesty) - We have **not** produced a minimal MiniBrowser/standalone reproducer. Given the crash window is purely teardown-after-close and lives entirely in WebKit's Skia GL-context destruction + libc TLS-destructor path, we expect it is reproducible with essentially any GTK WebKit view destroyed on a Wayland/EGL session under repeated open/close — but we have not verified that claim, so we are not asserting it. - We did not bisect or test a patch. We are on 2.52.4; we did not test 2.52.5 or trunk (public 2.52.5 notes do not call out a fix here). ## Attachments - `backtraces/key-threads.md` — the relevant threads, annotated. - `backtraces/gdb-bt-520750.txt`, `backtraces/gdb-bt-523858.txt` — full `thread apply all bt` from gdb, symbolized against the nixpkgs webkitgtk build (function names + library paths). - `backtraces/coredumpctl-info-520750.txt`, `backtraces/coredumpctl-info-523858.txt` — systemd-coredump metadata + per-thread stacks + full loaded-module list. Raw `.core` files (~15.6M each, contain only mock/test data) are retained locally and can be provided privately to a maintainer on request.
chetrf2@gmail.com
Comment 5 2026-07-21 09:40:46 PDT
Created attachment 480700 [details] Key-treads
chetrf2@gmail.com
Comment 6 2026-07-21 09:41:33 PDT
Created attachment 480701 [details] Full symbolized 'thread apply all bt' — coredump 1
chetrf2@gmail.com
Comment 7 2026-07-21 09:42:04 PDT
Created attachment 480702 [details] Full symbolized 'thread apply all bt' — coredump 2
chetrf2@gmail.com
Comment 8 2026-07-21 09:42:41 PDT
Created attachment 480703 [details] systemd-coredump metadata + module list — coredump 1
chetrf2@gmail.com
Comment 9 2026-07-21 09:43:19 PDT
Created attachment 480704 [details] systemd-coredump metadata + module list — coredump 2
Michael Catanzaro
Comment 10 2026-07-23 15:10:52 PDT
> `WebKitWebProcess` prints `free(): corrupted unsorted chunks` to stderr and aborts (SIGABRT). I also see this routinely with Epiphany, and I agree it's probably the same underlying problem as the deadlock that I reported here.
qzji5b01n
Comment 11 2026-09-14 16:52:29 PDT
I isolated this outside WebKit and reproduced the same corruption with a standalone Mesa/EGL program. The reproducer initializes a surfaceless EGL display, repeatedly calls `eglQueryString()` from worker threads, and calls `exit(0)` from the main thread without joining them. On Mesa 26.0.8, 47/50 runs aborted with: free(): corrupted unsorted chunks Two controls produced no failures: replacing `exit(0)` with `_exit(0)`: 0/50; joining the EGL worker threads before `exit(0)`: 0/50. The difference is statistically significant (Fisher’s exact test, approximately `p = 2.3e-25`) and isolates the failure to Mesa’s `atexit` teardown while another thread is using an EGL display. The observed race is: 1. `eglQueryString()` enters `_eglLockDisplay()`. 2. `_eglLookupDisplay()` finds the display while holding `_eglGlobal.Mutex`, but releases the global mutex before returning (`eglapi.c:263–277`). 3. Before `_eglLockDisplay()` acquires `TerminateLock` and `disp->Mutex` (`eglapi.c:282–291`), the main thread enters Mesa’s exit handler. 4. `_eglFiniDisplay()` frees `disp` without acquiring those object locks (`egldisplay.c:187–217`). 5. The worker then executes `simple_mtx_lock(&disp->Mutex)` on the freed allocation. The allocator layout explains the observed corruption precisely. If `F` is the freed glibc chunk: ``` disp = F + 0x10 &disp->Mutex = F + 0x18 unsorted-bin bk = F + 0x18 ``` Therefore, after `free(disp)`, `disp->Mutex` occupies the same address as the freed chunk’s `bk` field. A GDB capture shows a worker blocked in: ``` futex_wait simple_mtx_lock _eglLockDisplay eglQueryString ``` while the main thread aborts from `__run_exit_handlers`. I also captured the overwrite without GDB using an `LD_PRELOAD` shim that intercepts `free(disp)` and inspects the chunk 800 µs later: * race active: 10/10 runs changed `bk` to `(high half of unsorted-bin head) | 0x2` or `| 0x0`; * workers joined before exit: 4/4 runs preserved `bk == fd` and did not crash. The `0x2` value comes from the contended path in `simple_mtx_lock`: the initial `cmpxchg(0, 1)` fails because the slot already contains the unsorted-bin `bk` pointer, after which `xchg(..., 2)` overwrites its low 32 bits. This directly matches the two WebKitWebProcess coredumps previously described: ``` 0x000074d8_00000002 0x00007e7c_00000000 ``` Thus, the WebKitGTK crash appears to be one manifestation of a Mesa EGL teardown race: `_eglLookupDisplay()` returns a display that can be freed by `_eglFiniDisplay()` before `_eglLockDisplay()` acquires the display’s locks. I can attach: * the standalone C reproducer; * the `LD_PRELOAD` capture shim; * the GDB backtrace; * the raw before/after chunk captures.
qzji5b01n
Comment 12 2026-09-14 17:24:35 PDT
/* * Minimal reproducer for a Mesa EGL shutdown race. * * _eglFiniDisplay(), called from Mesa's atexit handler, can free an * _EGLDisplay while another thread has returned from _eglLookupDisplay() * but has not yet locked disp->TerminateLock and disp->Mutex. * * The worker then locks a mutex inside the freed allocation. In glibc's * layout, this mutex overlaps the freed chunk's unsorted-bin bk field, * causing "free(): corrupted unsorted chunks" or SIGSEGV. * * Modes: * race Default. Calls exit() while EGL worker threads are active. * noexit Calls _exit(), bypassing atexit teardown. * joined Joins all workers before calling exit(). * * Usage: * repro-egl-race [delay_us] [hammer_threads] [mode] */ #define _GNU_SOURCE #include <dlfcn.h> #include <pthread.h> #include <stdint.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> typedef void *EGLDisplay; typedef void *EGLNativeDisplayType; typedef unsigned int EGLBoolean; typedef unsigned int EGLenum; typedef int32_t EGLint; typedef intptr_t EGLAttrib; #define EGL_DEFAULT_DISPLAY ((EGLNativeDisplayType)0) #define EGL_NO_DISPLAY ((EGLDisplay)0) #define EGL_PLATFORM_SURFACELESS_MESA 0x31DD #define EGL_VENDOR 0x3053 #define EGL_VERSION 0x3054 #define MAX_THREADS 64 static EGLDisplay (*p_eglGetDisplay)(EGLNativeDisplayType); static EGLDisplay (*p_eglGetPlatformDisplay)(EGLenum, void *, const EGLAttrib *); static EGLBoolean (*p_eglInitialize)(EGLDisplay, EGLint *, EGLint *); static const char *(*p_eglQueryString)(EGLDisplay, EGLint); static EGLDisplay g_dpy; static volatile int g_run = 1; static void *hammer_thread(void *arg) { (void)arg; while (g_run) { p_eglQueryString(g_dpy, EGL_VENDOR); } return NULL; } static void *churn_thread(void *arg) { (void)arg; while (g_run) { void *p = malloc(0x3000); if (p) { ((volatile unsigned char *)p)[0] = 1; free(p); } void *q = malloc(0x30); free(q); } return NULL; } /* * Performs allocator churn during process teardown, making heap corruption * more likely to become observable. */ static void post_exit_churn(void) { for (int i = 0; i < 512; i++) { void *p = malloc(0x3000); if (p) { ((volatile unsigned char *)p)[0] = 1; free(p); } } } int main(int argc, char **argv) { long delay_us = argc > 1 ? atol(argv[1]) : 4000; int n_hammer = argc > 2 ? atoi(argv[2]) : 6; const char *mode = argc > 3 ? argv[3] : "race"; if (n_hammer < 1) n_hammer = 1; if (n_hammer > MAX_THREADS - 2) n_hammer = MAX_THREADS - 2; void *lib = dlopen("libEGL.so.1", RTLD_NOW | RTLD_GLOBAL); if (!lib) { fprintf(stderr, "failed to load libEGL.so.1: %s\n", dlerror()); return 2; } p_eglGetDisplay = (EGLDisplay(*)(EGLNativeDisplayType))dlsym(lib, "eglGetDisplay"); p_eglGetPlatformDisplay = (EGLDisplay(*)(EGLenum, void *, const EGLAttrib *))dlsym(lib, "eglGetPlatformDisplay"); if (!p_eglGetPlatformDisplay) p_eglGetPlatformDisplay = (EGLDisplay(*)(EGLenum, void *, const EGLAttrib *))dlsym(lib, "eglGetPlatformDisplayEXT"); p_eglInitialize = (EGLBoolean(*)(EGLDisplay, EGLint *, EGLint *))dlsym(lib, "eglInitialize"); p_eglQueryString = (const char *(*)(EGLDisplay, EGLint))dlsym(lib, "eglQueryString"); if (!p_eglGetDisplay || !p_eglInitialize || !p_eglQueryString) { fprintf(stderr, "missing EGL symbols\n"); return 2; } EGLDisplay dpy = EGL_NO_DISPLAY; if (p_eglGetPlatformDisplay) dpy = p_eglGetPlatformDisplay(EGL_PLATFORM_SURFACELESS_MESA, NULL, NULL); if (!dpy) dpy = p_eglGetDisplay(EGL_DEFAULT_DISPLAY); if (!dpy) { fprintf(stderr, "eglGetDisplay failed\n"); return 2; } EGLint major = 0, minor = 0; if (!p_eglInitialize(dpy, &major, &minor)) { fprintf(stderr, "eglInitialize failed\n"); return 2; } g_dpy = dpy; fprintf(stderr, "EGL %d.%d vendor=%s version=%s\n", major, minor, p_eglQueryString(dpy, EGL_VENDOR), p_eglQueryString(dpy, EGL_VERSION)); atexit(post_exit_churn); pthread_t th[MAX_THREADS]; int n_churn = 2; int n_threads = 0; for (int i = 0; i < n_hammer + n_churn; ++i) { void *(*fn)(void *) = i < n_hammer ? hammer_thread : churn_thread; if (pthread_create(&th[n_threads], NULL, fn, NULL) == 0) ++n_threads; } if (delay_us > 0) usleep((useconds_t)delay_us); if (strcmp(mode, "noexit") == 0) _exit(0); if (strcmp(mode, "joined") == 0) { g_run = 0; for (int i = 0; i < n_threads; ++i) pthread_join(th[i], NULL); } exit(0); } __________________________________________________________________ The reproducer is working on my system. A sample run produced: EGL 1.5 vendor=Mesa Project version=1.5 malloc(): smallbin double linked list corrupted malloc(): smallbin double linked list corrupted malloc(): smallbin double linked list corrupted malloc(): smallbin double linked list corrupted malloc(): smallbin double linked list corrupted Segmentation fault (core dumped) The exact glibc diagnostic varies between runs (`corrupted unsorted chunks`, `smallbin double linked list corrupted`, or SIGSEGV), depending on where the corrupted heap metadata is first accessed. The important result is that the `race` mode corrupts the heap, while the `noexit` and `joined` control modes do not. Build command: cc -O2 -g -Wall -Wextra repro-egl-race.c \ -o repro-egl-race -ldl -lpthread Test modes: ./repro-egl-race 4000 6 race ./repro-egl-race 4000 6 noexit ./repro-egl-race 4000 6 joined Environment: Mesa 26.0.8 EGL 1.5 glibc 2.43 Intel Iris Xe / i915 Wayland
jay
Comment 13 2026-09-17 05:35:58 PDT
Seeing this on WPE as well, which may make it wider than the [GTK] in the title. Arch Linux, kernel 7.2.3, glibc 2.44, Mesa 26.2.2, AMD Phoenix1 (radeonsi). WPE WebKit from Playwright's bundled `webkit-2272` (Playwright 1.59.1, `browser.version()` reports 26.4), shipping `libWPEWebKit-2.0.so.1.10.0`, libwpe 1.9.3, WPEBackend-fdo 1.9.5. Same failure as comment 4: `free(): corrupted unsorted chunks` followed by SIGABRT in the web process at exit, after it has already produced correct results. The parent's exit status is 0 every time, which makes this easy to miss entirely — I only found it because systemd-coredump was logging the aborts. Both stacks in this bug show up: usually `__run_exit_handlers`, occasionally `__call_tls_dtors` on a secondary thread. What I haven't seen mentioned here is that the abort rate depends strongly on how long the page is left running before the browsing context is closed, and that it peaks rather than rising monotonically. Measured with Playwright: 4 contexts per run, one page load in each, context closed after a fixed wait. One "close" below is one context close; aborts counted from `coredumpctl`. wait closes aborts rate 0ms 40 1 2% 800ms 32 7 22% 1600ms 32 23 72% 3200ms 40 16 40% Forcing software rendering with `LIBGL_ALWAYS_SOFTWARE=1` at the 3200ms wait takes it to 29/32, or 91%. That looks consistent with the `_eglFiniDisplay()` race in comments 11-12: llvmpipe is slow enough that there is still GL work in flight at 3200ms, where the hardware driver has gone quiescent by then. If that reading is right, the peak is just "how much EGL work is outstanding at teardown", and the wait time is a proxy for it. Page content matters more than I expected, and I can't fully account for this. A minimal page of my own — continuous WebGL clear, a 2D canvas fill, one CSS animation — gives 0/32 at the 1600ms peak. The page I reproduce on reliably is <https://kub0.ai/>, at 33% (8/24 closes at 1600ms) with no page errors. It is a static marketing page with web fonts and a handful of compositing layers, nothing unusual. I have not narrowed which property is required, so the URL is load-bearing in the repro below. ```js // npm i playwright && npx playwright install webkit // WAIT=1600 node repro.js const { webkit } = require('playwright'); const WAIT = Number(process.env.WAIT ?? 1600); const URL = process.env.URL ?? 'https://kub0.ai/'; (async () => { const browser = await webkit.launch({ headless: true }); for (let i = 0; i < 4; i++) { const ctx = await browser.newContext({ viewport: { width: 1440, height: 1000 } }); const page = await ctx.newPage(); await page.goto(URL, { waitUntil: 'load' }); await page.waitForTimeout(WAIT); await ctx.close(); } await browser.close(); })(); ``` At a 1600ms wait most runs produce at least one abort. The script exits 0 either way, so watch `coredumpctl list` or the journal rather than the exit status. Running under `ulimit -c 0` keeps the journal records while writing no core files, which is convenient when iterating — 81 aborts here cost no disk. One caveat on the numbers: they are from an unsandboxed shell. The same script at the same wait, run inside a bubblewrap sandbox with `/dev/dri` still bound, drops to roughly 5% per close. I verified the abort counting works identically in both, so the sandbox appears to change the race rather than the measurement, but I don't know what about it does that.
bob
Comment 14 2026-09-28 10:30:47 PDT
Seeing the same class of exit-time EGL teardown race on the NVIDIA proprietary driver, via WebKitGTK. Here the failure is a SIGSEGV in a SkiaGPUWorker thread's TLS destructors rather than a deadlock/watchdog. Environment: webkit2gtk-4.1 2.52.6 (Arch Linux package), nvidia-utils 610.57.04, Wayland (Hyprland), kernel 7.2.5, glibc from Arch. Embedder is a Tauri 2 (wry) app; the crashing process is /usr/lib/webkit2gtk-4.1/WebKitWebProcess, as the embedding application quits. Intermittent: two crashes within 7 minutes, but most quits are clean. Kernel log: SkiaGPUWorker[115515]: segfault at 41bb25f0 ip 00007dd00199b0e5 sp 00007dbf7e7f6e50 error 4 in libnvidia-eglcore.so.610.57.04[79b0e5,7dd001419000+c46000] si_code: SEGV_MAPERR Crashing thread, SkiaGPUWorker, exiting and running thread-local destructors: #0 libnvidia-eglcore.so.610.57.04 + 0x79b0e5 #1 libwebkit2gtk-4.1.so.0 + 0x4fc5ca7 #2 libwebkit2gtk-4.1.so.0 + 0x4f61c28 #3 libwebkit2gtk-4.1.so.0 + 0x4f11eb6 #4 libwebkit2gtk-4.1.so.0 + 0x4f1212e #5 libwebkit2gtk-4.1.so.0 + 0x452a91a #6 __call_tls_dtors (libc.so.6) #7 libc.so.6 (thread exit) #8 libc.so.6 (clone) Main thread, in exit(), tearing down the NVIDIA EGL driver from libc exit handlers: #0-1 libc.so.6 (lock wait) #2-3 libnvidia-glsi.so.610.57.04 (_nv004glsi) #4-5 libnvidia-eglcore.so.610.57.04 #6-16 libEGL_nvidia.so.0 #17 libc.so.6 (exit handlers) #18 exit #19-20 __libc_start_main A second SkiaGPUWorker thread was in the same __call_tls_dtors path, blocked on a lock inside libnvidia-glsi. The other crash has the same shape, faulting at libnvidia-eglcore + 0x705749. Sorry the WebKit frames are unsymbolized: Arch's debuginfod returned nothing for them and NVIDIA ships no symbols. Offsets are into the Arch webkit2gtk-4.1 2.52.6-1 build of libwebkit2gtk-4.1.so.0. I still have both cores and can run commands against them if useful. So this isn't Mesa-specific: the per-thread GL context teardown in the Skia GPU workers' TLS destructors races the EGL display teardown in exit() on NVIDIA too, which supports removing these exit-time destructors (NeverDestroyed) rather than fixing it per driver.
Note You need to log in before you can comment on or make changes to this bug.