Skip to content

OpenGL: fix per-frame VAO leak in the ImGui GL3 backend - #2463

Merged
flyinghead merged 1 commit into
flyinghead:masterfrom
aldoborrero:fix/imgui-gl3-per-frame-vao-leak
Sep 4, 2026
Merged

OpenGL: fix per-frame VAO leak in the ImGui GL3 backend#2463
flyinghead merged 1 commit into
flyinghead:masterfrom
aldoborrero:fix/imgui-gl3-per-frame-vao-leak

Conversation

@aldoborrero

Copy link
Copy Markdown
Contributor

OpenGL: fix per-frame VAO leak in the ImGui GL3 backend

Fixes #2462.

The bug

ImGui_ImplOpenGL3_SetupRenderState generates the VAO into its vertex_array_object
parameter, which is passed by value:

static void ImGui_ImplOpenGL3_SetupRenderState(..., GLuint vertex_array_object)
{
    ...
    glGenVertexArrays(1, &vertex_array_object);   // writes the local copy only
    glBindVertexArray(vertex_array_object);
}

so the name never reaches the caller, ImGui_ImplOpenGL3_RenderDrawData, where it is still
0:

GLuint vertex_array_object = 0;
ImGui_ImplOpenGL3_SetupRenderState(..., vertex_array_object); // by value
...
if (vertex_array_object != 0)                    // still 0 — never true
    glDeleteVertexArrays(1, &vertex_array_object);

The guard is false, glDeleteVertexArrays never runs, and a VAO is leaked every frame.
The OSD overlay is drawn every frame, so on Mesa/radeonsi host memory grows unbounded until
OOM (~370–535 MiB/h in my testing). The Vulkan renderer's ImGui backend keeps persistent
buffers and is unaffected — which is the tell.

The fix

Cache the VAO in a single persistent handle (flycast uses one GL context) and drop the dead
delete. The vertex-attrib state is re-established every frame in SetupRenderState, so a
single reused VAO is correct.

Validation

Built and run on Linux / AMD Radeon 780M (radeonsi), OpenGL:

  • Before: RSS climbs linearly (~370–535 MiB/h), no plateau.
  • After: RSS plateaus — Sonic Adventure flat at ~200 MiB over 30 min; Crazy Taxi
    byte-identical over 10 min. Vulkan already flat, unchanged.

Rendering and the in-emulator ImGui menus verified visually with the cached VAO.

Analysis and reproduction assisted by an AI tool; the patch was built and verified on real hardware. (No LLM policy in this repo — remove if unwanted.)

@flyinghead

Copy link
Copy Markdown
Owner

Good catch on this problem. That might explain a number of OOM errors and GL driver crashes.

However your proposed solution brings a new issue: the VAO created is never deleted, even if the GL context is invalidated, the window destroyed and recreated, flycast switches to vulkan and back, etc. It likely will become invalid at some point.
What was done upstream is to create the VAO in ImGui_ImplOpenGL3_RenderDrawData and pass it to ImGui_ImplOpenGL3_SetupRenderState where it's bound. Then it is correctly deleted before ImGui_ImplOpenGL3_RenderDrawData returns.

@aldoborrero
aldoborrero force-pushed the fix/imgui-gl3-per-frame-vao-leak branch from 6c9a8e8 to 1819878 Compare September 3, 2026 18:55
ImGui_ImplOpenGL3_SetupRenderState generated the VAO into its
`vertex_array_object` parameter, which is passed by value, so the name never
reached ImGui_ImplOpenGL3_RenderDrawData. There the local was still 0, the
`if (vertex_array_object != 0)` guard was false, and glDeleteVertexArrays never
ran — a VAO was leaked every frame. The OSD is drawn every frame, so on Mesa
(radeonsi) host memory grows unbounded until OOM; Vulkan is unaffected.

Move the glGenVertexArrays back into RenderDrawData and pass the handle to
SetupRenderState for binding only, so the existing glDeleteVertexArrays runs
and the VAO is recreated each frame (staying valid across GL context changes),
matching the upstream ImGui backend.
@aldoborrero
aldoborrero force-pushed the fix/imgui-gl3-per-frame-vao-leak branch from 1819878 to 56d3db3 Compare September 3, 2026 19:23
@aldoborrero

Copy link
Copy Markdown
Contributor Author

Good point, and you're right; I confirmed it by live-switching OpenGL -> Vulkan -> OpenGL. I've updated the PR accordingly.

Thanks for your time.

@flyinghead
flyinghead merged commit d8a7680 into flyinghead:master Sep 4, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenGL renderer leaks a VAO every frame (ImGui GL3 backend) — unbounded host RAM growth, OOM

2 participants