Fix MPCVideoDec dangling-pointer crash on resolution-changing channel switch (D3D11/DXVA2 HEVC)

[nanamitm]

Problem

Switching a live channel from a lower resolution to a higher one (e.g. 4K to
8K) while using D3D11 (or DXVA2) hardware decoding of HEVC crashes nearly
every time, with an access violation inside ff_dxva2_commit_buffer's
memcpy. Reproduced with both MPC Video Renderer and EVR as the downstream
renderer, which rules out either renderer and points at MPCVideoDec.ax
itself. Same-resolution switches (4K to 4K) never crash.

Root cause

CMPCVideoDecFilter::FillAVPacket() in MPCVideoDec.cpp pointed
AVPacket->data directly at m_pFFBuffer instead of copying it, for any
codec that doesn't need PRORES's padded-buffer path. m_pFFBuffer is a
single buffer that ParseInternal() reuses (and may reallocate via
av_fast_realloc) on every subsequent call.

FFmpeg's DXVA2/D3D11 HEVC hwaccel keeps that raw pointer alive well past the
FillAVPacket()/avcodec_send_packet() call: dxva2_hevc_decode_slice()
(libavcodec/dxva2_hevc.c) stores it as ctx_pic->bitstream, and it isn't
actually read until the frame's end_frame()/GPU submission via
ff_dxva2_commit_buffer()'s memcpy(dxva_data, data, size) in
libavcodec/dxva2.c — i.e. asynchronously, after decode_slice() already
returned. If m_pFFBuffer gets reused or reallocated for a later packet
before that GPU submission completes, the hwaccel ends up reading through a
dangling or overwritten pointer.

A resolution change adds just enough extra latency
(CD3D11Decoder::ReInitD3D11Decoder() tearing down and rebuilding the
decoder/surface pool) for this race to be won reliably. Same-resolution
switches never hit that reinit path, so the race window stays narrow enough
to not normally reproduce — but the underlying bug is not actually specific
to resolution changes.

Fix

Force FillAVPacket() to make a real av_new_packet copy whenever the
filter is using DXVA2 or D3D11 hardware decoding, instead of aliasing the
shared buffer pointer, so each packet owns independent memory regardless
of how/when m_pFFBuffer gets reused. The existing PRORES oversized-buffer
copy-path behavior (pkt->size left as the padded size) is unchanged.

Testing

Repeated 4K -> 8K -> 4K -> 8K ... channel switching under D3D11 hardware
decoding (HEVC), with both MPC Video Renderer and EVR — previously crashed
on almost every switch to the higher resolution, no crashes after this fix
across many repeated switches.

[v0lt]

Switching a live channel from a lower resolution to a higher one (e.g. 4K to 8K)

Please provide a sample file or a link to a "live channel".

[Aleksoid1978]

Or record from live channel and upload.

[nanamitm]

Uploaded repro samples here, since this needs hardware/conditions I can't ship as a small synthetic clip:
https://github.com/nanamitm/MPC-BE/releases/tag/pr-1163-repro-samples

• Record_20260620-174601.ts — short 4K HEVC clip
• Record_20260620-174634.ts — short 8K HEVC clip

Repro steps (no TVTest/BonDriver needed, reproduces directly in MPC-BE):

1. Open the 4K clip with D3D11 hardware decoding enabled.
2. While it's playing, open the 8K clip (File > Open, without closing the first) to trigger the resolution-changing reconnect.

Without the fix this crashes reliably (observed in ff_dxva2_commit_buffer's memcpy, access violation reading the source pointer) due to the dangling-pointer race in FillAVPacket() described in the PR description. With the fix applied, repeated switching between the two clips back and forth did not reproduce the crash.

[nanamitm]

A few notes on reproduction conditions, in case it doesn't trigger right away:

• D3D11 hardware decoding must be selected in MPCVideoDec's options (not DXVA2, not software/FFmpeg-only). The fix also covers the DXVA2 path, but the race was specifically observed and reproduced via D3D11.
• Let the 4K clip play for a couple of seconds before opening the 8K clip, so decoding is actively in flight (not just-opened/paused).
• The resolution increase is what matters - it's CD3D11Decoder::ReInitD3D11Decoder() rebuilding the decoder/surface pool for the new size that adds enough latency to reliably lose the race. Switching between two clips of the same resolution won't reproduce it.
• If it doesn't crash on the very first try (GPU/driver speed can affect the race window), switching back and forth a couple of times (4K → 8K → 4K → 8K) reproduced it reliably in my testing.

[Aleksoid1978]

When you open new file - player closed current graph and all filters. For test need file with resolution changes in video stream.

[v0lt]

For test need file with resolution changes in video stream.

copy /b Record_20260620-174601.ts+Record_20260620-174634.ts test.ts

My MPC-BE crashes when playing test.ts.

[v0lt]

The patch doesn't crash.
But I have an RTX 5060. This graphics card can hardware decode HEVC 8k.
It would be good to test it on a graphics card that only supports HEVC 4k.

[Aleksoid1978]

The crash has been fixed, but in a slightly different way.