Avoid EC_DISPLAY_CHANGED for output-format-only renderer notifications in MPCVideoDec

[nanamitm]

Problem

With D3D11 hardware decoding selected, a connected video renderer (in my
case the MPC Video Renderer, but the underlying mechanism is
renderer-agnostic) can sometimes show no video at all - at TVTest's startup
auto-tune, or right after switching the renderer setting while a stream is
playing. Manually toggling playback off/on (forcing a fresh avcodec_open2())
"fixes" it.

Root cause

CMPCVideoDecFilter::ChangeOutputMediaFormat()'s renderer branch calls
NotifyEvent(EC_DISPLAY_CHANGED, (LONG_PTR)pPin, 0) to tell a connected
video renderer that the decoder's output media type changed.

EC_DISPLAY_CHANGED is meant for a presenter to announce that it just
reset its own D3D device - see SyncRenderer.cpp's OnResetDevice() and
EVRAllocatorPresenter.cpp, both of which send this event right after
ResetDevice(), with a comment citing Microsoft's documentation
("Presenter should send this message").

quartz.dll's default filter graph manager reacts to EC_DISPLAY_CHANGED by
automatically Stop()-ing (and restarting) the whole graph on its own
worker thread (CFGControl::WorkerDisplayChanged -> CFilterGraph::Stop()).
Since ChangeOutputMediaFormat(2) is reached the first time a D3D11 decode
session discovers its real surface format - i.e. during completely normal
first-time playback, not only on a deliberate resolution change - this
fires on every channel tune, forcing the output pin through
CompleteConnect()/CD3D11Decoder::PostConnect() again and recreating the
D3D11 device a 2nd (sometimes 3rd) time within under a second of each
other. The resulting GPU resource churn can leave a freshly recreated
allocator's buffer creation failing for several seconds
(CD3D11SurfaceAllocator::Commit() returning E_FAIL), which has no
recovery path and leaves decode permanently stuck producing no frames.

Confirmed directly via logging: NotifyEvent(EC_DISPLAY_CHANGED, ...)
returning, and the second CompleteConnect(PINDIR_OUTPUT) firing, were
consistently ~100ms apart across multiple repro runs. Also confirmed this
isn't 4K/8K-specific - the same multi-CompleteConnect() churn happens with
regular 2K H.264 content too, it just doesn't reliably lose the GPU-resource
race there since the surfaces involved are much smaller.

Fix

Send the new media type the same in-place way ReconnectOutput() already
does for resolution changes, instead of EC_DISPLAY_CHANGED: find a media
type the connected pin's QueryAccept() accepts, call
m_pOutput->SetMediaType(&mt), and set m_bSendMediaType = true so
CD3D11Decoder::DeliverFrame() carries it on the next delivered sample via
IMediaSample::SetMediaType() - no connection-level reconnect, no
EC_DISPLAY_CHANGED, no extra D3D11 device churn.

Testing

Repeated TVTest startup auto-tune and live switching between MPC Video
Renderer and EVR - both reliably showed no video before this fix, both work
immediately after.

[Aleksoid1978]

How to reproduce ?

I checked this code when I wrote it, and everything works correctly.
But what you're proposing sounds like nonsense, no offense intended.

Test file with 2 different video streams - https://drive.google.com/file/d/1Gfpyq_FjU9gGloPj-sWjq_vIzbYkPqZa/view?usp=sharing

[nanamitm]

Sorry for the confusion - you're right that this specific test doesn't reproduce it. I actually tested AVC_MIX.ts directly (seeking through the whole clip) and confirmed ChangeOutputMediaFormat() is never called while playing it.

The trigger isn't a resolution change. It's CD3D11Decoder::ReInitD3D11Decoder() detecting that the D3D11 surface pixel format changed (bChangeFormat, comparing the previous m_SurfaceFormat against the newly mapped DXGI format from c->sw_pix_fmt) - e.g. NV12 vs P010. AVC_MIX.ts is 8-bit H.264 throughout, so that branch (and ChangeOutputMediaFormat(2)'s renderer notification) never fires for it.

I don't have a small synthetic file that reliably triggers this, but I have exact timestamped logging from the real scenario (D3D11 + 10-bit HEVC) showing the mechanism directly, not as a hypothesis:

19:34:57.130 : ChangeOutputMediaFormat() #1 enter: nType=2
19:34:57.130 : ChangeOutputMediaFormat() #1 calling NotifyEvent(EC_DISPLAY_CHANGED, ...)
19:34:57.130 : ChangeOutputMediaFormat() #1 NotifyEvent(EC_DISPLAY_CHANGED, ...) -> hr=0x00000000
19:34:57.130 : ChangeOutputMediaFormat() #1 exit: hr=0x00000000
19:34:57.228 : CompleteConnect(PINDIR_OUTPUT) enter: m_bReinit=0   <- second connect, 98ms later

That second CompleteConnect(PINDIR_OUTPUT) only happens because of the NotifyEvent(EC_DISPLAY_CHANGED, ...) call right above it - nothing else touches the output pin's connection in between. This is quartz.dll's stock CFGControl::WorkerDisplayChanged() reacting to the event by Stop()-ing the graph on its own worker thread and reconnecting - documented EC_DISPLAY_CHANGED behavior, just triggered here for the wrong reason (an output-format notification, not an actual device reset).

If you want to verify directly: add one DLog() at the top of ChangeOutputMediaFormat()'s renderer branch and test with any 10-bit HEVC content - it should fire as soon as the surface format gets refined.

The actual user-visible symptom: a D3D11 video renderer ends up recreating its D3D11 device because of that unwanted second CompleteConnect(). On 8K-sized surfaces this occasionally lost a race against GPU resource cleanup and CD3D11SurfaceAllocator::Commit() returned E_FAIL for several seconds, leaving decode stuck with no video until the user toggled playback. This fix shouldn't change behavior for any setup where nothing actually resets a device - it only stops broadcasting a device-reset event when no device was reset.

[Aleksoid1978]

AVC_MIX.ts - need select different video streams.

[v0lt]

1. What is TVTest?
2. Have you tried LAV Video Decoder or any other decoders?
3. Have you tried using MPC Video Decoder in other video players?