feat(gta-streaming-five): give each resource its own pgRawStreamer#4072
feat(gta-streaming-five): give each resource its own pgRawStreamer#4072xalva98 wants to merge 1 commit into
Conversation
DaniGP17
left a comment
There was a problem hiding this comment.
LGTM, I've also tested and everything seems to work fine @prikolium-cfx
|
Thank you alva for the contribution and Dani for the review. |
|
Btw RedM failed to build with that. |
Sorry for missing that, checking right now |
|
@xalva98 uint32_t* RegisterRawStreamingFileWithTag(uint32_t* fileId, const char* fileName, bool unkTrue, const char* registerAs, bool errorIfFailed, const std::string& tag)
{
return RegisterRawStreamingFile(fileId, fileName, unkTrue, registerAs, errorIfFailed);
}
STREAMING_EXPORT rage::fiCollection* GetRawStreamerByIndex(uint16_t idx)
{
return nullptr;
}
STREAMING_EXPORT bool IsRawStreamerCollection(uint16_t idx)
{
return idx < 2;
}
STREAMING_EXPORT std::vector<RawStreamerInfo> GetRawStreamerInfos()
{
return {};
} |
should be fixed, thank you 👍 |
|
Sadly, after putting in some more testing each ressource having a streamer is most likely not possible. So I changed the approach to just increase to up to 32 (could be reduced even further technically). The streamers are shared, if they are about to overflow a new "bucket" is created so to speak. |
The high-volume raw asset types (ytd, ybn, yft, ydd, ydr, ymap) each get their own pgRawStreamer, allocated lazily from free sm_Collections slots and rolled over to a fresh streamer once one fills, so the 65535 entry limit no longer applies across the whole server. The stock streamers are untouched: base game content stays in slot 0 and the ytd/user mod streamers (1, 2) keep their original routing, so only tagged resource content is diverted.
Goal of this PR
Raise the raw-streaming file limit so large servers stop hitting
"pgRawStreamer is full".
A
pgRawStreameraddresses its entries with a 16-bit file index, so a singleraw streamer fills up at 65535 files. Servers that register a lot of loose
stream assets overflow that shared streamer, after which streaming breaks.
How is this PR achieving the goal
Each resource gets its own
pgRawStreamer, allocated lazily on the firststream file it registers, so the 65535 limit becomes per-resource instead of
shared across the whole server.
already available at registration time
(
GetRawStreamerForFileWithTag/RegisterRawStreamingFileWithTaginStreaming.cpp). Files that would otherwise land in the shared game (0) orytd (1) streamer are diverted;
faux_pack/addons:and untagged fileskeep the existing shared streamers.
index, so per-resource streamers just occupy their own collection slots and
total capacity scales with the number of streaming resources.
sm_Collectionsslots (packfiles alreadyclaim some by the time resources load, so a free slot is searched for rather
than assumed).
MaxPackfilesis read from the game's packfileslot-allocation loop so the bound follows the game across builds.
further resources bucket into streamers already allocated, so packfiles are
never starved - this should ensure that nothing can cause crashes in the future.
IsRawHandleis now a membership test (per-resource streamers sit atnon-contiguous collection indices), and the "Loaded pgRawStreamer assets"
devtools window lists every streamer with its owning resource and fill level.
This PR applies to the following area(s)
FiveM, Streaming
Successfully tested on
Game builds: 3258


Platforms: Windows
Pattern
B8 ? ? ? ? 66 3B D0 72has been found in all game builds up to 3788Tested locally with a bunch of copied .ytd files
Checklist
Fixes issues
Fixes the "pgRawStreamer is full" (65535 raw file) limit on large servers.
PS: This idea was taken from prikolium
Note: You can't have infinite pgRawStreamers, there's an upper limit (which should increase every gamebuild)
For 3258: MaxPackfiles (3928) − reserve (512) ≈ 3,416 ressources, so unless you have 3400 ressources on your server you should be golden.