Foundation Classes - Fix Resource_Manager::Debug and Storage_Schema current-data data races - #1399
Conversation
…aces (v1.15.18) Fixes the two upstream OCCT foundation-layer races #371's confirmation harness turned up (OCCT#1398): Resource_Manager::Debug is now a std::atomic<bool>, and Storage_Schema::ICurrentData() is guarded by a new recursive mutex spanning construction, Write(), and the type-binding accessors. Kernel-only change (Scripts/patches/0016); OCCT.xcframework rebuilt, OCCTBridge.xcframework unchanged. Upstream fix proposed as Open-Cascade-SAS/OCCT#1399 (fixes OCCT#1398). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
@gsdali I think |
…urrent-data data races Resource_Manager::Resource_Manager(const char*, bool) writes a file-scope static bool Debug on every construction with zero synchronization; two Resource_Manager instances constructed concurrently on different threads (e.g. two independent TDocStd_Application instances each lazily building their own via Resources()/DefineFormat()) race on this write. Debug becomes std::atomic<bool>. Storage_Schema::ICurrentData() is a function-local static Handle(Storage_Data) shared by every Storage_Schema in the process and mutated with no synchronization: Write() sets it for the duration of one store, and any Storage_Schema construction calls Clear() -> ICurrentData().Nullify(), including throwaway instances built only to read header info (PCDM_ReadWriter_1::ReadReferenceCounter, ReadReferences and ReadDocumentVersion, all reached on every Open()). One instance's construction can therefore null out an unrelated instance's in-flight Write() on another thread. - Replace the global with a mutable Handle(Storage_Data) myCurrentData field on Storage_Schema; no locking is needed once the state is per instance. - Remove the private statics ICurrentData() and ISetCurrentData(), which had no reference outside the class. - Drop the static from AddPersistent()'s TCollection_AsciiString aTypeName: it is assigned from the tName argument and read two lines later, so the static only saved an allocation while adding a second unsynchronized process-wide object. Every Storage_Schema in the tree is constructed locally by its caller (PCDM_StorageDriver::Write, and PCDM_ReadWriter_1 at three sites), none is cached or shared, and Storage_CallBack::Add/Write/Read all take the driving schema as an argument, so the BindType/AddPersistent/PersistentToAdd calls made from callbacks run against that same instance. The other Storage_Schema users in the tree use only the statics CheckTypeMigration() and ICreationDate(), neither of which reads the current data. Both races were found while testing a private TDocStd_Application-per-caller usage pattern: multiple independent application instances constructed concurrently, never sharing state. See the linked issue for the ThreadSanitizer writeup and reproducer.
09a6117 to
773fa3e
Compare
|
Done, and thanks for the pointer: I checked the assumptions the field relies on before changing it:
It also fixes strictly more than the mutex did. Under the lock, a throwaway schema built by One extra change rode along in the same commit, because removing the lock re-exposes it: ThreadSanitizer: 0 races at 8x50, 8x30 and 10x60 with the field, matching what the mutex revision gave, against 13 race warnings plus an abort on the unpatched kernel at 8x30. Re-run after the On whether to split: I have kept |
Description
Fixes #1398.
Resource_Manager::Resource_Manager(const char*, bool)(Resource_Manager.cxx:109) writes afile-scope
static bool Debugon every construction with zero synchronization. TwoResource_Managerinstances constructed concurrently on different threads (e.g. two independentTDocStd_Applicationinstances each lazily building their own viaResources()/DefineFormat())race on this write. Fixed by making
Debugastd::atomic<bool>: it is a plain process-widedebug-logging flag, not per-instance intent, so an atomic is sufficient here.
Storage_Schema::ICurrentData()(Storage_Schema.cxx:802) is a function-local staticHandle(Storage_Data), shared by everyStorage_Schemain the process and mutated with nosynchronization anywhere.
Write()sets it for the duration of one store, and anyStorage_Schemaconstruction callsClear()->ICurrentData().Nullify()in its constructor,including throwaway instances built only to read header info
(
PCDM_ReadWriter_1::ReadReferenceCounter,ReadReferencesandReadDocumentVersion, all reachedunconditionally from a plain
TDocStd_Application::Open()viaCDF_Application::Retrieve). Oneinstance's construction can therefore null out a different, unrelated instance's in-flight
Write()on another thread, or race a concurrent load's own throwaway construction.Fixed by removing the global rather than guarding it, per @gkv311's review below: the handle
becomes a
mutable Handle(Storage_Data) myCurrentDatafield onStorage_Schema, so there is noshared state left to synchronize.
Write()assigns the field,Clear()nullifies its own, andHasTypeBinding()/BindType()/TypeBinding()/AddPersistent()/PersistentToAdd()read itthrough
*this. The private staticsICurrentData()andISetCurrentData()are removed.mutablebecause all of those methods areconst.Nothing in the tree relies on the state being process-wide:
Storage_Schemais constructed locally by its caller and used only there(
PCDM_StorageDriver::Write, andPCDM_ReadWriter_1at three sites). None is cached or shared.Storage_CallBack::Add/Write/Readall take the driving schema as an argument, so theBindType/AddPersistent/PersistentToAddcalls made from per-type callbacks during aWrite()run against that same instance.
Storage_Schemausers in the tree (BinLDrivers,XmlMDF,StdLDrivers) use only thestatics
CheckTypeMigration()andICreationDate(), neither of which reads the current data.ICurrentData()andISetCurrentData()were bothprivatewith no reference outside the class,so removing them is not an API change.
The same commit drops the
staticfromAddPersistent()'sTCollection_AsciiString aTypeName, asecond unsynchronized process-wide object in the same class. It is assigned from the
tNameargument and read two lines later, so
staticonly ever saved an allocation, and it was previouslycovered only incidentally by the mutex this revision removes.
Both races are invisible under the traditional single shared
XCAFApp_Application/TDocStd_Applicationusage pattern, because thereResources()'s own lazy-init mutex happens toserialize
Resource_Manager/Storage_Schemaconstruction down to "runs once, ever, for the wholeprocess." They only become reachable once an application is constructed privately per caller or
thread (the pattern this project's own
TDocStd_Applicationdocumentation recommends over theXCAFApp_Application::GetApplication()singleton) and multiple independent instances are builtconcurrently. See the linked issue for the full TSan writeup and a standalone reproducer.
No public API signature changes. The revision is a net reduction: 15 insertions, 31 deletions.
Type of change
How Has This Been Tested?
ThreadSanitizer (minimal-module build: FoundationClasses + ModelingData + ModelingAlgorithms +
DataExchange,
RelWithDebInfo,-fsanitize=thread -g), each round building a brand-new privateTDocStd_Application, defining formats, populating and saving a document, closing it, then aseparate brand-new private application opening, verifying and closing it. No two threads ever
share an application or schema instance.
Resource_Manager::Resource_Manager, 11xStorage_Schema::Storage_Schema'sClear()call) at 8 threads x 30 rounds, 0 functionalfailures.
Re-run after the
aTypeNamechange with the same result.Also run against the wider concurrency suite that the downstream project maintains for this kernel
(10 scenarios covering the fixes from #1374, #1388, #1390, #1394 and #1397 plus this one): all
clean, no regression. Full downstream test suite (4666 tests) clean against a production build of
the patched kernel.
Reproducer and full writeup:
https://github.com/SecondMouseAU/OCCTSwift/tree/main/Scripts/repro/374-resource-manager-storage-schema-race
Checklist:
clang-format --dry-run --Werrorclean onevery touched file)
internal-only change)