fix: fail fast when amendment blocked instead of degraded state#707
Draft
sublimator wants to merge 5 commits into
Draft
fix: fail fast when amendment blocked instead of degraded state#707sublimator wants to merge 5 commits into
sublimator wants to merge 5 commits into
Conversation
- signalStop() for graceful shutdown when unsupported amendment activates - early shutdown ~1 minute before expected activation to avoid race - try/catch in switchLastClosedLedger to survive unknown field crashes during shutdown window - show amendment warning to all RPC users, not just admin Fixes: #706
4cf2be8 to
1e6cda4
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## dev #707 +/- ##
=======================================
- Coverage 77.3% 77.3% -0.0%
=======================================
Files 837 837
Lines 78585 78602 +17
Branches 11559 11572 +13
=======================================
Hits 60738 60738
- Misses 17837 17854 +17
Partials 10 10
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Amendment-blocked nodes currently enter a degraded state: they stop validating
but keep running, serving data from ledgers produced under rules they don't
understand. This can lead to crashes
(#706) when the node encounters
unknown serialized fields in post-amendment ledgers.
This PR replaces the degraded state with a graceful shutdown. Operators get
warnings during the 2-week amendment majority window, and the server exits
cleanly rather than continuing to operate in an unsafe state.
Problem
When an unsupported amendment activates,
STObject::setthrowsstd::runtime_erroron unknown fields:📍
src/libxrpl/protocol/STObject.cpp:249-257Crash path (issue #706)
The JUMP path in
switchLastClosedLedgercallsprocessClosedLedger, whichiterates
view.txsto compute fee metrics — deserializing every transaction:📍
src/xrpld/app/misc/detail/TxQ.cpp:109-116This triggers
deserializeTxPlusMeta→STTx→STObject::set→ crash.doAdvancehas a try/catch that survives this, butswitchLastClosedLedgerdid not.
Why fail-fast is safer than the degraded state
The amendment-blocked degraded state was introduced in 2017 but has not been
exercised in practice — binaries on the network tend to support all known
fields. When it is triggered by a genuinely unknown amendment, the behavior
is unsafe.
Post-amendment ledgers are stored before the amendment check
setValidLedgerstores new validated ledgers before checking amendmentblocked status. Post-amendment ledgers accumulate and become queryable:
📍
src/xrpld/app/ledger/detail/LedgerMaster.cpp:278-291Inconsistent RPC behavior
The RPC condition check only blocks RPCs that require network/ledger state:
📍
src/xrpld/rpc/detail/Handler.h:85-89But the majority of RPCs — including all account and ledger queries — are
registered with
NO_CONDITIONand bypass this check:📍
src/xrpld/rpc/detail/Handler.cpp:84-95Along with
ledger_data,ledger_entry,ledger_header,book_offers,gateway_balances,transaction_entry, and many more.This means an amendment-blocked node gives inconsistent responses — some
RPCs return
rpcAMENDMENT_BLOCKEDwhile others serve data from ledgersproduced under rules the server doesn't understand. It also means other
deserialization paths may be exposed to unknown fields:
pubLedger→AcceptedLedgerconstructor (with active subscribers):📍
src/xrpld/app/misc/NetworkOPs.cpp:2922-2929📍
src/xrpld/app/ledger/AcceptedLedger.cpp:26-48LedgerReplayconstructor (during ledger replay sync):📍
src/xrpld/app/ledger/detail/LedgerReplay.cpp:25-36LedgerHistory::log_one(on ledger mismatch):📍
src/xrpld/app/ledger/LedgerHistory.cpp:158-178A clean shutdown avoids all of these concerns.
Changes
1. Graceful shutdown on amendment activation
setAmendmentBlocked()now callssignalStop()to initiate a clean shutdowninstead of leaving the server in a degraded CONNECTED mode. The
amendmentBlocked_flag is still set so RPCs getrpcAMENDMENT_BLOCKEDduring the shutdown window. Skipped in standalone mode for test compatibility.
Logs at fatal level (
NetworkOPs:FTL) so the reason is unmissable.📍
src/xrpld/app/misc/NetworkOPs.cpp:1632-16472. Early shutdown before amendment activation
When an unsupported amendment is within 1 minute of its expected activation
time, shut down early. Since amendments require a 2-week majority period,
operators have had ample warning. The early shutdown avoids the server
encountering post-amendment ledger data it cannot deserialize.
📍
src/xrpld/app/ledger/detail/LedgerMaster.cpp:316-3273. Try/catch in switchLastClosedLedger
Safety net for the case where the server missed the early shutdown window
(e.g., it was restarted right as the amendment activates). Only swallows the
exception when amendment blocked; rethrows otherwise:
📍
src/xrpld/app/misc/NetworkOPs.cpp:1806-18174. Amendment warning visible to all RPC users
The unsupported-majority warning was previously admin-only. Now all RPC
users see it, giving operators the best chance to upgrade during the 2-week
window.
Test plan
ripple.server.ServerStatusandripple.app.AmendmentBlockedtests pass (standalone mode skips
signalStop)shuts down gracefully instead of crashing
server_inforesponsesduring the majority period
NetworkOPs:FTLlog line appears on shutdown