Add guidance on signal concurrency limits to the Temporal Cloud System Limits docs

[dallastexas92]

Temporal Affiliation

• I am a Temporal employee

Content Suggestion

On this docs page detailing Temporal system limits, it details that a single Workflow execution may receive up to 10,000 Signals, but our docs omit the other limit/recommendation about how many signals can be received at any given time (per second). Adding the following would be hugely beneficial and comes straight from Maxim

Temporal Signal Concurrency Guidelines:

• The recommended guidance is to limit signals to ≤5 per second sustained for optimal workflow performance
• The theoretical limit is tied to database latency (1/database_latency), so ~20/sec with 50ms latency
• All updates to a workflow are performed under a single lock, and workflows need resources for other operations beyond just signal processing - executing workflow tasks, activities, etc., which all require database updates
• If a signal is received when completing (or continuing as new), then the workflow task is rejected and the whole workflow should be recovered. The constant stream of signals might prevent the workflow from ever closing, which is why 5/sec is the practical recommendation rather than the theoretical max