Add Service Bus SKU configuration to Terraform resources

[marrobi]

Introduce a configurable SKU for the Azure Service Bus, allowing users to select between Standard and Premium options.

The Standard SKU does not support VNet integration hence is exposed publically, also message size is limited. however this will significantly reduce costs for development scenarios.

Have a feeling upgrade will fail due from prior version and needs testing.

TODO: Add Service tags to firewall network rules.

[github-actions]

Unit Test Results

0 tests   0 ✅  0s ⏱️
0 suites  0 💤
0 files    0 ❌

Results for commit c0c3c4b.

♻️ This comment has been updated with latest results.

[marrobi]

Depends on #4342

[marrobi]

I added the firewall rules. Things seem to work, however large messages is an outstanding issue. Copilot says:

---

Service Bus Standard SKU Message Size Issues

Problem

Switching from Premium to Standard SKU reduces the maximum message size from 2048 KB to 256 KB. Several components currently send messages that exceed this limit.

Affected Components

Resource Request Messages

• Files: api_app/service_bus/resource_request_sender.py, api_app/models/domain/resource.py
• Issue: Resource-request messages carry the entire resource properties and, for cascaded installs, every dependency. Templates embedding scripts, certificates, or CAS JSON routinely exceed Standard's 256 KB envelope, so those jobs never reach the Resource Processor.

Deployment Status Updates

• File: resource_processor/vmss_porter/runner.py
• Issue: service_bus_message_generator inlines full Porter logs/errors plus the parsed outputs array. Terraform failures easily top 300 KB, so Standard would return MessageSizeExceeded, leaving Porter runs aborted and the API waiting forever for completion events.

Pipeline Steps

• File: api_app/service_bus/deployment_status_updater.py
• Issue: Each additional pipeline step calls send_deployment_message, re-sending large resource definitions. Multi-step operations therefore multiply payload size even if a single step might have fit.

Airlock Step Results

• Files: api_app/service_bus/airlock_request_status_update.py, airlock_processor/StatusChangedQueueTrigger, BlobCreatedTrigger, ScanResultTrigger
• Issue: Step-result messages include request_files arrays with name+size per file. Large import/export requests or malware verdicts with long status_message values exceed Standard limits quickly.

Data Deletion & Event Grid Payloads

• Files: airlock_processor/DataDeletionTrigger and other Service Bus triggers
• Issue: Deserialize full Event Grid payloads containing copied_from histories. Multiple copy hops inflate that metadata until it approaches the 256 KB cap.

Infrastructure Configuration

• File: core/terraform/servicebus.tf
• Issue: Only raises max_message_size_in_kilobytes when SKU = Premium (2048 KB). Dropping to Standard removes the override so every queue reverts to the default 256 KB without any chunking/offload logic in Python.

Impact

Without shrinking/offloading payloads, the Standard SKU will break:

1. workspacequeue - Resource requests for large templates or cascaded operations will be rejected
2. deploymentstatus - Messages carrying verbose Porter outputs/errors will fail, breaking status propagation
3. Airlock queues (airlock-step-result, etc.) - Will overflow when enumerating many files or attaching rich verdict metadata

Required Changes

Need to implement payload offloading to blob storage and include only references in Service Bus messages when content exceeds ~200 KB threshold.