Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Prydwen - SST Physical Channel Library

Prydwen is a physical channel library for SST-based simulations. It provides the pluggable transport layer used by Mordred routers and NICs, allowing the physical link implementation to be swapped without modifying routing or flow-control logic.

An abstract subcomponent API and generic implementation allow for other networking libraries (for example, merlin) to eventually be expanded to use this library.

While the only physical transport layer implemented today is UCIe, the model is expandable to other protocols.

Prydwen registers a single SST element library (prydwen) containing all components. UCIe-specific components follow the naming convention prydwen.ucie<Thing>:

SST name Description
prydwen.genericPhysChannel Generic raw-link PhysChannelAPI implementation
prydwen.uciePhysChannel UCIe adapter-layer PhysChannelAPI with FLIT + flow control
prydwen.ucieInterfaceSN UCIe SimpleNetwork subcomponent for multi-endpoint topologies
prydwen.ucieTestEndpointSN Test endpoint component for ucieInterfaceSN

Note

The UCIe timings have not been validated against another tool/test suite/approach; that is extremely high on the TODO list.


Requirements

  • CMake >= 3.19
  • SST >= 15.0
  • C++17 compiler
  • sst and sst-config must be on PATH

Building

mkdir build && cd build
cmake ..
make -j
make install
make test

If using Mordred with Prydwen (default case, but not required), Prydwen must be built and registered first — Mordred's build system references prydwen/src/ for the PhysChannelAPI.h header.


Relationship to Mordred

PhysChannelAPI and GenericPhysChannel originated in the Mordred repo and were moved here to separate the physical transport layer from the NoC logic. Mordred's MordredNicPC and RtrPortControlPC components still use PhysChannelAPI but now find the header via Mordred's PRYDWEN_SRC_DIR CMake variable (see mordred/src/CMakeLists.txt).

When writing Mordred simulation scripts, use prydwen.genericPhysChannel for the raw-link transport.


Repository Layout

src/             PhysChannelAPI + GenericPhysChannel
src/ucie/        UCIe specialization (compiled into the same prydwen library)
tests/           SST Python test scripts
scripts/         SST version-detection helper scripts

Each transport layer should have a separate folder in src.


Components

PhysChannelAPI (abstract)

Abstract base class (SST::Prydwen::PhysChannelAPI) for physical channel subcomponents. Defined in src/PhysChannelAPI.h. Subcomponent slot API:

SST_ELI_REGISTER_SUBCOMPONENT_API(SST::Prydwen::PhysChannelAPI, int /*num_vns*/)

Implementations provide: send(), recv(), sendUntimedData(), recvUntimedData(), spaceToSend(), requestToReceive(), setNotifyOnReceive(), isNetworkInitialized(), getLinkBW(), getFlitPayloadBytes().

Lifecycle forwarding requirement. SST does not automatically propagate init(), setup(), complete(), or finish() to SubComponents loaded with ComponentInfo::SHARE_NONE. Any parent component that loads a SimpleNetwork subcomponent (e.g. mordredNicPC) with SHARE_NONE must explicitly forward all four lifecycle calls to the subcomponent in its own overrides. If init() is not forwarded, uciePhysChannel never runs its UCIe handshake, the link stays uninitialized, and mordred's credit exchange silently fails. mordred.mordredTestEP and merlin.test_nic both forward all four calls correctly and serve as reference implementations.

getFlitPayloadBytes() returns the maximum payload bytes that fit in a single physical flit, allowing callers to decompose large events before calling send(). Returns 0 if the implementation has no flit-size constraint (the default for non-UCIe implementations).


prydwen.genericPhysChannel

A minimal PhysChannelAPI implementation. Wraps any SST::Event* in a thin PhysChannelLinkEvent envelope (carrying the VN tag) and forwards it over a raw SST::Link. Both ends of a link must use a compatible channel implementation.

Parameters

Parameter Description Default
port_name Name of the SST link to configure (e.g. "port0") "port"
verbose Verbosity level 0

prydwen.uciePhysChannel

A multi-stack UCIe adapter-layer PhysChannelAPI implementation. Applies the full UCIe wire protocol — FLIT serialization, credit-based flow control, and the Init/Agree handshake — to raw SST::Event* objects. Acts as a drop-in replacement for prydwen.genericPhysChannel in Mordred simulations.

Key parameters

Parameter Description Default
endpoint_id Numeric ID for this die endpoint 0
link_latency Die-to-die propagation + adapter latency "2ns"
port_name SST link port name ("port" when inside mordredNicPC) "ucie_link"
num_stacks Number of UCIe protocol stacks (1 or 2) 1
flit_format UCIe FLIT format 1–6 5
multi_protocol_mode Stack mux mode when num_stacks=2: "static" or "enhanced" "enhanced"
verbose Verbosity level 5

prydwen.ucieInterfaceSN

An SST::Interfaces::SimpleNetwork subcomponent that carries multiple virtual networks over a UCIe link using FLIT multiplexing. For multi-endpoint topologies.

Credit accounting for large messages. Each send() call deducts ceil(size_bytes / flit_payload_bytes) credits from the sender's available pool. The receiver returns the same number of credits upon receipt so the pool remains balanced. Messages that fit within a single flit slot cost 1 credit; messages larger than flit_payload_bytes (e.g. > 240 B for format 5) cost multiple credits but are still transmitted as a single wire event. credits_per_vn must be large enough to keep the desired number of large messages in-flight simultaneously.


prydwen.ucieTestEndpointSN

A test component that drives traffic across multiple VNs through ucieInterfaceSN. Useful for verifying credit flow and BW-limit behavior.


Tests

Tests live in tests/ and are run via make test from the build directory. All tests require the prydwen library to be registered in SST; the physch tests additionally require the mordred library.

Test script Description
test_ucie_single_stack_basic.py Minimal single-stack smoke test, ucieTestEndpointSN driver
test_ucie_static.py Static (Multi_Protocol_Enable) NOP-insertion mode
test_ucie_multistack.py 2-stack point-to-point, ucieTestEndpointSN, enhanced mode
test_ucie_enhanced_bw_limit_01.py Enhanced mode BW limit "0,1": stack 1 limited
test_ucie_enhanced_bw_limit_10.py Enhanced mode BW limit "1,0": stack 0 limited
test_ucie_enhanced_bw_limit_11.py Enhanced mode BW limit "1,1": both stacks limited
test_ucie_enhanced_bw_asymmetric_neg.py Asymmetric OR negotiation: endpoints advertise opposite per-stack limits
test_ucie_large_message.py Messages > flit_payload_bytes; exercises multi-credit deduction/return
test_ucie_loopback.py Self-addressed sends; no wire traffic, credits not consumed
test_ucie_flit_format2.py FLIT format 2 (68B wire / 64B payload) at 16 GT/s
test_ucie_multimodule.py num_modules=2 bonded; verifies BW and serialization-delay scaling
test_ucie_merlin_nic.py Single-stack with merlin.test_nic
test_ucie_physch_basic.py 2-router Mordred mesh with UCIePhysChannel links
test_ucie_physch_3x3.py 3×3 Mordred mesh, 18 endpoints, UCIePhysChannel
test_ucie_physch_multistack.py 2-router Mordred mesh, num_stacks=2; uses mordred.mordredTestEP

Acknowledgments

This work was supported by the U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research program under project 84245 — Democratization of Co-design for Energy-Efficient Heterogeneous Computing (DeCoDe) at Pacific Northwest National Laboratory (PNNL). PNNL is a multi-program national laboratory operated for the U.S. Department of Energy (DOE) by Battelle Memorial Institute under Contract No. DE-AC05-76RL01830.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages