Urma transport - #14
Open
Gzure wants to merge 54 commits into
Open
Conversation
* change the timeout checker bthread to timer bthread * refine ProgressiveReadTimeoutReader class hold SocketId and read_timeout_ms fields * refine socektId access method, change HandleIdleProgressiveReader belong and logic
* change the timeout checker bthread to timer bthread * refine ProgressiveReadTimeoutReader class hold SocketId and read_timeout_ms fields * refine socektId access method, change HandleIdleProgressiveReader belong and logic
Add a fourth transport (SOCKET_MODE_URMA) that uses openEuler's URMA (Unified Remote Memory Access) SDK for remote-memory RPC, modeled on the existing RdmaTransport design. Targets super-node (UB-bus-interconnected) clusters as Route B of apache#3217, complementing the OBMM-based UBRing transport (Route C double-backend) for large-packet / cross-node paths. Design (mirrors RdmaTransport / UBShmTransport): - UrmaTransport + urma::UrmaEndpoint two-layer structure - Tri-state URMA_ON/OFF/UNKNOWN with TCP fallback - Dual-plane handshake (TCP control + URMA data), v2 binary / v3 protobuf - Dual-window credit flow control, URMA_OPC_SEND send path, JFC poll recv - import_seg-before-import_jetty (TP routing fix, per yalantinglibs) - Dedicated buffer pool (mmap + urma_register_seg, IOBuf allocator swap) CI without URMA hardware: - Bundled link-time mock (src/brpc/urma/mock_urma.cpp), modeled on Mooncake's mock_urma.cpp, provides fake urma_* symbols when liburma is absent - CMake falls back to the mock (URMA_USE_MOCK=1) instead of FATAL_ERROR - brpc_urma_unittest covers handshake wire-format, state machine, fallback, and mock init/post/poll smoke tests Referenced implementations: - yalantinglibs urma_socket.hpp (event mode, buffer pool, import order) - ubs-comm UrmaApi dlopen bindings + RAII wrappers - Mooncake mock_urma.cpp (mock pattern) Proposal: docs/cn/urma_proposal.md (with mermaid diagrams + mock design §7.1)
…hake 1. SDK headers: delete vendored urma_api.h/urma_types.h/urma_opcode.h. Use system-installed URMA SDK via #include "urma_api.h" (same pattern as RDMA's #include <infiniband/verbs.h>). CMake uses find_path(URMA_INCLUDE_PATH NAMES urma_api.h) + include_directories when not using the mock. 2. Handshake: remove v3 protobuf (urma_handshake.proto + UrmaHello message + v3 handshake classes). Keep only v2 binary handshake (HelloMessage Serialize/Deserialize via HostToNet/NetToHost + ReadFromFd/WriteToFd), matching RDMA's default v2 approach. Remove urma_client_handshake_version gflag. Handshake is now plain free functions, not a class hierarchy. 3. Remove stale friend declarations for deleted handshake classes.
Bring back the v3 protobuf handshake alongside v2 binary, matching RDMA's dual-version approach. The v3 path uses a plain protobuf message (UrmaHello, cc_generic_services=false) serialized over TCP via ReadFromFd/WriteToFd -- not an RPC service. - Restore urma_handshake.proto (UrmaHello message) - Restore UrmaHandshakeClient/ServerV2/V3 classes + factory functions - Restore FillLocalHelloV3/WriteHelloV3/ReadAndParseHelloV3 in endpoint - Restore urma_client_handshake_version gflag - Restore v3 protobuf round-trip test - Keep system include pattern (#include "urma_api.h", not vendored) - Restore v3 references in docs/proposal
# Conflicts: # CMakeLists.txt
protobuf proto2 only supports uint32/uint64 (not uint8/uint16). Change tp_type field from uint8 to uint32. The C++ code already casts to/from uint8_t where needed.
…ation When URMA SDK is not installed (URMA_USE_MOCK=1, CI default), the transport source files and mock_urma.cpp still need the URMA type definitions (urma_jfc_t, urma_jetty_t, etc.) to compile. Restore the 3 SDK headers (urma_api.h, urma_types.h, urma_opcode.h) under src/brpc/urma/sdk/ and add that directory to the include path ONLY in mock mode. When the real SDK is installed, the system include path is used instead (matching RDMA's pattern).
The system URMA SDK installs headers at /usr/include/ub/umdk/urma/ (not flat under /usr/include). Update: - find_path: search for 'urma/urma_api.h' (not 'urma_api.h') - All source includes: '#include "urma/urma_api.h"' (not '"urma_api.h"') - SDK header internal includes: 'urma/urma_types.h' (not 'urma_types.h') - Bundled mock headers: move from sdk/ to sdk/urma/ to match layout This way find_path resolves to /usr/include/ub/umdk when the real SDK is installed, and the mock's sdk/ directory provides the same urma/ subdir structure.
The URMA SDK installs headers at /usr/include/ub/umdk/urma/urma_api.h (two levels under /usr/include). CMake's default find_path does not search this non-standard subdirectory. Add PATHS and PATH_SUFFIXES to cover common install layouts: PATHS /usr/include /usr/local/include PATH_SUFFIXES ub/umdk ub/umdk/urma umdk umdk/urma
…matches 1. Add friend declarations for urma::UrmaEndpoint/UrmaConnect/ UrmaHandshakeClient/ServerV2/V3 in socket.h (same as RDMA). Without these, UrmaEndpoint cannot access Socket private members (_read_buf, _transport, _app_connect, _user, _fd, _io_event, WaitEpollOut, WakeAsEpollOut). 2. Add HELLO_BODY_LEN and HELLO_PACKET_LEN constants to v2_wire namespace in urma_handshake.h (referenced by FillLocalHelloV2). 3. Add WriteToFd(IOBuf&) overload declaration to urma_endpoint.h (implementation exists in .cpp but was not declared in .h). 4. Fix ProcessNewMessage call: last_msg must be InputMessageClosure, not bool (matching RDMA's pattern). 5. Replace memset on non-trivial types (ParsedHello, HelloMessage) with value-initialization to avoid -Wclass-memaccess warnings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: resolve
Problem Summary:
What is changed and the side effects?
Changed:
add UrmaTransport impl
Side effects:
Performance effects:
Breaking backward compatibility:
Check List: