diff --git a/discv5/discv5-rationale.md b/discv5/discv5-rationale.md index 1b51b4a8..c480906d 100644 --- a/discv5/discv5-rationale.md +++ b/discv5/discv5-rationale.md @@ -48,11 +48,13 @@ Discovery v4 trusts other nodes to return neighbors according to an agreed dista metric. Mismatches in implementation can make it hard for nodes to join the network, or lead to network fragmentation. -#### 1.1.6 Secondary topic-based node index +#### 1.1.6 Topic-based Service Discovery -The protocol must support discovery of nodes via an arbitrary topic identifier. Finding -nodes belonging to a topic should be as fast or faster than finding a node with a certain -ID. +The protocol must support discovery of nodes that participate in a higher-level service or topic. A topic is identified by a fixed-length topic identifier in the same key space as node IDs. + +Topic-based service discovery should reuse the ordinary Node Discovery v5 network rather than requiring each service to operate its own bootstrap network or discovery DHT. + +Finding nodes that participate in a topic should be efficient even when the topic is supported by only a small fraction of the discovery network. At the same time, topic discovery should preserve the security benefits of sampling from a large global discovery network and should not concentrate discovery for a topic at a small set of predictable nodes. #### 1.1.7 Change replay prevention @@ -131,7 +133,7 @@ These attacks rely on being able to create many real nodes, or spoof many logica for a small number of physical endpoints, to form a large, isolated area of the network under the control of the malicious actor. The victim's discovery findings are directed into that part of the network, either to manipulate their traffic or to fully isolate them -from the network. +from the network. Additional security goals specific to topic-based service discovery (TopDisc) are listed in the Topic-based Service Discovery Protocol v5 rationale section below. ## Version Interoperability / Upgrade Paths @@ -176,8 +178,7 @@ discovery mechanism must be chosen. Another reason for UDP is communication latency: participants in the discovery protocol must be able to communicate with a large number of other nodes within a short time frame to establish and maintain the neighbor set and must perform regular liveness checks on -their neighbors. For the topic advertisement system, registrants collect tickets and must -use them as soon as the ticket expires to place an ad in a topic queue. +their neighbors. For TopDisc, advertisers collect tickets from registrars and retry registration after the indicated waiting time. Low-latency request/response interactions help advertisers renew advertisements and maintain placements despite churn. These protocol interactions are difficult to implement in a TCP setting where connections require multiple round-trips before application data can be sent and the connection @@ -207,7 +208,7 @@ understandable while providing a distributed database that scales with the numbe participants. Our system also relies on the routing table to allow enumeration and random traversal of the whole network, i.e. all participants can be found. Most importantly, having a structured network with routing enables thinking about DHT 'address space' and -'regions of address space'. These concepts are used to build the [topic-based node index]. +'regions of address space'. These concepts are used by TopDisc to construct service tables centred on topic identifiers. Kademlia is often criticized as a naive design with obvious weaknesses. We believe that most issues with simple Kademlia can be overcome by careful programming and the benefits @@ -219,8 +220,7 @@ The well-known 'sybil attack' is based on the observation that creating node ide essentially free. In any system using a measure of proximity among node identities, an adversary may place nodes close to a chosen node by generating suitable identities. For basic node discovery through network enumeration, the 'sybil attack' poses no significant -challenge. Sybils are a serious issue for the topic-based node index, especially for -topics provided by few participants, because the index relies on node distance. +challenge. Sybils are a serious issue for topic-based service discovery, especially for topics provided by few participants. An adversary may try to generate node IDs close to a chosen topic identifier, dominate registrars selected for that topic, or flood registrar ad caches with advertisements under its control. An 'eclipse attack' is usually based on generating sybil nodes with the goal of polluting the victim node's routing table. Once the table is overtaken, the victim has no way to @@ -342,78 +342,214 @@ disturb the operation of the protocol. Session keys per node-ID/IP generally pre replay across sessions. The `request-id`, mirrored in response packets, prevents replay of responses within a session. -## The Topic Index - -Using FINDNODE queries with appropriately chosen targets, the entire DHT can be sampled by -a random walk to find all other participants. When building a distributed application, it -is often desirable to restrict the search to participants which provide a certain service. -A simple solution to this problem would be to simply split up the network and require -participation in many smaller application-specific networks. However, such networks are -hard to bootstrap and also more vulnerable to attacks which could isolate nodes. - -The topic index provides discovery by provided service in a different way. Nodes maintain -a single node table tracking their neighbors and advertise 'topics' on nodes found by -randomly walking the DHT. While the 'global' topic index can be also spammed, it makes -complete isolation a lot harder. To prevent nodes interested in a certain topic from -finding each other, the entire discovery network would have to be overpowered. - -To make the index useful, searching for nodes by topic must be efficient regardless of the -number of advertisers. This is achieved by estimating the topic 'radius', i.e. the -percentage of all live nodes which are advertising the topic. Advertisement and search -activities are restricted to a region of DHT address space around the topic's 'center'. - -We also want the index to satisfy another property: When a topic advertisement is placed, -it should last for a well-defined amount of time. This ensures nodes may rely on their -advertisements staying placed rather than worrying about keeping them alive. - -Finally, the index should consume limited resources. Just as the node table is limited in -number and size of buckets, the size of the index data structure on each node is limited. - -### Why should advertisers wait? - -Advertisers must wait a certain amount of time before they can be registered. Enforcing -this time limit prevents misuse of the topic index because any topic must be important -enough to outweigh the cost of waiting. Imagine a group phone call: announcing the -participants of the call using topic advertisement isn't a good use of the system because -the topic exists only for a short time and will have very few participants. The waiting -time prevents using the index for this purpose because the call might already be over -before everyone could get registered. - -### Dealing with Topic Spam - -Our model is based on the following assumptions: - -- Anyone can place their own advertisements under any topics and the rate of placing ads - is not limited globally. The number of active ads for any node is roughly proportional - to the resources (network bandwidth, mostly) spent on advertising. -- Honest actors whose purpose is to connect to other honest actors will spend an adequate - amount of efforts on registering and searching for ads, depending on the rate of newly - established connections they are targeting. If the given topic is used only by honest - actors, a few registrations per minute will be satisfactory, regardless of the size of - the subnetwork. -- Dishonest actors may want to place an excessive amount of ads just to disrupt the - discovery service. This will reduce the effectiveness of honest registration efforts by - increasing the topic radius and/or topic queue waiting times. If the attacker(s) can - place a comparable amount or more ads than all honest actors combined then the rate of - new (useful) connections established throughout the network will reduce proportionally - to the `honest / (dishonest + honest)` registration rates. - -This adverse effect can be countered by honest actors increasing their registration and -search efforts. Fortunately, the rate of established connections between them will -increase proportionally both with increased honest registration and search efforts. If -both are increased in response to an attack, the required factor of increased efforts from -honest actors is proportional to the square root of the attacker's efforts. - -### Detecting a useless registration attack - -In the case of a symmetrical protocol, where nodes are both searching and advertising -under the same topic, it is easy to detect when most of the found ads turn out to be -useless and increase both registration and query frequency. It is a bit harder but still -possible with asymmetrical (client-server) protocols, where only clients can easily detect -useless registrations, while advertisers (servers) do not have a direct way of detecting -when they should increase their advertising efforts. One possible solution is for servers -to also act as clients just to test the server capabilities of other advertisers. It is -also possible to implement a feedback system between trusted clients and servers. +# Topic-based Service Discovery Protocol v5 - Rationale + +This section explains the rationale for TopDisc, the topic-based service discovery extension to Node Discovery v5. TopDisc is based on the DISC-NG design described in [DISC-NG]. + +TopDisc addresses the problem of discovering peers that participate in a particular decentralised service or application. The participants of such a service form a service-specific overlay, but the discovery mechanism should not require each service to operate a separate discovery network. In this document, a topic is the protocol-level identifier for a service. The terms topic-based discovery and service discovery refer to the same mechanism: discovering peers for the service identified by a topic. + +## Service Discovery Performance Goals + +The following performance goals are specific to TopDisc service discovery. + +### Progressive Lookup Toward the Topic Identifier + +Lookup should be able to find advertisements without immediately concentrating requests at the nodes closest to the topic identifier. + +TopDisc lookup starts from buckets far from the topic identifier and progresses towards buckets closer to it. In each bucket, the discoverer queries up to `Klookup` registrars. This allows popular topics to be discovered before reaching the closest buckets, reducing hotspots near the topic identifier. + +For less popular topics, lookup can continue toward buckets closer to the topic identifier, where advertisers and discoverers are more likely to overlap. This provides a structured search process while avoiding the cost of blind random sampling across the whole discovery network. + +### Efficient Discovery for Small Topics + +Topic discovery should remain efficient even when the target topic is advertised by only a small fraction of nodes in the global discovery network. + +Random sampling over the ordinary node discovery network is robust, but it may require many probes before finding enough peers for a rare topic. TopDisc reduces this cost by allowing advertisers to place topic advertisements at registrars and allowing discoverers to query registrars selected from topic-centred service tables. + +### Bounded Registrar Storage + +A registrar should be able to bound the amount of memory it dedicates to topic discovery. TopDisc addresses this through a bounded ad cache with capacity `C`. Advertisements are soft state and expire after duration `E`, so storage is reclaimed automatically unless advertisers renew their advertisements. + +### Stateless Registration Operations + +A registrar should not be required to store any state for advertisers waiting to be admitted. TopDisc uses tickets to carry pending-registration state back to the advertiser. The registrar can validate a returning advertiser using the ticket, without keeping per-request state for every pending registration attempt. + +### Compact Responses + +TopDisc responses should remain small enough for UDP-based discovery. Advertisements returned by a registrar are capped by `Freturn`. Auxiliary ENRs are selected with an implementation-defined total cap, and the recommended selection rule returns at most one auxiliary ENR per requested topic-distance. This keeps responses compact while still helping requesters improve their service tables. + +### Incremental Service-Table Improvement + +A node should be able to start TopDisc operations before it has a complete service table for a topic. A service table is soft state. It is initially derived from the ordinary node table and then refined using auxiliary ENRs returned by TopDisc responses. This allows lookup and advertisement placement to begin with partial knowledge and improve over time. + +## Service Discovery Security Goals + +The following security goals are specific to TopDisc service discovery. They complement the Node Discovery v5 security goals listed above. + +### Advertisement Flooding + +A malicious advertiser may attempt to fill registrar ad caches with its own advertisements, or with advertisements for services it controls, so that honest advertisements are delayed or excluded. + +TopDisc mitigates this attack by using bounded ad caches and waiting-time-based admission control. Advertisements that increase cache occupancy or reduce cache diversity receive longer waiting times. + +### Registrar Resource Exhaustion + +A malicious node may send many registration attempts, retries, or malformed requests in an attempt to exhaust registrar memory, CPU, or bandwidth. + +TopDisc mitigates memory exhaustion by avoiding unbounded per-request registrar state for pending registrations. Pending registration state is carried by advertisers in registrar-authenticated tickets. Registrars may also apply local rate limits, request validation, and temporary exclusion policies for nodes that repeatedly fail TopDisc operations. + +### Service Censorship + +A malicious actor may attempt to prevent honest advertisements for a target topic from being discovered. + +This may be attempted by flooding registrars with competing advertisements, by returning incomplete or misleading lookup responses, or by trying to control nodes in parts of the key space relevant to a topic. + +TopDisc mitigates this risk by distributing advertisement placement across the topic-centred key space, by using multiple registrars, and by allowing discoverers to query registrars across multiple buckets rather than relying on a single topic-specific location. + +### Service Eclipse + +A malicious actor may attempt to make discoverers of a target topic receive only advertisements for malicious nodes. + +TopDisc reduces the effectiveness of this attack by requiring discoverers to collect advertisements from multiple registrars and by encouraging diversity in registrar ad caches. Parameters such as `Flookup`, `Freturn`, `Klookup`, and `Kregister` control the trade-off between lookup cost and diversity of sources. + +### Service-Table Poisoning + +A malicious registrar may return misleading auxiliary ENRs in TopDisc responses in order to pollute the requester's service table. + +Implementations mitigate this by validating returned ENRs, requiring a supported TopDisc capability before inserting nodes into service tables, applying ordinary Node Discovery v5 liveness checks, and temporarily excluding nodes that repeatedly fail TopDisc operations. Auxiliary ENRs are routing information, not lookup results, and should not be treated as proof that a node participates in the requested topic. + +### Advertisement Redirection + +A malicious node may attempt to advertise an ENR that directs discoverers to a victim endpoint or to a node that does not actually participate in the advertised service. + +TopDisc relies on ENR validation and on the self-signed nature of node records to prevent intermediaries from modifying advertised node information. Applications using discovered advertisements should still perform their normal service-level checks before relying on the discovered peer. + +# Rationale + +## Why Not Use Separate Discovery Networks? + +A simple way to discover service-specific peers would be to run a separate discovery network for each service. Nodes interested in a service would join that service's discovery network directly. + +This approach makes each service responsible for its own bootstrapping and security. New or small services would have few participants and would therefore be easier to isolate or eclipse. Running many small discovery networks would also duplicate infrastructure and fragment the global peer-discovery ecosystem. + +TopDisc instead reuses the ordinary Node Discovery v5 network. Services benefit from the existing global discovery network, and nodes can discover peers for many services without joining a separate discovery DHT for each one. + +## Why Not Use Only Random Sampling? + +Ordinary Node Discovery v5 can be used to sample nodes from the global discovery network. A service-specific protocol can then check whether each sampled node supports the desired topic or service. + +This approach has a useful security property: the search is spread across the global discovery network rather than being concentrated at a small set of predictable service-specific nodes. However, it is inefficient when the target topic is supported by only a small fraction of nodes. In that case, many unrelated nodes must be contacted before enough useful service peers are found. + +TopDisc keeps the benefit of using the global discovery network, but adds explicit topic advertisements so that discoverers do not need to test many unrelated nodes. + +## Why Not Store Advertisements Only Near the Topic Identifier? + +A simple DHT-style design would store all advertisements for a topic at the nodes whose node IDs are closest to the topic identifier. Advertisers and discoverers would then know where to place and retrieve advertisements. + +This design is efficient, but it concentrates load and trust near the topic identifier. Popular topics would create hotspots around their identifiers. More importantly, an adversary could generate node IDs close to a chosen topic identifier and attempt to control advertisement storage or lookup results for that topic. + +TopDisc therefore does not rely only on the closest nodes to a topic identifier. Advertisers maintain placements across buckets of a topic-centred table. Discoverers query registrars starting from buckets far from the topic identifier and progress towards buckets closer to it, querying up to `Klookup` registrars per bucket. This keeps discovery distributed while still ensuring that advertiser and discoverer walks converge toward the topic identifier when more search effort is needed. + +## Why Use Topic-Centred Tables? + +The ordinary node table is centred on the local node ID. It is useful for maintaining the global discovery network and for finding nodes close to arbitrary node IDs. For topic discovery, however, advertisers and discoverers need a shared reference point: the topic identifier. + +A service table is centred on the topic identifier rather than on the local node ID. This gives advertisers and discoverers for the same topic a compatible view of the key space. An advertiser uses the service table to choose registrars for advertisement placement. A discoverer uses the service table to choose registrars for lookup. + +Service tables do not replace the ordinary node table. They are derived from ordinary node discovery state and refined using auxiliary ENRs returned by TopDisc responses. + +## Why Use Registrars? + +A registrar is a TopDisc-capable node that stores admitted advertisements and returns them to discoverers. + +Registrars decouple advertisers from discoverers. Advertisers do not need to be online at the exact moment a discoverer performs a lookup, as long as their advertisements remain stored at registrars. Discoverers do not need to contact arbitrary nodes and test their service membership one by one; they can query registrars for advertisements that have already been placed. + +Because any TopDisc-capable node can act as a registrar, the design does not depend on a central registry or trusted topic-specific bootstrap node. + +## Why Use an Ad Cache? + +A registrar stores admitted advertisements in an ad cache. The ad cache has bounded capacity, and each advertisement expires after a fixed duration. The fixed advertisement lifetime gives advertisers predictable soft state. Once admitted, an advertisement remains available. Advertisers can therefore schedule renewal rather than continuously re-registering at every moment. + +The cache bounds registrar storage and makes advertisements soft state. Advertisers must renew or replace advertisements over time, which allows the system to adapt to churn, failures, and changes in service participation. + +The ad cache is separate from the service table. The service table is used to select registrars for registration and lookup. The ad cache is the registrar's local storage of advertisements that can be returned to discoverers. + +## Why Use Admission Control? + +Registrars have finite storage. If every registration request were admitted immediately, popular topics or malicious advertisers could dominate the ad cache and crowd out other advertisements. + +TopDisc uses admission control to decide when an advertisement may enter the cache. Admission is based on a waiting-time function. Advertisements that would increase cache occupancy or reduce diversity receive longer waiting times. + +This makes flooding more expensive, promotes diversity across topics and IP prefixes, and helps ensure that less popular topics can still obtain representation in registrar caches. + +## Why Should Advertisers Wait? + +Waiting time is the registrar's main admission-control mechanism. + +The waiting time makes it costly to flood registrars with advertisements, limits the rate at which the ad cache fills, and gives the registrar a way to prefer advertisements that improve cache diversity. An advertiser that receives a waiting time must come back later with a valid ticket before the advertisement can be admitted. + +Waiting also avoids a simple replacement-policy problem. If registrars used only policies such as least-recently-used replacement, an attacker could repeatedly send new advertisements to evict honest ones. With waiting-time admission, the attacker must pay the cost of waiting before advertisements are admitted. + +## Why Use Tickets? + +Tickets allow a registrar to enforce waiting without storing unbounded per-request state. + +A ticket records the advertisement binding and registrar-generated timing information needed to show that the advertiser has waited. The ticket is carried by the advertiser and returned to the registrar on retry. The registrar verifies the ticket and recomputes the waiting time against the current cache state. + +This design prevents pending registrations from consuming unbounded registrar memory. If an advertiser never returns, the registrar does not need to clean up per-request state for that advertiser. + +Tickets are bound to the advertisement being registered. This prevents a ticket issued for one topic or advertised ENR from being reused to register a different advertisement. + +## Why Recompute Waiting Times? + +The state of the ad cache may change while an advertiser is waiting. Advertisements may expire, new advertisements may be admitted, and the diversity of the cache may change. + +For this reason, the waiting time in a ticket is not binding. When the advertiser retries, the registrar recomputes the waiting time using the current cache state. The advertiser is admitted only if its accumulated waiting time is sufficient according to the recomputed value. + +This prevents stale tickets from forcing admission under cache conditions that no longer justify it. + +## Why Include Service Similarity? + +The service-similarity component increases waiting time when the incoming advertisement is for a topic that is already well represented in the registrar's ad cache. + +This prevents popular topics from crowding out less represented topics. It also helps less popular topics obtain cache entries even when the total registration demand is high. + +## Why Include IP Similarity? + +A malicious actor may create many node identities from a small number of physical hosts or IP prefixes. Counting only node IDs would not distinguish this behaviour from a diverse set of independent advertisers. + +The IP-similarity component increases waiting time for advertisements whose IP prefixes are already overrepresented in the cache. This discourages a small number of IP prefixes from dominating the ad cache. + +This approach is more flexible than a fixed rule such as "only one node per prefix". Fixed prefix limits can harm honest users behind NATs, shared hosting providers, or other common infrastructure. A similarity score instead increases cost gradually as concentration increases. + +## Why Use a Safety Constant? + +If an advertisement is for an underrepresented topic and comes from an underrepresented IP prefix, the service-similarity and IP-similarity components may both be small. + +The safety constant ensures that the waiting time does not become zero in such cases. This provides a baseline admission cost and helps prevent the ad cache from being filled too quickly by advertisements that appear diverse only because they use random topic identifiers or diverse IP prefixes. + +## Why Use a Waiting-Time Lower Bound? + +If waiting times could decrease freely as the ad cache changes, advertisers would be incentivised to repeatedly request new tickets in the hope of obtaining a shorter wait. This would create unnecessary traffic and processing load. + +The lower-bound mechanism ensures that a new waiting time cannot improve on a previous waiting time by more than the elapsed time. It removes the incentive to repeatedly request new tickets while keeping registrar state bounded. + +The lower-bound state is maintained only for bounded structures, such as topics already present in the ad cache and prefixes represented in the IP similarity tree. + +## Why Return Auxiliary ENRs? + +A service table is initially derived from the ordinary node table. However, the ordinary node table is centred on the local node ID, not on the topic identifier. It may therefore contain few nodes in buckets that are important for a particular topic. TopDisc responses can include auxiliary ENRs selected from the responder's view of the topic-centred key space. Advertisers and discoverers can use these ENRs to refine their local service tables. Returned ENRs are auxiliary routing information, not lookup results. Implementations must validate them and check TopDisc capability in the ENR before inserting them into service tables. + +When a request includes topic-distances where the requester has free space, the responder returns at most one auxiliary ENR per requested distance. This keeps responses compact and avoids overrepresenting a single bucket. It also limits the ability of a malicious responder to flood the requester’s service table with many ENRs from one distance. + +## Why Support Mixed Deployments? + +TopDisc is an extension to Node Discovery v5. During deployment, some nodes may support only ordinary Node Discovery v5 while others support both ordinary Node Discovery v5 and TopDisc. + +Nodes that do not support TopDisc remain useful for maintaining the ordinary discovery network. They are not selected for TopDisc registration or lookup operations, but they can still participate in ordinary node discovery. + +TopDisc-capable nodes advertise support in their ENR. This allows implementations to build service tables from ordinary node discovery state while selecting only nodes that can handle TopDisc messages. + # References @@ -450,6 +586,10 @@ also possible to implement a feedback system between trusted clients and servers *Low-Resource Eclipse Attacks on Ethereum’s Peer-to-Peer Network.* 2018.\ +- Michał Król, Onur Ascigil, Sergi Rene, Alberto Sonnino, Matthieu Pigaglio, Ramin Sadre, Felix Lange, and Etienne Rivière. + *DISC-NG: Robust Service Discovery in the Ethereum Global Network.* 2024 IEEE 9th European Symposium on Security and Privacy (EuroS&P), pp. 193–215. IEEE, 2024.\ + + [wire protocol]: ./discv5-wire.md -[topic-based node index]: ./discv5-theory.md#topic-advertisement [node records]: ../enr.md +[TopDisc theory]: ./discv5-theory.md#topic-based-service-discovery diff --git a/discv5/discv5-theory.md b/discv5/discv5-theory.md index 1b33fca4..67083da5 100644 --- a/discv5/discv5-theory.md +++ b/discv5/discv5-theory.md @@ -6,532 +6,788 @@ This document explains the algorithms and data structures used by the protocol. ## Nodes, Records and Distances -A participant in the Node Discovery Protocol is represented by a 'node record' as defined -in [EIP-778]. The node record keeps arbitrary information about the node. For the purposes -of this protocol, the node must at least provide an IP address (`"ip"` or `"ip6"` key) and -UDP port (`"udp"` key) in order to have it's record relayed in the DHT. +A participant in the Node Discovery Protocol is represented by a 'node record' as defined in [EIP-778]. +The node record keeps arbitrary information about the node. For the purposes of this protocol, the node +must at least provide an IP address (`"ip"` or `"ip6"` key) and UDP port (`"udp"` key) in order to have +it's record relayed in the DHT. -Node records are signed according to an 'identity scheme'. Any scheme can be used with -Node Discovery Protocol, and nodes using different schemes can communicate. +Node records are signed according to an 'identity scheme'. Any scheme can be used with Node Discovery +Protocol, and nodes using different schemes can communicate. The identity scheme of a node record defines +how a 32-byte 'node ID' is derived from the information contained in the record. -The identity scheme of a node record defines how a 32-byte 'node ID' is derived from the -information contained in the record. The 'distance' between two node IDs is the bitwise -XOR of the IDs, taken as the big-endian number. +The 'distance' between two node IDs is the bitwise XOR of the IDs, taken as the big-endian number. distance(n₁, n₂) = n₁ XOR n₂ -In many situations, the logarithmic distance (i.e. length of differing suffix in bits) is -used in place of the actual distance. +In many situations, the logarithmic distance (i.e. length of differing suffix in bits) is used in place of the +actual distance. logdistance(n₁, n₂) = log2(distance(n₁, n₂)) ### Maintaining The Local Node Record -Participants should update their record, increase the sequence number and sign a new -version of the record whenever their information changes. This is especially important for -changes to the node's IP address and port. Implementations should determine the external -endpoint (the Internet-facing IP address and port on which the node can be reached) and -include it in their record. +Participants should update their record, increase the sequence number and sign a new version of the record +whenever their information changes. This is especially important for changes to the node's IP address and port. -If communication flows through a NAT device, the UPnP/NAT-PMP protocols or the mirrored -UDP envelope IP and port found in the [PONG] message can be used to determine the external -IP address and port. - -If the endpoint cannot be determined (e.g. when the NAT doesn't support 'full-cone' -translation), implementations should omit IP address and UDP port from the record. +Implementations should determine the external endpoint (the Internet-facing IP address and port on which +the node can be reached) and include it in their record. If communication flows through a NAT device, the +UPnP/NAT-PMP protocols or the mirrored UDP envelope IP and port found in the [PONG] message can be used +to determine the external IP address and port. If the endpoint cannot be determined (e.g. when the NAT +doesn't support 'full-cone' translation), implementations should omit IP address and UDP port from the record. ## Sessions -Discovery communication is encrypted and authenticated using session keys, established in -the handshake. Since every node participating in the network acts as both client and -server, a handshake can be initiated by either side of communication at any time. +Discovery communication is encrypted and authenticated using session keys, established in the handshake. +Since every node participating in the network acts as both client and server, a handshake can be initiated +by either side of communication at any time. ### Handshake Steps #### Step 1: Node A sends message packet -In the following definitions, we assume that node A wishes to communicate with node B, -e.g. to send a FINDNODE message. Node A must have a copy of node B's record in order to -communicate with it. +In the following definitions, we assume that node A wishes to communicate with node B, e.g. to send a +FINDNODE message. Node A must have a copy of node B's record in order to communicate with it. -If node A has session keys from prior communication with B, it encrypts its request with -those keys. If no keys are known, it initiates the handshake by sending an ordinary -message packet with random message content. +If node A has session keys from prior communication with B, it encrypts its request with those keys. +If no keys are known, it initiates the handshake by sending an ordinary message packet with random +message content. - A -> B FINDNODE message packet encrypted with unknown key + A -> B FINDNODE message packet encrypted with unknown key #### Step 2: Node B responds with challenge -Node B receives the message packet and extracts the source node ID from the packet header. -If node B has session keys from prior communication with A, it attempts to decrypt the -message data. If decryption and authentication of the message succeeds, there is no need -for a handshake and node B can simply respond to the request. - -If node B does not have session keys or decryption is not successful, it must initiate a -handshake by responding with a [WHOAREYOU packet]. +Node B receives the message packet and extracts the source node ID from the packet header. If node B has +session keys from prior communication with A, it attempts to decrypt the message data. If decryption and +authentication of the message succeeds, there is no need for a handshake and node B can simply respond to +the request. -It first generates a unique `id-nonce` value and includes it in the packet. Node B also -checks if it has a copy of node A's record. If it does, it also includes the sequence -number of this record in the challenge packet, otherwise it sets the `enr-seq` field to -zero. +If node B does not have session keys or decryption is not successful, it must initiate a handshake by +responding with a [WHOAREYOU packet]. It first generates a unique `id-nonce` value and includes it in the +packet. Node B also checks if it has a copy of node A's record. If it does, it also includes the sequence +number of this record in the challenge packet, otherwise it sets the `enr-seq` field to zero. -Node B must also store the A's record and the WHOAREYOU challenge for a short duration -after sending it to node A because they will be needed again in step 4. +Node B must also store the A's record and the WHOAREYOU challenge for a short duration after sending it to +node A because they will be needed again in step 4. - A <- B WHOAREYOU packet including id-nonce, enr-seq + A <- B WHOAREYOU packet including id-nonce, enr-seq #### Step 3: Node A processes the challenge -Node A receives the challenge sent by node B, which confirms that node B is alive and is -ready to perform the handshake. The challenge can be traced back to the request packet -which solicited it by checking the `nonce`, which mirrors the request packet's `nonce`. +Node A receives the challenge sent by node B, which confirms that node B is alive and is ready to perform +the handshake. The challenge can be traced back to the request packet which solicited it by checking the +`nonce`, which mirrors the request packet's `nonce`. -Node A proceeds with the handshake by re-sending the FINDNODE request as a [handshake -message packet]. This packet contains three parts in addition to the message: -`id-signature`, `ephemeral-pubkey` and `record`. +Node A proceeds with the handshake by re-sending the FINDNODE request as a [handshake message packet]. +This packet contains three parts in addition to the message: `id-signature`, `ephemeral-pubkey` and `record`. The handshake uses the unmasked WHOAREYOU challenge as an input: - challenge-data = masking-iv || static-header || authdata + challenge-data = masking-iv || static-header || authdata -Node A can now derive the new session keys. To do so, it first generates an ephemeral key -pair on the elliptic curve used by node B's identity scheme. As an example, let's assume -the node record of B uses the "v4" scheme. In this case the `ephemeral-pubkey` will be a -public key on the secp256k1 curve. +Node A can now derive the new session keys. To do so, it first generates an ephemeral key pair on the +elliptic curve used by node B's identity scheme. As an example, let's assume the node record of B uses the +"v4" scheme. In this case the `ephemeral-pubkey` will be a public key on the secp256k1 curve. - ephemeral-key = random private key generated by node A - ephemeral-pubkey = public key corresponding to ephemeral-key + ephemeral-key = random private key generated by node A + ephemeral-pubkey = public key corresponding to ephemeral-key -The ephemeral key is used to perform Diffie-Hellman key agreement with node B's static -public key and the session keys are derived from it using the HKDF key derivation -function. +The ephemeral key is used to perform Diffie-Hellman key agreement with node B's static public key and the +session keys are derived from it using the HKDF key derivation function. - dest-pubkey = public key corresponding to node B's static private key - secret = ecdh(dest-pubkey, ephemeral-key) - kdf-info = "discovery v5 key agreement" || node-id-A || node-id-B - prk = HKDF-Extract(secret, challenge-data) - key-data = HKDF-Expand(prk, kdf-info) - initiator-key = key-data[:16] - recipient-key = key-data[16:] + dest-pubkey = public key corresponding to node B's static private key + secret = ecdh(dest-pubkey, ephemeral-key) + kdf-info = "discovery v5 key agreement" || node-id-A || node-id-B + prk = HKDF-Extract(secret, challenge-data) + key-data = HKDF-Expand(prk, kdf-info) + initiator-key = key-data[:16] + recipient-key = key-data[16:] -Node A creates the `id-signature`, which proves that it controls the private key which -signed its node record. The signature also prevents replay of the handshake. +Node A creates the `id-signature`, which proves that it controls the private key which signed its node +record. The signature also prevents replay of the handshake. - id-signature-text = "discovery v5 identity proof" + id-signature-text = "discovery v5 identity proof" id-signature-input = id-signature-text || challenge-data || ephemeral-pubkey || node-id-B - id-signature = id_sign(sha256(id-signature-input)) - -Finally, node A compares the `enr-seq` element of the WHOAREYOU challenge against its own -node record sequence number. If the sequence number in the challenge is lower, it includes -its record into the handshake message packet. + id-signature = id_sign(sha256(id-signature-input)) -The request is now re-sent, with the message encrypted using the new session keys. +Finally, node A compares the `enr-seq` element of the WHOAREYOU challenge against its own node record +sequence number. If the sequence number in the challenge is lower, it includes its record into the handshake +message packet. The request is now re-sent, with the message encrypted using the new session keys. - A -> B FINDNODE handshake message packet, encrypted with new initiator-key + A -> B FINDNODE handshake message packet, encrypted with new initiator-key #### Step 4: Node B receives handshake message -When node B receives the handshake message packet, it first loads the node record and -WHOAREYOU challenge which it sent and stored earlier. +When node B receives the handshake message packet, it first loads the node record and WHOAREYOU challenge +which it sent and stored earlier. If node B did not have the node record of node A, the handshake message +packet must contain a node record. A record may also be present if node A determined that its record is +newer than B's current copy. If the packet contains a node record, B must first validate it by checking the +record's signature. -If node B did not have the node record of node A, the handshake message packet must -contain a node record. A record may also be present if node A determined that its record -is newer than B's current copy. If the packet contains a node record, B must first -validate it by checking the record's signature. +Node B then verifies the `id-signature` against the identity public key of A's record. After that, B can +perform the key derivation using its own static private key and the `ephemeral-pubkey` from the handshake +packet. Using the resulting session keys, it attempts to decrypt the message contained in the packet. -Node B then verifies the `id-signature` against the identity public key of A's record. +If the message can be decrypted and authenticated, Node B considers the new session keys valid and responds +to the message. In our example case, the response is a `NODES` message: -After that, B can perform the key derivation using its own static private key and the -`ephemeral-pubkey` from the handshake packet. Using the resulting session keys, it -attempts to decrypt the message contained in the packet. - -If the message can be decrypted and authenticated, Node B considers the new session keys -valid and responds to the message. In our example case, the response is a `NODES` message: - - A <- B NODES encrypted with new recipient-key + A <- B NODES encrypted with new recipient-key #### Step 5: Node A receives response message -Node A receives the message packet response and authenticates/decrypts it with the new -session keys. If decryption/authentication succeeds, node B's identity is verified and -node A also considers the new session keys valid. +Node A receives the message packet response and authenticates/decrypts it with the new session keys. If +decryption/authentication succeeds, node B's identity is verified and node A also considers the new session +keys valid. ### Identity-Specific Cryptography in the Handshake -Establishment of session keys is dependent on the [identity scheme] used by the recipient -(i.e. the node which sends WHOAREYOU). Likewise, the signature over `id-sig-input` is made -by the identity key of the initiator. It is not required that initiator and recipient use -the same identity scheme in their respective node records. Implementations must be able to -perform the handshake for all supported identity schemes. - -At this time, the only supported identity scheme is "v4". +Establishment of session keys is dependent on the [identity scheme] used by the recipient (i.e. the node +which sends WHOAREYOU). Likewise, the signature over `id-sig-input` is made by the identity key of the +initiator. It is not required that initiator and recipient use the same identity scheme in their respective +node records. Implementations must be able to perform the handshake for all supported identity schemes. -`id_sign(hash)` creates a signature over `hash` using the node's static private key. The -signature is encoded as the 64-byte array `r || s`, i.e. as the concatenation of the -signature values. - -`ecdh(pubkey, privkey)` creates a secret through elliptic-curve Diffie-Hellman key -agreement. The public key is multiplied by the private key to create a secret ephemeral -key `eph = pubkey * privkey`. The 33-byte secret output is `y || eph.x` where `y` is -`0x02` when `eph.y` is even or `0x03` when `eph.y` is odd. +At this time, the only supported identity scheme is "v4". `id_sign(hash)` creates a signature over `hash` +using the node's static private key. The signature is encoded as the 64-byte array `r || s`, i.e. as the +concatenation of the signature values. `ecdh(pubkey, privkey)` creates a secret through elliptic-curve +Diffie-Hellman key agreement. The public key is multiplied by the private key to create a secret ephemeral +key `eph = pubkey * privkey`. The 33-byte secret output is `y || eph.x` where `y` is `0x02` when `eph.y` +is even or `0x03` when `eph.y` is odd. ### Handshake Implementation Considerations -Since a handshake may happen at any time, UDP packets may be reordered by transmitting -networking equipment, implementations must deal with certain subtleties regarding the -handshake. - -In general, implementations should keep a reference to all sent request packets until the -request either times out, is answered by the corresponding response packet or answered by -WHOAREYOU. If WHOAREYOU is received as the answer to a request, the request must be -re-sent as a handshake packet. +Since a handshake may happen at any time, UDP packets may be reordered by transmitting networking equipment, +implementations must deal with certain subtleties regarding the handshake. -If an implementation supports sending concurrent requests, multiple responses may be -pending when WHOAREYOU is received, as in the following example: +In general, implementations should keep a reference to all sent request packets until the request either +times out, is answered by the corresponding response packet or answered by WHOAREYOU. If WHOAREYOU is +received as the answer to a request, the request must be re-sent as a handshake packet. - A -> B FINDNODE - A -> B PING - A -> B TOPICQUERY - A <- B WHOAREYOU (nonce references PING) +If an implementation supports sending concurrent requests, multiple responses may be pending when WHOAREYOU +is received, as in the following example: -When this happens, all buffered requests can be considered invalid (the remote end cannot -decrypt them) and the packet referenced by the WHOAREYOU `nonce` (in this example: PING) -must be re-sent as a handshake. When the response to the re-sent is received, the new -session is established and other pending requests (example: FINDNODE, TOPICQUERY) may be -re-sent. + A -> B FINDNODE + A -> B PING + A -> B TOPICQUERY + A <- B WHOAREYOU (nonce references PING) -Note that WHOAREYOU is only ever valid as a response to a previously sent request. If -WHOAREYOU is received but no requests are pending, the handshake attempt can be ignored. +When this happens, all buffered requests can be considered invalid (the remote end cannot decrypt them) and +the packet referenced by the WHOAREYOU `nonce` (in this example: PING) must be re-sent as a handshake. When +the response to the re-sent is received, the new session is established and other pending requests (example: +FINDNODE, TOPICQUERY) may be re-sent. Note that WHOAREYOU is only ever valid as a response to a previously +sent request. If WHOAREYOU is received but no requests are pending, the handshake attempt can be ignored. -Another important issue is the processing of message packets while a challenge is -received: consider the case where node A has sent a packet that B cannot decrypt, and B -has responded with WHOAREYOU. +Another important issue is the processing of message packets while a challenge is received: consider the case +where node A has sent a packet that B cannot decrypt, and B has responded with WHOAREYOU. - A -> B FINDNODE - A <- B WHOAREYOU + A -> B FINDNODE + A <- B WHOAREYOU -Node B is now waiting for a handshake message packet to complete the new session, but -instead receives another ordinary message packet. +Node B is now waiting for a handshake message packet to complete the new session, but instead receives +another ordinary message packet. - A -> B ORDINARY MESSAGE PACKET + A -> B ORDINARY MESSAGE PACKET -In this case, implementations should respond with a new WHOAREYOU challenge referencing -the message packet. +In this case, implementations should respond with a new WHOAREYOU challenge referencing the message packet. ### Session Cache -Nodes should store session keys for communication with other recently-seen nodes. Since -sessions are ephemeral and can be re-established whenever necessary, it is sufficient to -store a limited number of sessions in an in-memory LRU cache. - -To prevent IP spoofing attacks, implementations must ensure that session secrets and the -handshake are tied to a specific UDP endpoint. This is simple to implement by using the -node ID and IP/port as the 'key' into the in-memory session cache. When a node switches -endpoints, e.g. when roaming between different wireless networks, sessions will have to be -re-established by handshaking again. This requires no effort on behalf of the roaming node -because the recipients of protocol messages will simply refuse to decrypt messages from -the new endpoint and reply with WHOAREYOU. - -The number of messages which can be encrypted with a certain session key is limited -because encryption of each message requires a unique nonce for AES-GCM. In addition to the -keys, the session cache must also keep track of the count of outgoing messages to ensure -the uniqueness of nonce values. Since the wire protocol uses 96 bit AES-GCM nonces, it is -strongly recommended to generate them by encoding the current outgoing message count into -the first 32 bits of the nonce and filling the remaining 64 bits with random data -generated by a cryptographically secure random number generator. +Nodes should store session keys for communication with other recently-seen nodes. Since sessions are ephemeral +and can be re-established whenever necessary, it is sufficient to store a limited number of sessions in an +in-memory LRU cache. To prevent IP spoofing attacks, implementations must ensure that session secrets and the +handshake are tied to a specific UDP endpoint. This is simple to implement by using the node ID and IP/port as +the 'key' into the in-memory session cache. + +When a node switches endpoints, e.g. when roaming between different wireless networks, sessions will have to +be re-established by handshaking again. This requires no effort on behalf of the roaming node because the +recipients of protocol messages will simply refuse to decrypt messages from the new endpoint and reply with +WHOAREYOU. + +The number of messages which can be encrypted with a certain session key is limited because encryption of each +message requires a unique nonce for AES-GCM. In addition to the keys, the session cache must also keep track +of the count of outgoing messages to ensure the uniqueness of nonce values. Since the wire protocol uses 96 bit +AES-GCM nonces, it is strongly recommended to generate them by encoding the current outgoing message count into +the first 32 bits of the nonce and filling the remaining 64 bits with random data generated by a cryptographically +secure random number generator. ## Node Table -Nodes keep information about other nodes in their neighborhood. Neighbor nodes are stored -in a routing table consisting of 'k-buckets'. For each `0 ≤ i < 256`, every node keeps a -k-bucket for nodes of `logdistance(self, n) == i`. The Node Discovery Protocol uses `k = -16`, i.e. every k-bucket contains up to 16 node entries. The entries are sorted by time -last seen — least-recently seen node at the head, most-recently seen at the tail. +Nodes keep information about other nodes in their neighborhood. Neighbor nodes are stored in a routing table +consisting of 'k-buckets'. For each `0 ≤ i < 256`, every node keeps a k-bucket for nodes of +`logdistance(self, n) == i`. The Node Discovery Protocol uses `k = 16`, i.e. every k-bucket contains up to +16 node entries. The entries are sorted by time last seen — least-recently seen node at the head, +most-recently seen at the tail. -Whenever a new node N₁ is encountered, it can be inserted into the corresponding bucket. -If the bucket contains less than `k` entries N₁ can simply be added as the first entry. If -the bucket already contains `k` entries, the liveness of the least recently seen node in -the bucket, N₂, needs to be revalidated. If no reply is received from N₂ it is considered -dead, removed and N₁ added to the front of the bucket. +Whenever a new node N₁ is encountered, it can be inserted into the corresponding bucket. If the bucket contains +less than `k` entries N₁ can simply be added as the first entry. If the bucket already contains `k` entries, the +liveness of the least recently seen node in the bucket, N₂, needs to be revalidated. If no reply is received from +N₂ it is considered dead, removed and N₁ added to the front of the bucket. -Neighbors of very low distance are unlikely to occur in practice. Implementations may omit -k-buckets for low distances. +Neighbors of very low distance are unlikely to occur in practice. Implementations may omit k-buckets for low +distances. ### Table Maintenance In Practice -Nodes are expected to keep track of their close neighbors and regularly refresh their -information. To do so, a lookup targeting the least recently refreshed bucket should be -performed at regular intervals. - -Checking node liveness whenever a node is to be added to a bucket is impractical and -creates a DoS vector. Implementations should perform liveness checks asynchronously with -bucket addition and occasionally verify that a random node in a random bucket is live by -sending [PING]. When the PONG response indicates that a new version of the node record is -available, the liveness check should pull the new record and update it in the local table. +Nodes are expected to keep track of their close neighbors and regularly refresh their information. To do so, +a lookup targeting the least recently refreshed bucket should be performed at regular intervals. -If a node's liveness has been verified many times, implementations may consider occasional -non-responsiveness permissible and assume the node is live. +Checking node liveness whenever a node is to be added to a bucket is impractical and creates a DoS vector. +Implementations should perform liveness checks asynchronously with bucket addition and occasionally verify that +a random node in a random bucket is live by sending [PING]. When the PONG response indicates that a new version +of the node record is available, the liveness check should pull the new record and update it in the local table. +If a node's liveness has been verified many times, implementations may consider occasional non-responsiveness +permissible and assume the node is live. -When responding to FINDNODE, implementations must avoid relaying any nodes whose liveness -has not been verified. This is easy to achieve by storing an additional flag per node in -the table, tracking whether the node has ever successfully responded to a PING request. +When responding to FINDNODE, implementations must avoid relaying any nodes whose liveness has not been verified. +This is easy to achieve by storing an additional flag per node in the table, tracking whether the node has ever +successfully responded to a PING request. -In order to keep all k-bucket positions occupied even when bucket members fail liveness -checks, it is strongly recommended to maintain a 'replacement cache' alongside each -bucket. This cache holds recently-seen nodes which would fall into the corresponding bucket -but cannot become a member of the bucket because it is already at capacity. Once a bucket -member becomes unresponsive, a replacement can be chosen from the cache. +In order to keep all k-bucket positions occupied even when bucket members fail liveness checks, it is strongly +recommended to maintain a 'replacement cache' alongside each bucket. This cache holds recently-seen nodes which +would fall into the corresponding bucket but cannot become a member of the bucket because it is already at capacity. +Once a bucket member becomes unresponsive, a replacement can be chosen from the cache. ### Lookup -A 'lookup' locates the `k` closest nodes to a node ID. - -The lookup initiator starts by picking `α` closest nodes to the target it knows of from -the local table. The initiator then sends [FINDNODE] requests to those nodes. `α` is an -implementation-defined concurrency parameter, typically `3`. As NODES responses are -received, the initiator resends FINDNODE to nodes it has learned about from previous -queries. Of the `k` nodes the initiator has heard of closest to the target, it picks `α` -that it has not yet queried and sends FINDNODE to them. The lookup terminates when the -initiator has queried and gotten responses from the `k` closest nodes it has seen. - -To improve the resilience of lookups against adversarial nodes, the algorithm may be -adapted to perform network traversal on multiple disjoint paths. Not only does this -approach benefit security, it also improves effectiveness because more nodes are visited -during a single lookup. The initial `k` closest nodes are partitioned into multiple -independent 'path' buckets, and ​concurrent FINDNODE​ requests executed as described above, -with one difference: results discovered on one path are not reused on another, i.e. each -path attempts to reach the closest nodes to the lookup target independently without -reusing intermediate results found on another path. Note that it is still necessary to -track previously asked nodes across all paths to keep the paths disjoint. +A 'lookup' locates the `k` closest nodes to a node ID. The lookup initiator starts by picking `α` closest nodes +to the target it knows of from the local table. The initiator then sends [FINDNODE] requests to those nodes. + +`α` is an implementation-defined concurrency parameter, typically `3`. As NODES responses are received, the +initiator resends FINDNODE to nodes it has learned about from previous queries. Of the `k` nodes the initiator +has heard of closest to the target, it picks `α` that it has not yet queried and sends FINDNODE to them. The +lookup terminates when the initiator has queried and gotten responses from the `k` closest nodes it has seen. + +To improve the resilience of lookups against adversarial nodes, the algorithm may be adapted to perform network +traversal on multiple disjoint paths. Not only does this approach benefit security, it also improves effectiveness +because more nodes are visited during a single lookup. + +The initial `k` closest nodes are partitioned into multiple independent 'path' buckets, and concurrent FINDNODE +requests executed as described above, with one difference: results discovered on one path are not reused on another, +i.e. each path attempts to reach the closest nodes to the lookup target independently without reusing intermediate +results found on another path. Note that it is still necessary to track previously asked nodes across all paths to +keep the paths disjoint. ### Lookup Protocol -This section shows how the wire protocol messages can be used to perform a lookup -interaction against a single node. +This section shows how the wire protocol messages can be used to perform a lookup interaction against a single node. -Node `A` is looking for target `x`. It selects node `B` from the local table or -intermediate lookup results. To query for nodes close to `x` on `B`, node `A` computes the -query distance `d = logdistance(B, x)` and sends its request. +Node `A` is looking for target `x`. It selects node `B` from the local table or intermediate lookup results. +To query for nodes close to `x` on `B`, node `A` computes the query distance `d = logdistance(B, x)` and sends +its request. - A -> B FINDNODE [d] + A -> B FINDNODE [d] -Node `B` responds with multiple nodes messages containing the nodes at the queried -distance. +Node `B` responds with multiple nodes messages containing the nodes at the queried distance. - A <- B NODES [N₁, N₂, N₃] - A <- B NODES [N₄, N₅] + A <- B NODES [N₁, N₂, N₃] + A <- B NODES [N₄, N₅] -Depending on the value of `d` and the content of `B`s table, the response to the initial -query might contain very few nodes or no nodes at all. Should this be the case, `A` varies -the distance to retrieve more nodes from adjacent k-buckets on `B`: +Depending on the value of `d` and the content of `B`s table, the response to the initial query might contain very +few nodes or no nodes at all. Should this be the case, `A` varies the distance to retrieve more nodes from adjacent +k-buckets on `B`: - A -> B FINDNODE [d+1] + A -> B FINDNODE [d+1] `B` responds with more nodes: - A <- B NODES [N₆, N₇] + A <- B NODES [N₆, N₇] + +Node `A` now sorts all received nodes by distance to the lookup target and proceeds by repeating the lookup +procedure on another, closer node. + +# Topic-based Service Discovery + +## Overview + +Node Discovery v5 maintains the global Discv5 discovery network and each node's local node table. Applications use this node discovery substrate to discover peers that participate in higher-level services, so that the participants of each service form a service-specific overlay. + +Currently, a node can search for service-specific peers by sampling nodes through node discovery and then checking whether the sampled nodes support the desired service. This check is outside the ordinary node discovery algorithm: depending on the application, service support may be inferred from information in the ENR, discovered by establishing a devp2p/RLPx connection and negotiating supported subprotocols, or determined by a service-specific protocol query. + +Discv5's random-sampling approach preserves the security benefits of sampling from the global discovery network, because the search is not concentrated around a small set of service-specific locations in the DHT keyspace. However, it is inefficient, especially when the target service is supported by only a small fraction of nodes. + +Discv5 topic-based discovery (**TopDisc**) extends Discovery v5 with topic-based service discovery. It allows nodes to advertise participation in a service and allows other nodes to discover those advertisements while reusing the existing Node Discovery v5 node table, ENR mechanism, packet format, and authenticated session machinery. + +## Co-existence with Node Discovery + +TopDisc is layered on top of Discovery v5. A TopDisc-capable node first uses the Node Discovery v5 to join the global discovery network, populate its local node table, and learn TopDisc-capable nodes. These nodes are then used to bootstrap TopDisc. As TopDisc registration and lookup operations proceed, TopDisc-capable nodes can also return additional nodes to improve those service-specific tables, as described below. + +A TopDisc failure does not by itself imply an ordinary Discovery v5 failure. A node may be usable for ordinary +node discovery but unusable for TopDisc registration or lookup. Conversely, a node that fails ordinary Discovery +v5 liveness checks ceases to be eligible for insertion into TopDisc service tables. + +## TopDisc Capability + +A node indicates support for TopDisc by including the [`topic-discovery`][topic-discovery-entry] entry in its ENR. + + topic-discovery = + +The value of `topic-discovery` is an unsigned integer identifying the supported TopDisc protocol version. Nodes whose ENR does not contain `topic-discovery`, or whose `topic-discovery` value is not supported by the local implementation are not inserted into service tables and are not selected for TopDisc registration or lookup requests. + +[topic-discovery-entry]: ../enr-entries/topic-discovery.md + +## Services and Service Identifiers + +TopDisc operates on 32-byte service identifiers. A service identifier denotes a higher-level service, subnetwork, overlay, or protocol-specific discovery target. + +Service identifiers are in the same 256-bit identifier space as Node IDs. This allows TopDisc to apply the Node Discovery v5 XOR distance function between service identifiers and node IDs. + +The mapping from higher-level service names or application-specific parameters to service identifiers is defined by the relevant service binding. Such parameters MAY include, for example, protocol name, network name, fork identifier, client capability, subnet identifier, or other service-specific values. + +This document does not define a canonical derivation rule for service identifiers. + +## Node Roles + +A node that advertises TopDisc capability in its ENR acts as a registrar, subject to local policy and resource limits. + +A **registrar** accepts TopDisc registration and lookup requests, admits advertisements into its local **ad cache**, and returns admitted advertisements to discoverers. + +A TopDisc-capable node may also act as an advertiser, a discoverer, or both. + +An **advertiser** participates in a service and registers advertisements for that service with registrars. A node acts as an advertiser only for services that it chooses to advertise. + +A **discoverer** queries registrars to obtain advertisements for a target service. A node acts as a discoverer only when it is looking up peers for a service. + +The roles are not mutually exclusive. A single node can simultaneously act as a registrar, advertise one or more services, and discover peers for one or more services. + +## Service Tables + +A service table `B(s)` is a per-service node table centred on service identifier `s`, rather than on the local node ID. + +Similar to the ordinary Node Table, `B(s)` is divided into distance buckets. The difference is the reference point used to assign nodes to buckets. In the ordinary node table, a node `n` is placed according to `logdistance(self, n)`, where `self` is the local node ID. In a service table, the same node `n` is placed according to `logdistance(s, n)`, where `s` is the service identifier. + +For each `0 ≤ i < 256`, bucket `bᵢ(s)` contains TopDisc-capable nodes whose node IDs are at logarithmic distance `i` from the service identifier: + + bᵢ(s) = { n | logdistance(s, n) = i } + +Thus, `B(s)` gives the local node a service-centred view of the discovery network. Buckets closer to `s` contain nodes whose IDs are closer to the service identifier, while buckets farther from `s` contain nodes from progressively larger regions of the key space. + +A node may maintain a service table for each service identifier for which it performs TopDisc operations. Advertisers use `B(s)` as an advertise table for service `s`; discoverers use `B(s)` as a search table for service `s`. + +The registrar ad cache is separate from service tables. A registrar does not need to maintain `B(s)` for every service represented in its ad cache. + +### Bootstrap from Ordinary Node Discovery + +A node does not start TopDisc advertisement placement or lookup from an empty service table. + +The node first joins the Node Discovery v5 network using the standard bootstrapping procedure. It populates its ordinary local node table through the existing `PING`, `PONG`, `FINDNODE`, lookup, refresh, and liveness mechanisms. + +For a service identifier `s`, the initial service table + + B(s) = { b₀(s), b₁(s), ..., b₂₅₅(s) } + +is derived from the ordinary node table. When constructing this initial service table, implementations should: + +1. take nodes currently known in the ordinary node table; +2. discard nodes whose ENR does not advertise a supported TopDisc version; +3. discard nodes whose liveness has not been verified by the ordinary Node Discovery v5 table-maintenance rules; +4. discard nodes currently excluded by local TopDisc usability policy; +5. insert each remaining node into the corresponding bucket of `B(s)`. + +The resulting `B(s)` is soft state. It need not be complete before TopDisc operations begin. If the ordinary node table contains too few nodes that can be used for TopDisc operations, implementations should continue ordinary Node Discovery v5 refresh and lookup operations until more candidates are learned. + +### Ongoing Maintenance of `B(s)` + +A service table is maintained from two sources: + +1. the ordinary Discovery v5 node table; and +2. **auxiliary ENRs** learned through TopDisc responses. + +When a newly verified node advertising TopDisc capability is learned through ordinary discovery, it becomes +eligible for insertion into relevant service tables. + +Responses to TopDisc registration and lookup requests may include auxiliary ENRs selected from the responder's +view of the service table. Such ENRs may be inserted into `B(s)` only after ENR validation, capability checking, +and any local TopDisc usability checks. + +Over time, this causes `B(s)` to become better aligned with the service identifier than the ordinary local node +table, while still remaining anchored in ordinary Discovery v5 state. + +### Auxiliary ENR Selection + +TopDisc responses may include auxiliary ENRs. Auxiliary ENRs are routing information used by the requester to improve its local service table `B(s)`; they are not service lookup results. + +A request may carry a list of topic-distances at which the requester's service table has free space. A topic-distance is a logarithmic distance from the service identifier `s` to a node ID, and therefore identifies a bucket of `B(s)`. + +When such a list is present, the registrar SHOULD use it as a hint for auxiliary ENR selection. For each requested topic-distance `d`, the registrar SHOULD return at most one ENR for a TopDisc-capable node `n` such that: + + logdistance(s, n) = d + +The registrar may obtain such ENRs from any local source of known TopDisc-capable nodes, including an existing service table `B(s)`, the ordinary node table filtered to TopDisc-capable nodes, or an implementation-local cache. A registrar is not required to maintain a service table for every service represented in its ad cache. + +A recommended selection algorithm is: + +1. iterate over the topic-distances supplied by the requester; +2. for each distance `d`, select at most one known TopDisc-capable node whose node ID satisfies `logdistance(s, n) = d`; +3. prefer pseudo-random selection when multiple eligible ENRs are available for the same distance; +4. skip ENRs that fail local validation or TopDisc usability checks; +5. stop when all requested distances have been considered or when an implementation-defined cap on auxiliary ENRs has been reached. + +Selecting at most one ENR per requested distance keeps responses compact and spreads coverage across the requester's free buckets. It avoids overrepresenting a single distance and reduces the ability of a responder to fill a response with many ENRs from one bucket. + +If the registrar has no eligible ENR for a requested distance, it omits that auxiliary ENR. + +The same auxiliary-ENR selection rule applies to registration responses and lookup responses. + +### TopDisc Liveness and Temporary Exclusion + +A node advertising TopDisc capability is not automatically a usable registrar for every TopDisc operation. It may +time out, return malformed responses, reject requests, or fail to implement the extension correctly. + +Implementations should therefore maintain TopDisc-level usability state for nodes in `B(s)`. + +If a node in `B(s)` repeatedly fails to answer registration or lookup requests, times out, or returns malformed +responses, implementations should temporarily exclude that node from selection for TopDisc operations. Temporary +exclusion may be implemented by removing the node from `B(s)`, suppressing its selection for a backoff period, +or assigning it lower selection priority. + +Temporary exclusion is not a permanent blacklist. After the backoff period expires, the node may become eligible +for re-insertion into `B(s)` if it is still present in the ordinary node table, still advertises TopDisc capability, +and still satisfies ordinary Discovery v5 liveness requirements. + +Failure of a TopDisc request does not by itself require removal from the ordinary Discovery v5 node table. +Retention in the ordinary node table continues to follow ordinary Discovery v5 liveness and table-maintenance +rules. + +## Registrar Behaviour + +### Ad Cache -Node `A` now sorts all received nodes by distance to the lookup target and proceeds by -repeating the lookup procedure on another, closer node. +A registrar stores admitted advertisements in a bounded ad cache. -## Topic Advertisement +Each admitted advertisement expires after duration `E`. The ad cache has capacity `C`. A registrar stores at most +one active advertisement for the same advertiser and service. -The topic advertisement subsystem indexes participants by their provided services. A -node's provided services are identified by arbitrary strings called 'topics'. A node -providing a certain service is said to 'place an ad' for itself when it makes itself -discoverable under that topic. Depending on the needs of the application, a node can -advertise multiple topics or no topics at all. Every node participating in the discovery -protocol acts as an advertisement medium, meaning that it accepts topic ads from other -nodes and later returns them to nodes searching for the same topic. +The ad cache is not a service table. It is registrar-local storage used to answer service lookup requests. -### Topic Table +An ad cache entry contains at least: -Nodes store ads for any number of topics and a limited number of ads for each topic. The -data structure holding advertisements is called the 'topic table'. The list of ads for a -particular topic is called the 'topic queue' because it functions like a FIFO queue of -limited length. The image below depicts a topic table containing three queues. The queue -for topic `T₁` is at capacity. +- the service identifier; +- the advertiser ENR; +- the expiry time; +- any advertisement payload defined by the service binding or wire-format document. -![topic table](./img/topic-queue-diagram.png) +Implementations may maintain additional indices for efficient retrieval by service, expiry time, advertiser, and +IP prefix. -The queue size limit is implementation-defined. Implementations should place a global -limit on the number of ads in the topic table regardless of the topic queue which contains -them. Reasonable limits are 100 ads per queue and 50000 ads across all queues. Since ENRs -are at most 300 bytes in size, these limits ensure that a full topic table consumes -approximately 15MB of memory. +Expired advertisements are removed automatically. Once an advertisement expires, it is no longer returned in +lookup responses. -Any node may appear at most once in any topic queue, that is, registration of a node which -is already registered for a given topic fails. Implementations may impose other -restrictions on the table, such as restrictions on the number of IP-addresses in a certain -range or number of occurrences of the same node across queues. +If a registrar receives a registration request for an advertisement that is already present in its ad cache, the +registrar may treat the request as a renewal or ignore it, depending on the renewal semantics specified by the +wire-format document. The registrar must not store duplicate active advertisements for the same advertiser and +service. + +### Advertisements + +An advertisement states that the node identified by an ENR participates in a service/topic. + +An advertisement is defined as: + + ad = [topic, ENR] + +where: + +- `topic` is the 32-byte service/topic identifier; +- `ENR` is the advertised node record. + +The advertisement digest is: + + ad-digest = H(ad) + +where `H` is the digest function defined for ticket construction. + +If future versions allow service-specific advertisement payloads, the advertisement definition becomes: + + ad = [topic, ENR, payload] + +### Admission Control + +Registrars use admission control to decide whether and when an incoming registration request for an advertisement may be admitted to the +ad cache. + +Admission is based on a **waiting-time mechanism**. If an advertisement is not admitted immediately, the registrar +returns a ticket and a waiting time. This mechanism promotes diversity in the ad cache and avoids requiring +registrars to keep unbounded per-request state for pending registrations. + +When a registrar receives a registration request, it computes a waiting time from the current state of the ad +cache and the incoming advertisement. If the effective remaining waiting time is less than or equal to zero, the +advertisement is admitted. Otherwise, the registrar returns a ticket and a waiting time. + +The registrar does not need to keep per-request state for pending registrations. Instead, the state needed to +prove accumulated waiting time is carried by the advertiser in the registrar-issued ticket. + +The waiting time in a ticket is not binding. On each retry, the registrar recomputes the waiting time using the +current ad cache state. The advertiser is admitted only when its accumulated waiting time is sufficient according +to the recomputed waiting time. + +If an advertiser retries too early, retries too late, omits the latest ticket, or presents an invalid ticket, the +registrar treats the request as a new registration attempt or rejects it, depending on the wire-format rules. ### Tickets -Ads should remain in the queue for a constant amount of time, the `target-ad-lifetime`. To -maintain this guarantee, new registrations are throttled and registrants must wait for a -certain amount of time before they are admitted. When a node attempts to place an ad, it -receives a 'ticket' which tells them how long they must wait before they will be accepted. -It is up to the registrant node to keep the ticket and present it to the advertisement -medium when the waiting time has elapsed. +A ticket is a registrar-issued, registrar-authenticated object that allows an advertiser to retry a registration attempt after waiting. + +A ticket is bound to a specific advertisement and registrar. The advertisement is denoted `ad` and includes the service identifier, the advertised ENR, and any service-specific advertisement payload. The ticket contains a digest of this advertisement, denoted `adDigest`. + +The advertiser uses the latest ticket issued by the registrar when retrying the registration. A ticket issued for one advertisement or registrar MUST NOT be accepted for a different registration request. In particular, a ticket issued for one service, advertised ENR, or advertisement payload MUST NOT be reused for another. + +The ticket is opaque to the advertiser. The advertiser does not interpret the ticket contents; it stores the latest ticket returned by the registrar and presents it in the next registration attempt to the same registrar. The exact ticket encoding, digest function, authentication mechanism, signature format, and signature domain are specified in the wire-format document. + +Algorithmically, a ticket contains enough authenticated information for the registrar to verify: + +- `adDigest`: the digest of the advertisement to which the ticket applies; +- `tinit`: the registrar-local time at which the ticket was first issued; +- `tmod`: the registrar-local time at which the ticket was last modified; +- `twait`: the remaining waiting duration reported to the advertiser; +- `auth`: registrar authentication over the ticket contents. + +The registrar authenticates the ticket, for example by signing the ticket body or applying a registrar-local MAC. A registrar MUST reject a ticket that fails authentication or that was not issued by that registrar. + +The timestamps `tinit` and `tmod` are generated and interpreted only by the registrar. They are local to the registrar and are not compared against the advertiser's clock. The only timing value used by the advertiser is the relative waiting duration `twait` reported by the registrar in the registration response. + +A retry is valid only during the registration window associated with the ticket: + + tmod + twait ≤ now ≤ tmod + twait + δ + +where `now` is the registrar's current local time when processing the retry, and `δ` is the registration window duration. + +The advertiser does not use `tmod` to schedule the retry and does not need its clock to be synchronised with the registrar's clock. The advertiser waits for the relative duration `twait` using its own local timer, then retries with the latest ticket. The registrar validates the retry window using its own clock when the ticket is presented again. + +When a returning advertiser presents a valid ticket, the registrar computes the accumulated waiting duration as: + + waited = now - tinit + +where `now` and `tinit` are both interpreted according to the registrar's local clock. + +If the retry is too early, too late, does not include the latest ticket, includes a ticket whose `adDigest` does not match the advertisement in the current registration request, or includes an invalid ticket, the registrar SHOULD reject the request or treat it as a new registration attempt according to the registration rules. + +Because the ticket is bound to `adDigest`, changing the service identifier, advertised ENR, or service-specific advertisement payload during a registration attempt requires starting a new registration attempt or obtaining a new ticket for the updated advertisement. + +### Waiting-Time Function + +The waiting-time function determines how long an advertisement must wait before admission. + +The function depends on the current ad cache occupancy, the number of cached advertisements for the same service, +the IP similarity of the advertiser, and a safety constant. The waiting time shapes the contents of the ad cache +and provides flow control at the registrar. + +The waiting time is: + + w(ad) = E * 1 / (1 - c/C)^Pocc * ( c(ad.service)/c + score(ad.IP) + G ) + +where: + +- `E` is the advertisement expiry duration; +- `C` is the ad cache capacity; +- `c` is the current number of advertisements in the cache; when `c = 0`, the service-similarity term is defined as `0`. +- `c(ad.service)` is the number of cached advertisements for the service being advertised; +- `score(ad.IP)` is the IP similarity score computed from the advertiser's IP (ad.IP) in the advertised ENR; +- `Pocc` is the occupancy exponent; +- `G` is the safety constant. + +The occupancy component increases the waiting time as the ad cache approaches capacity. This limits the rate at +which new advertisements can be admitted. + +The service-similarity component increases the waiting time for services that are already well represented in the +cache. This makes it easier for less represented services to obtain cache entries. + +The IP-similarity component increases the waiting time for advertisers whose IP prefixes are overrepresented in +the cache. This discourages a small number of IP prefixes from dominating the registrar's ad cache. + +The safety constant prevents the waiting time from becoming zero when the cache is sparsely populated and the +incoming advertisement appears dissimilar to existing entries. + +When a returning advertiser presents a valid ticket, the registrar computes the effective remaining waiting time as: + + tremaining = w(current-cache, ad) - (now - tinit) + +where `tinit` is the ticket creation time. The advertisement is admitted when `tremaining ≤ 0`. + +### IP Similarity Tree + +Registrars maintain an IP-prefix tree over the IP addresses of admitted advertisements. + +The tree is used to compute the IP similarity score. Prefixes that are overrepresented in the ad cache increase +the score, causing advertisements from similar IP prefixes to receive higher waiting times. + +For IPv4, the tree has 32 levels below the root. Each edge corresponds to one bit of the IP address, and each +vertex stores a counter. The root counter represents the total number of IP addresses represented in the ad cache. + +To score an IP address, the registrar walks the tree along the bits of the address. At each level, the registrar +compares the observed counter for the corresponding prefix with the counter expected in a perfectly balanced tree. +Prefixes that are overrepresented contribute to the score. + +The resulting score is normalised to the interval `[0, 1]`. + +The tree is updated when advertisements are admitted or expire. When an advertisement is admitted, counters along +the path corresponding to the advertiser's IP address are incremented. When the advertisement expires or is removed, +those counters are decremented. + +For IPv6, the same construction applies over 128 address bits. Implementations that support both IPv4 and IPv6 should define whether separate trees are maintained or whether addresses are mapped into a common representation. + +### Waiting-Time Lower Bound + +Registrars enforce a lower bound on waiting times so that an advertiser cannot obtain a better effective waiting +time by repeatedly requesting new tickets. + +Without a lower bound, a pending advertiser could repeatedly request new tickets in the hope that cache changes +cause a lower waiting time. The lower-bound rule ensures that a new waiting time cannot improve on the old one +by more than the elapsed time. + +Registrars do not maintain unbounded per-request lower-bound state for pending registrations. Lower-bound state is +maintained only for bounded structures, such as: + +- services already represented in the ad cache; and +- prefixes represented in the IP similarity tree. + +When a service `s` enters the ad cache for the first time, the registrar initialises lower-bound state for `s`. +When a later ticket request for `s` arrives, the registrar computes the service waiting-time component and applies +the stored lower bound before issuing a new ticket. + +For IP addresses, lower-bound state is maintained at the IP-tree vertex corresponding to the longest prefix match +already present in the tree, without introducing new vertices only for pending requests. If multiple IP addresses +map to the same vertex, the registrar aggregates their lower-bound state using a maximum. + +## Advertiser Behaviour + +### Advertisement Placement + +For each service `s` that a node advertises, the advertiser attempts to maintain up to `Kregister` active or +pending registrations in each bucket of its advertise table `B(s)`. + +Candidate registrars are selected from the corresponding bucket of the advertise table. Placement starts from +the furthest bucket from `s` and progresses towards the closest bucket. + +The advertiser maintains per-bucket state for ongoing and active registrations. This state records the registrars +for which an advertisement attempt is already pending or active, so that the advertiser does not repeatedly choose +the same registrar when filling the same bucket. + +Within a placement cycle, registrar selection should not repeatedly return the same registrar from the same bucket. + +For each bucket `bᵢ(s)`, while the number of active or pending registrations in that bucket is below `Kregister`, +the advertiser selects a candidate registrar from `bᵢ(s)` and starts a registration attempt. + +If no eligible registrar remains in the bucket, the advertiser stops trying to fill that bucket during the current +placement cycle. The bucket may be attempted again later after the service table is refreshed or after temporary +exclusions expire. + +Advertisement placement is continuous soft state. Advertisers periodically repeat placement and renewal so that +advertisements remain available despite expiry, churn, registrar failure, and changes in the service table. + +### Registration Procedure + +An advertiser registers an advertisement by sending a registration request to a selected registrar. The corresponding wire-format request is specified in [REGTOPIC]. + +The first registration request for an advertisement is sent without a ticket. The registrar either admits the advertisement immediately or returns a ticket and a waiting time. Ticket and confirmation responses are specified in [TICKET] and [REGCONFIRMATION]. + +If the registrar returns a ticket, the advertiser waits before retrying with the latest ticket. A single waiting interval SHOULD NOT exceed `E`, the advertisement expiry duration. If the registrar still cannot admit the advertisement after the retry, it may return an updated ticket and a new waiting time. + +The protocol does not define a fixed maximum total duration for a registration attempt. An implementation MAY abandon an attempt after a local timeout, after a configured maximum number of retries, or if the registrar is considered unusable according to local TopDisc liveness policy. + +A registration attempt fails if the registrar is unreachable, rejects the request, returns malformed responses, the registration window is missed, a local retry or timeout limit is reached, or the registrar is otherwise considered unusable according to local TopDisc liveness policy. On failure, the advertiser removes the registrar from the pending state for that bucket and may select another registrar. + +When registration succeeds, the advertisement remains stored by the registrar until it expires, unless it is removed earlier by registrar policy. + +Registration responses may include auxiliary ENRs selected from the registrar's view of the service table. The advertiser may use these ENRs to update its advertise table `B(s)` after validating the ENRs, checking TopDisc capability, and applying local TopDisc usability policy. The recommended auxiliary-ENR selection rule is described in [Auxiliary ENR Selection](#auxiliary-enr-selection). + +### Renewal -The waiting time constant is: +An admitted advertisement remains stored until its expiry time `E`. - target-ad-lifetime = 15min +Advertisers should renew advertisements before expiry, or continuously maintain enough active and pending +registrations, to preserve the target number of placements in each bucket. -The assigned waiting time for any registration attempt is determined according to the -following rules: +A renewal is a new registration attempt for an advertisement already admitted or about to expire. The exact renewal +encoding and the treatment of duplicate registrations are specified by the wire-format document. -- When the table is full, the waiting time is assigned based on the lifetime of the oldest - ad across the whole table, i.e. the registrant must wait for a table slot to become - available. -- When the topic queue is full, the waiting time depends on the lifetime of the oldest ad - in the queue. The assigned time is `target-ad-lifetime - oldest-ad-lifetime` in this - case. -- Otherwise the ad may be placed immediately. +If a renewal fails, the advertiser may attempt to register with another eligible registrar in the same bucket. -Tickets are opaque objects storing arbitrary information determined by the issuing node. -While details of encoding and ticket validation are up to the implementation, tickets must -contain enough information to verify that: +Advertisers should treat registrations as soft state. The target is not to permanently bind a service to a fixed +set of registrars, but to maintain sufficient active or pending placements across the service-centred key space. -- The node attempting to use the ticket is the node which requested it. -- The ticket is valid for a single topic only. -- The ticket can only be used within the registration window. -- The ticket can't be used more than once. +## Discoverer Behaviour -Implementations may choose to include arbitrary other information in the ticket, such as -the cumulative wait time spent by the advertiser. A practical way to handle tickets is to -encrypt and authenticate them with a dedicated secret key: +### Lookup Procedure - ticket = aesgcm_encrypt(ticket-key, ticket-nonce, ticket-pt, '') - ticket-pt = [src-node-id, src-ip, topic, req-time, wait-time, cum-wait-time] - src-node-id = node ID that requested the ticket - src-ip = IP address that requested the ticket - topic = the topic that ticket is valid for - req-time = absolute time of REGTOPIC request - wait-time = waiting time assigned when ticket was created - cum-wait = cumulative waiting time of this node +A discoverer looking for service `s` queries registrars selected from its search table `B(s)`. The corresponding wire-format request is specified in [TOPICQUERY]. Lookup proceeds bucket by bucket, starting from the bucket furthest from `s` and progressing towards buckets closer to `s`. For each bucket `bᵢ(s)`, the discoverer selects candidate registrars from that bucket and queries up to `Klookup` of them. -### Registration Window +A registrar may return advertisements for service `s`. The discoverer validates the returned advertisements, extracts the advertised ENRs, and de-duplicates them by advertiser identity. Advertisements are candidate results for the target service. The lookup terminates when the discoverer has collected enough distinct advertisers for its local or service-specific purpose, or when no unqueried registrars remain. This local target is denoted `Flookup` in this document. The value of `Flookup` is determined by the application, service binding, or local implementation policy, rather than by the TopDisc lookup procedure itself. If fewer than `Flookup` advertisers are found before all available candidate registrars are exhausted, the lookup returns the valid advertisers collected so far. -The image below depicts a single ticket's validity over time. When the ticket is issued, -the node keeping it must wait until the registration window opens. The length of the -registration window is 10 seconds. The ticket becomes invalid after the registration -window has passed. +The same registrar should not be queried repeatedly during a single lookup unless the implementation has exhausted other candidates and chooses to retry according to local policy. If a queried registrar is unreachable, times out, or returns a malformed response, the discoverer treats that query as failed and continues with another candidate. Repeated failures may cause the registrar to be temporarily excluded from TopDisc operations. -![ticket validity over time](./img/ticket-validity.png) +Lookup responses may also include auxiliary ENRs. These ENRs are not lookup results; they are auxiliary routing information used to improve the discoverer's search table `B(s)`. The recommended auxiliary-ENR selection rule is described in [Auxiliary ENR Selection](#auxiliary-enr-selection). -Since all ticket waiting times are assigned to expire when a slot in the queue opens, the -advertisement medium may receive multiple valid tickets during the registration window and -must choose one of them to be admitted in the topic queue. The winning node is notified -using a [REGCONFIRMATION] response. +### Lookup Responses -Picking the winner can be achieved by keeping track of a single 'next ticket' per queue -during the registration window. Whenever a new ticket is submitted, first determine its -validity and compare it against the current 'next ticket' to determine which of the two is -better according to an implementation-defined metric such as the cumulative wait time -stored in the ticket. +A registrar receiving a lookup request for service `s` returns advertisements for that service from its ad cache. A registrar receiving a [TOPICQUERY] request for service `s` returns advertisements and auxiliary ENRs using the response format defined for `TOPICQUERY` in the wire specification. -### Advertisement Protocol +The registrar MUST NOT return expired advertisements. If more than `Freturn` advertisements for the service are present in its ad cache, the registrar SHOULD return a pseudo-random subset of at most `Freturn` advertisements. The selection procedure SHOULD avoid deterministic bias towards the same advertisers across repeated lookup requests. -This section explains how the topic-related protocol messages are used to place an ad. +A registrar may also return auxiliary ENRs selected from its view of the service table for `s`. These ENRs are not lookup results; they are auxiliary routing information used to improve future registration and lookup operations. -Let us assume that node `A` provides topic `T`. It selects node `C` as advertisement -medium and wants to register an ad, so that when node `B` (who is searching for topic `T`) -asks `C`, `C` can return the registration entry of `A` to `B`. +The requester uses returned auxiliary ENRs to update its local service table `B(s)` after validating the ENRs, checking TopDisc capability, and applying local TopDisc usability policy. The exact encoding of returned advertisements, auxiliary ENRs, and any topic-distance list is specified in the wire-format document. -Node `A` first attempts to register without a ticket by sending [REGTOPIC] to `C`. +### Updating the Search Table During Lookup - A -> C REGTOPIC [T, ""] +The discoverer updates `B(s)` using auxiliary ENRs returned in lookup responses. -`C` replies with a ticket and waiting time. +An ENR learned through lookup is eligible for insertion into `B(s)` only if: - A <- C TICKET [ticket, wait-time] +1. the ENR is valid; +2. the ENR advertises TopDisc capability; +3. the node is not temporarily excluded by local TopDisc usability policy; +4. the node satisfies ordinary Discovery v5 liveness requirements, or is scheduled for ordinary liveness verification. -Node `A` now waits for the duration of the waiting time. When the wait is over, `A` sends -another registration request including the ticket. `C` does not need to remember its -issued tickets since the ticket is authenticated and contains enough information for `C` -to determine its validity. +Implementations may insert learned ENRs immediately with an unverified flag and verify liveness asynchronously. - A -> C REGTOPIC [T, ticket] +## Parameters -Node `C` replies with another ticket. Node `A` must keep this ticket in place of the -earlier one, and must also be prepared to handle a confirmation call in case registration -was successful. +The TopDisc algorithms use the following parameters: - A <- C TICKET [ticket, wait-time] +| Parameter | Meaning | Default | +|---|---|---| +| `Kregister` | Target number of active or pending registrations per bucket | `5` | +| `Klookup` | Maximum number of registrar queries per bucket during lookup | `5` | +| `Freturn` | Maximum number of advertisements returned by one registrar | `10` | +| `Flookup` | Optional local or service-specific target for the number of distinct advertisers to collect during lookup | `30` | +| `E` | Advertisement expiry duration | `15 min` | +| `C` | Registrar ad cache capacity | `1000` | +| `δ` | Registration retry window duration | **`TBD`** | +| `Pocc` | Occupancy exponent in the waiting-time function | `10` | +| `G` | Safety constant in the waiting-time function | `10^-7` | -Node `C` waits for the registration window to end on the queue and selects `A` as the node -which is registered. Node `C` places `A` into the topic queue for `T` and sends a -[REGCONFIRMATION] response. +## Implementation Considerations - A <- C REGCONFIRMATION [T] +### ENR Freshness -### Ad Placement And Topic Radius - -Since every node may act as an advertisement medium for any topic, advertisers and nodes -looking for ads must agree on a scheme by which ads for a topic are distributed. When the -number of nodes advertising a topic is at least a certain percentage of the whole -discovery network (rough estimate: at least 1%), ads may simply be placed on random nodes -because searching for the topic on randomly selected nodes will locate the ads quickly enough. +Advertisers should send their current ENR when registering an advertisement. -However, topic search should be fast even when the number of advertisers for a topic is -much smaller than the number of all live nodes. Advertisers and searchers must agree on a -subset of nodes to serve as advertisement media for the topic. This subset is simply a -region of the node ID address space, consisting of nodes whose Kademlia address is within a -certain distance to the topic hash `sha256(T)`. This distance is called the 'topic -radius'. - -Example: for a topic `f3b2529e...` with a radius of 2^240, the subset covers all nodes -whose IDs have prefix `f3b2...`. A radius of 2^256 means the entire network, in which case -advertisements are distributed uniformly among all nodes. The diagram below depicts a -region of the address space with topic hash `t` in the middle and several nodes close to -`t` surrounding it. Dots above the nodes represent entries in the node's queue for the -topic. +Registrars should store the ENR that was admitted and return that ENR in lookup responses until the advertisement +expires or is renewed. -![diagram explaining the topic radius concept](./img/topic-radius-diagram.png) +### Clocks -To place their ads, participants simply perform a random walk within the currently -estimated radius and run the advertisement protocol by collecting tickets from all nodes -encountered during the walk and using them when their waiting time is over. +TopDisc does not require clock synchronisation between advertisers and registrars. -### Topic Radius Estimation +Tickets carry registrar-generated timing information. Advertisers only need to wait for the duration indicated by +the registrar before retrying. -Advertisers must estimate the topic radius continuously in order to place their ads on -nodes where they will be found. The radius mustn't fall below a certain size because -restricting registration to too few nodes leaves the topic vulnerable to censorship and -leads to long waiting times. If the radius were too large, searching nodes would take too -long to find the ads. +### Response Splitting -Estimating the radius uses the waiting time as an indicator of how many other nodes are -attempting to place ads in a certain region. This is achieved by keeping track of the -average time to successful registration within segments of the address space surrounding -the topic hash. Advertisers initially assume the radius is 2^256, i.e. the entire network. -As tickets are collected, the advertiser samples the time it takes to place an ad in each -segment and adjusts the radius such that registration at the chosen distance takes -approximately `target-ad-lifetime / 2` to complete. +TopDisc responses may be split across multiple packets when supported by the wire protocol. -## Topic Search +Implementations should collect all response packets belonging to the same request until the announced response count +is reached or the request times out. -Finding nodes that provide a certain topic is a continuous process which reads the content -of topic queues inside the approximated topic radius. This is a much simpler process than -topic advertisement because collecting tickets and waiting on them is not required. +### Wire Encoding -To find nodes for a topic, the searcher generates random node IDs inside the estimated -topic radius and performs Kademlia lookups for these IDs. All (intermediate) nodes -encountered during lookup are asked for topic queue entries using the [TOPICQUERY] packet. +This document describes algorithms and data structures. -Radius estimation for topic search is similar to the estimation procedure for -advertisement, but samples the average number of results from TOPICQUERY instead of -average time to registration. The radius estimation value can be shared with the -registration algorithm if the same topic is being registered and searched for. +The exact encoding of TopDisc messages, ticket signatures, request identifiers, response splitting, returned +advertisements, neighbour ENRs, and any application-specific advertisement payload is specified in the wire-format +document. -[EIP-778]: ../enr.md -[identity scheme]: ../enr.md#record-structure -[handshake message packet]: ./discv5-wire.md#handshake-message-packet-flag--2 -[WHOAREYOU packet]: ./discv5-wire.md#whoareyou-packet-flag--1 -[PING]: ./discv5-wire.md#ping-request-0x01 -[PONG]: ./discv5-wire.md#pong-response-0x02 -[FINDNODE]: ./discv5-wire.md#findnode-request-0x03 [REGTOPIC]: ./discv5-wire.md#regtopic-request-0x07 -[REGCONFIRMATION]: ./discv5-wire.md#regconfirmation-response-0x09 -[TOPICQUERY]: ./discv5-wire.md#topicquery-request-0x0a +[TICKET]: ./discv5-wire.md#ticket-response-0x08 +[REGCONFIRMATION]: ./discv5-wire.md#regconfirmation-response-0x08 +[TOPICQUERY]: ./discv5-wire.md#topicquery-request-0x09 +[TOPICNODES]: ./discv5-wire.md#topicnodes-response-0x0a +[NODES]: ./discv5-wire.md#nodes-response-0x04 diff --git a/discv5/discv5-wire.md b/discv5/discv5-wire.md index 039d5c32..41bf15ce 100644 --- a/discv5/discv5-wire.md +++ b/discv5/discv5-wire.md @@ -203,13 +203,19 @@ the result set. The recommended result limit for FINDNODE queries is 16 nodes. message-type = 0x04 total = total number of responses to the request -NODES is the response to a FINDNODE or TOPICQUERY message. Multiple NODES messages may be -sent as responses to a single query. Implementations may place a limit on the allowed -maximum for `total`. If exceeded, additional responses may be ignored. +NODES is sent as a response to FINDNODE, REGTOPIC, or TOPICQUERY. Multiple NODES messages +may be sent as responses to a single query. Implementations may place a limit on the +allowed maximum for `total`. If exceeded, additional responses may be ignored. When handling NODES as a response to FINDNODE, the recipient should verify that the received nodes match the requested distances. +When NODES appears as a response to REGTOPIC or TOPICQUERY, it carries auxiliary ENRs +selected from the responder's view of the service table for the requested topic. These +ENRs are routing information for the requester to populate or refresh its own service +table `B(s)`. They are not themselves topic registrants; the actual registered nodes are +returned via TOPICNODES. + ### TALKREQ Request (0x05) message-data = [request-id, protocol, request] @@ -234,62 +240,102 @@ response data. ### REGTOPIC Request (0x07) -**NOTE: the content and semantics of this message are not final.** -**Implementations should not respond to or send these messages.** + message-data = [request-id, topic, ENR, ticket, + [topic-distance₁, topic-distance₂, ..., topic-distanceₙ]] + message-type = 0x07 + topic = 32-byte service / topic identifier + ENR = current node record of sender + ticket = opaque byte array containing a ticket previously issued by the + recipient registrar; empty (`0x80`) on first attempt + topic-distanceₙ = positive integer log2 distance from `topic` where the sender's + service table `B(topic)` still has space - message-data = [request-id, topic, ENR, ticket] - message-type = 0x07 - node-record = current node record of sender - ticket = byte array containing ticket content +REGTOPIC asks the recipient registrar to register the sender (identified by `ENR`) for +service `topic`. If the sender has a ticket from a previous registration attempt with this +registrar, it must present the ticket; otherwise `ticket` is the empty byte array. -REGTOPIC attempts to register the sender for the given topic. If the requesting node has a -ticket from a previous registration attempt, it must present the ticket. Otherwise -`ticket` is the empty byte array (RLP: `0x80`). The ticket must be valid and its waiting -time must have elapsed before using the ticket. +The `topic-distance` list carries the sender's "send me ENRs at these distances" hint to +the recipient: when returning auxiliary ENRs, the recipient should prefer ENRs whose log2 +distance to `topic` matches one of the listed values, so the response helps the sender +populate its service table. -REGTOPIC is always answered by a TICKET response. The requesting node may also receive a -REGCONFIRMATION response when registration is successful. It may take up to 10s for the -confirmation to be sent. +REGTOPIC is always answered with a single REGCONFIRMATION response. The recipient may +additionally send zero or more NODES responses carrying auxiliary ENRs selected from its +view of the service table. -### TICKET Response (0x08) +See the [theory section on tickets] and [theory section on registrar admission control] +for the registrar's waiting-time semantics. -**NOTE: the content and semantics of this message are not final.** -**Implementations should not respond to or send these messages.** +### REGCONFIRMATION Response (0x08) - message-data = [request-id, ticket, wait-time] + message-data = [request-id, total, ticket, wait-time] message-type = 0x08 - ticket = an opaque byte array representing the ticket - wait-time = time to wait before registering, in seconds - -TICKET is the response to REGTOPIC. It contains a ticket which can be used to register for -the requested topic after `wait-time` has elapsed. See the [theory section on tickets] for -more information. - -### REGCONFIRMATION Response (0x09) - -**NOTE: the content and semantics of this message are not final.** -**Implementations should not respond to or send these messages.** - - message-data = [request-id, topic] - message-type = 0x09 request-id = request-id of REGTOPIC + total = total number of responses (REGCONFIRMATION + NODES) to the request + ticket = ticket issued by the registrar for the next attempt; + empty byte array (RLP: `0x80`) when the registration was admitted + wait-time = milliseconds to wait before submitting the next REGTOPIC attempt + with the returned `ticket`. When `ticket` is empty, `wait-time` carries + the advertisement lifetime instead. + +REGCONFIRMATION is the response to REGTOPIC. It is sent immediately by the registrar and +plays two roles, distinguished by the length of `ticket`: + +- If `ticket` is the empty byte array, the advertisement has been admitted to the + registrar's ad cache. `wait-time` indicates the advertisement lifetime; the advertiser + should renew before that lifetime elapses to remain in the cache. +- If `ticket` is non-empty, the advertisement was not admitted on this attempt. The + sender must wait at least `wait-time` milliseconds and re-attempt the registration with + the returned `ticket`. See the [theory section on tickets] and the [theory section on + the waiting-time function]. + +The `total` field announces the total number of responses (this REGCONFIRMATION plus any +NODES messages carrying auxiliary ENRs) that the registrar will send for this request. + +### TOPICQUERY Request (0x09) + + message-data = [request-id, topic, + [topic-distance₁, topic-distance₂, ..., topic-distanceₙ]] + message-type = 0x09 + topic = 32-byte service / topic identifier + topic-distanceₙ = positive integer log2 distance from `topic` where the sender's + service table `B(topic)` still has space + +TOPICQUERY asks the recipient to return registered advertisers for the given `topic` from +its ad cache. The recipient sends zero or more TOPICNODES responses containing matching +advertiser ENRs, and may additionally send zero or more NODES responses carrying +auxiliary ENRs selected from its service-table view (see NODES). + +The `topic-distance` list serves the same purpose as in REGTOPIC: it tells the recipient +which log2 distances from `topic` the sender's service table still has room for, so the +recipient can choose useful auxiliary ENRs to include in its NODES responses. + +See the [theory section on lookup responses] for the discoverer-side termination semantics +(distinct-advertisers count) and the [theory section on parameters] for `Freturn`. + +### TOPICNODES Response (0x0A) -REGCONFIRMATION notifies the recipient about a successful registration for the given -topic. This call is sent by the advertisement medium after the time window for -registration has elapsed on a topic queue. + message-data = [request-id, total, [ENR, ...]] + message-type = 0x0a + request-id = request-id of TOPICQUERY + total = total number of responses (NODES + TOPICNODES) to the request -### TOPICQUERY Request (0x0A) +TOPICNODES is the dedicated response to TOPICQUERY carrying advertiser ENRs that are +currently registered for the requested topic in the recipient's ad cache. Multiple +TOPICNODES messages may be sent for a single TOPICQUERY. -**NOTE: the content and semantics of this message are not final.** -**Implementations should not respond to or send these messages.** +The `total` field announces the total number of responses (TOPICNODES messages plus any +NODES messages carrying auxiliary ENRs) the recipient will send for this request. +Implementations may place a limit on the allowed maximum for `total`; if exceeded, +additional responses may be ignored. - message-data = [request-id, topic] - message-type = 0x0a - topic = 32-byte topic hash +The recipient should return only non-expired advertisements from its ad cache. When the +ad cache contains more than `Freturn` advertisements for the topic, the recipient +selects which advertisements to return; the exact selection policy is implementation +defined. -TOPICQUERY requests nodes in the [topic queue] of the given topic. The recipient of this -request must send one or more NODES messages containing node records registered for the -topic. +TOPICNODES carries only registered advertisers. Auxiliary routing information for the +sender's service table is carried separately via NODES responses. ## Test Vectors @@ -299,5 +345,9 @@ A collection of test vectors for this specification can be found at [handshake section]: ./discv5-theory.md#handshake-steps [topic queue]: ./discv5-theory.md#topic-table [theory section on tickets]: ./discv5-theory.md#tickets +[theory section on registrar admission control]: ./discv5-theory.md#admission-control +[theory section on the waiting-time function]: ./discv5-theory.md#waiting-time-function +[theory section on lookup responses]: ./discv5-theory.md#lookup-responses +[theory section on parameters]: ./discv5-theory.md#parameters [EIP-778]: ../enr.md [discv5 wire test vectors]: ./discv5-wire-test-vectors.md diff --git a/discv5/discv5.md b/discv5/discv5.md index 13e2ab66..0692fcc1 100644 --- a/discv5/discv5.md +++ b/discv5/discv5.md @@ -15,24 +15,35 @@ entry point into the network. The system's design is loosely inspired by the Kademlia DHT, but unlike most DHTs no arbitrary keys and values are stored. Instead, the DHT stores and relays 'node records', which are signed documents providing information about nodes in the network. Node -Discovery acts as a database of all live nodes in the network and performs three basic +Discovery acts as a database of live nodes in the network and performs two basic functions: -- Sampling the set of all live participants: by walking the DHT, the network can be +- Sampling the set of live participants: by walking the DHT, the network can be enumerated. -- Searching for participants providing a certain service: Node Discovery v5 includes a - scalable facility for registering 'topic advertisements'. These advertisements can be - queried and nodes advertising a topic found. - Authoritative resolution of node records: if a node's ID is known, the most recent version of its record can be retrieved. +Node Discovery v5 also supports topic-based service discovery through TopDisc. TopDisc is +an extension layered on top of the ordinary node discovery network. It allows nodes to +advertise participation in a topic or service, and allows other nodes to discover those +advertisements while reusing the existing node table, ENR mechanism, packet format, and +authenticated session machinery. + +TopDisc is intended for discovering participants in higher-level services or overlays +without requiring each service to operate a separate discovery network. Nodes that support +TopDisc advertise this capability in their ENR. Nodes that do not support TopDisc remain +ordinary Node Discovery v5 participants and continue to contribute to the global discovery +network. + ## Specification Overview The specification has three parts: - [discv5-wire.md] defines the wire protocol. -- [discv5-theory.md] describes the algorithms and data structures. -- [discv5-rationale.md] contains the design rationale. +- [discv5-theory.md] describes the algorithms and data structures for ordinary node + discovery and topic-based service discovery. +- [discv5-rationale.md] contains the design rationale for ordinary node discovery and + topic-based service discovery. ## Comparison With Other Discovery Mechanisms @@ -40,27 +51,31 @@ Systems such as MDNS/Bonjour allow finding hosts in a local-area network. The No Discovery Protocol is designed to work on the Internet and is most useful for applications with a large number of participants spread across the Internet. -Systems using a rendezvous server: these systems are commonly used by desktop applications -or cloud services to connect participants to each other. While undoubtedly efficient, this -requires trust in the operator of the rendezvous server and these systems are prone to -censorship. Compared to a rendezvous server, The Node Discovery Protocol doesn't rely on a -single operator and places a small amount of trust in every participant. It becomes more -resistant to censorship as the size of the network increases and participants of multiple -distinct peer-to-peer networks can share the discovery network to further increase its -resilience. +Systems using a rendezvous server are commonly used by desktop applications or cloud +services to connect participants to each other. While efficient, this requires trust in +the operator of the rendezvous server and these systems are prone to censorship. Compared +to a rendezvous server, the Node Discovery Protocol does not rely on a single operator and +places a small amount of trust in every participant. It becomes more resistant to +censorship as the size of the network increases, and participants of multiple distinct +peer-to-peer networks can share the discovery network to further increase its resilience. + +TopDisc provides topic-based service discovery without introducing a central rendezvous +server or requiring every service to maintain a separate discovery network. It reuses the +ordinary Node Discovery v5 network as a shared discovery substrate, while adding +registrar-side admission control and bounded advertisement storage for service discovery. The Achilles heel of the Node Discovery Protocol is the process of joining the network: while any other node may be used as an entry point, such a node must first be located -through some other mechanism. Several approaches including scalable listing of initial -entry points in DNS or discovery of participants in the local network can be used for -reasonable secure entry into the network. +through some other mechanism. Several approaches, including scalable listing of initial +entry points in DNS or discovery of participants in the local network, can be used for +reasonably secure entry into the network. ## Comparison With Node Discovery v4 -- Topic advertisement was added. -- Arbitrary node metadata can be stored/relayed. -- Node identity crypto is extensible, use of secp256k1 keys isn't strictly required. -- The protocol no longer relies on the system clock. +- Topic-based service discovery through TopDisc was added. +- Arbitrary node metadata can be stored/relayed through ENRs. +- Node identity crypto is extensible; use of secp256k1 keys is not strictly required. +- The protocol no longer relies on the system clock for replay prevention. - Communication is encrypted, protecting topic searches and record lookups against passive observers. diff --git a/enr-entries/topic-discovery.md b/enr-entries/topic-discovery.md new file mode 100644 index 00000000..d7cd87a7 --- /dev/null +++ b/enr-entries/topic-discovery.md @@ -0,0 +1,44 @@ +# The "topic-discovery" ENR entry + +This specification defines the "topic-discovery" ENR entry, which signals that a node +implements the [TopDisc][topdisc] topic discovery capability for [Node Discovery v5][discv5] +and is willing to participate in topic service tables, registrations, and lookups. + +## Entry Format + + entry-key = "topic-discovery" + entry-value = version + +Where `version` is an unsigned integer identifying the supported TopDisc protocol version. +A node implementing the version described in the [TopDisc theory][topdisc] document sets: + + topic-discovery = 1 + +## Semantics + +A node MUST publish the `topic-discovery` entry in its ENR before it can be selected as a +registrar or as a target of topic-discovery queries by other nodes. Nodes whose ENR does +not contain `topic-discovery`, or whose `topic-discovery` value is not understood by the +local implementation, MUST NOT be inserted into local TopDisc service tables and MUST NOT +be selected for topic registration or lookup requests. + +The entry does not by itself indicate which services or topics a node advertises. Service +membership is established through TopDisc registrations and observed at runtime; the ENR +entry only signals capability and protocol version compatibility. + +Future, non-backwards-compatible revisions of the topic discovery capability MUST bump the +`version` value. Implementations that encounter an unknown version SHOULD treat the node +as if it did not publish the entry at all. Implementations MAY support multiple versions +simultaneously; the matching rule between local and remote versions is defined by local +policy. + +## Change Log + +### Initial version (2026) + +The initial version of the "topic-discovery" entry is proposed in this document, alongside +the [Discv5 wire][discv5-wire] and [TopDisc theory][topdisc] specifications. + +[discv5]: ../discv5/discv5.md +[topdisc]: ../discv5/discv5-theory.md +[discv5-wire]: ../discv5/discv5-wire.md diff --git a/enr.md b/enr.md index 57969ca6..37f29871 100644 --- a/enr.md +++ b/enr.md @@ -23,16 +23,19 @@ The key/value pairs must be sorted by key and must be unique, i.e. any key may b only once. The keys can technically be any byte sequence, but ASCII text is preferred. Key names in the table below have pre-defined meaning. -| Key | Value | -|:------------|:-------------------------------------------| -| `id` | name of identity scheme, e.g. "v4" | -| `secp256k1` | compressed secp256k1 public key, 33 bytes | -| `ip` | IPv4 address, 4 bytes | -| `tcp` | TCP port, big endian integer | -| `udp` | UDP port, big endian integer | -| `ip6` | IPv6 address, 16 bytes | -| `tcp6` | IPv6-specific TCP port, big endian integer | -| `udp6` | IPv6-specific UDP port, big endian integer | +| Key | Value | +|:------------------|:---------------------------------------------------| +| `id` | name of identity scheme, e.g. "v4" | +| `secp256k1` | compressed secp256k1 public key, 33 bytes | +| `ip` | IPv4 address, 4 bytes | +| `tcp` | TCP port, big endian integer | +| `udp` | UDP port, big endian integer | +| `ip6` | IPv6 address, 16 bytes | +| `tcp6` | IPv6-specific TCP port, big endian integer | +| `udp6` | IPv6-specific UDP port, big endian integer | +| `topic-discovery` | [TopDisc] capability version, unsigned integer | + +[TopDisc]: enr-entries/topic-discovery.md All keys except `id` are optional, including IP addresses and ports. A record without endpoint information is still valid as long as its signature is valid. If no `tcp6` /