Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/concepts/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Wiring Diagram consists of the following resources:
* __Server__: *any* physical server attached to the Fabric including Control Nodes
* __Connection__: *any* logical connection for devices
* usually it's a connection between two or more ports on two different devices
* for example: MCLAG Peer Link, Unbundled/MCLAG server connections, Fabric connection between spine and leaf
* for example: Fabric connection between spine and leaf, and server connections like Unbundled, Bundled, MCLAG, or ESLAG.
* __VLANNamespace__ -> non-overlapping VLAN ranges for attaching servers
* __IPv4Namespace__ -> non-overlapping IPv4 ranges for VPC subnets

Expand Down
33 changes: 10 additions & 23 deletions docs/install-upgrade/build-wiring.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ spec:

1. See the [list](../reference/profiles.md) of profile names
2. More information in the [User Guide](../user-guide/profiles.md#port-naming)
3. Could be MCLAG, ESLAG or nothing, more details in [Redundancy
3. Could be ESLAG or nothing, more details in [Redundancy
Groups](../user-guide/devices.md#redundancy-groups)

## Design Discussion
Expand Down Expand Up @@ -79,12 +79,11 @@ A connection represents the physical wires in your data center. They connect swi

#### Server Connections

A server connection is a connection used to connect servers to the fabric. The fabric will configure the server-facing port according to the type of the connection (MLAG, Bundle, etc). The configuration of the actual server needs to be done by the server administrator. The server port names are not validated by the fabric and used as metadata to identify the connection. A server connection can be one of:
A server connection is a connection used to connect servers to the fabric. The fabric will configure the server-facing port according to the type of the connection (Unbundled, Bundled, ESLAG, etc.). The configuration of the actual server needs to be done by the server administrator. The server port names are not validated by the fabric and used as metadata to identify the connection. A server connection can be one of:

- *Unbundled* - A single cable connecting switch to server.
- *Bundled* - Two or more cables going to a single switch, a LAG or similar.
- *MCLAG* - Two cables going to two different switches, also called dual homing. The switches will need a fabric link between them.
- *ESLAG* - Two to four cables going to different switches, also called multi-homing. If four links are used there will need to be four switches connected to a single server with four NIC ports.
- *ESLAG* - Two to four cables going to different switches, also called multi-homing (EVPN-MH). If four links are used there will need to be four switches connected to a single server with four NIC ports.

``` mermaid
graph TD
Expand All @@ -95,45 +94,33 @@ graph TD
L3([Leaf 3])
L4([Leaf 4])
L5([Leaf 5])
L6([Leaf 6])
L7([Leaf 7])

TS1[Server1]
TS2[Server2]
TS3[Server3]
TS4[Server4]

S1 & S2 ---- L1 & L2 & L3 & L4 & L5 & L6 & L7
S1 & S2 ---- L1 & L2 & L3 & L4 & L5
L1 <-- Bundled --> TS1
L1 <-- Bundled --> TS1
L1 <-- Unbundled --> TS2
L2 <-- MCLAG --> TS3
L3 <-- MCLAG --> TS3
L4 <-- ESLAG --> TS4
L5 <-- ESLAG --> TS4
L6 <-- ESLAG --> TS4
L7 <-- ESLAG --> TS4
L2 <-- ESLAG --> TS3
L3 <-- ESLAG --> TS3
L4 <-- ESLAG --> TS3
L5 <-- ESLAG --> TS3

subgraph VPC 1
TS1
TS2
TS3
TS4
end

subgraph MCLAG
L2
L3
end

subgraph ESLAG
L2
L3
L4
L5
L6
L7
end

```
#### Fabric Connections

Expand Down
3 changes: 3 additions & 0 deletions docs/reference/profiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ features and port naming scheme.

## Switch Feature Matrix

!!! warning "MCLAG Deprecation"
MCLAG is being deprecated in favor of ESLAG (EVPN Multi-Homing) for multi-homing. While still supported, it is recommended to use ESLAG for new deployments.

The following table shows which features are supported by each switch profile:

| Switch Profile | Subinterfaces | ACLs | L2VNI | L3VNI | RoCE | MCLAG | ESLAG | QPN |
Expand Down
16 changes: 7 additions & 9 deletions docs/troubleshooting/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,20 @@ command:
```console
core@control-1 ~ $ kubectl fabric inspect fabric
Switches:
NAME PROFILE ROLE GROUPS SERIAL STATE GEN APPLIED HEARTBEAT
leaf-01 Virtual Switch server-leaf mclag-1 000000000 Ready 1/1 4 minutes ago 15 seconds ago
leaf-02 Virtual Switch server-leaf mclag-1 000000000 Ready 1/1 3 minutes ago 19 seconds ago
leaf-03 Virtual Switch server-leaf eslag-1 000000000 Ready 2/2 5 minutes ago 12 seconds ago
leaf-04 Virtual Switch server-leaf eslag-1 000000000 Ready 2/2 3 minutes ago 17 seconds ago
leaf-05 Virtual Switch server-leaf 000000000 Ready 2/2 5 minutes ago 9 seconds ago
spine-01 Virtual Switch spine 000000000 Ready 1/1 3 minutes ago 19 seconds ago
spine-02 Virtual Switch spine 000000000 Ready 2/2 4 minutes ago 1 second ago
NAME PROFILE ROLE GROUPS SERIAL STATE GEN APPLIED HEARTBEAT
leaf-01 Virtual Switch server-leaf eslag-1 0000000000000000000 Ready 1/1 10 minutes ago 22 seconds ago
leaf-02 Virtual Switch server-leaf eslag-1 0000000000000000000 Ready 1/1 21 minutes ago 19 seconds ago
leaf-03 Virtual Switch server-leaf 0000000000000000000 Ready 1/1 38 minutes ago 10 seconds ago
spine-01 Virtual Switch spine 0000000000000000000 Ready 1/1 15 minutes ago 10 seconds ago
spine-02 Virtual Switch spine 0000000000000000000 Ready 1/1 45 minutes ago 24 seconds ago
```

The output above is from the virtual testing environment. In a deployment of physical
switches, the profile would match the profile of the switch, and the correct
serial number would be displayed.

The `GROUP` column will be populated if you have redundancy configured on the
switches, either MCLAG, or ESLAG.
switches, such as ESLAG (EVPN Multi-Homing).

The `GEN` column shows the applied/current generation. If the numbers are equal
then there are no pending changes for the switches.
Expand Down
136 changes: 72 additions & 64 deletions docs/user-guide/connections.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,37 +58,11 @@ spec:
port: s5248-01/E1/2
```

### MCLAG

MCLAG server connections are used to connect servers to a pair of switches using multiple ports (Dual-homing).
Switches should be configured as an MCLAG pair which requires them to be in a single redundancy group of type `mclag`
and a Connection with type `mclag-domain` between them. MCLAG switches should also have the same `spec.ASN` and
`spec.VTEPIP`. The server interfaces should be configured for 802.3ad LACP.

```yaml
apiVersion: wiring.githedgehog.com/v1beta1
kind: Connection
metadata:
name: server-1--mclag--s5248-01--s5248-02
namespace: default
spec:
mclag:
links: # Defines multiple links between a single server and a pair of switches
- server:
port: server-1/enp2s1
switch:
port: s5248-01/E1/1
- server:
port: server-1/enp2s2
switch:
port: s5248-02/E1/1
```

### ESLAG

ESLAG server connections are used to connect servers to the 2-4 switches using multiple ports (Multi-homing). Switches
should belong to the same redundancy group with type `eslag`, but contrary to the MCLAG case, no other configuration is
required. The server interfaces should be configured for 802.3ad LACP.
ESLAG (EVPN Multi-Homing) server connections are used to connect servers to 2-4 switches using multiple ports.
This is the recommended approach for multi-homing. Switches should belong to the same redundancy group with
type `eslag`. The server interfaces should be configured for 802.3ad LACP.

```yaml
apiVersion: wiring.githedgehog.com/v1beta1
Expand All @@ -98,7 +72,7 @@ metadata:
namespace: default
spec:
eslag:
links: # Defines multiple links between a single server and a 2-4 switches
links: # Defines multiple links between a single server and 2-4 switches
- server:
port: server-1/enp2s1
switch:
Expand Down Expand Up @@ -172,40 +146,6 @@ spec:
port: s5248-04/E1/56
```

### MCLAG-Domain

MCLAG-Domain connections define a pair of MCLAG switches with Session and Peer link between them. Switches should be
configured as an MCLAG, pair which requires them to be in a single redundancy group of type `mclag` and Connection with
type `mclag-domain` between them. MCLAG switches should also have the same `spec.ASN` and `spec.VTEPIP`.

```yaml
apiVersion: wiring.githedgehog.com/v1beta1
kind: Connection
metadata:
name: s5248-01--mclag-domain--s5248-02
namespace: default
spec:
mclagDomain:
peerLinks: # Defines multiple links between a pair of MCLAG switches for Peer link
- switch1:
port: s5248-01/E1/12
switch2:
port: s5248-02/E1/12
- switch1:
port: s5248-01/E1/13
switch2:
port: s5248-02/E1/13
sessionLinks: # Defines multiple links between a pair of MCLAG switches for Session link
- switch1:
port: s5248-01/E1/14
switch2:
port: s5248-02/E1/14
- switch1:
port: s5248-01/E1/15
switch2:
port: s5248-02/E1/15
```

## Connecting Fabric to the outside world

Connections in this section provide connectivity to the outside world. For example, they can be connections to the
Expand Down Expand Up @@ -290,3 +230,71 @@ spec:
ip: 172.30.128.8/31
port: spine-01/E1/5
```

## Deprecated Connections

### MCLAG

!!! warning "Deprecated"
MCLAG is being deprecated. Use [ESLAG](#eslag) for multi-homing instead.

MCLAG server connections are used to connect servers to a pair of switches using multiple ports (Dual-homing).
Switches should be configured as an MCLAG pair which requires them to be in a single redundancy group of type `mclag`
and a Connection with type `mclag-domain` between them. MCLAG switches should also have the same `spec.ASN` and
`spec.VTEPIP`. The server interfaces should be configured for 802.3ad LACP.

```yaml
apiVersion: wiring.githedgehog.com/v1beta1
kind: Connection
metadata:
name: server-1--mclag--s5248-01--s5248-02
namespace: default
spec:
mclag:
links: # Defines multiple links between a single server and a pair of switches
- server:
port: server-1/enp2s1
switch:
port: s5248-01/E1/1
- server:
port: server-1/enp2s2
switch:
port: s5248-02/E1/1
```

### MCLAG-Domain

!!! warning "Deprecated"
MCLAG is being deprecated. Use [ESLAG](#eslag) for multi-homing instead.

MCLAG-Domain connections define a pair of MCLAG switches with Session and Peer link between them. Switches should be
configured as an MCLAG, pair which requires them to be in a single redundancy group of type `mclag` and Connection with
type `mclag-domain` between them. MCLAG switches should also have the same `spec.ASN` and `spec.VTEPIP`.

```yaml
apiVersion: wiring.githedgehog.com/v1beta1
kind: Connection
metadata:
name: s5248-01--mclag-domain--s5248-02
namespace: default
spec:
mclagDomain:
peerLinks: # Defines multiple links between a pair of MCLAG switches for Peer link
- switch1:
port: s5248-01/E1/12
switch2:
port: s5248-02/E1/12
- switch1:
port: s5248-01/E1/13
switch2:
port: s5248-02/E1/13
sessionLinks: # Defines multiple links between a pair of MCLAG switches for Session link
- switch1:
port: s5248-01/E1/14
switch2:
port: s5248-02/E1/14
- switch1:
port: s5248-01/E1/15
switch2:
port: s5248-02/E1/15
```
44 changes: 9 additions & 35 deletions docs/user-guide/devices.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ spec:
- some-group
redundancy: # Optional field to define that switch belongs to the redundancy group
group: eslag-1 # Name of the redundancy group
type: eslag # Type of the redundancy group, one of mclag or eslag
type: eslag # Type of the redundancy group, should be eslag
enableAllPorts: true # Optional field to enable all ports on the switch by default
portAutoNegs: # Used for rj45 copper ports, and 800G ports for link conditioning
E1/18: true
Expand Down Expand Up @@ -139,17 +139,14 @@ spec: {}
## Redundancy Groups

Redundancy groups are used to define the redundancy between switches. It's a regular `SwitchGroup` used by multiple
switches and currently it could be MCLAG or ESLAG (EVPN MH / ESI). A switch can only belong to a single redundancy
group.
switches. ESLAG (EVPN Multi-Homing) is the recommended approach and supports up to 4 switches. A switch can only
belong to a single redundancy group.

MCLAG is only supported for pairs of switches and ESLAG is supported for up to 4 switches. Multiple types of redundancy
groups can be used in the fabric simultaneously.
Connections with type `eslag` are used to define the server connections to switches. They are only supported if the
switch belongs to a redundancy group with the corresponding type.

Connections with types `mclag` and `eslag` are used to define the servers connections to switches. They are only
supported if the switch belongs to a redundancy group with the corresponding type.

In order to define a MCLAG or ESLAG redundancy group, you need to create a `SwitchGroup` object and assign it to the
switches using the `redundancy` field.
To define an ESLAG redundancy group, create a `SwitchGroup` object and assign it to the switches using the
`redundancy` field.

Example of switch configured for ESLAG:

Expand All @@ -174,31 +171,8 @@ spec:
...
```

And example of switch configured for MCLAG:

```{.yaml .annotate linenums="1" title="MCLAG-switchgroup.yaml"}
apiVersion: wiring.githedgehog.com/v1beta1
kind: SwitchGroup
metadata:
name: mclag-1
namespace: default
spec: {}
---
apiVersion: wiring.githedgehog.com/v1beta1
kind: Switch
metadata:
name: s5248-01
namespace: default
spec:
...
redundancy:
group: mclag-1
type: mclag
...
```

In case of MCLAG it's required to have a special connection with type `mclag-domain` that defines the peer and session
links between switches. For more details, see [Connections](./connections.md).
!!! warning "MCLAG Deprecated"
MCLAG is being deprecated. Use ESLAG for multi-homing instead.

## Servers

Expand Down
17 changes: 5 additions & 12 deletions docs/user-guide/host-settings.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,19 +10,12 @@ additional details on options and behavior, consult the [kernel bonding driver][
[nmanager]: https://networkmanager.dev/docs/admins/
[netplan]: https://documentation.ubuntu.com/server/explanation/networking/configuring-networks/index.html

## MCLAG / ESLAG
## Multi-homing

The multi-chassis LAG architecture is a way to provide device redundancy
in a network architecture. At the physical layer, an MCLAG topology is a single
server connected to two different switches and, those switches are directly connected
to each other in addition to being connected to the rest of the fabric.

ESLAG is a similar technology to MCLAG, with the beneficial difference that the
switches do not need to be directly connected to each other. There can be up to 4
switches in an ESLAG group, whereas MCLAG is always two switches.

Regardless of whether MCLAG or ESLAG is chosen, the host must configure its two
(or more) ports using LACP (IEEE 802.3ad).
ESLAG (EVPN Multi-Homing) is the recommended way to provide device redundancy.
A server connects to multiple switches (up to 4) without requiring the switches
to be directly connected to each other. The host must configure its ports using
LACP (IEEE 802.3ad).

### Server Settings

Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/shrink-expand.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ a spine is being added, it shares the same ASN as the existing spines. For an
IPv4 address increment the largest IP by one, keep the same netmask.

!!! note
If the`Switch` will be used in `ESLAG` or `MCLAG` groups, appropriate groups should exist. Redundancy groups should
If the`Switch` will be used in `ESLAG` or `MCLAG` (deprecated) groups, appropriate groups should exist. Redundancy groups should
be specified in the `Switch` object before creation.

#### Expanding Example
Expand Down
Loading