Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/install/harvester-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,10 @@ DNS resolution is unavailable in the `after-install-chroot stage`, and the `name
```yaml
os:
after_install_chroot_commands:
- "rm -f /etc/resolv.conf && echo 'nameserver 8.8.8.8' | sudo tee /etc/resolv.conf"
- "echo 'nameserver 8.8.8.8' | sudo tee /etc/resolv.conf"
- "mkdir /usr/local/bin"
- "curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 700 get_helm.sh && ./get_helm.sh"
- "rm -f /etc/resolv.conf && ln -s /var/run/netconfig/resolv.conf /etc/resolv.conf"
- "rm -f /etc/resolv.conf"
```


Expand Down
2 changes: 1 addition & 1 deletion docs/install/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ Harvester nodes require the following port connections or inbound rules. Typical
| TCP | 9796 | Harvester management and compute nodes | Node-exporter |
| TCP | 30000-32767 | Harvester management and compute nodes | NodePort port range |
| TCP | 22 | Harvester management and compute nodes | sshd |
| UDP | 68 | Harvester management and compute nodes | Wicked |
| UDP | 68 | Harvester management and compute nodes | NetworkManager |
| TCP | 3260 | Harvester management and compute nodes | iscsid |

### Port Requirements for Integrating Harvester with Rancher
Expand Down
171 changes: 54 additions & 117 deletions docs/install/update-harvester-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,179 +12,116 @@ description: How to update Harvester configuration after installation
<link rel="canonical" href="https://docs.harvesterhci.io/v1.6/install/update-harvester-configuration"/>
</head>

Harvester's OS has an immutable design, which means most files in the OS revert to their pre-configured state after a reboot. The Harvester OS loads the pre-configured values of system components from configuration files during the boot time.
Harvester's OS has an immutable design, which means most files in the OS revert to their pre-configured state after a reboot. The Harvester OS loads the pre-configured values of system components from configuration files during the boot time.

This page describes how to edit some of the most-requested Harvester configurations. To update a configuration, you must first update the runtime value in the system and then update configuration files to make the changes persistent between reboots.
To update a configuration, you must first update the runtime value in the system and then update configuration files to ensure that changes persist between reboots.

However, starting with v1.7.0, network configuration changes made using the `nmcli` tool persist automatically because the `/etc/NetworkManager` directory is now included in the list of persistent paths.

:::note

If you upgrade from a version before `v1.1.2`, the `cloud-init` file in examples will be `/oem/99_custom.yaml`. Please substitute the value if needed.

:::

## DNS servers
## Password of user `rancher`

### Runtime change

1. Log in to a Harvester node and become root. See [how to log into a Harvester node](../troubleshooting/os.md#how-to-log-in-to-a-harvester-node) for more details.
1. Edit `/etc/sysconfig/network/config` and update the following line. Use a space to separate DNS server addresses if there are multiple servers.

```
NETCONFIG_DNS_STATIC_SERVERS="8.8.8.8 1.1.1.1"
```
1. [Log in to a Harvester node](../troubleshooting/os.md#how-to-log-in-to-a-harvester-node) using the `rancher` user account.

1. Update and reload the configuration with the following command:

```
netconfig update
```

1. Confirm the file `/etc/resolv.conf` contains the correct DNS servers with the `cat` command:

```
cat /etc/resolv.conf
```
1. Restart rke2-coredns:
```
kubectl rollout restart deployment/rke2-coredns-rke2-coredns -n kube-system
```
1. Confirm rke2-coredns was rolled out successfully:
```
kubectl rollout status deployment/rke2-coredns-rke2-coredns -n kube-system
```
1. Reset the password for the `rancher` user account by running the command `passwd`.

### Configuration persistence

Beginning with v1.1.2, the persistent name of the cloud-init file is `/oem/90_custom.yaml`. Harvester now uses a newer version of Elemental, which creates the file during installation.

When upgrading from an earlier version to `v1.1.2` or later, Harvester retains the old file name (`/oem/99_custom.yaml`) to avoid confusion. You can manually rename the file to `/oem/90_custom.yaml` if necessary.

1. Backup the elemental `cloud-init` file `/oem/90_custom.yaml` as follows:

```
cp /oem/90_custom.yaml /oem/install/90_custom.yaml.$(date --iso-8601=minutes)
```

1. Edit `/oem/90_custom.yaml` and update the value under the yaml path `stages.initramfs[0].commands`. The `commands` array must contain a line to manipulate the `NETCONFIG_DNS_STATIC_SERVERS` config. Add the line if the line doesn't exist.

The following example adds a line to change the `NETCONFIG_DNS_STATIC_SERVERS` config:

```
stages:
initramfs:
- commands:
- sed -i 's/^NETCONFIG_DNS_STATIC_SERVERS.*/NETCONFIG_DNS_STATIC_SERVERS="8.8.8.8 1.1.1.1"/' /etc/sysconfig/network/config
```

Replace the DNS server addresses and save the file. Harvester sets up new servers after rebooting.
1. Edit `/oem/90_custom.yaml` and update the yaml path `stages.initramfs[0].users.rancher.passwd`.

For information about specifying the `rancher` user account password in an encrypted form, see [`os.password`](./harvester-configuration.md#ospassword).

## NTP servers

We introduce the new mechanism for the NTP configuration in Harvester v1.2.0.
We introduced a new mechanism for NTP configuration in Harvester v1.2.0.

For more information about NTP settings in Harvester v1.2.0 and later versions, see the [NTP servers](../host/host.md#ntp-configuration).
For more information about NTP settings in Harvester v1.2.0 and later versions, see [NTP servers](../host/host.md#ntp-configuration).

## Password of user `rancher`
## DNS servers

### Runtime change
1. [Log in to a Harvester node](../troubleshooting/os.md#how-to-log-in-to-a-harvester-node) using the root account.
1. Check the management interface configuration, and then run either of the following commands:

1. Log in to a Harvester node as user `rancher`. See [how to log into a Harvester node](../troubleshooting/os.md#how-to-log-in-to-a-harvester-node) for more details.
1. To reset the password for the user `rancher`, run the command `passwd`.
- Not configured to use a VLAN:

```
nmcli con modify bridge-mgmt ipv4.dns 8.8.8.8,1.1.1.1 && nmcli device reapply mgmt-br
```

### Configuration persistence
- Configured to use a VLAN:

1. Backup the elemental `cloud-init` file `/oem/90_custom.yaml` as follows:
You must replace `VLAN_ID` with the actual ID. To view a list of configured connections and devices, run the command `nmcli con`.

```
cp /oem/90_custom.yaml /oem/install/90_custom.yaml.$(date --iso-8601=minutes)
nmcli con modify vlan-mgmt ipv4.dns 8.8.8.8,1.1.1.1 && nmcli device reapply mgmt-br.VLAN_ID
```

1. Edit `/oem/90_custom.yaml` and update the yaml path `stages.initramfs[0].users.rancher.passwd`. Refer to the configuration [`os.password`](./harvester-configuration.md#ospassword) for details on how to specify the password in an encrypted form.

1. Verify that the file `/etc/resolv.conf` contains the correct DNS servers by running the `cat` command.

## Bonding slaves

You can update the slave interfaces of Harvester's management bonding interface `mgmt-bo`.

### Runtime change

1. Log in to a Harvester node and become root. See [how to log into a Harvester node](../troubleshooting/os.md#how-to-log-in-to-a-harvester-node) for more details.
1. Identify the interface names with the following command:

```
ip a
cat /etc/resolv.conf
```

1. Edit `/etc/sysconfig/network/ifcfg-mgmt-bo` and update the lines associated with bonding slaves and bonding mode:
1. Restart the `rke2-coredns` deployment.

```
BONDING_SLAVE_0='ens5'
BONDING_SLAVE_1='ens6'
BONDING_MODULE_OPTS='miimon=100 mode=balance-tlb '
kubectl rollout restart deployment/rke2-coredns-rke2-coredns -n kube-system
```

1. Restart the network with the `wicked ifreload` command:
1. Verify that the `rke2-coredns` deployment was rolled out successfully.

```
wicked ifreload mgmt-bo
kubectl rollout status deployment/rke2-coredns-rke2-coredns -n kube-system
```

:::caution
## Bonding slaves

A mistake in the configuration may disrupt the SSH session.
You can update the slave interfaces of Harvester's management bonding interface `mgmt-bo`.

:::
1. Log in to a Harvester node and become root. See [how to log into a Harvester node](../troubleshooting/os.md#how-to-log-in-to-a-harvester-node) for more details.
1. Identify the interface names with the following command:

### Configuration persistence
```
$ nmcli device
DEVICE TYPE STATE CONNECTION
mgmt-br bridge connected bridge-mgmt
...
mgmt-bo bond connected bond-mgmt
ens6 ethernet connected bond-slave-ens6
ens7 ethernet disconnected --
...
```

1. Use the `nmcli` tool to create a connection for the interface and attach it to the management bond.

1. Backup the elemental cloud-init file `/oem/90_custom.yaml` as follows:
Example:

```
cp /oem/90_custom.yaml /oem/install/90_custom.yaml.$(date --iso-8601=minutes)
```
$ nmcli con add type bond-slave ifname ens7 master mgmt-bo
Connection 'bond-slave-ens7' (5a379328-178a-4167-b065-b5426facd659) successfully added.

1. Edit `/oem/90_custom.yaml` and update the yaml path `stages.initramfs[0].files`. More specifically, update the content of the `/etc/sysconfig/network/ifcfg-mgmt-bo` file and edit the `BONDING_SLAVE_X` and `BONDING_MODULE_OPTS` entries accordingly:
1. You should now be able to see the device is connected:

```
stages:
initramfs:
- ...
files:
- path: /etc/sysconfig/network/ifcfg-mgmt-bo
permissions: 384
owner: 0
group: 0
content: |+
STARTMODE='onboot'
BONDING_MASTER='yes'
BOOTPROTO='none'
POST_UP_SCRIPT="wicked:setup_bond.sh"


BONDING_SLAVE_0='ens5'
BONDING_SLAVE_1='ens6'

BONDING_MODULE_OPTS='miimon=100 mode=balance-tlb '

DHCLIENT_SET_DEFAULT_ROUTE='no'

encoding: ""
ownerstring: ""
- path: /etc/sysconfig/network/ifcfg-ens6
permissions: 384
owner: 0
group: 0
content: |
STARTMODE='hotplug'
BOOTPROTO='none'
encoding: ""
ownerstring: ""
$ nmcli device
DEVICE TYPE STATE CONNECTION
mgmt-br bridge connected bridge-mgmt
...
mgmt-bo bond connected bond-mgmt
ens6 ethernet connected bond-slave-ens6
ens7 ethernet connected bond-slave-ens7
```

:::note

If you didn't select an interface during installation, you must add an entry to initialize the interface. Please check the `/etc/sysconfig/network/ifcfg-ens6` file creation in the above example. The file name should be `/etc/sysconfig/network/ifcfg-<interface-name>`.

:::
3 changes: 2 additions & 1 deletion docs/logging/harvester-logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ In Harvester's practice, the `Logging`, `Audit` and `Event` shares one architect
The Harvester logging infrastructure allows you to aggregate Harvester logs into an external service such as [Graylog](https://www.graylog.org), [Elasticsearch](https://www.elastic.co/elasticsearch/), [Splunk](https://www.splunk.com/), [Grafana Loki](https://grafana.com/oss/loki/) and others.

### Collected Logs

See below for a list logs that are collected:
- Logs from all cluster `Pods`
- Kernel logs from each `node`
Expand All @@ -55,7 +56,7 @@ See below for a list logs that are collected:
- `rke2-agent`
- `rancherd`
- `rancher-system-agent`
- `wicked`
- `NetworkManager`
- `iscsid`

:::note
Expand Down
55 changes: 29 additions & 26 deletions docs/networking/best-practice.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,35 +40,38 @@ You can follow the steps below and run them in each node step by step.

Before any action is taken, it is important to collect the current network information and status.

- Harvester network configuration: By default, Harvester creates a bond interface named `mgmt-bo` for the management network and one new bond interface for each cluster network. Harvester saves network configuration details in the file `/oem/90_custom.yaml`.
- Harvester network configuration: By default, Harvester creates a bond interface named `mgmt-bo` for the management network. On top of that is a bridge interface named `mgmt-br`, which may optionally use a VLAN. Each cluster network also has one new bond interface. You can view the current connection details using the `nmcli` tool.

Example: A NIC named `ens3` was added to the `mgmt-bo` bond interface.
Example:

```
- path: /etc/sysconfig/network/ifcfg-mgmt-bo
permissions: 384
owner: 0
group: 0
content: |+
STARTMODE='onboot'
BONDING_MASTER='yes'
BOOTPROTO='none'
POST_UP_SCRIPT="wicked:setup_bond.sh"
BONDING_SLAVE_0='ens3'
BONDING_MODULE_OPTS='miimon=100 mode=active-backup '
DHCLIENT_SET_DEFAULT_ROUTE='no'
encoding: ""
ownerstring: ""

- path: /etc/sysconfig/network/ifcfg-ens3
permissions: 384
owner: 0
group: 0
content: |
STARTMODE='hotplug'
BOOTPROTO='none'
encoding: ""
ownerstring: ""
$ nmcli

mgmt-br.2017: connected to vlan-mgmt
"mgmt-br.2017"
vlan, 5C:B9:01:89:C2:F5, sw, mtu 1500
ip4 default
inet4 10.115.55.20/21
route4 10.115.48.0/21 metric 400
route4 default via 10.115.55.254 metric 400

...

mgmt-bo: connected to bond-mgmt
"mgmt-bo"
bond, 5C:B9:01:89:C2:F5, sw, mtu 1500
master mgmt-br

mgmt-br: connected to bridge-mgmt
"mgmt-br"
bridge, 5C:B9:01:89:C2:F5, sw, mtu 1500

eno50: connected to bond-slave-eno50
"Intel 82599ES SFI/SFP+"
ethernet (ixgbe), 5C:B9:01:89:C2:F5, hw, sriov, mtu 1500
master mgmt-bo

...
```

- Physical NICs: You can use the command `ip link` to retrieve related information, including the state of each NIC and the corresponding master (if applicable).
Expand Down
Loading