Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .env.model.public
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
# chat db networking
HERMES_MODEL_DB_FIXED_IP=
HERMES_MODEL_DB_FIXED_IP=
HERMES_MODEL_CLOUD_MARIA_IP=
88 changes: 87 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,94 @@ What: socket.io, Notion, OpenAI, Whatsapp/Telegram/etc APIs

## Getting Started

### Mail + Nextcloud deployment (Docker Compose now, k3s via `kompose`)

1. Copy each `*.public` file to its matching `*.private` file and fill in the required values.
2. Run the deployment in order from the repository root:

```bash
cp .env.common.public .env.common.private
cp .env.mail.public .env.mail.private
cp .env.cloud.public .env.cloud.private
cp .env.ingress.public .env.ingress.private
cp .env.model.public .env.model.private

./init-networks.sh
./init-weagle.sh
./init-model.sh
./init-mail.sh # local mail storage
./init-cloud.sh
```
```

Use `./init-mail.sh setup:storage` only when the mail S3 variables are set and `s3fs` is installed. For a single-command bring-up, run `./init.sh`.

For k3s generation, run `./init-k8s-config.sh` after the same private env files are present. The generated manifests depend on the compose service ports, so the mail template now exports SMTP/Submission/IMAP/IMAPS/POP3/POP3S/Sieve explicitly for Traefik and `kompose`.

### MUST-NOT-BE-EMPTY variables

#### Common (`.env.common.private`)

| Variable | Why it is required |
| --- | --- |
| `HERMES_APEX` | Base mail domain for certificates and mailserver identity. |
| `HERMES_HOSTNAME` | Public FQDN used by Traefik and Nextcloud. |
| `HERMES_CERT_FILE` | Mounted TLS certificate filename. |
| `HERMES_CERT_KEY_FILE` | Mounted TLS private key filename. |
| `HERMES_MAIL_DOMAINS` | Domains provisioned in docker-mailserver. |
| `HERMES_MAIL_MAIN_HOSTNAME` | Mail host used for mailbox creation and DNS guidance. |
| `HERMES_DNS_RESOLVER` | Resolver injected into mail/cloud containers. |
| `HERMES_INGRESS_SUBNET` | Trusted proxy subnet for Nextcloud behind Traefik. |
| `HERMES_CLOUD_DB_HOST` | MariaDB host for Nextcloud. |
| `HERMES_PORT_PREFIX` | Prefix used when compose is converted into k8s services. |

#### Mail (`.env.mail.private`)

| Variable | Why it is required |
| --- | --- |
| `HERMES_MAIL_CERT_TYPE` | `docker-mailserver` TLS mode (`manual` for the bundled cert mount). |
| `HERMES_MAIN_MAILBOX` | Main administrator mailbox reference. |
| `HERMES_MAIL_INITIAL_BOXES` | Comma-separated local parts for optional bootstrap mailboxes. |
| `HERMES_MAIL_INITIAL_BOXES_DEFAULT_PASSWORD` | Password used when `setup:mailboxes` is invoked. |

Optional only when using S3-backed mail storage: `HERMES_MAIL_S3_BUCKET`, `HERMES_MAIL_S3_HOST`, `HERMES_MAIL_S3_KEY`, `HERMES_MAIL_S3_SECRET`.

#### Cloud (`.env.cloud.private`)

| Variable | Why it is required |
| --- | --- |
| `HERMES_CLOUD_BASEPATH` | Nextcloud path prefix (default `/cloud`). |
| `HERMES_CLOUD_DB_NAME` | Nextcloud MariaDB database name. |
| `HERMES_CLOUD_DB_USER` | Nextcloud MariaDB user. |
| `HERMES_CLOUD_DB_PASS` | Nextcloud MariaDB password. |
| `HERMES_CLOUD_DB_REDIS` | Redis hostname reachable on `hermes-net-model-cloud`. |

Optional S3/object-store values: `HERMES_CLOUD_S3_BUCKET`, `HERMES_CLOUD_S3_HOST`, `HERMES_CLOUD_S3_REGION`, `HERMES_CLOUD_S3_KEY`, `HERMES_CLOUD_S3_SECRET`.

#### Ingress (`.env.ingress.private`)

| Variable | Why it is required |
| --- | --- |
| `HERMES_TRAEFIK_BASE_AUTH` | Protects the Traefik dashboard. |
| `HERMES_CLOUDFLARE_EMAIL` | ACME DNS challenge account. |
| `HERMES_CLOUDFLARE_API_KEY` | ACME DNS challenge credential. |

#### Model (`.env.model.private`)

| Variable | Why it is required |
| --- | --- |
| `HERMES_MODEL_CLOUD_MARIA_IP` | Static IP assigned to `hermes-model-mariadb` on `hermes-net-model-cloud`. |

### Fresh-server notes

- `./init-mail.sh` is idempotent for local storage and only mounts S3 when `setup:storage` is requested.
- `./init-cloud.sh` now writes `cloud/data/cloud/config/hermes.config.php` from `cloud/_config.php`, so fresh installs no longer depend on an already-existing Nextcloud `config.php`.
- The mail Traefik TCP routers use wildcard SNI matching for plain TCP/STARTTLS protocols and passthrough only on implicit TLS ports.

### Troubleshooting

- If `init-mail.sh` or `init-cloud.sh` exits early, fill in the missing required env variable reported by the script.
- If `setup:storage` fails, confirm `s3fs` is installed and that `mail/data/email-data` is not already mounted.
- If Nextcloud is reachable but redirects incorrectly, verify `HERMES_HOSTNAME`, `HERMES_CLOUD_BASEPATH`, and `HERMES_INGRESS_SUBNET`, then rerun `./init-cloud.sh` to regenerate `hermes.config.php`.

Documentation: [Research Paper](https://angeloreale.notion.site/Lady-Science-100-Days-of-Products-Day-012-DreamLetter-Angelo-Reale-078155e635a747e8b06ba1c67ec28bfe?pvs=4)

Expand Down
10 changes: 9 additions & 1 deletion cloud/_config.php
Original file line number Diff line number Diff line change
@@ -1,13 +1,21 @@
<?php
$CONFIG = array (
'htaccess.RewriteBase' => '${HERMES_CLOUD_BASEPATH}',
'overwrite.cli.url' => 'https://${HERMES_HOSTNAME}${HERMES_CLOUD_BASEPATH}',
'overwritehost' => '${HERMES_HOSTNAME}',
'overwriteprotocol' => 'https',
'overwritewebroot' => '${HERMES_CLOUD_BASEPATH}',
'memcache.local' => '\OC\Memcache\APCu',
'memcache.distributed' => '\OC\Memcache\Redis',
'memcache.locking' => '\OC\Memcache\Redis',
'redis' => [
'host' => '${HERMES_CLOUD_DB_REDIS}',
'port' => 6379,
],
'trusted_proxies' =>
array (
0 => '${HERMES_INGRESS_SUBNET}',
),
'apps_paths' =>
array (
0 =>
Expand All @@ -23,4 +31,4 @@
'writable' => true,
),
),
);
);
3 changes: 0 additions & 3 deletions cloud/_docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,6 @@ services:
- "./data/cloud/custom_apps:/var/www/html/custom_apps"
- "./data/cloud/config:/var/www/html/config"
- "./data/cloud/data:/var/www/html/data"
external_links:
- hermes-model-mariadb
- hermes-model-redis
environment:
PUID: 1000
PGID: 1000
Expand Down
67 changes: 51 additions & 16 deletions init-cloud.sh
Original file line number Diff line number Diff line change
@@ -1,26 +1,29 @@
#!/bin/bash
set -euo pipefail
# init mail
echo -e "\033[0;62m\033[0;49;35m"
set -a && source .env.common.private && set +a
set -a && source .env.cloud.private && set +a
root_dir="$(pwd)"
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
set -a && source "$script_dir/.env.common.private" && set +a
set -a && source "$script_dir/.env.cloud.private" && set +a
root_dir="$script_dir"

log () {
local level="${2:-}"
echo -e "\033[0;49;35m"
# warning
if [[ "$2" == "2" ]]; then
if [[ "$level" == "2" ]]; then
echo -e "\033[35;43m$1\033[0m"
fi
# error
if [[ "$2" == "1" ]]; then
if [[ "$level" == "1" ]]; then
echo -e "\033[35;41m\033[33m$1\033[0m"
fi
# success
if [[ "$2" == "0" ]]; then
if [[ "$level" == "0" ]]; then
echo -e "\033[35;42m$1\033[0m"
fi
# normal
if [[ "$2" == "" ]]; then
if [[ "$level" == "" ]]; then
echo -e "\033[35;46m$1\033[0m"
fi
}
Expand All @@ -44,26 +47,58 @@ take () {
while true; do echo -n .; sleep 1; done | pv -s $1 -S -F '%t %p' > /dev/null
}

require_env () {
local name="$1"
if [[ -z "${!name:-}" ]]; then
log "dp::hermes::cloud::(error)::Missing required environment variable: $name" 1
exit 1
fi
}

validate_cloud_env () {
for name in \
HERMES_HOSTNAME \
HERMES_CLOUD_BASEPATH \
HERMES_CLOUD_DB_NAME \
HERMES_CLOUD_DB_USER \
HERMES_CLOUD_DB_PASS \
HERMES_CLOUD_DB_HOST \
HERMES_CLOUD_DB_REDIS \
HERMES_DNS_RESOLVER \
HERMES_INGRESS_SUBNET
do
require_env "$name"
done
}

# Cloud
validate_cloud_env
mkdir -p "$root_dir/cloud/data/cloud/html" "$root_dir/cloud/data/cloud/custom_apps" "$root_dir/cloud/data/cloud/config" "$root_dir/cloud/data/cloud/data"

log "dp::hermes::hermes_net_cloud::(busy):: Preparing Claudia (Storage, Calendar, Mail Web Client: Nextcloud)." 2

log "dp::hermes::hermes_net_cloud::(busy):: Preparing Claudia: Consolidating files."
origin="./cloud/_docker-compose.yml"
destination="./cloud/docker-compose.yml"
tmpfile=$(mktemp --tmpdir=.)
cp -p $origin $tmpfile
cat $origin | envsubst > $tmpfile && mv $tmpfile $destination
origin="$root_dir/cloud/_docker-compose.yml"
destination="$root_dir/cloud/docker-compose.yml"
tmpfile=$(mktemp --tmpdir="$root_dir")
envsubst < "$origin" > "$tmpfile" && mv "$tmpfile" "$destination"

log "dp::hermes::hermes_net_cloud::(busy):: Preparing Claudia: Consolidating files."
sed -i "s/'host' => 'redis'/'host' => '$HERMES_CLOUD_DB_REDIS'/g" cloud/data/cloud/config/config.php
sed -i "s/'htaccess.RewriteBase' => '/'/'htaccess.RewriteBase' => '$HERMES_CLOUD_BASEPATH'/g" cloud/data/cloud/config/config.php
cloud_config_destination="$root_dir/cloud/data/cloud/config/hermes.config.php"
# Limit substitution to Hermes placeholders so PHP symbols like $CONFIG stay intact.
envsubst '${HERMES_CLOUD_BASEPATH} ${HERMES_HOSTNAME} ${HERMES_CLOUD_DB_REDIS} ${HERMES_INGRESS_SUBNET}' \
< "$root_dir/cloud/_config.php" > "$cloud_config_destination"
if [[ ! -s "$cloud_config_destination" ]]; then
log "dp::hermes::cloud::(error)::Failed to generate $cloud_config_destination" 1
exit 1
fi

# dir setup
take 5 "dp::hermes::hermes_net_cloud::(busy):: Launching Docker Compose Swarms."


cd cloud
cd "$root_dir/cloud"
docker compose up -d
cd $root_dir
cd "$root_dir"

log "dp::hermes::hermes_net_cloud::(idle)::all good." 0
Loading