Skip to content

local-stack.md

Latest commit

 

History

History
203 lines (157 loc) · 9.48 KB

local-stack.md

File metadata and controls

203 lines (157 loc) · 9.48 KB
Raw

Local monitoring stack (Docker Compose)

How to run the whole thing locally on a Mac — the scraper, InfluxDB, and Grafana — keep it running across reboots, and split Grafana or InfluxDB out to another host later. Everything here is free/open-source; the only cost is a little RAM.

The supporting files live in the repo:

Architecture & the networking decision

Three components with very different network needs — this drives the whole design:

Component Talks to Needs your home LAN?
scraper (arris_stats.py) the modem at 192.168.100.1, and InfluxDB Yes — must reach the modem
InfluxDB receives from scraper, serves Grafana No — only other containers + your Mac
Grafana reads InfluxDB No — only InfluxDB

Only the scraper touches your home network, and that's where Docker on Mac bites:

  • Docker Desktop for Mac runs containers inside a Linux VM, not on your Mac's network directly. Outbound to a LAN device like 192.168.100.1 often works via NAT, but it is not guaranteed, and network_mode: host does not behave like it does on Linux — on Mac it binds to Docker's VM, not your Mac's Wi-Fi/Ethernet, so it won't reliably reach the modem. (macvlan has the same VM problem.)

So the reliable design is: run InfluxDB + Grafana in Compose, and run the scraper natively on the Mac host. The scraper then uses the same network path your browser uses to hit 192.168.100.1, and writes to InfluxDB via the published port at localhost:8086. No container-to-LAN guesswork. A fully-containerized variant is described at the end for hosts (e.g. Linux) where it works.

+----------------------- your Mac (host network) ------------------------+
|                                                                        |
|  scraper (venv + launchd) --HTTP--> 192.168.100.1  (modem, home LAN)   |
|        |                                                               |
|        +--writes--> localhost:8086                                     |
|                        |                                               |
|   +---- Docker Compose (bridge network "modemnet") ----+               |
|   |   influxdb:8086  <--reads--  grafana:3000          |               |
|   +----------------------------------------------------+               |
|                         localhost:3000 (browser)                       |
+------------------------------------------------------------------------+

Run it

# 1. Config: copy the env file and set a Grafana password
cp .env.example .env
$EDITOR .env

# 2. Bring up InfluxDB + Grafana
docker compose up -d
docker compose ps          # both should be running / healthy

# 3. Point the scraper at the stack (src/config.ini), host-side => localhost:
#    destination = influxdb
#    influx_host = localhost
#    modem_model = cm1000
#    modem_url = http://192.168.100.1/DocsisStatus.asp
#    modem_auth_required = True
#    modem_password = <your modem pw>
#    poll_event_log = True

# 4. Run the scraper on the host (uses your real home network)
source venv/bin/activate
python3 src/arris_stats.py --config src/config.ini --debug

Grafana is at http://localhost:3000 (admin / your .env password) with the InfluxDB datasource already provisioned. Import grafana/sb8200_grafana.json under Dashboards → Import (it works as-is for the CM1000).

Keep it running

Two independent pieces: the containers, and the host-side scraper.

Containers — already handled by restart: unless-stopped. Also enable Docker Desktop → Settings → General → "Start Docker Desktop when you log in" so they return after a reboot.

Scraper — use macOS's native supervisor, launchd, via deploy/com.arris-stats.plist:

# Edit the plist: replace /ABSOLUTE/PATH/TO/arris_cable_modem_stats with your clone path
cp deploy/com.arris-stats.plist ~/Library/LaunchAgents/com.arris-stats.plist
$EDITOR ~/Library/LaunchAgents/com.arris-stats.plist

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.arris-stats.plist
launchctl enable  gui/$(id -u)/com.arris-stats
# stop/remove later:
launchctl bootout gui/$(id -u)/com.arris-stats

KeepAlive restarts the process if it dies. The scraper also re-authenticates in place on an expired modem session, so it self-heals without needing a restart.

Login vs. boot: the LaunchAgent above starts at login (fine if the Mac auto-logs-in or stays logged in). For a dedicated headless "monitor Mac" that should start the scraper at boot, before any login, use the LaunchDaemon template deploy/com.arris-stats.daemon.plist instead (installed to /Library/LaunchDaemons, runs as a service, drops to your user via UserName). Install one of the two, not both — the setup steps are in the file's header comment.

On Linux (e.g. Linux Mint / Ubuntu), use systemd instead of launchd: deploy/arris-stats.service is a system service that starts at boot, restarts on failure, and logs to the journal (journalctl -u arris-stats -f). Setup steps are in the unit file's header comment. Docker Compose for InfluxDB + Grafana works the same on Linux; and because Linux Docker has native networking (no VM), a container there can reach the modem — but running the scraper under systemd is still the simplest, and keeps the modem read on the host network.

Verify all three:

docker compose ps                                              # influx + grafana healthy
curl -s localhost:8086/ping -o /dev/null -w '%{http_code}\n'   # 204 = influx up
curl -s localhost:3000/api/health                              # {"database":"ok",...}
launchctl list | grep arris-stats                              # scraper managed by launchd
tail -f arris-stats.log                                        # scraper writing

Carveout A — split Grafana out later

Grafana's only tie to this stack is the datasource URL. To move it to a separate host (or reuse an existing Grafana):

  1. Keep InfluxDB's 8086 reachable from where Grafana now lives (it already maps 8086 to your Mac; a remote Grafana points at http://<mac-lan-ip>:8086).
  2. Point the new Grafana's InfluxDB datasource at that URL instead of http://influxdb:8086.
  3. Remove the grafana service (and its provisioning mount + grafana-data volume) from docker-compose.yml. Nothing else changes — InfluxDB doesn't care where Grafana is.

Keep your dashboards exported as JSON in the repo (grafana/sb8200_grafana.json) so any Grafana reproduces the views by importing them. Grafana's own state lives in the grafana-data volume; that's the only thing to back up or migrate.

Carveout B — split InfluxDB out later

InfluxDB is the shared datastore, so it has two consumers to repoint:

  1. Stand up InfluxDB 1.8 on the new host (same image, or the native package).
  2. Repoint the scraper: influx_host = <new-host-ip> in src/config.ini (or the influx_host env var). Already fully parameterized — no code change.
  3. Repoint Grafana: change the datasource URL to http://<new-host-ip>:8086.
  4. Remove the influxdb service from docker-compose.yml.

Data migration (pick per how much history you care about):

  • Fresh start — for modem stats this is often fine; just let it repopulate.
  • Move the volume — stop InfluxDB, copy the influxdb-data named volume across.
  • Backup/restore — docker exec influxdb influxd backup -portable /tmp/bk, copy it out, influxd restore on the new host.

The data already lives in the named volume influxdb-data (portable), and both consumers reference InfluxDB by host/URL (never hard-coded), so the split is config-only.

Design note: both carveouts are one-liners because every cross-component link is a network address, not a code dependency — scraper→InfluxDB is influx_host, Grafana→InfluxDB is the datasource URL. Keep it that way (don't bake influxdb/localhost into code) and any piece can move freely.

Alternative: run the scraper in Compose too

Only if the container can reach the modem. Test first:

docker run --rm curlimages/curl -sS -m 5 http://192.168.100.1/GenieLogin.asp | head -c 200

If that prints HTML, uncomment the arris-stats service in docker-compose.yml and set MODEM_PASSWORD in .env. Note it uses influx_host=influxdb (the service name), not localhost. If the test does not return HTML, stay with the host-side scraper — don't fight Docker Desktop's VM networking.

No database at all?

If you just want to eyeball data without InfluxDB, set destination = file in the config and the scraper appends each poll to file_path (default modem_stats.jsonl) as JSON Lines. See the "No InfluxDB? Dump to a file" section in the README.

Command cheat-sheet

docker compose up -d          # start influx + grafana
docker compose ps             # status/health
docker compose logs -f grafana
docker compose down           # stop (keeps volumes/data)
docker compose down -v        # stop AND delete data (careful)
launchctl list | grep arris   # scraper status