Skip to content

Improve CrowdSec hardening and reverse proxy troubleshooting #917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TechHutTV wants to merge 2 commits into
base: main
Choose a base branch
from
+270 −22
Open

Improve CrowdSec hardening and reverse proxy troubleshooting #917

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7b8d77e
Improve CrowdSec setup, recovery, monitoring, and access-log document…
TechHutTV Aug 11, 2026
c5d45c3
Refine CrowdSec dashboard recovery and access-log field documentation
TechHutTV Aug 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 28 additions & 7 deletions src/pages/manage/reverse-proxy/access-logs.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ NetBird logs every request and connection that passes through your reverse proxy

## Viewing access logs

Access logs are available in the NetBird dashboard under **Activity** > **Proxy Events**. This view displays a table of all HTTP requests and L4 connections that have passed through your reverse proxy services, with filters to narrow down results by time range, status, or other fields.
Access logs are available in the NetBird dashboard under **Reverse Proxy** > **Access Logs**. This view displays a table of all HTTP requests and L4 connections that have passed through your reverse proxy services, with filters to narrow down results by time range, status, or other fields.

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Update the screenshot alt text to match the renamed view.

Line 12 now names the page Access Logs, but the adjacent image at Line 15 still says Proxy Events table. Update the alt text so screen-reader users receive the current dashboard name.

Proposed fix 
-    <img src="/docs-static/img/manage/reverse-proxy/access-logs/access-logs-table.png" alt="Proxy Events table showing reverse proxy access log entries" className="imagewrapper"/>
+    <img src="/docs-static/img/manage/reverse-proxy/access-logs/access-logs-table.png" alt="Access Logs table showing reverse proxy access log entries" className="imagewrapper"/>
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/pages/manage/reverse-proxy/access-logs.mdx` at line 12, Update the
adjacent image alt text near the Access Logs description to use the renamed
“Access Logs” dashboard view instead of “Proxy Events table,” while preserving
the existing image and surrounding documentation.


<p>
<img src="/docs-static/img/manage/reverse-proxy/access-logs/access-logs-table.png" alt="Proxy Events table showing reverse proxy access log entries" className="imagewrapper"/>
Expand Down Expand Up @@ -37,9 +37,9 @@ Every log entry (HTTP and L4) shares a common set of fields. Some fields are onl
| **Bytes Downloaded** | Bytes sent from backend to client | Yes | Yes |
| **Source IP** | The client's IP address | Yes | Yes |
| **Location** | Country, city, and subdivision based on source IP geolocation | Yes | Yes |
| **Auth Method** | Authentication method used (SSO, password, PIN, header, or none) | Yes | N/A |
| **Auth Method** | Authentication method used (SSO, password, PIN, header, or none). For denied requests, carries the restriction code instead (e.g. `ip_restricted`, `crowdsec_ban`) | Yes | N/A |
| **User** | The authenticated user's ID (if SSO was used) | Yes | N/A |
| **Reason** | Reason for denial, if applicable | Yes | Yes |
| **Reason** | `Authentication failed` when authentication or an access restriction rejected the request, or `Request failed` when an authenticated request returned `4xx`/`5xx`. Not a specific denial code: see the note under [Deny reasons](#deny-reasons) | Yes | No |
Comment on lines +40 to +42

@coderabbitai coderabbitai Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

ref="${NETBIRD_REF:?Set NETBIRD_REF to the matching NetBird server commit or release}"
base="https://raw.githubusercontent.com/netbirdio/netbird/${ref}"

for path in \
  proxy/auth/auth.go \
  proxy/internal/auth/middleware.go \
  proxy/internal/accesslog/logger.go \
  management/internals/modules/reverseproxy/accesslogs/accesslogentry.go
do
  printf '\n--- %s ---\n' "$path"
  curl -fsSL "$base/$path" |
    rg -n -C 4 \
      'MethodPassword|MethodPIN|MethodOIDC|MethodHeader|attemptedMethod|SetAuthMethod|DenyReason|AuthMechanism|AuthMethodUsed|Authentication failed|Request failed'
done

Repository: netbirdio/docs

Length of output: 244

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- changed file status ---'
git status --short
printf '%s\n' '--- relevant file lines ---'
cat -n src/pages/manage/reverse-proxy/access-logs.mdx | sed -n '1,180p'
printf '%s\n' '--- recent repository references ---'
rg -n 'NETBIRD_REF|/api/events/proxy|auth_method_used|Denied L4|Deny reasons' . -g '*.mdx' -g '*.md' -g '*.yml' -g '*.yaml' -g '*.json' | head -200

Repository: netbirdio/docs

Length of output: 13427

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- changed file status ---'
git status --short
printf '%s\n' '--- relevant file lines ---'
cat -n src/pages/manage/reverse-proxy/access-logs.mdx | sed -n '1,180p'
printf '%s\n' '--- repository references ---'
rg -n 'NETBIRD_REF|/api/events/proxy|auth_method_used|Denied L4|Deny reasons' . -g '*.mdx' -g '*.md' -g '*.yml' -g '*.yaml' -g '*.json' | head -200

Repository: netbirdio/docs

Length of output: 13420

🏁 Script executed:

#!/bin/bash
set -euo pipefail

fetch() {
  local ref="$1" path="$2"
  printf '\n--- %s %s ---\n' "$ref" "$path"
  curl -fsSL "https://raw.githubusercontent.com/netbirdio/netbird/${ref}/${path}" |
    nl -ba |
    rg -n -C 8 \
      'MethodPassword|MethodPIN|MethodOIDC|MethodHeader|attemptedMethod|SetAuthMethod|DenyReason|AuthMechanism|AuthMethodUsed|Authentication failed|Request failed|reason|auth_method_used|metadata|AccessLog'
}

for ref in main 1e7b16db; do
  fetch "$ref" proxy/auth/auth.go
  fetch "$ref" proxy/internal/auth/middleware.go
  fetch "$ref" proxy/internal/accesslog/logger.go
  fetch "$ref" management/internals/modules/reverseproxy/accesslogs/accesslogentry.go
done

Repository: netbirdio/docs

Length of output: 247

🏁 Script executed:

#!/bin/bash
set -euo pipefail

fetch() {
  local ref="$1" path="$2" tmp
  tmp="$(mktemp)"
  curl -fsSL "https://raw.githubusercontent.com/netbirdio/netbird/${ref}/${path}" >"$tmp"
  printf '\n--- %s %s ---\n' "$ref" "$path"
  awk '{printf "%6d\t%s\n", NR, $0}' "$tmp" |
    rg -n -C 8 \
      'MethodPassword|MethodPIN|MethodOIDC|MethodHeader|attemptedMethod|SetAuthMethod|DenyReason|AuthMechanism|AuthMethodUsed|Authentication failed|Request failed|reason|auth_method_used|metadata|AccessLog'
  rm -f "$tmp"
}

for ref in main 1e7b16db; do
  fetch "$ref" proxy/auth/auth.go
  fetch "$ref" proxy/internal/auth/middleware.go
  fetch "$ref" proxy/internal/accesslog/logger.go
  fetch "$ref" management/internals/modules/reverseproxy/accesslogs/accesslogentry.go
done

Repository: netbirdio/docs

Length of output: 50370

🏁 Script executed:

#!/bin/bash
set -euo pipefail

ref="1e7b16db"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT

for spec in \
  "proxy/internal/auth/middleware.go middleware.go" \
  "proxy/internal/accesslog/logger.go logger.go" \
  "management/internals/modules/reverseproxy/accesslogs/accesslogentry.go accesslogentry.go"
do
  path="${spec% *}"
  file="${spec##* }"
  curl -fsSL "https://raw.githubusercontent.com/netbirdio/netbird/${ref}/${path}" > "$tmp/$file"
done

python3 - "$tmp" <<'PY'
import re
import sys
from pathlib import Path

root = Path(sys.argv[1])
auth = (root / "middleware.go").read_text()
logger = (root / "logger.go").read_text()
entry = (root / "accesslogentry.go").read_text()

checks = {
    "raw auth methods": all(
        f'Method{name} Method = "{value}"' in auth.replace("auth.", "")
        for name, value in (
            ("Password", "password"),
            ("PIN", "pin"),
            ("OIDC", "oidc"),
            ("Header", "header"),
        )
    ),
    "failed credentials retain attempted method": (
        "attemptedMethod = scheme.Type().String()" in auth
        and "cd.SetAuthMethod(attemptedMethod)" in auth
    ),
    "restriction denial stores restriction code": (
        "cd.SetAuthMethod(reason)" in auth
    ),
    "L4 denial maps to auth mechanism": (
        "le.AuthMechanism = entry.DenyReason" in logger
        and "le.AuthSuccess = false" in logger
    ),
    "L4 denial does not set reason": (
        "if a.Protocol == \"\" || a.Protocol == AccessLogProtocolHTTP" in entry
    ),
    "reason classification": all(
        text in entry
        for text in (
            'a.Reason = "Authentication failed"',
            'a.Reason = "Request failed"',
            'if !serviceLog.GetAuthSuccess()',
            'else if serviceLog.GetResponseCode() >= 400',
        )
    ),
    "empty auth method omitted": (
        "if a.AuthMethodUsed != \"\"" in entry
        and "authMethod = &a.AuthMethodUsed" in entry
    ),
}

for name, passed in checks.items():
    print(f"{'PASS' if passed else 'FAIL'}: {name}")

if not all(checks.values()):
    raise SystemExit(1)
PY

Repository: netbirdio/docs

Length of output: 408

🏁 Script executed:

#!/bin/bash
set -euo pipefail

ref="1e7b16db"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT

for spec in \
  "proxy/internal/auth/middleware.go middleware.go" \
  "proxy/internal/accesslog/logger.go logger.go" \
  "management/internals/modules/reverseproxy/accesslogs/accesslogentry.go accesslogentry.go"
do
  path="${spec% *}"
  file="${spec##* }"
  curl -fsSL "https://raw.githubusercontent.com/netbirdio/netbird/${ref}/${path}" > "$tmp/$file"
done

python3 - "$tmp" <<'PY'
import sys
from pathlib import Path

root = Path(sys.argv[1])
auth = (root / "middleware.go").read_text()
logger = (root / "logger.go").read_text()
entry = (root / "accesslogentry.go").read_text()

checks = {
    "raw auth methods": all(
        f'Method{name} Method = "{value}"' in auth.replace("auth.", "")
        for name, value in (
            ("Password", "password"),
            ("PIN", "pin"),
            ("OIDC", "oidc"),
            ("Header", "header"),
        )
    ),
    "failed credentials retain attempted method": (
        "attemptedMethod = scheme.Type().String()" in auth
        and "cd.SetAuthMethod(attemptedMethod)" in auth
    ),
    "restriction denial stores restriction code": (
        "cd.SetAuthMethod(reason)" in auth
    ),
    "L4 denial maps to auth mechanism": (
        "le.AuthMechanism = entry.DenyReason" in logger
        and "le.AuthSuccess = false" in logger
    ),
    "L4 denial does not set reason": (
        "if a.Protocol == \"\" || a.Protocol == AccessLogProtocolHTTP" in entry
    ),
    "reason classification": all(
        text in entry
        for text in (
            'a.Reason = "Authentication failed"',
            'a.Reason = "Request failed"',
            'if !serviceLog.GetAuthSuccess()',
            'else if serviceLog.GetResponseCode() >= 400',
        )
    ),
    "empty auth method omitted": (
        "if a.AuthMethodUsed != \"\"" in entry
        and "authMethod = &a.AuthMethodUsed" in entry
    ),
}

for name, passed in checks.items():
    print(f"{'PASS' if passed else 'FAIL'}: {name}")

if not all(checks.values()):
    raise SystemExit(1)
PY

Repository: netbirdio/docs

Length of output: 408

🏁 Script executed:

#!/bin/bash
set -euo pipefail

ref="1e7b16db"
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT

for spec in \
  "proxy/auth/auth.go auth.go" \
  "proxy/internal/auth/middleware.go middleware.go" \
  "proxy/internal/accesslog/logger.go logger.go" \
  "management/internals/modules/reverseproxy/accesslogs/accesslogentry.go accesslogentry.go"
do
  path="${spec% *}"
  file="${spec##* }"
  curl -fsSL "https://raw.githubusercontent.com/netbirdio/netbird/${ref}/${path}" > "$tmp/$file"
done

python3 - "$tmp" <<'PY'
import sys
from pathlib import Path

root = Path(sys.argv[1])
auth_types = (root / "auth.go").read_text()
middleware = (root / "middleware.go").read_text()
logger = (root / "logger.go").read_text()
entry = (root / "accesslogentry.go").read_text()

checks = {
    "raw auth methods": all(
        f'Method{name} Method = "{value}"' in auth_types
        for name, value in (
            ("Password", "password"),
            ("PIN", "pin"),
            ("OIDC", "oidc"),
            ("Header", "header"),
        )
    ),
    "failed credentials retain attempted method": (
        "attemptedMethod = scheme.Type().String()" in middleware
        and "cd.SetAuthMethod(attemptedMethod)" in middleware
    ),
    "restriction denial stores restriction code": (
        "cd.SetAuthMethod(reason)" in middleware
    ),
    "L4 denial maps to auth mechanism": (
        "le.AuthMechanism = entry.DenyReason" in logger
        and "le.AuthSuccess = false" in logger
    ),
    "L4 denial does not set reason": (
        "if a.Protocol == \"\" || a.Protocol == AccessLogProtocolHTTP" in entry
    ),
    "reason classification": all(
        text in entry
        for text in (
            'a.Reason = "Authentication failed"',
            'a.Reason = "Request failed"',
            'if !serviceLog.GetAuthSuccess()',
            'else if serviceLog.GetResponseCode() >= 400',
        )
    ),
    "empty auth method omitted": (
        "if a.AuthMethodUsed != \"\"" in entry
        and "authMethod = &a.AuthMethodUsed" in entry
    ),
}

for name, passed in checks.items():
    print(f"{'PASS' if passed else 'FAIL'}: {name}")

if not all(checks.values()):
    raise SystemExit(1)
PY

Repository: netbirdio/docs

Length of output: 408

🏁 Script executed:

#!/bin/bash
set -euo pipefail

tmp="$(mktemp)"
trap 'rm -f "$tmp"' EXIT
curl -fsSL 'https://raw.githubusercontent.com/netbirdio/netbird/1e7b16db/proxy/auth/auth.go' > "$tmp"

printf '%s\n' '--- declaration lines ---'
sed -n '12,24p' "$tmp"

python3 - "$tmp" <<'PY'
import sys
from pathlib import Path

text = Path(sys.argv[1]).read_text()
for line in text.splitlines():
    if "Method" in line and ("password" in line or "pin" in line or "oidc" in line or "header" in line):
        print(repr(line))
PY

Repository: netbirdio/docs

Length of output: 542

🏁 Script executed:

#!/bin/bash
set -euo pipefail

tmp="$(mktemp)"
trap 'rm -f "$tmp"' EXIT
curl -fsSL 'https://raw.githubusercontent.com/netbirdio/netbird/1e7b16db/proxy/internal/accesslog/logger.go' > "$tmp"

printf '%s\n' '--- AuthSuccess assignments and log construction ---'
awk '{printf "%6d\t%s\n", NR, $0}' "$tmp" |
  rg -n -C 10 'AuthSuccess\s*:|AuthSuccess|LogHTTP|LogL4|logEntry\{'

printf '%s\n' '--- public/auth middleware references ---'
for path in \
  proxy/internal/auth/middleware.go \
  proxy/internal/proxy/proxy.go \
  proxy/internal/proxy/handler.go
do
  url="https://raw.githubusercontent.com/netbirdio/netbird/1e7b16db/$path"
  if curl -fsSL "$url" > "$tmp"; then
    printf '\n--- %s ---\n' "$path"
    awk '{printf "%6d\t%s\n", NR, $0}' "$tmp" |
      rg -n -C 8 'AuthSuccess|LogHTTP|AuthMechanism|public|authenticate' || true
  fi
done

Repository: netbirdio/docs

Length of output: 12851

Align access-log documentation with the raw /api/events/proxy fields.

  • Use oidc, password, pin, and header. Empty auth_method_used values are omitted.
  • For credential failures, auth_method_used stores the attempted method when available. It does not store detailed causes.
  • reason is Authentication failed for authentication failures and Request failed for other HTTP 4xx/5xx responses.
  • Denied L4 entries store the restriction code in auth_method_used and omit reason.
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/pages/manage/reverse-proxy/access-logs.mdx` around lines 40 - 42, Update
the access-log field descriptions in the table around “Auth Method,” “User,” and
“Reason” to match the raw /api/events/proxy fields: list auth methods as oidc,
password, pin, and header; state that empty auth_method_used values are omitted
and credential failures retain only the attempted method when available; clarify
the exact reason values; and document that denied L4 entries store the
restriction code in auth_method_used while omitting reason.

Source: MCP tools


## Understanding log entries

Expand All @@ -48,18 +48,18 @@ Every log entry (HTTP and L4) shares a common set of fields. Some fields are onl
HTTP log entries fall into three categories based on the status code:

- **Allowed requests**: successful requests show a `2xx` status code along with the authentication method used to access the service.
- **Denied requests**: failed authentication or access restriction blocks show `401` or `403` status codes with a reason explaining why the request was denied (e.g., invalid password, missing SSO session, IP restricted, country restricted).
- **Denied requests**: failed authentication or access restriction blocks show `401` or `403` status codes with `reason` set to `Authentication failed`. The specific cause (invalid password, missing SSO session, IP restricted, country restricted, CrowdSec verdict) is carried in `auth_method_used`, not in `reason`.
- **Errors**: backend errors or proxy issues show `5xx` status codes. These typically indicate that the target service is unreachable or returned an error.

### L4 log entries

L4 entries are logged when the connection closes and record the total bytes transferred in each direction and the connection duration. L4 entries do not have HTTP status codes.

Denied L4 connections (blocked by access restrictions) are logged immediately with a deny reason. Since L4 services do not support authentication, denials come from access restrictions only.
Denied L4 connections (blocked by access restrictions) are logged immediately. L4 entries carry no `reason` value, so the restriction code identifies the denial. Since L4 services do not support authentication, denials come from access restrictions only.

### Deny reasons

The following deny reasons can appear for both HTTP and L4 services:
The following deny reasons identify why a connection was rejected. Note that for HTTP services these values are not carried in the entry's `reason` field: see the note below the table.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

| Reason | Description |
|--------|-------------|
Expand All @@ -73,7 +73,28 @@ The following deny reasons can appear for both HTTP and L4 services:

All CrowdSec decision types (ban, captcha, throttle) result in a connection denial in enforce mode. The proxy does not serve captcha challenges or apply rate limiting: the decision type is recorded for informational purposes only.

When CrowdSec is in **observe** mode, the verdict appears in the log metadata but the deny reason field is empty (the connection is allowed). In the dashboard, these entries render with an observe-mode badge on the reason cell and show the underlying decision type (ban, captcha, throttle, unavailable) on hover. This lets you audit what CrowdSec would block without affecting traffic. For a self-test workflow, see [Testing the integration](/selfhosted/maintenance/crowdsec#testing-the-integration).
<Note>
For HTTP services, the deny code from the table above is recorded in the `auth_method_used` field, and the entry's `reason` field carries a synthesized generic value rather than the specific code. This applies to every access restriction, not only CrowdSec:

```json
{ "status_code": 403, "reason": "Authentication failed", "auth_method_used": "ip_restricted" }
```

```json
{
"status_code": 403,
"reason": "Authentication failed",
"auth_method_used": "crowdsec_ban",
"metadata": { "crowdsec_verdict": "crowdsec_ban" }
}
```
Comment thread
coderabbitai[bot] marked this conversation as resolved.

When reading entries through `GET /api/events/proxy`, match on `auth_method_used` (and `metadata.crowdsec_verdict` for CrowdSec specifically) rather than `reason`.

Observe-mode entries carry the normal status code and record both `crowdsec_mode` and `crowdsec_verdict` in `metadata`. Because the connection is allowed, CrowdSec itself contributes no `reason`, but the field can still be populated by a later stage of the request such as an authentication or backend failure. Treat `metadata.crowdsec_mode` as the signal that an entry is an observe-mode verdict, not the absence of `reason`.
</Note>

When CrowdSec is in **observe** mode, the verdict appears in the log metadata and CrowdSec adds no deny reason of its own (the connection is allowed). In the dashboard, these entries render with an observe-mode badge on the reason cell and show the underlying decision type (ban, captcha, throttle, unavailable) on hover. This lets you audit what CrowdSec would block without affecting traffic. For a self-test workflow, see [Testing the integration](/selfhosted/maintenance/crowdsec#testing-the-integration).

## Use cases

Expand Down
107 changes: 101 additions & 6 deletions src/pages/selfhosted/maintenance/crowdsec-dashboard.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,15 +96,16 @@ services:
- '--experimental.plugins.bouncer.version=v1.6.0'
```

If your generated Compose project network is named `netbird_netbird`, make sure Traefik uses that Docker network:
The generated Compose file already sets `--providers.docker.network` in the Traefik command block, normally to `netbird`, matching the network key in the Compose file. Leave it as generated and do not add a second copy of the flag.

```yaml
services:
traefik:
command:
- '--providers.docker.network=netbird_netbird'
Only change it if Traefik logs warnings about a missing Docker network. Compose prefixes the network name with the project name, which defaults to the install directory name, so the actual name varies by deployment. Check it before editing:

```bash
docker network ls | grep netbird
```

An install in `/root` produces `root_netbird`; one in `/opt/netbird` produces `netbird_netbird`.

Under `services.traefik.depends_on`, make Traefik wait until CrowdSec is healthy:

```yaml
Expand Down Expand Up @@ -299,10 +300,104 @@ curl -ks -A 'Mozilla/5.0 NetBirdDashboardCheck' -o /dev/null -w '%{http_code}\n'

The response should be `200`.

## Verifying the middleware stays attached

This protection fails open. If the middleware is detached for any reason, most commonly a label edit applied with `docker compose restart` instead of `docker compose up -d`, which does not recreate the container, requests stop being inspected and nothing reports an error. The dashboard loads, authentication works, and all containers report healthy. The only symptom is that a known malicious probe returns `404` instead of `403`.

After any change to the labels, recreate the affected containers and re-run the probe:

```bash
docker compose up -d dashboard netbird-server
curl -ks -o /dev/null -w '%{http_code}\n' "https://$NETBIRD_DOMAIN/.env"
```

A `403` confirms the middleware is in the request path. A `404` means it is not.

To surface this automatically, monitor the same probe from outside the deployment and treat `403` as the healthy response. Point an uptime monitor or a cron job at:

```text
https://<your-domain>/.env
```

Alert when the response is anything other than `403`. This is an inverted check, alerting on the absence of a block rather than on an outage, but it is the only signal that confirms the protection layer is still in the request path.
Comment on lines +316 to +322

@coderabbitai coderabbitai Bot Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu

base="https://<your-domain>"
for path in /.env /api/.env; do
  code="$(curl -ks -o /dev/null -w '%{http_code}' "${base}${path}")"
  test "$code" = 403 || exit 1
done

Repository: netbirdio/docs

Length of output: 152

🏁 Script executed:

#!/bin/bash
set -eu

file="src/pages/selfhosted/maintenance/crowdsec-dashboard.mdx"
printf '%s\n' '--- target section ---'
sed -n '300,345p' "$file"

printf '%s\n' '--- route and middleware references ---'
rg -n -C 3 'middleware|/api|/oauth2|\\.env|router|Traefik' "$file"

Repository: netbirdio/docs

Length of output: 16088

🏁 Script executed:

#!/bin/bash
set -eu

file="src/pages/selfhosted/maintenance/crowdsec-dashboard.mdx"

printf '%s\n' '--- complete route configuration ---'
sed -n '115,165p' "$file"

printf '%s\n' '--- probes and router labels ---'
rg -n -C 2 'https?://.*(\\.env|api)|routers\\.|middlewares=' "$file"

printf '%s\n' '--- repository references to the same router names ---'
rg -n -C 2 'netbird-dashboard-api|netbird-dashboard-crowdsec|PathPrefix\\(`/api`\\)|PathPrefix\\(`/oauth2`\\)' .

Repository: netbirdio/docs

Length of output: 7790

Probe both protected routers. The /.env check covers only the dashboard router. The guide also protects /api and /oauth2 through a separate router.

Add /api/.env to the monitor, or state that the check covers only the dashboard router. Keep the unauthenticated Traefik API disabled.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/pages/selfhosted/maintenance/crowdsec-dashboard.mdx` around lines 316 -
322, Update the external monitoring guidance around the
`https://<your-domain>/.env` probe to also cover the protected `/api` router by
adding `https://<your-domain>/api/.env`, or explicitly state that the existing
check covers only the dashboard router. Preserve the instruction to keep the
unauthenticated Traefik API disabled.

Source: MCP tools


<Warning>
Do not enable the Traefik API (`--api.insecure=true`) for this purpose. The
API serves the full dynamic configuration without authentication, including
middleware plugin settings such as `crowdsecLapiKey`. Even with port `8080`
unpublished, every container on the Docker network could then read the
CrowdSec bouncer key.
</Warning>

<Warning>
This is detection only. Nothing here restores enforcement automatically: the
probe tells you protection has stopped, and reattaching the middleware
remains a manual step.
</Warning>

## Recovering from a dashboard lockout

If AppSec or an IP decision blocks you from the dashboard, the block also covers `/api` and `/oauth2`, so it cannot be lifted from the UI. All recovery is performed over SSH on the host.

First identify which of the two mechanisms is blocking you, because they are handled differently:

```bash
docker compose exec crowdsec cscli decisions list
```

If your address is listed, an **IP decision** is blocking you. If it is not, the block is an **AppSec** match on the request itself: AppSec inspects requests and does not create decisions, so `cscli decisions delete` has no effect in that case. Confirm with `cscli metrics show appsec` and `cscli alerts list`, where AppSec matches appear with `kind` set to `waf`.

Allowlisting the address resolves both cases. `cscli allowlists add` fails with `allowlist '<name>' not found` if the list does not already exist, so create it first if you have not set one up:

```bash
docker compose exec crowdsec cscli allowlists create netbird-admins -d "Addresses that should never be blocked"
docker compose exec crowdsec cscli allowlists add netbird-admins <your-ip> -d "admin"
```

Adding an address that currently carries a decision expires that decision immediately. Allowlisted addresses are also exempted from AppSec blocking, though that takes effect on the Traefik bouncer's next stream refresh rather than instantly, so allow up to a minute. See [Allowlisting addresses](/selfhosted/maintenance/crowdsec#allowlisting-addresses) for details.

If an IP decision is present and you want it gone without allowlisting the address permanently, delete it directly:

```bash
docker compose exec crowdsec cscli decisions delete --ip <your-ip>
```

The Traefik bouncer runs in `stream` mode and refreshes periodically, so allow up to a minute for access to return.

If the dashboard is still unreachable and you need immediate access, detach the middleware. It is attached to **two** routers and both must be commented out:

```yaml
services:
dashboard:
labels:
# - traefik.http.routers.netbird-dashboard.middlewares=netbird-dashboard-crowdsec@docker

netbird-server:
labels:
# - traefik.http.routers.netbird-dashboard-api.middlewares=netbird-dashboard-crowdsec@docker
```

```bash
docker compose up -d dashboard netbird-server
```

<Warning>
Commenting out only the `dashboard` label leaves the dashboard reachable while
`/api` and `/oauth2` continue to return `403`. The interface loads but cannot
authenticate or fetch data, which is easily mistaken for a different fault.
Detach both labels, or neither.
</Warning>

Restore both labels once the underlying issue is resolved, recreate the containers, and confirm with the `/.env` probe above.

## Troubleshooting

If all dashboard requests return `403` immediately after startup, Traefik may have started before CrowdSec LAPI and AppSec were ready. Confirm that the CrowdSec health check is present and that Traefik uses `depends_on.condition: service_healthy`.

If CrowdSec is stopped or unreachable while `crowdsecAppsecUnreachableBlock=true` is set, all dashboard requests are denied by design.

Separately, a reverse proxy service in `enforce` mode denies all connections while its bouncer has not completed its initial sync, for example when the proxy starts with the LAPI unavailable. See [Enforcement modes](/selfhosted/maintenance/crowdsec#enforcement-modes).

If Traefik logs warnings about a missing Docker network, check the actual network name:

```bash
Expand Down
Loading
Loading