Skip to content

Tag ADIT telemetry per service in observability overlay#340

Open
samuelvkwong wants to merge 2 commits intomainfrom
feat/per-service-otel-tagging
Open

Tag ADIT telemetry per service in observability overlay#340
samuelvkwong wants to merge 2 commits intomainfrom
feat/per-service-otel-tagging

Conversation

@samuelvkwong
Copy link
Copy Markdown
Collaborator

@samuelvkwong samuelvkwong commented Apr 27, 2026

Summary

  • Restructures `docker-compose.override.yml.example` so `service.namespace` and `deployment.environment` are defined once at the top in a shared YAML anchor.
  • Each of the 6 services merges the shared anchor and adds only its own `SERVICE_COMPONENT` env var (`init`, `web`, `default_worker`, `dicom_worker`, `mass_transfer_worker`, `receiver`).
  • Header documents the per-stack `sed` workflow for switching `production` → `staging`/`development` after copying the example.

Why

OpenObserve telemetry was tagged with a single `service.name` per stack, collapsing all containers under one identity. With this overlay every signal carries:

Attribute Value
`service.name` `${STACK_NAME}` (e.g. `adit_prod`, `adit_staging`)
`service.component` the compose service that emitted it
`service.namespace` `openradx`
`deployment.environment` `production` (default) / `staging` / `development`

This makes telemetry filterable per stack, per component, and per deployment environment.

Dependency

`service.component` is set via a custom `SERVICE_COMPONENT` env var that the `adit-radis-shared` telemetry helper reads — see openradx/adit-radis-shared#186. `pyproject.toml` will need updating to reference that branch (or a follow-up tag) before this PR's effect is visible end-to-end.

Sibling PR

Test plan

  • `docker compose config --quiet` parses the new overlay successfully
  • All 6 services produce unique `SERVICE_COMPONENT` values matching their compose keys
  • `OTEL_RESOURCE_ATTRIBUTES` is identical across services and `OTEL_SERVICE_NAME` resolves to `${STACK_NAME}`
  • After deploying with the new shared-lib version, verify in OpenObserve that records carry the four attributes above

🤖 Generated with Claude Code

Restructure the overlay so service.namespace and deployment.environment live
once in a shared env anchor; each service merges that anchor and adds only
its own SERVICE_COMPONENT value (consumed by adit_radis_shared/telemetry.py).
service.name comes from OTEL_SERVICE_NAME=${STACK_NAME}, dynamically resolved
by the existing CLI helper.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 27, 2026

Warning

Rate limit exceeded

@samuelvkwong has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 56 minutes and 26 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: efcc58ae-7196-4dcd-b3b0-fb580536c504

📥 Commits

Reviewing files that changed from the base of the PR and between f1003ff and 0dce7a4.

📒 Files selected for processing (1)
  • docker-compose.override.yml.example
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/per-service-otel-tagging

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the observability configuration in docker-compose.override.yml.example by splitting network and environment settings into separate anchors and adding OpenTelemetry resource attributes for each service. The review feedback suggests enhancing the flexibility and portability of this configuration by leveraging existing environment variables from the .env file and using default value syntax for OTEL settings, rather than relying on manual file modifications or hardcoded values.

Comment on lines +13 to +17
# 2. If this is a staging or development deployment, edit the
# `deployment.environment=production` value inside `OTEL_RESOURCE_ATTRIBUTES`
# below to `staging` or `development`:
# sed -i 's/deployment.environment=production/deployment.environment=staging/' \
# docker-compose.override.yml
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The sed -i command is not portable across different operating systems (e.g., macOS/BSD vs. GNU/Linux). Instead of requiring manual file modification or fragile scripts, you can leverage the ENVIRONMENT variable already defined in the .env file to set the deployment environment dynamically. This simplifies the setup process and makes it more robust.

#   2. The deployment environment is automatically set from the `ENVIRONMENT` 
#      variable in your `.env` file (defaults to `production`). 
#      To override it, change `ENVIRONMENT` in your `.env` file.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gemini The ENVIRONMENT variable in .env describes which django settings ADIT should use, either development or production, not the deployment environment.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the clarification, @samuelvkwong. I understand now that deployment.environment in OTEL_RESOURCE_ATTRIBUTES is distinct from the ENVIRONMENT variable used for Django settings. Given that this is a manual configuration step for the deployment environment, I recommend updating the documentation to explicitly mention that this is a manual override required for non-production environments, rather than suggesting it be handled via the existing .env file.

Comment on lines +30 to +32
OTEL_PYTHON_DJANGO_EXCLUDED_URLS: "health/,static/.*"
OTEL_SERVICE_NAME: ${STACK_NAME}
OTEL_RESOURCE_ATTRIBUTES: "service.namespace=openradx,deployment.environment=production"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Hardcoding these values prevents users from overriding them via the .env file. Using the ${VAR:-DEFAULT} pattern provides better flexibility. Additionally, using ${ENVIRONMENT} allows the deployment environment to be synchronized with the rest of the application settings automatically, avoiding the need for manual sed commands.

  OTEL_PYTHON_DJANGO_EXCLUDED_URLS: ${OTEL_PYTHON_DJANGO_EXCLUDED_URLS:-health/,static/.*}
  OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME:-${STACK_NAME}}
  OTEL_RESOURCE_ATTRIBUTES: "service.namespace=openradx,deployment.environment=${ENVIRONMENT:-production}"

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gemini See #340 (comment)
Also each deployment environment will have its own project directory and respective .env file. The deployment environment will be set once per .env file and will not change.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. Given that the deployment environment is managed per-stack and is static for the lifetime of that specific .env file, the current approach of manual configuration is acceptable. I will keep this in mind for future reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant