Skip to content

feat: Centralize logging setup with structlog integration#179

Open
khvn26 wants to merge 10 commits intomainfrom
feat/logging-improvements
Open

feat: Centralize logging setup with structlog integration#179
khvn26 wants to merge 10 commits intomainfrom
feat/logging-improvements

Conversation

@khvn26
Copy link
Member

@khvn26 khvn26 commented Mar 13, 2026

Closes #31

What changed

Logging setup moves from Django settings into flagsmith-common, running before Django loads. Both stdlib and structlog now route through a single ProcessorFormatter, so all output (app code, structlog, Gunicorn error logs) shares the same format and ISO timestamps.

Key additions

  • setup_logging() in common.core.logging — configures stdlib via dictConfig, then wires structlog through stdlib using LoggerFactory (not PrintLoggerFactory). Accepts application_loggers for per-package level control and extra_foreign_processors for extending the foreign_pre_chain.
  • _SentryFriendlyProcessorFormatter — subclass of ProcessorFormatter that snapshots and restores record.msg/record.args across formatting, so Sentry's LoggingIntegration (which reads these in a finally block after handlers run) sees the original message template for proper event grouping.
  • sentry_processor in common.core.sentry — structlog processor that sets Sentry context from event dicts. Hardwired into the structlog processor chain. Since structlog routes through stdlib, Sentry's LoggingIntegration automatically captures ERROR+ logs.
  • application_loggers allowlist — root logger stays at WARNING to suppress third-party noise; listed app loggers get the requested level. DEBUG overrides everything. Configurable via APPLICATION_LOGGERS env var (comma-separated).
  • make_gunicorn_access_processor() in common.gunicorn.loggingforeign_pre_chain processor that extracts structured fields (path, method, status, duration, extra items) from Gunicorn access log records. Configurable via ACCESS_LOG_EXTRA_ITEMS env var.
  • Gunicorn error logs propagate to root — no separate handler/formatter needed.
  • Gunicorn access logs — JSON mode propagates to root for structured output via the access processor; generic mode keeps Gunicorn's own %(message)s handler for pure CLF output.
  • TTY-aware colorsConsoleRenderer only enables ANSI colors when stdout is a TTY.
  • sentry-sdk added as an explicit dependency in the common-core group.
  • environs used in ensure_cli_env for env var parsing.

What it removes

  • GunicornAccessLogJsonFormatter and GunicornAccessLogJsonRecord — replaced by the foreign_pre_chain processor.
  • Separate formatter management for Gunicorn error logs.
  • LOGGING_DEFAULT_GENERIC_FORMAT constant — ConsoleRenderer handles generic formatting.
  • PROMETHEUS_HISTOGRAM_BUCKETS and PROMETHEUS_HTTP_SERVER_RESPONSE_SIZE_HISTOGRAM_BUCKETS Django settings — histogram buckets now use library defaults directly, removing import-time Django settings access that prevented pre-Django logging setup.

What it keeps

  • JsonFormatter and JsonRecordJsonRecord is still the schema definition used by map_event_to_json_record.
  • Gunicorn access log CLF format in generic mode.
  • Gunicorn constants (WSGI_EXTRA_SUFFIX_TO_CATEGORY, wsgi_extra_key_regex) in their original location.

Migration guide for flagsmith/flagsmith

1. Replace the LOGGING dict in settings/common.py

# Remove the entire LOGGING = { ... } block (lines ~615-668) and replace with:
LOGGING = {"version": 1, "disable_existing_loggers": False}

Django's dictConfig(LOGGING) call during django.setup() becomes a no-op.

2. Set env vars

APPLICATION_LOGGERS=app,features,integrations,task_processor,app_analytics,webhooks,gunicorn
ACCESS_LOG_EXTRA_ITEMS={flagsmith.route}e,{origin}i,{access-control-allow-origin}o

3. Delete api/util/logging.py

It duplicates JsonFormatter which now lives in common.core.logging.

@khvn26 khvn26 requested a review from a team as a code owner March 13, 2026 19:28
@khvn26 khvn26 requested review from Zaimwa9 and removed request for a team March 13, 2026 19:28
@codecov-commenter
Copy link

codecov-commenter commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 99.72900% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 96.44%. Comparing base (398c83c) to head (b355ee1).

Files with missing lines Patch % Lines
src/common/gunicorn/processors.py 96.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #179      +/-   ##
==========================================
+ Coverage   96.12%   96.44%   +0.32%     
==========================================
  Files          92       97       +5     
  Lines        3222     3540     +318     
==========================================
+ Hits         3097     3414     +317     
- Misses        125      126       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Logging improvements

2 participants