feat(analytics): add permission-gated login-activity read api#511
Conversation
|
@claude-review |
|
Claude finished @zaira-bibi's task in 2m 18s —— View job Claude Code Review
Reviewed against base ✅ Strengths
🟡 Medium —
|
|
|
||
| # PostgreSQL: read the pre-rolled continuous aggregate. | ||
| _PG_SQL = text( | ||
| "SELECT to_char(day, 'YYYY-MM-DD') AS day, login_count FROM login_activity_daily ORDER BY day DESC LIMIT :days" | ||
| ) | ||
|
|
||
| # SQLite (tests/e2e): aggregate raw_events directly — same semantics as the cagg. | ||
| _SQLITE_SQL = text( | ||
| "SELECT date(occurred_at) AS day, count(*) AS login_count " | ||
| "FROM raw_events " | ||
| "WHERE event_type = 'user.logged_in' " | ||
| "GROUP BY date(occurred_at) " | ||
| "ORDER BY day DESC " | ||
| "LIMIT :days" | ||
| ) |
There was a problem hiding this comment.
[Medium] LIMIT :days caps the number of returned rows, not a calendar window.
Both variants order by day DESC and LIMIT :days, so days=30 returns up to 30 days that had at least one login — not "the last 30 calendar days." If there are gap days (days with zero logins), the result can reach back well beyond 30 calendar days, and there is no lower date bound. The endpoint docstring in analytics.py ("for the last days days") reinforces the calendar-window reading, so a dashboard labelled "last 30 days" may silently show older data.
Two consequences worth deciding on explicitly:
- No date floor — consider adding
WHERE occurred_at >= now() - (:days || ' days')::interval(PG) /date(occurred_at) >= date('now', '-' || :days || ' days')(SQLite) if a true window is intended, or rename/clarify to "most recent N data points" if row-capping is intended. - No zero-fill — days with no logins are simply absent from the series; the frontend must be prepared to handle gaps.
Either behaviour is defensible, but the naming (days) and docstring should match the actual semantics.
| from sqlalchemy import text | ||
| from sqlmodel.ext.asyncio.session import AsyncSession | ||
|
|
||
|
|
There was a problem hiding this comment.
[Low] Make the UTC-bucketing assumption explicit. The two variants "must stay semantically identical," but they bucket differently: PG's time_bucket('1 day', occurred_at) on a timestamptz column buckets in UTC, while SQLite's date(occurred_at) buckets on whatever offset the stored string carries. They agree only as long as events are stored/interpreted as UTC (which the current ingest path does). A one-line note here that day boundaries are UTC on both paths would prevent a future timezone-parameterization on the PG side from silently diverging from SQLite.
|
|
||
|
|
||
| # PostgreSQL: read the pre-rolled continuous aggregate. | ||
| _PG_SQL = text( |
There was a problem hiding this comment.
Lets avoid adding raw SQL in code
There was a problem hiding this comment.
went with raw SQL here on purpose as the postgres query reads the login_activity_daily continuous aggregate, which isn't (and shouldn't be) registered on analytics_metadata, reason being that the autogenerate will try to create/drop it and it doesn't exist on sqlite either. also worth noting that the two can't share a query (because of different source relation and day bucketing), so it's two selects either way.
the sqlite query could be written in sqlAlchemy core, but then the readers couldn't diff them by eye.
| # backfill (refresh_continuous_aggregate) is run out-of-band if history matters. | ||
| op.execute( | ||
| """ | ||
| CREATE MATERIALIZED VIEW login_activity_daily |
There was a problem hiding this comment.
Could we check if we can do this via the ORM
There was a problem hiding this comment.
unfortunately sqlAlchemy does not have DDL construct for materialized views at all.
| conn = op.get_bind() | ||
| conn.execute( | ||
| sa.text( | ||
| "DELETE FROM role_permission WHERE permission = :permission " |
dc0aea7 to
8b10f75
Compare
2cbdee4 to
1713ec5
Compare
1ca310b to
eb7d0ad
Compare
| """ | ||
| SELECT add_continuous_aggregate_policy( | ||
| 'login_activity_daily', | ||
| start_offset => INTERVAL '3 days', |
There was a problem hiding this comment.
This start_offset combined with WITH NO DATA means anything older than 3 days at migration time never gets materialized: once the first policy run advances the watermark, real-time aggregation stops covering those old buckets and they disappear from the view entirely. Since user.logged_in has been emitting for a while, a deployed instance silently loses its pre-migration history here (the SQLite path keeps full history, so the two dialects drift). The comment above mentions an out-of-band backfill, but nothing in the repo runs one. Creating the view WITH DATA, or a one-off refresh_continuous_aggregate(login_activity_daily, NULL, NULL), would fix it.
There was a problem hiding this comment.
we'd need a refresh_continuous_aggregate(xyz_activity, NULL, NULL) for every cagg that's created, which is the wrong shape. i can either do (a) start_offset => NULL in the policy itself to refresh from the beginning of time or (b) one generic backfill command to discover all caggs and full-refresh each one. i lean towards (b) as (a) will cause each refresh to re-scan full history, which will grow expensive as data accumulates. what do you say?
|
|
||
| # PostgreSQL: read the pre-rolled continuous aggregate. | ||
| _PG_SQL = text( | ||
| "SELECT to_char(day, 'YYYY-MM-DD') AS day, login_count FROM login_activity_daily ORDER BY day DESC LIMIT :days" |
There was a problem hiding this comment.
time_bucket buckets on UTC midnight, but to_char renders a timestamptz in the session TimeZone, and we don't pin one on the analytics engine. On any server west of UTC every label shifts one day back, so today's logins show up under yesterday. The SQLite path is effectively UTC, so tests will never catch this. to_char(day AT TIME ZONE 'UTC', 'YYYY-MM-DD') makes it deterministic.
| "WHERE event_type = 'user.logged_in' " | ||
| "GROUP BY date(occurred_at) " | ||
| "ORDER BY day DESC " | ||
| "LIMIT :days" |
There was a problem hiding this comment.
LIMIT :days returns the N most recent days that had logins, not "the last N days" as the endpoint doc says, so a quiet stretch pushes the window arbitrarily far back. It also leaves this SQLite branch scanning all of raw_events on every call. A WHERE occurred_at >= ... bound on both queries would match the documented contract and cap the scan.
6730a05 to
8f657f5
Compare
eb7d0ad to
cc50a5b
Compare
d2da0d1 to
1941720
Compare
d2dbced to
9db4564
Compare
Prove the analytics read side end-to-end for the user.logged_in event: a Postgres continuous-aggregate rollup, a dialect-aware read function, and a permission-gated read endpoint. The read path only ever touches analytics tables/rollups, never the app DB. - add login_activity_daily TimescaleDB continuous aggregate (SQLite no-op) - declare and export the analytics.read permission - add dialect-aware get_login_activity (reads the cagg on Postgres, aggregates raw_events on SQLite), exposed via app.lib.analytics - add GET /api/v1/analytics/login-activity gated by analytics.read - grant analytics.read to the seeded admin role - document the permission (README) and read API (CLAUDE.md) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
9db4564 to
e3a84c8
Compare
What
Add a permission-gated read API for daily login activity — the first read-side slice of the analytics subsystem, built on the
user.logged_inevent.Changes
login_activity_dailyTimescaleDB continuous aggregate (Postgres-only; no-op on SQLite)analytics.readpermissionget_login_activityread function, exposed viaapp.lib.analyticsGET /api/v1/analytics/login-activity, gated byanalytics.readanalytics.readto the seededadminroleanalytics.read(README) and the read API (CLAUDE.md)How to Test
make test.backend.pytest— all green (1316 passed), includingtests/analytics/test_login_activity_read.pyand the migration no-op guard intests/analytics/test_analytics_migrations.py.make mypy— clean;make lint.backendandmake lint.format.backend check=1— clean.uv run alembic headsanduv run alembic -c alembic_analytics.ini headseach show one head.make services.up,make migrations):make db-shell-analytics→\d+ login_activity_dailyandSELECT * FROM timescaledb_information.continuous_aggregates;show the aggregate + refresh policy.make db-shell→ confirm theadminrole has aanalytics.readgrant.GET /api/v1/analytics/login-activity?days=30; a non-admin gets403.Notes
c5b6e911f9d6; analytics continuous aggregate287f281e6558. Runmake migrations.raw_eventsdirectly. The Postgres cagg and the admin grant can only be verified on real Postgres (step 4)./dashboard/analyticspage + typed client) is intentionally out of scope here and will follow in a separate PR.This PR description was written with the assistance of an LLM (Claude).