Skip to content

feat(analytics): add permission-gated login-activity read api#511

Merged
zaira-bibi merged 1 commit into
mainfrom
zaira/feat/analytics-milestone-1
Jul 14, 2026
Merged

feat(analytics): add permission-gated login-activity read api#511
zaira-bibi merged 1 commit into
mainfrom
zaira/feat/analytics-milestone-1

Conversation

@zaira-bibi

Copy link
Copy Markdown
Contributor

What

Add a permission-gated read API for daily login activity — the first read-side slice of the analytics subsystem, built on the user.logged_in event.

Changes

  • feat(migrations): add login_activity_daily TimescaleDB continuous aggregate (Postgres-only; no-op on SQLite)
  • feat(core): declare and export the analytics.read permission
  • feat(analytics): add dialect-aware get_login_activity read function, exposed via app.lib.analytics
  • feat(api): add GET /api/v1/analytics/login-activity, gated by analytics.read
  • feat(migrations): grant analytics.read to the seeded admin role
  • docs: document analytics.read (README) and the read API (CLAUDE.md)

How to Test

  1. make test.backend.pytest — all green (1316 passed), including tests/analytics/test_login_activity_read.py and the migration no-op guard in tests/analytics/test_analytics_migrations.py.
  2. make mypy — clean; make lint.backend and make lint.format.backend check=1 — clean.
  3. Single migration heads: uv run alembic heads and uv run alembic -c alembic_analytics.ini heads each show one head.
  4. On real Postgres/TimescaleDB (make services.up, make migrations):
  • make db-shell-analytics\d+ login_activity_daily and SELECT * FROM timescaledb_information.continuous_aggregates; show the aggregate + refresh policy.
    • make db-shell → confirm the admin role has a analytics.read grant.
    • Log in as an admin and call GET /api/v1/analytics/login-activity?days=30; a non-admin gets 403.

Notes

  • Migrations required (two lineages): app grant c5b6e911f9d6; analytics continuous aggregate 287f281e6558. Run make migrations.
  • The continuous aggregate is TimescaleDB-only — a no-op on SQLite (tests/e2e), where the read path aggregates raw_events directly. The Postgres cagg and the admin grant can only be verified on real Postgres (step 4).
  • No breaking changes, no new env vars, no dependency bumps.
  • Frontend dashboard (the /dashboard/analytics page + typed client) is intentionally out of scope here and will follow in a separate PR.

This PR description was written with the assistance of an LLM (Claude).

@zaira-bibi zaira-bibi self-assigned this Jul 8, 2026
@zaira-bibi zaira-bibi added the enhancement New feature or request label Jul 8, 2026
@zaira-bibi

Copy link
Copy Markdown
Contributor Author

@claude-review

@claude

claude Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Claude finished @zaira-bibi's task in 2m 18s —— View job


Claude Code Review

  • Read changed files & gather context
  • Security & permissions review
  • Correctness & logic review
  • Migrations review
  • Tests & docs review
  • Post feedback

Reviewed against base origin/zaira/feat/analytics-plugin-event-schemas. This is a clean, well-scoped, well-documented PR — the read/write split, dialect-aware read path, permission gating, and the two-lineage migration story are all handled thoughtfully, and the test coverage is solid (buckets, days limit, permitted/forbidden/unauthenticated). Two non-blocking correctness/clarity points below; no security issues found.

✅ Strengths

  • Layering is correct — read queries in app/core/analytics/reads.py, re-exported through app/lib/analytics, consumed by the API. No reaching into internals.
  • Permission gating is textbook: analytics.read declared via Permission.create(), exported through app/lib/permissions, gated with require_in_global_scope(), granted to admin via an idempotent INSERT … SELECT migration. The now() literal-SQL pattern matches the parent migration 9477cf5a43a0, and keeping permission strings as literals in the migration (not imports) is the right call.
  • Dialect-aware read path with a matching no-op guard on the cagg migration, plus a test (test_login_activity_cagg_migration_is_noop_on_sqlite) that proves the analytics lineage still upgrades to head on SQLite.
  • Parameterized SQL (:days) — no injection surface. Query(ge=1, le=365) bounds the input.
  • Docs (README + CLAUDE.md) and the regenerated frontend/lib/api/generated.ts were updated alongside the code.

🟡 Medium — days limits rows, not a calendar window

get_login_activity orders by day DESC and LIMIT :days, so days=30 returns up to 30 days that had ≥1 login — not "the last 30 calendar days." With gap days the series can reach back further than 30 calendar days, and there's no lower date bound. The endpoint docstring ("for the last days days") reinforces the calendar-window reading, so a dashboard labelled "last 30 days" could silently show older data. Also note days with zero logins are absent (no zero-fill) — the frontend must handle gaps. Either behaviour is fine, but pick one and make the naming + docstring match. Details inline on reads.py.

🟢 Low — make the UTC-bucketing assumption explicit

The comment says the PG and SQLite variants "must stay semantically identical," but time_bucket('1 day', occurred_at) (timestamptz → UTC buckets) vs date(occurred_at) (buckets on the stored offset) agree only while events are UTC. A one-line note that both bucket on UTC would guard against future divergence. Inline on reads.py.

Nits

  • reads.py: session.bind.dialect.name assumes bind is non-None; fine at runtime for a request-scoped session and mypy is reported clean, so just flagging for awareness.

Overall

👍 Approvable once the days semantics (Medium) are confirmed as intended or adjusted — that's the only item I'd want resolved before merge; the rest are clarity nits. Nice work.
· zaira/feat/analytics-milestone-1

Comment on lines +25 to +39

# PostgreSQL: read the pre-rolled continuous aggregate.
_PG_SQL = text(
"SELECT to_char(day, 'YYYY-MM-DD') AS day, login_count FROM login_activity_daily ORDER BY day DESC LIMIT :days"
)

# SQLite (tests/e2e): aggregate raw_events directly — same semantics as the cagg.
_SQLITE_SQL = text(
"SELECT date(occurred_at) AS day, count(*) AS login_count "
"FROM raw_events "
"WHERE event_type = 'user.logged_in' "
"GROUP BY date(occurred_at) "
"ORDER BY day DESC "
"LIMIT :days"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Medium] LIMIT :days caps the number of returned rows, not a calendar window.

Both variants order by day DESC and LIMIT :days, so days=30 returns up to 30 days that had at least one login — not "the last 30 calendar days." If there are gap days (days with zero logins), the result can reach back well beyond 30 calendar days, and there is no lower date bound. The endpoint docstring in analytics.py ("for the last days days") reinforces the calendar-window reading, so a dashboard labelled "last 30 days" may silently show older data.

Two consequences worth deciding on explicitly:

  1. No date floor — consider adding WHERE occurred_at >= now() - (:days || ' days')::interval (PG) / date(occurred_at) >= date('now', '-' || :days || ' days') (SQLite) if a true window is intended, or rename/clarify to "most recent N data points" if row-capping is intended.
  2. No zero-fill — days with no logins are simply absent from the series; the frontend must be prepared to handle gaps.

Either behaviour is defensible, but the naming (days) and docstring should match the actual semantics.

from sqlalchemy import text
from sqlmodel.ext.asyncio.session import AsyncSession


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Low] Make the UTC-bucketing assumption explicit. The two variants "must stay semantically identical," but they bucket differently: PG's time_bucket('1 day', occurred_at) on a timestamptz column buckets in UTC, while SQLite's date(occurred_at) buckets on whatever offset the stored string carries. They agree only as long as events are stored/interpreted as UTC (which the current ingest path does). A one-line note here that day boundaries are UTC on both paths would prevent a future timezone-parameterization on the PG side from silently diverging from SQLite.



# PostgreSQL: read the pre-rolled continuous aggregate.
_PG_SQL = text(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets avoid adding raw SQL in code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

went with raw SQL here on purpose as the postgres query reads the login_activity_daily continuous aggregate, which isn't (and shouldn't be) registered on analytics_metadata, reason being that the autogenerate will try to create/drop it and it doesn't exist on sqlite either. also worth noting that the two can't share a query (because of different source relation and day bucketing), so it's two selects either way.

the sqlite query could be written in sqlAlchemy core, but then the readers couldn't diff them by eye.

# backfill (refresh_continuous_aggregate) is run out-of-band if history matters.
op.execute(
"""
CREATE MATERIALIZED VIEW login_activity_daily

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we check if we can do this via the ORM

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately sqlAlchemy does not have DDL construct for materialized views at all.

conn = op.get_bind()
conn.execute(
sa.text(
"DELETE FROM role_permission WHERE permission = :permission "

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-milestone-1 branch from dc0aea7 to 8b10f75 Compare July 8, 2026 10:16
@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-plugin-event-schemas branch 2 times, most recently from 2cbdee4 to 1713ec5 Compare July 8, 2026 10:36
@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-milestone-1 branch 3 times, most recently from 1ca310b to eb7d0ad Compare July 8, 2026 10:56
"""
SELECT add_continuous_aggregate_policy(
'login_activity_daily',
start_offset => INTERVAL '3 days',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This start_offset combined with WITH NO DATA means anything older than 3 days at migration time never gets materialized: once the first policy run advances the watermark, real-time aggregation stops covering those old buckets and they disappear from the view entirely. Since user.logged_in has been emitting for a while, a deployed instance silently loses its pre-migration history here (the SQLite path keeps full history, so the two dialects drift). The comment above mentions an out-of-band backfill, but nothing in the repo runs one. Creating the view WITH DATA, or a one-off refresh_continuous_aggregate(login_activity_daily, NULL, NULL), would fix it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we'd need a refresh_continuous_aggregate(xyz_activity, NULL, NULL) for every cagg that's created, which is the wrong shape. i can either do (a) start_offset => NULL in the policy itself to refresh from the beginning of time or (b) one generic backfill command to discover all caggs and full-refresh each one. i lean towards (b) as (a) will cause each refresh to re-scan full history, which will grow expensive as data accumulates. what do you say?

Comment thread sparkth/core/analytics/reads.py Outdated

# PostgreSQL: read the pre-rolled continuous aggregate.
_PG_SQL = text(
"SELECT to_char(day, 'YYYY-MM-DD') AS day, login_count FROM login_activity_daily ORDER BY day DESC LIMIT :days"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

time_bucket buckets on UTC midnight, but to_char renders a timestamptz in the session TimeZone, and we don't pin one on the analytics engine. On any server west of UTC every label shifts one day back, so today's logins show up under yesterday. The SQLite path is effectively UTC, so tests will never catch this. to_char(day AT TIME ZONE 'UTC', 'YYYY-MM-DD') makes it deterministic.

"WHERE event_type = 'user.logged_in' "
"GROUP BY date(occurred_at) "
"ORDER BY day DESC "
"LIMIT :days"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LIMIT :days returns the N most recent days that had logins, not "the last N days" as the endpoint doc says, so a quiet stretch pushes the window arbitrarily far back. It also leaves this SQLite branch scanning all of raw_events on every call. A WHERE occurred_at >= ... bound on both queries would match the documented contract and cap the scan.

@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-plugin-event-schemas branch 3 times, most recently from 6730a05 to 8f657f5 Compare July 9, 2026 09:20
@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-milestone-1 branch from eb7d0ad to cc50a5b Compare July 9, 2026 09:41
@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-plugin-event-schemas branch 2 times, most recently from d2da0d1 to 1941720 Compare July 9, 2026 10:19
Base automatically changed from zaira/feat/analytics-plugin-event-schemas to main July 9, 2026 10:43
@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-milestone-1 branch 3 times, most recently from d2dbced to 9db4564 Compare July 13, 2026 08:22
Prove the analytics read side end-to-end for the user.logged_in event:
a Postgres continuous-aggregate rollup, a dialect-aware read function,
and a permission-gated read endpoint. The read path only ever touches
analytics tables/rollups, never the app DB.

- add login_activity_daily TimescaleDB continuous aggregate (SQLite
no-op)
- declare and export the analytics.read permission
- add dialect-aware get_login_activity (reads the cagg on Postgres,
  aggregates raw_events on SQLite), exposed via app.lib.analytics
- add GET /api/v1/analytics/login-activity gated by analytics.read
- grant analytics.read to the seeded admin role
- document the permission (README) and read API (CLAUDE.md)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@zaira-bibi zaira-bibi force-pushed the zaira/feat/analytics-milestone-1 branch from 9db4564 to e3a84c8 Compare July 13, 2026 08:47
@zaira-bibi zaira-bibi merged commit 9d933a4 into main Jul 14, 2026
5 checks passed
@zaira-bibi zaira-bibi deleted the zaira/feat/analytics-milestone-1 branch July 14, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants