Skip to content

feat: add performance logging for ingestion phases and GeoIP lookups#24

Merged
fukusuket merged 3 commits into
mainfrom
add-perf-log
Jun 16, 2026
Merged

feat: add performance logging for ingestion phases and GeoIP lookups#24
fukusuket merged 3 commits into
mainfrom
add-perf-log

Conversation

@fukusuket

Copy link
Copy Markdown
Collaborator

Closed #14

@fukusuket fukusuket requested a review from Copilot June 16, 2026 15:10
@fukusuket fukusuket self-assigned this Jun 16, 2026
@fukusuket fukusuket added the enhancement New feature or request label Jun 16, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds optional ingestion performance instrumentation to the Rust ingester so operators can see where time is spent (discovery, dedup-map load, parsing, insertion, GeoIP, batch marking) and estimate throughput.

Changes:

  • Add --perf-log flag to ingester ingest and print a perf breakdown to stderr on completion.
  • Track per-phase timings in IngestStats via a new PhaseTimings struct.
  • Time GeoIP lookups during insertion and surface that timing to the ingestion pipeline.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File Description
ingester/src/main.rs Adds --perf-log CLI flag and prints perf breakdown/throughput when enabled.
ingester/src/ingest.rs Introduces PhaseTimings and instruments discovery/dedup/parse/recv-wait/insert/batch-mark.
ingester/src/db.rs Times GeoIP lookups during inserts and returns (rows_inserted, geoip_ns) from insert_events_with_geo.

Comment thread ingester/src/ingest.rs
Comment thread ingester/src/ingest.rs
Comment thread ingester/src/ingest.rs
Comment thread ingester/src/ingest.rs
Comment thread ingester/src/db.rs
Comment thread ingester/src/main.rs
Comment thread ingester/src/main.rs
@fukusuket fukusuket marked this pull request as ready for review June 16, 2026 15:20
@fukusuket fukusuket merged commit 5686641 into main Jun 16, 2026
4 checks passed
@fukusuket fukusuket deleted the add-perf-log branch June 16, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Add performance logging

2 participants