Skip to content

feat(prommap): shared Prometheus metric-mapping package (PIPE-1486, BP-471) - #319

Open
Dylan-M wants to merge 3 commits into
mainfrom
dylanmyers/pipe-1486-prometheus-metric-mapping-package-shared
Open

Dylan-M wants to merge 3 commits into
mainfrom
dylanmyers/pipe-1486-prometheus-metric-mapping-package-shared

Conversation

@Dylan-M

@Dylan-M Dylan-M commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Proposed Change

Adds internal/prommap, the shared layer the two Prometheus outputs (prometheus-remote-write, prometheus-scrape) build on. It maps an embed MetricPoint to the in-memory model both the scrape (text) and remote-write (protobuf) encoders serialize from.

Type mapping: gauge and sum to gauge, counter to counter (_total), histogram to histogram (_bucket/_sum/_count). Histogram buckets are cumulative with le labels, including +Inf. Names are sanitized to the Prometheus grammar. Labels are sorted, le last. HELP, TYPE, unit, and millisecond timestamps come from the MetricPoint.

Checklist
  • Changes are tested
  • CI has passed

@Dylan-M
Dylan-M requested review from a team as code owners September 23, 2026 22:33
Comment thread internal/prommap/prommap.go Outdated
// le labels, including +Inf), plus _sum and _count.
func Map(mp embed.MetricPoint) (MetricFamily, error) {
base := sanitizeName(mp.Name)
labels := sortedLabels(mp.Metadata.Attributes)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Labels come only from Metadata.Attributes; Metadata.Resource, where host identity lives, is never read. With hostmetrics at workers: 4 against a real Prometheus, system_memory_usage ends up with 4 series ({state=…}) instead of 16, and there's no host label. Every simulated host writes into the same series. In remote-write this causes 400 duplicate sample for timestamp … overrides not allowed, which drops the whole batch (5 of them in a 15s run). In scrape, hosts overwrite each other.

Resource attributes need to become labels, or at least job/instance plus target_info, following the OTel→Prometheus convention.

Comment thread internal/prommap/prommap.go Outdated
samples = append(samples,
Sample{Name: base + "_bucket", Labels: withLE(labels, "+Inf"), Value: float64(cumulative), TimestampMS: tsMS},
Sample{Name: base + "_sum", Labels: labels, Value: mp.HistogramSum, TimestampMS: tsMS},
Sample{Name: base + "_count", Labels: labels, Value: float64(mp.HistogramCount), TimestampMS: tsMS},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_count comes from HistogramCount instead of the cumulative +Inf total, so the two can disagree.

@Dylan-M
Dylan-M force-pushed the dylanmyers/pipe-1486-prometheus-metric-mapping-package-shared branch from 06e4933 to 313d507 Compare September 28, 2026 18:08
@Dylan-M
Dylan-M requested a review from eKuG September 28, 2026 18:09
@eKuG
eKuG added this pull request to stack #326 September 28, 2026 18:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants