Repository navigation
fix(otlp-grpc): send generator resource and real severity on logs, metrics and traces - #329
Open
adnanrahic wants to merge 2 commits into
Open
adnanrahic wants to merge 2 commits into
adnanrahic wants to merge 2 commits into
Conversation
…trics and traces The OTLP gRPC output dropped the resource every generator attaches (Metadata.Resource): logs and traces were always sent with a hard-coded service.name=blitz resource, and metrics with buildMetricRequest(metrics, nil). host.name, telemetry.source, apache.format and the rest never reached the collector, so pipelines could not route or filter on them. Per-record Metadata.Attributes on logs were dropped too. Each queued log record, metric and span now carries its generator's resource. A batch is split into one ResourceLogs / ResourceMetrics / ResourceSpans per distinct resource, in order of first appearance. service.name=blitz is still added unless the generator sets its own. mapSeverityNumber only matched exact uppercase DEBUG/INFO/WARN/ERROR/FATAL, so lowercase levels (kubernetes, apache error), Windows Event Log level names (Warning, Error, Critical), PostgreSQL WARNING/PANIC/LOG and syslog notice/crit all became INFO. Matching is now case-insensitive and covers those names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…et host
Since generators draw their host identity from the simulated environment
(PIPE-1036), hostmetrics took whatever system SystemForKey("hostmetrics")
selected and used that system's OS, ignoring generator.hostmetrics.os. With
the default randomized fleet that is often a Windows host, so
BLITZ_GENERATOR_HOSTMETRICS_OS=linux (also the default) still produced
Windows metrics.
Add Environment.SystemForKeyWithOS, the same key-hashed selection restricted
to systems running a given OS, and use it for hostmetrics with the configured
OS (empty means linux). When no system runs that OS the generator falls back
to its synthetic host built from OS and Hostname.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dylan-M
requested changes
Oct 7, 2026
Dylan-M
left a comment
Contributor
There was a problem hiding this comment.
Suggest closing. These fixes are already in progress.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Change
The OTLP gRPC output dropped data every generator already provides, and the hostmetrics generator ignored its configured OS. Collectors couldn't route on the source of a record, and most severities arrived as INFO.
1. Resource was never sent (logs, metrics, traces)
Generators attach a resource (
host.name,telemetry.source,apache.format, ...) inMetadata.Resource, but the OTLP gRPC output ignored it:service.name=blitzresource; the record carried only the body plus emptyenvironment/locationattributes. Per-recordMetadata.Attributeswere dropped too.buildMetricRequest(metrics, nil), so host metrics arrived withouthost.name.service.name=blitzresource.Found while running Blitz into a Bindplane gateway that routes on
resource.attributes["telemetry.source"]: no condition ever matched, so every log fell through to the catch-all route.Each queued log record, metric and span now carries its generator's resource (generic
entry[T]inresource.go). At send time the batch is split into oneResourceLogs/ResourceMetrics/ResourceSpansper distinct resource, in order of first appearance.service.name=blitzis still added unless the generator sets its own. Log records now also carryMetadata.Attributes.2. Severity matching was uppercase-only
mapSeverityNumberonly matched exactDEBUG/INFO/WARN/ERROR/FATAL, so these all became INFO:warn,error(lowercase)crit,error,warn,noticeWarning,Error,Critical,VerboseWARNING,PANIC,LOG,NOTICEMatching is now case-insensitive and covers those names. Unknown levels still map to INFO. Severity text is unchanged.
3.
generator.hostmetrics.oswas ignoredSince generators draw their host from the simulated environment (PIPE-1036), hostmetrics used whatever system
SystemForKey("hostmetrics")selected, including that system's OS. With the default randomized fleet that is often Windows, soBLITZ_GENERATOR_HOSTMETRICS_OS=linux(also the default) still produced Windows metrics.New
Environment.SystemForKeyWithOSdoes the same key-hashed selection restricted to systems running a given OS. Hostmetrics uses it with the configured OS (empty means linux), and falls back to its synthetic host when no system runs that OS.Verification
TestEnvironmentSystemForKeyWithOSandTestHostMetricsIdentityHonorsConfiguredOS(OS-restricted host selection),resource_test.go(resource reaches the OTLP resource for logs, metrics and traces; grouping by resource; no duplicateservice.name) andseverity_test.go(table of every level name above).go test ./...,go vet,gofmtandrevive -config .revive.tomlare clean.hostmetrics,tracesandnginxgenerators against an OTel Collector with the debug exporter. All three arrived withhost.nameandtelemetry.sourceon the resource. Against a Bindplane gateway, the routing connector stopped sending everything to its catch-all route, and Kuberneteswarn/errorlines now matchseverity_number >= SEVERITY_NUMBER_WARN. The hostmetrics generator, previously started asos.type: windowsdespiteBLITZ_GENERATOR_HOSTMETRICS_OS=linux, now starts aslinux.Checklist
🤖 Generated with Claude Code