Add support for logging to InfluxDB#290
Open
michelemin wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an InfluxDB logging backend to Plaid’s existing logging pipeline and updates data generators to emit simple processed-count timeseries metrics through that pipeline.
Changes:
- Added a new
influxdblogger module and extended logging configuration/error handling to support it. - Changed the logger trait and dispatch loop to use async
send_logcalls, with a Tokio runtime started inside the logging thread. - Updated data generators and example tailers to pass the logger through and emit
*_dg_processedmetrics.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
runtime/plaid/src/logging/webhook.rs |
Updated webhook logger trait implementation to async and cleaned webhook-specific comments. |
runtime/plaid/src/logging/stdout.rs |
Updated stdout logger trait implementation to async. |
runtime/plaid/src/logging/splunk.rs |
Updated Splunk logger trait implementation to async. |
runtime/plaid/src/logging/mod.rs |
Wired in InfluxDB config/error types, made the logging loop async, and added runtime startup / Influx dispatch. |
runtime/plaid/src/logging/influxdb.rs |
New InfluxDB backend implementation for timeseries log delivery. |
runtime/plaid/src/data/mod.rs |
Passed the logger into DG processing and emitted *_dg_processed timeseries points. |
runtime/plaid/examples/tailers/github.rs |
Updated example call site for the new get_and_process_dg_logs signature. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+73
to
+89
| let line = format!( | ||
| "{},{} {} {}", | ||
| table, | ||
| tags.map(|t| t | ||
| .into_iter() | ||
| .map(|(k, v)| format!("{}={}", k, v)) | ||
| .collect::<Vec<_>>() | ||
| .join(",")) | ||
| .unwrap_or_default(), | ||
| fields | ||
| .into_iter() | ||
| .map(|(k, v)| format!("{}={}", k, v)) | ||
| .collect::<Vec<_>>() | ||
| .join(","), | ||
| timestamp.map(|t| t.to_string()).unwrap_or_default() | ||
| ); | ||
|
|
Comment on lines
+120
to
+123
| .send_line_protocol( | ||
| &measurement, | ||
| None, | ||
| [("value", value.to_string().as_str())].into(), |
Owner
There was a problem hiding this comment.
I think this is wrong given that it..does compile. I believe the .into() fixes it
Comment on lines
+290
to
+292
| if let Some(logger) = &influxdb_logger { | ||
| if let Err(_) = logger.send_log(&log).await { | ||
| error!("Could not send logs to InfluxDB"); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add InfluxDB as a supported logging backend and wire data generators to emit simple timeseries metrics through the existing logging pipeline.
Changes
*_dg_processedtimeseries points from data generators with the number of processed logs