feat(export): include accumulated warnings in output json (#113)#116
feat(export): include accumulated warnings in output json (#113)#116aryanputta wants to merge 1 commit into
Conversation
Active TraceWarnings accumulated during processing are now collected from the stage contexts and written to otherData.issues in the exported json. Important warnings no longer get lost when console output is ignored or drowned out. Resolves IBM#113. Signed-off-by: Aryan Putta <aryansputta@gmail.com>
lasch
left a comment
There was a problem hiding this comment.
Need to hold this for a bit, sorry. There's a design change coming with respect to the verification pipeline that would simplify the approach.
Comment on current approach:
- not yet convinced that additional tracking of contexts in the processing is needed. Kind of stores the contexts twice.
|
Understood, happy to hold for the verification-pipeline change. On the double-storage point: the second store exists only because warnings raised during processing have no path back to the context at export time today. If the redesign gives the exporter access to the context's warnings directly, I would drop the processor-side tracking entirely and just read from the context. Does the new pipeline expose that, or should I rebase onto it once it lands and resimplify then? |
|
I just created #117 introducing a path from context warnings to exporter using metadata events. The basics should probably work for the regular processing pipeline too. |
Closes #113.
What
Accumulated
TraceWarnings that fire during processing are now exported into the output json underotherData.issues, keyed by warning name:{ "otherData": { ... "issues": { "zero_gap_time": "Detected 3 identical timestamps between consecutive events. ...", "flex_mismatch": "CAT: Found 21 categorization mismatches compared to FLEX src classifier." } } }This follows the preferred approach in the issue: reuse the existing
TraceWarningclass and let the exporter emit the active warnings, so problems are not lost when the console output is ignored or drowned out.How
AbstractContext.collect_warnings()returns each active warning asname -> formatted text(mirrors the existingprint_warnings).EventProcessorkeeps its stage contexts pastdrain()and aggregates their active warnings viacollect_warnings().AbstractTraceExporter.export_issues()is a no-op default;JsonFileTraceExporteroverrides it to write intootherData(inherited by the TensorBoard exporter, propagated per rank). Empty issue sets add no section.export_issues(processor.collect_warnings())after drain and before flush.Tests
test_collect_warningsfor the context-level collection.test_exporter.pyverifies the json exporter writesotherData.issuesand skips the section when there are no warnings.Scope note
This covers the pipeline stage warnings. Ingestion-side warnings live on the importer and could be folded in as a follow-up if you want them in the same section.