fix(ingest/mode): link chartless queries to reports and gate charts on report chart_count - #19382
Open
Ynabokov wants to merge 3 commits into
Open
fix(ingest/mode): link chartless queries to reports and gate charts on report chart_count#19382Ynabokov wants to merge 3 commits into
Ynabokov wants to merge 3 commits into
Conversation
Contributor
|
Linear: ING-3361 Thanks for your contribution! We have created an internal ticket to track this PR. A member of the core DataHub team will be assigned to review it within the next few business days - you will get a follow-up comment once a reviewer is assigned. |
Contributor
There was a problem hiding this comment.
1 issue found across 4 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="metadata-ingestion/src/datahub/ingestion/source/mode.py">
<violation number="1" location="metadata-ingestion/src/datahub/ingestion/source/mode.py:2007">
P2: Chart fetching is gated entirely on the report-level chart_count. If report_chart_count coerces to 0 while the report does contain query charts (missing key, non-numeric, or a stale/under-counted value), all chart metadata and the report→chart→query lineage for that report are dropped, and every query is mislabeled chartless with a direct datasetEdge. Consider treating chart_count as an optimization hint rather than authoritative, or verifying against the fetched query data.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes Mode report↔query lineage broken by #17357.
PR #17357 gated the per-query chart fetch on
query.get("chart_count"). Per the Mode API,chart_countis a report-level field - the query object does not expose it (the Report object (https://mode.com/developer/api-reference/analytics/reports/) haschart_count; the Query object (https://mode.com/developer/api-reference/analytics/queries/) has onlyexplorations_countandreport_imports_count). So in production the gate was always 0,_get_chartswas never called, and reports received no chart/query lineage. #17357 passed CI only because its fixture added chart_count to the query object, which doesn't exist in the real API.Changes
chart_countDashboardInfo.datasetEdges(a query otherwise reaches the report only through its charts, so chartless queries were orphaned).datasetEdgesis used becauseDashboardInfo.datasetsis deprecated.