feat: tnc connectors implementation [CM-1010]#3894
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
mbani01
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
services/apps/snowflake_connectors/src/integrations/tnc/courses/transformer.ts
Outdated
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/tnc/certificates/transformer.ts
Outdated
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/tnc/courses/buildSourceQuery.ts
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/tnc/certificates/buildSourceQuery.ts
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/tnc/certificates/buildSourceQuery.ts
Show resolved
Hide resolved
services/apps/snowflake_connectors/src/integrations/tnc/certificates/buildSourceQuery.ts
Show resolved
Hide resolved
| if (!IS_PROD_ENV) { | ||
| select += ` AND cms.slug = 'cncf'` | ||
| } |
There was a problem hiding this comment.
Nitpick: Can we add a comment here just to explain that this is being added for staging/testing purposes?
| sourceId: certificateId, | ||
| sourceParentId: (row.COURSE_ID as string | null) || undefined, | ||
| member: { | ||
| displayName: learnerName || email, |
There was a problem hiding this comment.
If we were to fallback to the email, could we remove the domain? And simply add the initial part of the email?
| sourceId: courseActionId, | ||
| sourceParentId: (row.COURSE_ID as string | null) || undefined, | ||
| member: { | ||
| displayName: learnerName || email, |
services/apps/snowflake_connectors/src/integrations/tnc/courses/transformer.ts
Outdated
Show resolved
Hide resolved
| sourceId: enrollmentId, | ||
| sourceParentId: (row.COURSE_ID as string | null) || undefined, | ||
| member: { | ||
| displayName: learnerName || email, |
services/apps/snowflake_connectors/src/integrations/tnc/courses/transformer.ts
Outdated
Show resolved
Hide resolved
| return null | ||
| } | ||
|
|
||
| const certificateId = (row.CERTIFICATE_ID as string)?.trim() |
There was a problem hiding this comment.
Missing sourceId validation in certificates and enrollments transformers
Medium Severity
The certificates transformer extracts certificateId via (row.CERTIFICATE_ID as string)?.trim() and the enrollments transformer extracts enrollmentId via (row.ENROLLMENT_ID as string)?.trim() — neither validates the value before assigning it to activity.sourceId. If the row value is null/undefined, the result is undefined, violating the IActivityData interface where sourceId: string is required (not optional). The courses transformer correctly validates courseActionId and returns null if missing, but the other two transformers skip this check, risking activities with undefined sourceId being passed downstream.
Additional Locations (1)
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
| SELECT account_id, account_name, website, domain_aliases, NULL AS LOGO_URL, NULL AS INDUSTRY, NULL AS N_EMPLOYEES | ||
| FROM analytics.bronze_fivetran_salesforce_b2b.accounts | ||
| WHERE website IS NOT NULL | ||
| )` |
There was a problem hiding this comment.
SQL CTE constants duplicated across three files
Low Severity
The CDP_MATCHED_SEGMENTS and ORG_ACCOUNTS SQL CTE constants are identically copy-pasted across all three TNC buildSourceQuery files (certificates, courses, enrollments). LFID_COALESCE is also duplicated between certificates and enrollments. These could live in a shared TNC SQL fragments module to avoid divergence during future updates.
Additional Locations (2)
2 participants
This pull request adds support for the TNC platform in the Snowflake connectors integration. It introduces new activity types, data sources, SQL queries, and transformers to ingest and process TNC certification, enrollment, and course action data. The changes include both backend database updates and significant additions to the data ingestion pipeline.
Key changes:
1. TNC Platform Integration:
2. Data Source Query Builders:
buildSourceQueryfunctions for TNC certificates, enrollments, and course actions, providing SQL logic to extract and join relevant data from Snowflake, including handling environment-specific filters and incremental syncs. [1] [2] [3]3. Data Transformation Logic:
4. Database Schema Update:
activityTypestable to support tracking of certification enrollments, training enrollments, issued certifications, and course/exam attempts.These changes lay the groundwork for ingesting and processing TNC data, enabling richer analytics and reporting for certification and training activities.
Note
Medium Risk
Adds a new production data ingestion path with complex Snowflake SQL and new activity typing, which could affect export volume, correctness, or downstream attribution if queries/field mappings are wrong.
Overview
Adds TNC support to the Snowflake connectors pipeline, registering three new exports (enrollments, certificates, and course actions) with dedicated Snowflake query builders and row transformers that emit unified activities (including member identity/org resolution, segment routing, and incremental backfill logic).
Introduces new
tncactivity types/scoring (enrolled-*,issued-certification,attempted-*) and wires TNC into organization source/attribute precedence. ImprovesTransformerBase.safeTransformRowlogging to capture error message/stack plus row keys for easier debugging when rows are skipped.Written by Cursor Bugbot for commit bdf6faf. This will update automatically on new commits. Configure here.