feat(Base Set): move thirdParty ids onto variants_detailed + additive schema/server retrocompat - #1392
feat(Base Set): move thirdParty ids onto variants_detailed + additive schema/server retrocompat#1392lamplis wants to merge 6 commits into
Conversation
…e cardmarket metadata
…for migrated cards
🃏 102 Cards ChangedDetails: 102 modified Alakazam (base1-1) (found using en)
File: [data/Base/Base Set/1.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/1.ts) Mewtwo (base1-10) (found using en)
File: [data/Base/Base Set/10.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/10.ts) Lightning Energy (base1-100) (found using en)
File: [data/Base/Base Set/100.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/100.ts) Psychic Energy (base1-101) (found using en)
File: [data/Base/Base Set/101.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/101.ts) Water Energy (base1-102) (found using en)
File: [data/Base/Base Set/102.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/102.ts) Nidoking (base1-11) (found using en)
File: [data/Base/Base Set/11.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/11.ts) Ninetales (base1-12) (found using en)
File: [data/Base/Base Set/12.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/12.ts) Poliwrath (base1-13) (found using en)
File: [data/Base/Base Set/13.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/13.ts) Raichu (base1-14) (found using en)
File: [data/Base/Base Set/14.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/14.ts) Venusaur (base1-15) (found using en)
File: [data/Base/Base Set/15.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/15.ts) Zapdos (base1-16) (found using en)
File: [data/Base/Base Set/16.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/16.ts) Beedrill (base1-17) (found using en)
File: [data/Base/Base Set/17.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/17.ts) Dragonair (base1-18) (found using en)
File: [data/Base/Base Set/18.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/18.ts) Dugtrio (base1-19) (found using en)
File: [data/Base/Base Set/19.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/19.ts) Blastoise (base1-2) (found using en)
File: [data/Base/Base Set/2.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/2.ts) Electabuzz (base1-20) (found using en)
File: [data/Base/Base Set/20.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/20.ts) Electrode (base1-21) (found using en)
File: [data/Base/Base Set/21.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/21.ts) Pidgeotto (base1-22) (found using en)
File: [data/Base/Base Set/22.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/22.ts) Arcanine (base1-23) (found using en)
File: [data/Base/Base Set/23.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/23.ts) Charmeleon (base1-24) (found using en)
File: [data/Base/Base Set/24.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/24.ts) Dewgong (base1-25) (found using en)
File: [data/Base/Base Set/25.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/25.ts) Dratini (base1-26) (found using en)
File: [data/Base/Base Set/26.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/26.ts) Farfetch'd (base1-27) (found using en)
File: [data/Base/Base Set/27.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/27.ts) Growlithe (base1-28) (found using en)
File: [data/Base/Base Set/28.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/28.ts) Haunter (base1-29) (found using en)
File: [data/Base/Base Set/29.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/29.ts) Chansey (base1-3) (found using en)
File: [data/Base/Base Set/3.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/3.ts) Ivysaur (base1-30) (found using en)
File: [data/Base/Base Set/30.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/30.ts) Jynx (base1-31) (found using en)
File: [data/Base/Base Set/31.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/31.ts) Kadabra (base1-32) (found using en)
File: [data/Base/Base Set/32.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/32.ts) Kakuna (base1-33) (found using en)
File: [data/Base/Base Set/33.ts](https://github.com/tcgdex/cards-database/blob/3cd5ba024f95ecf300f6bf36756234eba0427983/data/Base/Base Set/33.ts)
|
|
Hi @thomas-bassett , |
| rarity?: string; | ||
| set: { name: string }; | ||
| hasImage: boolean; | ||
| cardmarketIds?: number[]; |
There was a problem hiding this comment.
Can we remove all edits within .github.
There isn't a reason to change the comments or actions for this
There was a problem hiding this comment.
I added a link to cardmarket to test the cardmarket thirdparty id
If you don't want it, I'll remove it @FalconChipp
There was a problem hiding this comment.
Please. Up to Aivor to change the git workflows.
If anything, put it in a separate pr and let him review when he gets the time
| working-directory: .github/scripts | ||
| permissions: | ||
| pull-requests: write | ||
| contents: read |
| subtype: "unlimited", | ||
| subtype: "1999-2000-copyright", | ||
| thirdParty: { | ||
| cardmarket: 273696, |
There was a problem hiding this comment.
How are you sourcing this information? I checked just this one because it's very a specific subtype and its just linking to the standard
There was a problem hiding this comment.
For me subtype: "1999-2000-copyright" is the basic normal card.
there is no "normal" variant so I mapped it to the standard cardmarket card.
But if "1999-2000-copyright" is not the basic original card, should I create a basic "normal" variant instead?
There was a problem hiding this comment.
I found default basic card seem to be the "unlimited" (https://youtu.be/FIgZahAte7A?si=aY-stT7SDftmVQ4v)
Do you agree? @thomas-bassett
There was a problem hiding this comment.
the 1999-2000 copyright version is the 4th print and likely is just thrown in on cardmarket to be 1 id.
but the "base" version of the card the way i looked at things should be the first print of the card.
in this card the first edition shadowless would be the first card printed
…dback Restore workflow/comment script files to base content and replace the Base Set migration with the corrected variant-level mapping so marketplace IDs stay on the intended existing variants.
|
Hello, |
|
The windows job often fails. |
FalconChipp
left a comment
FalconChipp
left a comment
There was a problem hiding this comment.
Git workflow changes removed
| variants: [ | ||
| { | ||
| type: "holo", | ||
| subtype: "1999-2000-copyright", |
There was a problem hiding this comment.
why are you switching this around. I think you missed my point in the other comment. the base version "should" be the first print.
thomas-bassett
left a comment
thomas-bassett
left a comment
There was a problem hiding this comment.
Please clean up the PR. the description still has all of the old changes in it. same with title.
Also if we are going to move the ids over can get get all of the IDs so we can merge it once? Falcon has some tools that help with the extraction
lamplis
changed the title
|
Thanks Thomas. I cleaned up the PR title and description so they now match the actual 107-file diff. On the broader "move all of the ids over once" point: I kept this PR scoped as the proof point for the full shape change, since it is the first one that combines (1) the additive That is also the same cadence Falcon has been using in the recent merged variant-migration PRs ( If you would rather I close this and reopen as one giant PR instead, say the word and I will do that. The tradeoff is just that the additive schema/server review then gets mixed into a much larger data-only migration. |
| } | ||
|
|
||
| /** | ||
| * Retrocompat helper: reconstruct the deprecated root-level `thirdParty` |
There was a problem hiding this comment.
this already exsists just not at compile time
Did you? i still see all the server changes not needed in a Data PR. these MUST be split up unless abosulty nessasary like a new feature. I dont think the server changes you made are nessasary but its a mess im not sure what changes are what. this should be like 3 differnt prs (data,cmversion,thirdparty fallback) although the last one isnt needed. also looking at #1381 all variants have been given third-party ids. this is what I meant about filling out all the ids. If we don't do this now we will just need to come back and do them again later so lets get them done now. I am going to close this PR for now. please split this up into small concise PRs and we can go from there. |
4 participants
Summary
Migrate every card in
data/Base/Base Set/(102 files) so thatthirdParty(cardmarket,tcgplayer) lives onvariants_detailed[*]instead of at the card root, plus the minimal additive schema widening and server-side reconstruction needed to keep both the V2 API response and the compiled JSON artifact byte-compatible for existing consumers.Files in this PR (107 total)
data/Base/Base Set/*.ts(102 files)interfaces.d.ts,meta/definitions/api.d.tsserver/src/V2/Components/Card.ts,server/compiler/utils/cardUtil.ts,server/compiler/utils/translationUtil.tsData changes
thirdPartyblocks are moved into the canonicalvariants_detailedentry for each Base Set card.Schema changes (additive only)
CardmarketVersiontoken type (V${number}|BASE|LIBRE) added toVariantType. Used for placeholder variants emitted by the Cardmarket sync when a productId cannot be matched to a canonical source variant.'Yellow A'literal added toVariantTypefor alternate-art sibling printings that share a name and number with a base printing but use a local id suffixed with"a"(for exampleXY150a,PHF024a,FFI055a).variant_detailed.cardmarketLabels?: string[]for the raw OCR-derived labels used by human reviewers to reconcile placeholder variants to canonical type / subtype / stamp.Server retrocompat
server/src/V2/Components/Card.ts):resolveRootPricingFromVariantsreconstructs root-levelcardmarket/tcgplayerpricing fromvariants_detailed[0].thirdParty, falling back to the first variant that carries a populated pricing block.variant.thirdParty.thirdParty(object-variant promos, trainer kits) continue to work unchanged.server/compiler/utils/cardUtil.ts):resolveRootThirdPartyFromVariantssynthesises the deprecated rootthirdPartyfield from the firstvariants_detailedentry that carries one, so that direct JSON consumers keep seeing the same shape.thirdPartyis only@deprecatedin theCardinterface docstring; no behavior change for clients.server/compiler/utils/translationUtil.ts: small additive support for the new variant tokens (CardmarketVersion,Yellow A).Retrocompat guarantees
card.pricing.cardmarket/card.pricing.tcgplayercontinue to resolve correctly for migrated cards via the new variant-based reconstruction. Non-migrated cards keep using the existing rootthirdPartycode path with zero behavior change for them.thirdPartyis preserved byte-for-byte on migrated cards through the new compile-time synthesis helper. Verified onbase1-1(Alakazam) andbase1-10(Mewtwo): same{cardmarket, tcgplayer}object as before the migration. Non-migratedbase2-1(Clefable) is unchanged.Scope (intentional)
This PR is intentionally scoped to a single set so the variant move plus additive schema and server retrocompat shape can be reviewed end-to-end on a small, well-understood data surface (Base Set, all printings). Once merged, the remaining root-
thirdPartycards can be moved in a single mechanical follow-up PR using the same canonical-variant landing rule, since the schema and server pieces only need to land once.Validation
Locally ran the full upstream CI gate (
.github/workflows/test.yml), including Bruno integration:bun install --frozen-lockfile(root + server)bun run validate(root TS,tsc --noEmit)cd server && bun run compilecould not load filewarnings come from the pre-existing empty-path corner inserver/compiler/utils/util.tsand are not introduced here)cd server && bun run --bun validate(server TS,tsgo --noEmit)cd .bruno && bru run --env Developpementcard,cards,serie,series,set,sets), and regression tickets #467, #471, #474, #475, #489, #562, #566, #567Compiled JSON spot check (
server/generated/en/cards.json):base1-1Alakazam (migrated): rootthirdParty = {cardmarket: 273696, tcgplayer: 42346}, reconstructed fromvariants_detailed[0].thirdParty.base1-10Mewtwo (migrated): same pattern, retrocompat confirmed.base2-1Clefable (Base Set 2, not in this PR): unchanged, no regression on non-migrated cards.