Add support for zod validation error callback in schema stream

[RollForReflex]

Was noodling on #78 a bit last night:

While we don't want to defacto throw validation errors (the point of the package after all is type-safety), we could provide a simple callback once the stream ends to say "hey, here's the errors we found" if the zod schema provided is strict.

This gives the caller the power to do whatever they want but they now have both the completed data via the stream and validation errors they may or may not care about.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 4b31c44

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
island-ai-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 27, 2024 8:23pm

[roodboi]

love it - I tried to compose these packages in a few different ways before they all ended up in the current form but the high level on how things are organized/responsibilities of each are this:

schema stream: pure streaming json parser - the schema input allows us to stub the final object which enables you to start reading from the response asap - value here is that you can define what the expected output will be so as it streams through you have a stub that is readable testObject.nestedObject.someKey for instance will be in the structure from the start with default value for the type or a provided default ( https://github.com/hack-dance/island-ai/blob/main/public-packages/schemaStream/src/utils/streaming-json-parser.ts#L100 )

zod stream: given a zod schema pass through to schema stream for streaming json over the wire - we don't do the fullzod schema.parse since that throws but we safe parse and update the _meta object on the response to show u in realtime what paths in the object have completed and are/are not valid (
https://github.com/hack-dance/island-ai/blob/main/public-packages/zod-stream/src/structured-stream.client.ts#L79 )

instructor: uses both under the hood and implements retry logic/actual schema validation/parsing and extends the openai sdk to make it as close as possible to what people are already using which means to adopt you only need to change an import ( https://github.com/instructor-ai/instructor-js )

[roodboi]

instead of passing a conditional callback that depends on another flag - would it be easier for us to just allow an onComplete here and just by default pass through the safeparse results always?

feels like making sure that isStrictSchmea is set on the instance AND havaing to pass the callback might be a bit confusing/unintuitive

vs say just adding

.parse({ onComplete })

and we conditional add the onEnd wiht a safeParse inside of it? We could also jsut pasas through the finala result and any other meta we have for free

  const stream = parser.parse({
    onComplete:({ isValid, errors = [], data }) => {
      errors = zodError.errors
    }
  })

[RollForReflex]

yeah I dig that, that way you can do whatever you want

[RollForReflex]

love it, updated