Updates: 7 new posts for 2026-08 - #156
Open
vannyle wants to merge 43 commits into
Open
Conversation
Specs the pipeline that turns Alex's qualifying X posts into a monthly company-voice blog post, gated on his informal review before merge.
Addresses Alex's concern that some source posts are too thin to make a worthwhile blog post: raises the classify bar from "on-topic" to "stands alone for a reader with zero context," makes drafting explicitly editorial instead of reformatting, scales the length filter back to a cheap one-liner check, and lets a run produce multiple posts (grouped) instead of forcing everything into one.
Bare directory args to `node --test` don't recurse into matching test files on this Node version - it tries to require the directory path itself and fails with MODULE_NOT_FOUND. Explicit globs fix it.
…slug/env hardening) - Quote and escape the summary field in publish-post.js's frontmatter the same way title already is, so a summary containing a colon no longer produces invalid YAML. - Make blog-draft branch/PR names unique per run (append GITHUB_RUN_ID, falling back to Date.now()) so a same-month re-run no longer collides with an existing branch/PR. - Sanitize the LLM-provided slug (lowercase, collapse non-alphanumeric runs to a hyphen, trim leading/trailing hyphens) before using it in the post folder name. - Add an assertRequiredEnv check at the top of main() in run.js so a missing X_API_BEARER_TOKEN/OPENAI_API_KEY/SLACK_WEBHOOK_URL fails fast with a clear error instead of an obscure downstream failure.
Proves the YAML-escaping fix by round-tripping a colon-containing summary through the real frontmatter parser Gatsby uses, not just checking the output string shape.
Replaces the openai SDK with @anthropic-ai/sdk across the classify and draft steps. The Anthropic API has no json_object response mode, so the JSON contract is now enforced by structured outputs (output_config.format) with real schemas instead of a prompt-level request. Adds anthropic-json.js to share the request shape between both steps and to turn refusals and token-limit truncation into named errors, so they surface as Slack alerts rather than JSON parse failures.
Rewrites the classify and draft tasks around @anthropic-ai/sdk and structured outputs, and adds a new Task 7 for the shared anthropic-json helper (renumbering the tasks after it). Also corrects the prerequisites: the key is now an Anthropic one, and X removed its free tier in Feb 2026.
The listing predated the review-fixes commit, so it was missing assertRequiredEnv, the empty-group filter, and the GITHUB_RUN_ID branch suffix. Spliced in from the real file rather than retranscribed, so the two are byte-identical apart from the file-annotation comment.
A dry run stops after drafting, so demanding SLACK_WEBHOOK_URL up front blocked local testing for a webhook that is never called. Also skips the failure notification when no webhook is configured, so a dry-run error isn't buried under a second failure from posting to an undefined URL.
A dry run against real June 2026 posts surfaced two problems. Drafts referred to "Alex Barashkov, our CEO" in the third person while being published under his byline; the draft prompt now states who the narrator is. And a group re-covering an existing post slipped through, so the classify schema now requires a per-group already_covered verdict that the code filters on, instead of trusting the model to omit it silently.
Near-duplicate detection is a judgment call the classify step should not make alone, so the PR gate is where it belongs. The checklist names the failure modes an actual dry run produced rather than generic advice, and lives where the reviewer already is instead of in the Actions log.
classifyAndGroupPosts now returns { groups, skipped } so a group dropped
as a duplicate reaches the reviewer instead of vanishing into the Actions
log — a wrong drop was previously as invisible as a duplicate slipping
through. Extracts the PR body into a tested module rather than building
it inline, since the skipped section is a branch real data rarely hits.
Adds sync-plan-code.js, which rewrites every code listing in the plan
from the file it documents. The listings had drifted repeatedly, in
files this branch never touched.
Adds the media expansion to the timeline call, assigns filenames before drafting so the model references names that exist, downloads them into the post folder, and strips any reference that failed to download rather than shipping a broken image. A survey of 35 days of posts found no author-supplied alt text on any of 42 media items, so the model writes it — it renders as the visible caption. publishPost becomes async because the body can only be finalised once the downloads are known. Video stays excluded: it is 23 of those 42 items and still needs a hosting decision.
Requests variants/width/height from the X API, picks the highest-bitrate mp4, and emits the site's <Video> component with the exact attributes it requires. The poster frame is downloaded into the post folder, where gatsby-node picks it up by filename like any other image. The mp4 is hotlinked from video.twimg.com rather than rehosted, because the pipeline has no write access to the S3 bucket existing posts use. Those URLs can rot, so a published video may stop playing later — hence the new PR checklist item asking the reviewer to play them. A <Video> whose poster failed to download is dropped entirely: a missing poster throws in video.jsx and would fail the whole site build.
1200x630, 276KB — down from a 3840x2160 7.2MB placeholder that was copied into every generated post folder.
A real end-to-end run failed at gh pr create with 'GraphQL: not all refs are readable' — git push returns before GitHub has indexed the new ref. The identical command succeeded moments later. A CI runner is faster than a laptop, so this is more likely there, not less; it would have failed the first scheduled run. Retries only that error, with backoff, so a bad token still fails fast into the Slack alert instead of stalling. Also passes --base main explicitly rather than depending on the repo's default branch. git-pr was untested on the theory that mocking child_process only tests the mock; the retry policy is real logic, so runImpl/sleepImpl are now injectable.
gatsby-node.js:135 collects posters with name: { regex: "/video-cover/" }.
Posters were named video-1-cover.jpg, which does not match, so the file
downloaded correctly but was invisible to the query — leaving videoCovers
empty and making video.jsx:18 throw, which fails the entire site build
rather than degrading to a missing image.
Renamed to video-cover-N.jpg, matching both the query and the convention
existing posts already use. A test now asserts poster names satisfy that
same regex, so the coupling can't silently break again.
Found by a real gatsby build; stripUnusableVideos did not catch it because
the download succeeded — the file was simply unreachable by name.
A real run produced a second Toolcraft post whose summary reused the existing post's own 'starter kit and UI library' framing. The classify verdict was defensible — the group carried genuine news (toolcraft.sh, an AI harness overhaul) — so tightening the duplicate check would have suppressed a legitimate release announcement. The duplication was introduced one step later, by a drafter that had never been told an existing post already explains the product. Classify now names related existing posts per group, separately from the already_covered verdict, and read-existing-posts resolves each to its blog path via the site's own getBlogPostPath. The drafter is told to open with what is new and link to the background rather than reintroduce it. Verified live: the reused framing is gone, and 5 of 6 drafts now link to relevant existing posts — including ones I had not anticipated, like the Aval post citing the earlier ffmpeg video piece.
The blog is sentence case throughout ('Taking automated web page
screenshots with Puppeteer and Sharp'), but drafts drifted into Title Case
On Every Word on roughly a third of titles. The rule needs the
counter-example alongside it — naming the wrong form is what makes it
stick, and spelling out that AI/Blender/Novu stay capitalised for what they
are keeps proper nouns intact.
Verified live: all 8 titles came back in sentence case, including two that
were title-cased in the previous run.
Estimating the per-run bill was guesswork because thinking tokens bill as output and dominate the total — input is only about a tenth of it. The run now prints actual token counts and a dollar figure, so budget questions are answered by measurement instead of arithmetic. Prices are hardcoded and will drift; the token counts stay accurate regardless.
handlePlay called videoRef.current.play() immediately after setIsPlaying(true), but the <video> element is only rendered when isPlaying is true — so the ref was still null and the call threw. Awaiting a setState call does not wait for the re-render. The rejection also skipped setShowCover(false), so the cover stayed up and clicking play did nothing. Moved play() into an effect keyed on isPlaying, which runs after the element mounts, and restored the cover if the browser refuses playback so the reader gets the play button back instead of a dead frame. This is a site component, not the pipeline: it affects the existing comparing-gatsby-and-nextjs post too, which is the only other post using the poster path. Most posts use autoPlay and take a different branch, which is why this went unnoticed.
bestMp4 took the top of X's encoding ladder, so a post ended up serving a 3840x2160 file into a column that is 696px wide — and one clip offered 1600x1200 at 10,368 kbps next to 960x720 at 2,176. None of that resolution reaches the reader; it just costs them bandwidth and delays first frame. Now picks the smallest variant that still covers the column at 2x, falling back to the largest when every option is below that, and to bit rate when a url carries no dimensions. On the current month's videos that drops the 4K encode to 1920x1080 and leaves the already-reasonable ones untouched — 19% less bitrate overall.
X returns 403 for any request to video.twimg.com carrying a Referer from another domain. Browsers always send one and referrerPolicy is ignored on <video>, so the hotlinked src could never have worked from a real page — curl only succeeded because it sends no Referer, which masked this. Routes the mp4 through a /x-video/:path* rewrite in vercel.json, matching the /aval and /api external proxies already there. Vercel fetches server-side, so no browser Referer reaches X; verified that the origin returns 206 for a browser User-Agent with no Referer. Unverified until a preview deploy: whether Vercel forwards the client Referer upstream. If it does, this fails the same way and the fallback is a poster image linking to the post on X.
The Slack message read '6 new monthly blog drafts ready for review: undefined'. openDraftPr became async when the PR-create retry was added, but run.js kept destructuring it synchronously, so prUrl was a property of a Promise. The PR still got created, which is why this survived a real run. The message now leads with the PR link, lists each draft title, and notes how many posts were skipped as duplicates. The preview URL is deliberately not included: Vercel posts it as a comment on the PR after the build, so it does not exist at the moment this message is sent.
Video is served from X through the /x-video rewrite. X returns 403 for any request carrying a Referer from another domain, and Vercel forwards the browser's Referer upstream — verified on a preview deploy, where the same URL returns 206 with no Referer and 403 with one. With no Referer sent there is none to forward, and the proxy returns video/mp4. Scoped to the blog post template rather than set site-wide: the homepage and other routes are unaffected. The cost is that outbound links from articles no longer carry attribution to pixelpoint.io.
Drafts carried no outbound links at all. X rewrites every link as an opaque t.co shortlink, and handed one of those the model cannot tell what it points at, so it drops it. Requesting entities and substituting expanded_url gives real destinations — 46 of 51 recent posts now carry one. Announcements are often made by quoting someone else's post, which left the pipeline blind to both the link and the screenshot being pointed at: the Novu homepage post shipped with no media and no link for exactly this reason. referenced_tweets.id pulls the quoted post in; its media is used when the announcing post has none of its own, and its text is passed as context only — the draft prompt says to write about our work, not about the other person's post.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-generated monthly Updates draft(s). Review the Vercel preview(s) before merging.
Before merging