Feature/styling emails resume

.vscode/settings.json

@@ -211,6 +211,7 @@
     "miscategorized",
     "misconfiguring",
     "Misrouted",
+    "mjml",
     "mlflow",
     "moloco",
     "Moodle",

@types/mjml-template.d.ts

@@ -0,0 +1,4 @@
+declare module '*.mjml?raw' {
+  const content: string
+  export default content
+}

_TODO.md

@@ -31,4 +31,8 @@ https://aws.plainenglish.io/how-to-build-a-chatbot-using-aws-lex-and-lambda-in-2
 
 - `0/2000` characters should show number of characters left instead
 
-## Content Issues
+## Newsletter / MJML Templates
+
+- We need to make sure the images point to the full production URL, not a relative import
+- Need to move the unsubscribe link into an Action and handle it entirely within our website instead of on Hubspot
+- Need to add a newsletter publishing workflow as an action, using the newsletter static segment imported from Hubspot

docs/newsletter/example-issue-copy.md

@@ -0,0 +1,120 @@
+# The Kubernetes DNS Bug That Wasted 40 Engineering Hours
+
+*Monthly dispatch from Webstack Builders — platform engineering, DevOps, and cloud infrastructure*
+
+---
+
+## The Deep Dive — Debugging ndots and Why Your DNS Is Slower Than You Think
+
+Three weeks ago, a client's platform team started seeing intermittent 5xx errors from a handful of microservices. Latency percentiles looked normal. CPU and memory were fine. The on-call engineer checked the usual suspects — upstream dependencies, recent deploys, connection pool exhaustion — and found nothing.
+
+Forty engineering hours later, the root cause turned out to be DNS.
+
+**The symptoms were misleading.** Under moderate load, roughly 2% of HTTP requests to internal services would time out. The timeouts were evenly distributed across services, which made it look like a network issue rather than a resolution issue. The team spent a full day chasing a phantom connectivity problem between nodes before someone finally ran `tcpdump` on a pod's network namespace and noticed something odd: every single DNS lookup was generating five queries instead of one.
+
+**The culprit: `ndots:5`.** Kubernetes sets `ndots:5` in every pod's `/etc/resolv.conf` by default. This means that any hostname with fewer than five dots gets treated as a relative name, and the resolver appends each search domain before trying the name as-is. A lookup for `auth-service.production.svc.cluster.local` (four dots) would first try `auth-service.production.svc.cluster.local.production.svc.cluster.local`, then three more permutations, before finally resolving correctly on the fifth attempt.
+
+Under normal load, this is invisible. Under sustained traffic, it was multiplying DNS query volume by 5x and saturating CoreDNS pods that were sized for the expected query rate — not five times the expected query rate.
+
+**What made this hard to find.** The standard CoreDNS metrics — `coredns_dns_requests_total` and `coredns_dns_responses_total` — were elevated but didn't trigger alerts because the team's thresholds were based on historical averages that had gradually crept up. The metrics told the truth; nobody was looking at the right graph.
+
+**The fix was two lines.** In the pod spec's `dnsConfig`:
+
+```yaml
+dnsConfig:
+  options:
+    - name: ndots
+      value: "2"
+```
+
+This tells the resolver to treat any name with two or more dots as fully qualified, skipping the search domain dance. For internal service names that use the full `<service>.<namespace>.svc.cluster.local` format, this eliminates four unnecessary queries per lookup.
+
+The team also added explicit search domain entries to avoid breaking short names used in legacy configuration:
+
+```yaml
+dnsConfig:
+  searches:
+    - production.svc.cluster.local
+    - svc.cluster.local
+```
+
+**What should have caught this earlier.** Two monitoring changes went in immediately after the fix:
+
+- A Prometheus alert on `coredns_dns_requests_total` rate-of-change, not just absolute value. A 5x query spike in an hour is never normal.
+- A dashboard panel showing cache hit ratio alongside query volume. During the incident, cache hit rate had dropped to 31% — a clear signal that pods were hammering CoreDNS with queries that could never be cached because they were for nonexistent names.
+
+The takeaway isn't "change your ndots setting." It's that default configurations optimized for convenience can become performance landmines at scale, and the monitoring that catches them is rarely the monitoring you set up on day one.
+
+---
+
+## Quick Wins
+
+- **Terraform state backup before every apply.** Add this to your CI pipeline or local workflow. One line, zero regret when someone applies against the wrong workspace:
+
+  ```bash
+  terraform state pull > "tfstate-backup-$(date +%Y%m%d-%H%M%S).json" && terraform apply
+  ```
+
+- **Find abandoned Grafana dashboards.** This PromQL query surfaces dashboards with zero views in the last 90 days. Clean them out before your Grafana instance becomes a graveyard of dashboards nobody trusts:
+
+  ```text
+  grafana_db_dashboard_last_viewed_at < (time() - 86400 * 90)
+  ```
+
+  Run it against your Grafana metrics endpoint, or use the Grafana API: `GET /api/search?query=&sort=viewed-asc` and filter by `meta.lastViewedAt`.
+
+- **Pre-commit secrets scanning that works with monorepos.** Most `gitleaks` setups choke on monorepos because they scan the entire history on every commit. This `.pre-commit-config.yaml` entry scans only staged changes:
+
+  ```yaml
+  - repo: https://github.com/gitleaks/gitleaks
+    rev: v8.18.0
+    hooks:
+      - id: gitleaks
+        args: ["protect", "--staged"]
+  ```
+
+---
+
+## From the Blog
+
+Our latest article covers **structured logging with correlation IDs** — how to thread a single request identifier through every service in a call chain so that debugging distributed failures doesn't require cross-referencing timestamps across six different log streams.
+
+If that DNS investigation above had used correlation IDs, the team could have traced a single failing request from the API gateway through to the DNS timeout in minutes instead of hours. The post walks through implementation patterns for Node.js and Go services, with examples using OpenTelemetry's trace context propagation.
+
+[Read the full article →](#)
+
+---
+
+## What I'm Reading
+
+- **Cloudflare's routing incident post-mortem** — A BGP misconfiguration took down a significant chunk of their network for 17 minutes. The post-mortem is worth reading for the timeline alone: how a change that passed validation in staging behaved differently in production because of a subtle difference in route map evaluation order. [Read it →](#)
+
+- **OpenTelemetry Collector tail-sampling processor** — The new tail-sampling processor lets you make sampling decisions after a trace is complete, which means you can keep 100% of error traces and slow traces while sampling routine ones aggressively. If you're spending too much on trace storage, this is the feature to evaluate. [Read it →](#)
+
+- **Google SRE: A practical guide to SLO-based alerting** — Moves past the theory and into implementation. The section on multi-window, multi-burn-rate alerts is the clearest explanation of the concept available. If your alerts still fire on static thresholds, start here. [Read it →](#)
+
+---
+
+## One Thing to Try This Month
+
+Check your CoreDNS cache hit rate. If it's below 80%, you're probably hammering upstream resolvers unnecessarily — and you might be one traffic spike away from the exact scenario described in this issue's deep dive.
+
+Here's the Prometheus query:
+
+```text
+sum(rate(coredns_cache_hits_total[5m])) /
+(sum(rate(coredns_cache_hits_total[5m])) + sum(rate(coredns_cache_misses_total[5m])))
+```
+
+If the number is low, two things to check: your `ndots` setting (see above) and your CoreDNS Corefile's cache TTL. The default cache block caches positive responses for 30 seconds, which is usually too short for internal service names that rarely change. Bumping it to 300 seconds is safe for most clusters:
+
+```text
+cache 300
+```
+
+Add that line to the `Corefile` ConfigMap and roll the CoreDNS pods. Measure again the next day.
+
+---
+
+*Webstack Builders, Inc. — You're receiving this because you subscribed at webstackbuilders.com.*
+

docs/newsletter/example-issue-outline.md

@@ -0,0 +1,27 @@
+## Sample Issue Outline
+
+**Subject Line**: "The Kubernetes DNS Bug That Wasted 40 Engineering Hours"
+
+### 1 - Deep Dive: Debugging ndots and Why Your DNS Is Slower Than You Think
+
+Walk through a real scenario where the default Kubernetes `ndots:5` setting caused cascading DNS lookup failures under load. Cover the investigation process, the misleading metrics, the actual fix (adjusting ndots + adding search domain entries), and the monitoring changes that would have caught it earlier.
+
+### 2 - Quick Wins
+
+- **Terraform state backup**: A one-liner that snapshots your state file before every apply
+- **Grafana dashboard hygiene**: A PromQL query to find dashboards nobody has viewed in 90 days
+- **Git hook for secrets**: A pre-commit hook pattern using gitleaks that actually works with monorepos
+
+### 3 - From the Blog
+
+Highlight the latest article on structured logging with correlation IDs. Tie it back to the DNS debugging story (correlation IDs would have made the investigation faster).
+
+### 4 - What We're Reading
+
+- Cloudflare's post-mortem on their recent routing incident
+- The OpenTelemetry Collector's new tail-sampling processor and what it means for costs
+- A practical guide to SLO-based alerting from Google's SRE team
+
+### 5 - One Thing to Try This Month
+
+"Check your CoreDNS cache hit rate. If it's below 80%, you're probably hammering upstream resolvers unnecessarily. Here's the Prometheus query to check, and the CoreDNS Corefile change to fix it."

docs/newsletter/ideas.md

@@ -0,0 +1,29 @@
+# Recurring Content Ideas by Topic Area
+
+## Platform Engineering
+
+- Internal developer portal patterns (Backstage, custom solutions)
+- Golden paths vs. guardrails: finding the right balance
+- Platform team metrics that actually matter
+- Self-service infrastructure: what works, what doesn't
+
+## DevOps and CI/CD
+
+- Pipeline optimization techniques with real numbers
+- Monorepo vs. polyrepo build strategies
+- Feature flag infrastructure at scale
+- Deployment strategies beyond blue/green
+
+## SRE and Reliability
+
+- Error budget policies that teams actually follow
+- On-call rotation design for small teams
+- Chaos engineering on a budget
+- Incident response process improvements
+
+## Cloud Engineering
+
+- Multi-cloud networking pitfalls
+- Cost optimization strategies with measurable results
+- Infrastructure-as-code patterns and anti-patterns
+- Cloud provider feature comparisons (without vendor bias)

docs/newsletter/infrastructure.md

@@ -0,0 +1,35 @@
+# Newsletter Infrastructure
+
+You can access the list of names in the Newsletter segment and their email addresses via the HubSpot API on a **Starter plan**. However, there is a critical update to keep in mind: HubSpot has fully **deprecated legacy API keys** in favor of **Private Apps**.
+
+To retrieve your newsletter segment (static list), you will need to follow these steps:
+
+1. Create a Private App
+
+Instead of an API key, you must create a Private App within your HubSpot portal to generate an **Access Token**.
+
+- Navigate to **Settings > Integrations > Private Apps**.
+- Create a new app and select the necessary **Scopes** (permissions). For your goal, you will at minimum need `crm.lists.read` and `crm.objects.contacts.read`.
+- HubSpot will provide an **Access Token** that you will use in your API requests as a Bearer token.
+
+## Retrieve List Memberships
+
+Because you are using a "Newsletter" segment (list), the process is typically a two-step API call:
+
+1. **Get Member IDs**: Use the Lists API to get a list of all contact IDs belonging to your specific static list.
+
+   - **Endpoint:** `GET /crm/v3/lists/{listId}/memberships`
+
+2. **Fetch Contact Details**: Once you have the IDs, use the Contacts API to pull the actual names and email addresses for those specific IDs.
+
+   - **Endpoint:** `POST /crm/v3/objects/contacts/batch/read`
+
+3. Starter Plan Limitations to Watch
+
+- **Rate Limits:** On the Starter plan, you are limited to **100 requests per 10 seconds**.
+- **Daily Limit:** You have a generous **250,000 API calls per day**.
+- **Pagination:** If your newsletter list is large, the API will return results in "batches" (usually 100-250 at a time). You will need to check the `paging.next.after` value in the response to fetch the next set of contacts.
+
+**✅ Result Summary**
+
+You can use the **HubSpot API** on a **Starter plan** to programmatically extract your newsletter list members. You simply need to swap your old API key for a **Private App Access Token** and use the **Lists and Contacts API endpoints** to gather the names and emails.

docs/newsletter/newsletter-structure.md

@@ -0,0 +1,71 @@
+# Newsletter Content Plan
+
+## Overview
+
+The Webstack Builders newsletter is a monthly dispatch covering platform engineering, DevOps, SRE, and cloud engineering topics. Each issue delivers practical, opinionated content aimed at engineers and engineering leaders who build and operate production infrastructure.
+
+**Cadence**: Monthly (first Tuesday of each month)
+
+**Audience**: Platform engineers, DevOps practitioners, SREs, cloud architects, and engineering managers
+
+**Tone**: Direct, practical, opinionated. No fluff, no hype cycles. Real problems, real solutions.
+
+## Newsletter Structure
+
+Each issue follows a consistent format with five sections. Target length is 1,200-1,500 words (5-7 minute read).
+
+### Section 1 - The Deep Dive (500-600 words)
+
+The anchor piece. A single topic explored with enough depth to be immediately useful.
+
+Pick from recurring themes:
+
+- **Incident Teardowns**: Anonymized analysis of real production incidents. What broke, why the monitoring missed it, what the fix looked like, and what systemic changes prevented recurrence.
+- **Architecture Decisions**: A specific infrastructure choice (e.g., switching from polling to event-driven, adopting a service mesh, choosing a secrets manager) with the tradeoffs laid out honestly.
+- **Tool Deep Dives**: Hands-on evaluation of a specific tool or approach. Not a product review, but a "here's what it's actually like to operate this in production" piece.
+- **Process Improvements**: How to improve a specific engineering process (on-call rotations, incident response, deployment pipelines) with concrete before/after examples.
+
+**Example topics**:
+
+- "Why We Stopped Using Helm for Everything (and What Replaced It)"
+- "The Three Kubernetes Monitoring Gaps That Bit Us Last Quarter"
+- "Migrating from Jenkins to GitHub Actions: The Parts Nobody Warns You About"
+- "What Actually Happens When You Set CPU Limits Too Low"
+
+### Section 2 - Quick Wins (200-250 words)
+
+Three to four bite-sized, immediately actionable tips. Each one should be something a reader can implement the same day.
+
+**Format**: Brief description + code snippet or config example where relevant.
+
+**Example entries**:
+
+- A Prometheus query that catches container OOM kills before they cascade
+- A Terraform module pattern that eliminates drift in multi-environment setups
+- A GitHub Actions workflow optimization that cut CI time by 40%
+- A kubectl command chain for debugging intermittent pod failures
+
+### Section 3 - From the Blog (100-150 words)
+
+Highlight one or two recently published articles or downloadable guides from the Webstack Builders site. Brief context on why it matters and a direct link.
+
+### Section 4 - What We're Reading (150-200 words)
+
+Three to four curated links to notable content from across the industry. Prioritize:
+
+- Post-mortems and incident reports from companies willing to share
+- Technical blog posts with real implementation details
+- RFCs or proposals for emerging standards (OpenTelemetry, Gateway API, etc.)
+- Conference talks worth the time investment
+
+Each link gets a one-sentence annotation explaining why it's worth reading.
+
+### Section 5 - One Thing to Try This Month (100-150 words)
+
+A single, specific challenge or experiment for the reader. Something they can do in their own environment to improve reliability, observability, or developer experience.
+
+**Examples**:
+
+- "Run a game day: Pick your most critical service and simulate its primary database going read-only. How long until your team notices? How long until they recover?"
+- "Audit your alert rules: Count how many fired in the last 30 days. How many were actionable? Delete the ones that nobody acted on."
+- "Measure your deployment lead time: From merged PR to production traffic. If you can't measure it, that's the first thing to fix."