Skip to content

[bug] appendProductMdSource silently skips MD manifest update on missing or malformed frontmatter #30

Description

@kiannidev

Duplicate check

Item Relation
#12 (open) Not duplicate — scoped source storage layout (line/category vs product dir)
No open issue Silent MD manifest sync failure on approve/upload

Summary

After a successful PDF write and sources.yaml update, appendProductMdSource() updates the product MD sources: frontmatter list. If the MD file is missing or lacks valid --- frontmatter, the function returns without error. Callers (manualUpload, approve flow) still return { ok: true }, leaving sources.yaml and DB in sync but the canonical MD manifest stale.

Per CONTRIBUTING, the MD manifest is load-bearing for extraction — silent drift breaks downstream pipeline steps.


Steps to reproduce

  1. Corrupt or delete a product's frontmatter block in {slug}.md (keep file, remove --- delimiters).

  2. Upload a PDF:

    curl -sS -X POST http://localhost:3210/api/pipeline/sources/upload \
      -H 'Content-Type: application/json' \
      --data '{ ... valid upload body ... }'
  3. Response is success; sources.yaml has the new entry; product MD sources: list is unchanged — no error surfaced.

Silent early returns:

function appendProductMdSource(input: AppendMdSourceInput): void {
  if (!fs.existsSync(input.mdPath)) {
    return;
  }
  const fmMatch = raw.match(/^---\n([\s\S]*?)\n---\n?([\s\S]*)$/);
  if (!fmMatch) return;

Expected behavior

Missing MD or malformed frontmatter surfaces an error to the caller (4xx/5xx or { ok: false, error: "..." }). Upload/approve should not report full success when MD manifest was not updated.


Actual behavior

Best-effort no-op; API returns success.


Suggested fix

  1. Change appendProductMdSource to return Result<void> or throw on missing MD / bad frontmatter.
  2. Propagate error from manualUpload and approve callers.
  3. Add unit test: corrupt frontmatter → caller gets error, sources.yaml optionally rolled back.

PR scope

~30–50 lines + one test.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions