Skip to content

Latest commit

 

History

History
215 lines (167 loc) · 10.1 KB

INTEGRATION.md

File metadata and controls

215 lines (167 loc) · 10.1 KB

Integrating with Atomdrift Scan

Atomdrift Scan reads a file and calls it benign, suspicious, or hostile. It works offline and gives the same answer every time. It makes no network calls unless you ask it to.

This guide is for wiring it into a pipeline: a registry gating uploads, a CI check, an ingestion pipeline.

Start in one minute

# install (Linux/macOS via Homebrew)
brew install atomdrift-project/tap/scan

# or build from source — needs Rust 1.97+
git clone https://github.com/atomdrift-project/scan.git && cd scan && make install

# scan a file, a directory, or an archive (archives unpack automatically)
atomscan suspicious_package.tar.gz
atomscan ./incoming/

A clean run prints a one-line summary and exits 0. A finding is printed inline and changes the exit code. For most integrations, that exit code is the whole interface.

To refresh the models before a scan, add -u (a failed update is non-fatal); the models also auto-refresh when the local ruleset is over 24h stale. Disable that automatic refresh with --no-update, and silence the once-a-day update notice with SCAN_NO_UPDATE_CHECK=1.

Choose a path

Start with the CLI; move to the server when volume demands it.

Path Use when Cost Reference
CLI One-shot or batch, up to ~5 scans/min. Reloads the model on every run. atomscan --help
HTTP server Sustained traffic past ~5 scans/min. You run a long-lived process. SERVER_API.md
Rust library You are already in Rust and want in-process calls. More setup, weaker API stability. See Library. cargo doc — scan::Analyzer

A registry gating uploads on ingest should start with the CLI and grow into the HTTP server.

Gate on the exit code

The CLI turns a verdict into an exit code, so a gate is one line:

atomscan ./package/ || echo "flagged — block this upload"
Code Meaning
0 All clean.
1 At least one hostile file.
2 At least one suspicious file, none hostile.
3 An error occurred, nothing hostile or suspicious.
4 The rule set was incomplete — see below.
130 Interrupted (Ctrl-C) before every file was analyzed.

The most severe verdict wins: one hostile file in a tree returns 1 no matter what else is in it. A typical policy blocks on 1, sends 2 to a human, and pages on 3.

4 means the YARA engine ran degraded: rule sources failed to compile, or scanning was disabled mid-run after an engine panic. The scan finished, but with fewer rules than the trait set defines, so a clean or merely suspicious result proves nothing — the detection that would have fired may simply not have been loaded. It outranks 2 and 3 for that reason, and 1 outranks it in turn: a hostile verdict still stands, because missing rules can only cost you detections, never invent one.

Treat 4 as "run it again", not as a verdict. Gate on it the way you gate on a failed build rather than a flagged artifact, and do not let it fall into a >= 2 bucket that a policy reads as "suspicious, needs a human" — nobody can review a scan that did not fully happen.

Set the false-positive level

A level is a false-positive allowance, measured on the benign files the model was calibrated against:

  • L0 flags only what fired on none of them.
  • Ln (n ≥ 1) allows one false positive plus an estimated n − 1 more per 100 million benign files. L25 is one plus 24 per 100 million.

Higher levels are more sensitive: they catch more and cry wolf more. The levels are calibration targets, not guarantees: L0 means no false positive was observed, not that none can happen.

atomscan -l 0 ./pkg/       # zero observed calibration FPs; strictest
atomscan ./pkg/            # bundle default, normally L25: one FP + 24/100M extra
atomscan -l 5000 ./pkg/    # any calibrated point on the 0–25000 grid
atomscan -l 25000 ./pkg/   # most sensitive — catches more, cries wolf more

Choose by where a mistake hurts. A hard upload gate wants a strict level (-l 0, -l 1) to reduce false alerts. A triage queue feeding human reviewers can run a high level like -l 25000 and let people sort it out.

Whichever you pick is fixed when the process starts. A running server picks up a new setting with POST /_/reload — no restart.

Output formats

-f is a global flag, so it comes before the path:

atomscan -f json ./pkg/    # NDJSON, one envelope per file, including archive members
atomscan -f tiny ./pkg/    # compact text, built to feed a local LLM
atomscan ./pkg/            # default: human-readable terminal output

JSON gives you one line per file whatever the verdict. Read ml.lvl for the result (-1 means not flagged) and ml.prob for the score. The field names are short on purpose; JSON.md is the key. The envelope is stable within a major version.

A second opinion: --llm

--llm sends non-trivial samples to a local LLM and blends its read into the verdict (kept in the llm JSON section). It talks to an OpenAI-compatible endpoint at http://localhost:8000/v1 by default. Nothing leaves your network.

atomscan --llm ./pkg/

No model name is baked in. Unless you pin one with --llm-model, atomscan asks the endpoint what it serves (GET /v1/models) and takes the largest model listed — so whatever you started the server with is what gets used. If the endpoint lists nothing, the scan stops with an error rather than guessing.

We recommend Qwen/Qwen3.8-27B under vLLM; a Qwen3-class model works well generally, and ~9B is enough if that is what fits.

Point it elsewhere with --llm and --llm-model. --llm openrouter uses https://openrouter.ai/api/v1; the key is SCAN_LLM_KEY or ~/.tok/openrouter, and --llm-model defaults to openrouter/auto, OpenRouter's own router — nothing else from its billed catalog is ever guessed.

An endpoint that requires a bearer token is authenticated without any flag: put the token in ~/.tok/llm (mode 0600, first non-empty line) and atomscan sends it on every request. SCAN_LLM_KEY overrides the file. ~/.tok/llm is never sent to OpenRouter, which takes ~/.tok/openrouter or nothing. Without a token the endpoint answers 401, which costs the second opinion but not the scan: the verdict falls back to ML alone and the reason appears in the llm section. A server running under a service account reads ~/.tok/llm from that account's home.

Failing over between endpoints

Comma-separate --llm to name a chain, tried in order. For example, the Atomdrift Uplink hosted endpoint with OpenRouter behind it:

atomscan --llm 'https://llm.isotope13.ai/v1,openrouter' ./project

The first endpoint that answers wins, so an outage on the first costs a retry rather than the second opinion, and the billed public API is only reached when the free one cannot answer — put the endpoint you would rather not pay for last. Any refusal moves to the next: unreachable, timeout, 5xx, or a 4xx such as the 401 from a host you hold no token for. The llm section names the model that actually graded, not the one configured first.

Each link resolves independently. --llm-model pairs with --llm positionally — a blank slot asks that endpoint what it serves, and a single name (no comma) applies to the whole chain — and each takes its own key, so the example above asks the Uplink endpoint what it serves, authenticating with ~/.tok/llm, and sends openrouter/auto to OpenRouter with ~/.tok/openrouter; adding --llm-model ',qwen/qwen3.8-27b' would pin that second slot instead. A link that cannot be used (no key) is dropped with a warning while the rest carry on; only when nothing is usable does the scan stop, and a lone endpoint with a problem is still an error rather than a silent no-op. Which samples qualify is not a flag: a sample is sent when ML flagged it at any calibrated level — the model's own per-route cutoff — or when cleave surfaced a suspicious/hostile finding, whichever comes first.

Following references: --follow (experimental)

--follow retrieves references discovered inside the requested artifact, analyzes them, and folds their results back into its verdict. Choose declared manifest/lockfile dependencies (dependencies), packages and URLs named by install/download commands (references), or third-party CI actions (ci-actions, which also implies dependencies). A bare --follow and the default select dependencies,references; all also includes CI actions. With --follow-depth it follows hops: a script that pulls a loader that runs a curl | bash dropper is caught as one chain.

atomscan --follow=dependencies ./pkg/             # manifests and lockfiles only
atomscan --follow --follow-depth 3 ./pkg/          # dependencies and references
atomscan --follow=all --follow-depth 3 ./pkg/      # include CI actions too

This is the only analysis phase that makes additional network requests. Use --follow=none for air-gapped or transparent-proxy gates that already know which artifacts will be requested. Set the default per deployment with SCAN_FOLLOW instead of the flag if that suits you better.

Library

The in-process Rust API is the most direct path and the least stable. Until 1.0, minor releases may break it, so pin a commit of github.com/atomdrift-project/scan. Start with scan::Analyzer: load it once from a model bundle, then call scan_bytes or scan_file. Call scan::runtime::install_rayon_pool() at startup, so analysis runs on a pool with the deep stacks nested archives need. Results serialize to the same JSON envelope as every other path.