Nimbus Vestige (NV)

A forensic reconstruction engine for cloud and identity incident response.

Given fragmented cloud, SaaS, and identity telemetry — control-plane logs, sign-in events, token and consent activity — NV reconstructs how an intrusion most likely moved across accounts and services, enumerates the other most-likely paths it could have taken, and reports every step with calibrated, explainable confidence. It refuses to assert what the evidence cannot support.

Detection tells you that something happened. Nimbus Vestige tells you how — and what else.

Why this exists

Modern intrusions do not "break in." They log in. Identity is now the primary attack vehicle, implicated in the large majority of cloud IR investigations, and most intrusions span multiple surfaces — identity plus cloud plus SaaS plus endpoint. The telemetry that records them is fragmented and inconsistent, which already forces responders to reconstruct the story by hand from incomplete data.

That manual reconstruction is slow, and it fails in a predictable way: the responder anchors on the first plausible narrative and misses the real one. NV automates the reconstruction and directly attacks that failure mode by always presenting the ranked space of plausible paths, not a single story.

Crucially, NV does this with honesty as the product. The market's stated enemy is black-box confidence — a tool that asserts a conclusion without showing its work. Every number NV produces is traceable to the specific evidence that earned it, and anything it cannot support is withheld rather than guessed.

What it is — and is not

NV is not a detector, a scanner, an auditor, or an attack-runner. Those tools tell you that something happened and measure it. NV works backward — abductive reconstruction of the mechanism from patterns, across a fragmented identity estate.

• Not a detector — it doesn't fire alerts on activity; it explains how activity fits together.
• Not a SIEM — it enters through a narrow wedge (post-incident narrative reconstruction), not as a log platform.
• Not a rule engine — when nothing matches a known pattern, it still reconstructs, degrading gracefully instead of going blind.

Why it is valid long-term

1. Blue team regenerates; red team commoditizes. Offensive tooling finds a finite, patchable set of holes and is folding into automated CI/CD safety pipelines. Reconstructing how an intrusion happened never resolves — attackers keep inventing, so the need is permanent and self-renewing.

2. The engine is substrate-independent. The core logic — reconstruct the mechanism from patterns, with calibrated confidence and a stop rail — is committed to cloud/identity first, but ports to network, endpoint, and OT later. The target can change without rewriting the thesis.

3. Reconstruction enables hardening. Once you know how they got in — and which other doors were open — you build the defenses. The untaken-but-plausible paths are a hardening backlog, often worth more than the reconstruction itself, because most breaches exploit preventable exposure, not novel tradecraft.

4. It degrades gracefully on the novel intrusion — the very incident that matters most. A signature/rule engine goes blind on a zero-day; NV's abductive core still produces a most-likely path, honestly flagged as lower-confidence.

5. Honesty is a moat. Calibrated, case-based confidence with ranked alternatives is exactly what the market says it wants and what black-box competitors structurally cannot offer without redesign.

Architecture

Raw provider logs → normalized event/entity graph → reconstruction engine → JSON → GUI.

  O365 / Entra audit logs      nv_extract_identity_events.py
  AWS CloudTrail (IAM/STS/S3)   nv_extract_cloudtrail_events.py   (ingest + normalize)
          ▼
  normalized identity events (JSONL)  ← one event model, any substrate
          │  nv/graph.py                      (typed per-actor timelines)
          ▼
  reconstruction engine
     ├─ nv/patterns.py     known layer: ATT&CK identity pattern library
     ├─ nv/providers.py    provider packs: per-substrate op→ATT&CK vocab (Entra + AWS)
     ├─ nv/validation.py   Phase 3: provenance + integrity gate on every pattern
     ├─ nv/feeds.py        live ATT&CK STIX / TAXII / Sigma clients + scheduler
     ├─ nv/confidence.py   evidence-corroboration scorer (calibratable weights)
     ├─ nv/calibration.py  fit + measure confidence against labeled ground truth
     ├─ nv/reconstruct.py  most-likely chain + ranked competing paths + trust floor
     └─ nv/scope.py        authorized-scope gate (refuses unauthorized tenants/accounts)
          │  run_nv.py
          ▼
  reconstruction.json  ──►  nv_gui.html  (embedding canvas + two-column view + trust slider)

The GUI is a pure view layer — it renders engine output and lets the analyst move the trust floor. It contains no reconstruction logic of its own.

The confidence model (the credibility of the whole product)

Every step carries a confidence in [0, 1], built by evidence corroboration:

confidence = per-technique base rate
           + bonus for each independent corroborating signal
                 (source IP, device, successful outcome, temporal adjacency, broad-consent flags)
           − penalty for missing signals (e.g. no source IP to corroborate origin)

• verified — confidence ≥ 0.70 and two or more independent signals agree.
• pattern-only — above the trust floor but weakly or singly supported.
• withheld — below the trust floor; shown greyed, never silently guessed or hidden.

Path scores compose their links by geometric mean, so one weak link honestly drags a path down rather than being averaged away.

Every weight above — the per-technique base rates, the per-signal bonuses, the missing penalties — lives in a single PARAMS table in nv/confidence.py. Those are NV's v1 priors, and they are calibratable: nv/calibration.py can refit them against labeled ground truth and the engine will load the result, falling back to the v1 priors when no calibration is supplied (see below).

The trust floor

The withhold rule, baked into the engine's output contract — not just the UI. Below the floor, a step is withheld. Dragging the floor is the honesty-versus-coverage tradeoff made physical: up for strict/high-trust, down for permissive/high-coverage. The default floor is an editorial decision about where NV sits before the analyst touches it.

Calibrating confidence against ground truth

The v1 weights are defensible, but they are priors. nv/calibration.py tunes them against labeled ground truth — events where we know which ones were part of the intrusion and which were benign — and, crucially, measures whether tuning actually helped, so a calibration is adopted only if it lowers calibration error rather than just moving numbers around.

Calibrated confidence means one thing: NV's confidence for a step should equal the probability that the step really was on the attack path. So each labeled event's confidence is treated as a predicted probability, and the harness fits:

• per-operation base rates — the empirical hit rate for that operation, Bayesian- shrunk toward the v1 prior so small samples don't overfit; and
• per-signal bonuses and the two penalties — by bounded coordinate descent that minimizes Brier score with an L2 pull back toward the v1 values.

The verified/withheld thresholds are policy, not calibration, and are left untouched.

It reports Brier score, log-loss, ECE, AUC, a reliability table, and a trust-floor sweep (true-step recall vs benign false-positive rate at each floor), before and after — so the honesty-versus-coverage tradeoff is legible rather than a single number.

# calibrate against a real, labeled BadZure / MAAD-AF run
python3 calibrate.py --labeled run_labeled.jsonl --origin live \
    --run-label "badzure-2026-07 tenant-x" --out calibration.json

# demonstrate on the bundled documented-shape proxy corpus
python3 calibrate.py --out calibration.json

# run the engine on calibrated weights (opt-in; v1 priors are the default)
python3 run_nv.py events.jsonl out.json --authorize t.onmicrosoft.com \
    --calibration calibration.json

Ground truth source. The honest input is telemetry from a BadZure / MAAD-AF run in a real tenant, labeled by joining the unified audit log against the attack tool's own activity record (every event the tool caused is on-chain; everything else is benign — see LABELED_SCHEMA in nv/calibration.py). Until you point it at a real run, eval/calibration_cases.py supplies a documented-shape proxy corpus, and every artifact fitted from it is stamped source="proxy" in its provenance so a proxy fit can never be mistaken for a live one. calibration.proxy.json is that bundled proxy artifact.

On the bundled proxy corpus the fit is a clear improvement — Brier 0.398 → 0.045, ECE 0.576 → 0.081, AUC 0.81 → 0.91, and the benign false-positive rate at the 0.60 floor collapses from 0.69 to 0.00 (the v1 priors were badly over-confident on benign activity). The proxy is deliberately conservative; a live-tenant run is what produces weights you should actually deploy.

Trust integrity (Phase 3)

Two first-class controls protect the reconstruction from bad inputs:

• Authorized-scope gate (nv/scope.py) — NV refuses to run on any estate not on the authorized list. Run with --authorize <tenant-domain> per estate you are permitted to investigate.

• Pattern-source validation (nv/validation.py) — no pattern enters the library without passing: (1) trusted-source allowlist, (2) content checksum / tamper check, (3) schema well-formedness, (4) valid ATT&CK-style technique id. Every admitted pattern keeps an audit record; rejects are logged with a reason. A poisoned or malformed feed is stopped upstream before it can manufacture a false reconstruction. This is the same skepticism the engine applies to evidence, moved to the patterns themselves.

Keeping the knowledge current (staying ahead of attacker evolution)

A reconstruction engine is only as current as its pattern library plus its ability to handle what the library has never seen. NV addresses staleness on two independent fronts, by design:

1. The known layer stays fresh through a validated update pipeline

The library is not hardcoded — nv/patterns.py loads every pattern (including the builtin set) through nv/validation.py, so external feeds enter by the same trusted path. The live network clients that pull these feeds on a schedule are implemented in nv/feeds.py and driven by update_feeds.py. Sources, in trust tiers:

• MITRE ATT&CK (STIX) — the authoritative technique taxonomy. The connector reads the ATT&CK STIX index, pulls the latest enterprise release, and refreshes the authoritative technique name and tactic for every operation NV reasons over — dropping any whose technique ATT&CK has since revoked or deprecated. NV keeps ownership of the operation→technique binding and the base-rate prior; ATT&CK owns the taxonomy.
• Vetted CTI via TAXII 2.1 — a full discovery → api-root → collection → objects client. Pulls STIX attack-pattern objects from a vetted CTI collection and refreshes the techniques NV tracks. Weighted below ATT&CK.
• Sigma community ruleset — pulled as a git archive; each cloud/identity rule that names an O365/Entra operation and carries an attack.tXXXX tag becomes an operation→technique candidate, with a base rate derived from the rule's own severity level and then scaled down by the community source weight. Rules that don't map are skipped with a reason, never guessed.

On an operation collision the union is ordered by ascending trust before commit, so an authoritative ATT&CK binding always wins over a community one; lower-tier content is still admitted and recorded in the audit as corroboration. update_library(candidates) re-ingests the whole set through validation and rebuilds the library atomically, so a poisoned feed can never leave the library half-updated.

Two allowlists, not one. Validation already gates on the source tag; the connectors add a network-layer host allowlist, so a connector can only fetch from a trusted host over TLS — the transport equivalent of the source allowlist, and a guard against a hijacked or mistyped feed URL. Every fetch records the raw-payload sha256, URL and time as provenance, so a later re-pull can detect upstream mutation.

Why the validation layer matters more as the feeds grow: the more you automate updates, the more an auto-update pipeline becomes a trust risk in disguise — a wrong or poisoned feed injects false patterns and manufactures false reconstructions. NV treats ingestion as adversarial: allowlist, checksum, schema, and audit trail on every pattern, every update.

2. The abductive layer covers what no feed has yet named

Feeds always lag the newest tradecraft. The abductive core is the hedge: when an observed operation matches no library pattern, NV does not drop it — it reconstructs the most-likely path from first principles and flags it no known match at reduced confidence. This is validated in the eval suite (eval/ground_truth_cases.py), where a deliberately-unknown managed-identity token-theft step is caught, sequenced correctly, and honestly down-weighted below the trust floor.

Together these mean NV never fully goes stale: the known layer tracks the published frontier through a poisoning-resistant pipeline, and the abductive layer keeps the tool from going blind on the intrusion the frontier hasn't caught up to yet.

Suggested update cadence

nv/feeds.py carries a per-feed cadence and update_feeds.py only pulls what is due, so it can be dropped straight into cron or a systemd timer:

• ATT&CK STIX — 90 days (a few official releases a year).
• CTI/TAXII — 7 days (as the feed publishes), always through validation.
• Sigma — 30 days.

# run whatever is due, keeping state under ./.nv_feeds  (cron-friendly)
python3 update_feeds.py

# force all feeds now but only report what would change
python3 update_feeds.py --force --dry-run

# run fully offline against captured fixtures (no network)
python3 update_feeds.py --offline eval/fixtures --force

After any library change, re-run eval/run_eval.py to confirm known attack shapes still reconstruct, eval/validation_cases.py to confirm the gate still rejects bad input, and eval/feeds_offline_test.py to confirm the feed path is intact end to end.

A note on the demo data

The reconstruction shown in nv_gui.html is built from a public research dataset — the invictus-ir Office 365 unified audit log corpus — not from any company's live tenant. It exists so the engine can be demonstrated and dogfooded end-to-end without anyone's cooperation. To use NV for real, an organization wires in its own Entra/M365 audit data, as described below. No proprietary or customer data is included anywhere in this project.

Connecting your own data

NV runs wherever you run it; your logs never have to leave your environment. Four steps:

1 — Export your audit logs. NV reads the Microsoft 365 unified audit log. Source it from Microsoft Purview (Audit search → export CSV), the Search-UnifiedAuditLog Exchange Online PowerShell cmdlet, the Microsoft Graph auditLogs / signIns endpoints, or a Sentinel/SIEM export of OfficeActivity and SigninLogs.

2 — Normalize it into the event model NV reasons over:

python3 nv_extract_identity_events.py your_audit_export.csv your_events.jsonl

This keeps sign-ins, consent grants, service-principal and role changes, and mailbox access — including identity operations NV has never named, so they still reach the abductive layer — and drops the rest. It reads a CSV carrying the standard AuditData column, or JSON/JSONL where each record is an AuditData object (the shape the invictus-ir research corpus ships in).

3 — Reconstruct, scope-gated. The engine refuses to run on a tenant you haven't explicitly authorized:

python3 run_nv.py your_events.jsonl reconstruction.json \
    --authorize yourtenant.onmicrosoft.com

4 — View. Open nv_gui.html and click ⤒ load reconstruction.json to point it at the file the engine just produced — the same screen then shows your incident, your entities, and your confidence bands. Every dot on the embedding canvas re-runs the reconstruction for that entity on click.

Analyzing AWS CloudTrail instead

The engine is substrate-independent — only the operation vocabulary is provider-specific (nv/providers.py). The AWS path is the same three steps against CloudTrail:

python3 nv_extract_cloudtrail_events.py cloudtrail.json aws_events.jsonl
python3 run_nv.py aws_events.jsonl reconstruction.json --authorize aws:123456789012

The ingest keeps IAM/STS/sign-in events (novel IAM operations included, so they reach the abductive layer) plus S3 object reads, and scope-gates on the AWS account rather than a tenant domain. The same abductive core, confidence model, and trust floor apply unchanged.

The GUI also carries a demo-data banner and an in-app "Connecting your own data" guide, so anyone opening it understands the reconstruction is public sample data until they wire in their own.

Project layout

run_nv.py                        reconstruction engine entrypoint (scope-gated)
nv_extract_identity_events.py    ingest: M365/Entra unified audit log -> event model
nv_extract_cloudtrail_events.py  ingest: AWS CloudTrail -> event model      [iteration 3]
update_feeds.py     pull + validate threat-intel feeds on a cadence   [iteration 1]
calibrate.py        fit confidence weights from labeled ground truth  [iteration 2]
nv/                 the engine package
  graph.py  patterns.py  validation.py  confidence.py  reconstruct.py  scope.py
  providers.py      per-substrate op->ATT&CK vocabulary packs (Entra + AWS) [iteration 3]
  feeds.py          ATT&CK STIX / TAXII 2.1 / Sigma clients + scheduler [iteration 1]
  calibration.py    metrics + parameter fit                            [iteration 2]
eval/               evals + offline fixtures (no network, no tenant)
nv_gui.html         pure view layer (loads engine reconstruction.json)
calibration.proxy.json   bundled proxy calibration artifact           [iteration 2]

Run everything from the project root so the nv package is importable.

Running it

# 1. normalize raw O365/Entra audit logs into the event model
python3 nv_extract_identity_events.py auditrecords.csv nv_identity_events.jsonl

# 2. reconstruct (scope-gated; add --calibration calibration.json to use fitted weights)
python3 run_nv.py nv_identity_events.jsonl reconstruction.json \
    --trust-floor 0.6 --authorize your-tenant.onmicrosoft.com

# 3. view — open nv_gui.html (renders reconstruction.json)

Keep the pattern library current and tune confidence:

python3 update_feeds.py                       # pull + validate feeds that are due
python3 calibrate.py --labeled run.jsonl --origin live --out calibration.json

Evals

python3 eval/run_eval.py            # attack shapes reconstruct correctly
python3 eval/validation_cases.py    # poisoned patterns are rejected
python3 eval/feeds_offline_test.py  # feed connectors + scheduler (offline)
python3 eval/providers_aws_test.py  # AWS chain reconstructs through the same engine
python3 calibrate.py                # calibration harness on the bundled proxy corpus

Status

Working end-to-end on real data (invictus-ir O365 dataset), across the full plan:

Honest known gaps (next iterations)

• Live-tenant calibration numbers. The calibration harness is complete and the interface is proven, but the shipped calibration.proxy.json is fitted on documented- shape proxy data. Deployable weights require running the harness against a real BadZure / MAAD-AF run — an operational step for the adopter.
• Feed-driven vocabulary growth. The connectors refresh the operations NV already reasons over; letting a feed introduce a new operation (and wire it into the initial/pivot/collection classification) is future work.
• Second-provider depth. AWS is in as a proof of substrate-independence (pack + ingest + eval), but only IAM/STS/S3. Widening the AWS pack, adding provider-specific feed connectors, and a third substrate (GCP, Okta) are the next steps.

Author

Doby Baxter 2026