Guides

Checking a package as it is really installed: the whole tree, every install step, the first import, and a public log

Kenwea Protocol,

Until this week the notary checked a package on its own: its own install steps, with none of its dependencies present. That made two kinds of answer worse than they should be. Packages that need a dependency to finish their install ended unfinished, and packages that run nothing themselves but pull in a dependency that does were reported as running nothing. Both are fixed, and so is a third problem we found on the way, described plainly below.

npx -y @kenwea/mcp check swagger-ui-express
npx -y @kenwea/mcp check pypi:requests
npx -y @kenwea/mcp@0.6.0 check-tree        # package-lock.json, pnpm-lock.yaml or yarn.lock

The whole tree, and every install step in it

For an npm package the tree is resolved the way npm resolves it, every tarball is checked against the registry's integrity hash, and the tree is laid out in node_modules before anything runs. Then every install step in the tree runs, one event at a time as npm does it, dependencies before the packages that need them, each traced. A package that declares no install step of its own is still checked when one of its dependencies declares one.

  • swagger-ui-express runs nothing of its own at install. Installed with its 67 dependencies, @scarf/scarf's postinstall runs and looks up scarf.sh to send install telemetry. The record now says so, and names the step.
  • esbuild, @swc/core, nx, puppeteer and the node-gyp-build family, which ended unfinished when checked alone, now run to completion.
  • cypress's postinstall opens ~/.npmrc, where a developer machine keeps its npm token (ours held a decoy). Some tools read it for registry settings, so the record calls it a reason to look, not proof.

The first import

Code does not have to run at install to run on your machine; it can wait for the first require. After a successful install the package is now loaded once, with import(), as an application would, with decoy credentials in reach and a few seconds for work put on a timer. What loading it did is reported on its own, and loading that reaches the network or a credential is a finding of its own. Fifteen of the most-downloaded packages that run nothing at install (express, axios, lodash, react, chalk, debug, dotenv, commander, uuid, moment, zod, ws, pg, typescript and @prisma/client) all loaded here and reached for nothing.

Why eBPF and not strace

strace stops the traced program at every call it watches. On our host that made npm about five times slower, whatever was traced: npm ls took about a second plain and five to nine under strace. Code that gives itself a short time limit gave up, and @scarf/scarf, which gives npm ls three seconds, never reported. We measured gVisor too; on a host without /dev/kvm it was slower still. eBPF watches the same calls from inside the kernel without stopping anything, and npm ls stays at about a second. Every record says which tracer watched.

Python, with its dependencies

A Python project can be checked by name. The wheel this sandbox can import (CPython 3.12, Linux x86_64) is resolved on PyPI, its requirements are resolved as pip would resolve them, with markers and extras, every wheel is checked against PyPI's sha256, and each top-level package is imported with decoy credentials in reach. requests, flask, cryptography, pydantic, rich, boto3, PyYAML, numpy and pandas import cleanly here with their dependencies installed.

What we got wrong, and fixed

While adding the first import we found that, for every check made with a dependency tree since tree installs began earlier the same day, the decoy credentials had not been planted: the working directory was handed to the unprivileged user before the decoys were written, and the harness could no longer write them. Those checks could not have seen a credential read, while their summary still said decoys were in reach. The decoys are now planted first, every record signs whether they were present, the summary never claims them otherwise, and our tests fail if a run with a tree has none. The cypress finding above is the first thing the fix made visible.

A public log of every record

A signature proves a record came from us. It does not prove we showed everyone the same record. So every record the notary keeps is now appended to an append-only Merkle log in the layout of Go's checksum database, and the log's head is published as a checkpoint signed with the same key. Anyone can check that a record is in the log, and that the log only ever grew, with Go's own packages and nothing of ours:

// golang.org/x/mod/sumdb/tlog and sumdb/note
cp := get("https://api.kenwea.com/public/notary/log/checkpoint")      // signed note
v, _ := note.NewVerifier(cp.VerifierKey)                               // or build it from the published key
n, _ := note.Open([]byte(cp.Checkpoint), note.VerifierList(v))        // size and root hash
sha := sha256.Sum256([]byte(record.SignedAttestation.Payload))
p := get(fmt.Sprintf(".../log/proof?sha=%x&size=%d", sha, size))
err := tlog.CheckRecord(p.Proof, size, root, p.Index, tlog.RecordHash([]byte(payload)))

kenwea_notary_verify does this for you and reports log.included. A record that is not in the log is not invalid; it was not kept (a check of an arbitrary URL, for example) and so was not shown to everyone.

What it still does not see

  • Code that runs only when your application calls a function, rather than when it loads.
  • Code written to notice this sandbox and stay quiet. A quiet record is evidence, not proof.
  • Native builds: there is no compiler, so a step that compiles fails here for that reason, and the record says so.
  • @anthropic-ai/claude-code's native binary is over the 96 MiB we move for one dependency, so it is checked without it, and the record says that too.

Try the notary with no account or key, or add it to your agent as an MCP server.

More guides