/bruno
All writing

SecuritySelf-hosting8 min read

Nothing Installs at Our Company Until It's a Week Old

A frog bouncer in sunglasses and a SECURITY shirt holds up the ID of a package called vite 8.3.5 at the door of a club called node_modules. A sign reads "No entry under 7 days" and the frog says "Born on Wednesday? Come back next week." Behind a velvet rope wait boxes labelled rollup 4.65.1 and @swc/core, plus one in a hat and sunglasses labelled totally-safe 6.6.6.
Sorry vite, rules are rules.1-bit / hover to resolvehold to resolve

In the first twelve days after I put a proxy between our company and the public package registries, it refused 71 downloads. The ones I looked at were perfectly normal releases of vite, rollup, swc and caniuse-lite, and their only crime was being too new. That's the deal now: a new npm or PyPI release waits seven days before our gate hands it to anyone at work.

(Okay, the title fibs a little. pub packages wait three days, Maven and Go only get cached, and a few things don't go through the gate at all. "Nothing From npm or PyPI Installs at Our Company Until It's a Week Old" just didn't fit on the share card :P)

So here's why a week, how the gate works, and everything it broke along the way.

Why a Week

Every morning our Discord gets a digest of new advisories against the dependencies in our repos, and this year the news had a fresh supply-chain attack every few weeks. We've had a security incident of our own, too. So I took five of this year's hijacks that came with a proper timeline and wrote down one number for each: how long the poisoned version was up before it got pulled.

  • axios 1.14.1 (31 March): an attacker took over a maintainer's npm account and published two releases that pulled in a remote access trojan on install. Up for about three hours.
  • lightning 2.6.2 and 2.6.3 on PyPI (30 April): 42 minutes.
  • TanStack (11 May): 84 malicious versions across 42 @tanstack/* packages, published through TanStack's own release pipeline. npm removed the last tarball 4 hours 35 minutes after the first one went out.
  • keyv 6.0.0 (4 August), the start of the ChainDrop worm: about 1 hour 40 minutes until keyv was reverted, whilst the worm republished itself into hundreds of other packages within four hours.
  • litellm 1.107.0 on PyPI (21 October): 1 hour 43 minutes. Its payload sat in a .pth file, so it ran whenever Python started in that environment. You didn't even have to import litellm.
A hand-drawn bar chart. axios 1.14.1 was up about 3 hours, lightning 2.6.2 42 minutes, TanStack 4 hours 35 minutes, keyv 6.0.0 about 1 hour 40 minutes and litellm 1.107.0 1 hour 43 minutes. All five bars are slivers next to one long hatched bar for our 7-day wait.
Every one of these was gone before the end of day one.

All five were pulled within five hours, and the write-ups show who noticed: StepSecurity opened an issue on TanStack's repo 26 minutes after the first publish, and Lightning's own community raised the alarm within 42 minutes. The damage happens before that, because a fresh install can pick up a new version within seconds: Huntress saw its first axios infection 89 seconds after publish. A seven-day wait covers that whole window with days to spare.

And no, signed builds don't save you here. The malicious TanStack and keyv versions came out of the projects' own release workflows, with valid provenance attached. StepSecurity put it nicely: "Provenance proves which commit was built. It cannot prove the commit was authorized."

The Rules

EcosystemWaitWhat happens to a version that's too new
npm, pnpm, Yarn, Bun7 daysHidden from the version list. A download answers 404, even when a lockfile pins it
pip, uv7 daysSame as npm
pub (Dart, Flutter)3 daysHidden from the version list. A version already in pubspec.lock still downloads
Maven, Gradle, GononeCached, and can be blocked
CocoaPods, Google Maven, Docker imagesnoneNot routed through the gate

Two more things live in the gate:

  • A blocklist. One line with the exact version, say pkg:npm/chalk@5.6.1, a restart that takes a second or two, and that version disappears from the version list. Every download of it answers 403, cached copies included. It's empty so far, and I'd like to keep it that way.
  • A download log. Every pull is tied to a Google account, a service token, or runners for our self-hosted CI. So when something does go bad, "who already installed it?" is one Grafana query instead of a chat thread.
A hand-drawn diagram. Laptops with a personal token, our CI runners allowed by IP, and hosted CI with a service token all point into the gate. The gate holds the wait, the blocklist and the download log. npm, PyPI, pub.dev, Maven Central and Go modules feed into it from the right, and the gate sends its logs down to Grafana for dashboards and alerts.
Three ways in, five registries behind it, one place to look.

People get in with a personal token from a page behind our Google login. The self-hosted CI runners are let in by IP, and hosted CI uses a service token. On company Macs, a self-service app in Fleet sets up the common package managers in one go, so nobody has to edit an .npmrc by hand.

Why a Proxy and Not Just a Setting

Most package managers can do this on their own now, usually under the name cooldown or minimum release age (the settings are at the end). npm has had min-release-age since 11.10, and pnpm 11 even waits a day by default. I still went with a proxy.

A setting protects the repo it lives in, and a forgotten Dockerfile or a colleague's laptop never reads it. A proxy is one place that everything passes through, and our self-hosted CI runners aren't allowed to go around it. That same place can hold the blocklist and the download log, which no setting can do: a setting can't tell you who already installed something. And it keeps npm, PyPI, pub, Maven and Go in one config file, which matters when only about 60 of our 113 active repos are JavaScript (22 use Gradle or Maven, 16 use pub and 14 use PyPI).

Verdaccio was the first thing I looked at. It has had a minimum-age filter since 6.4, but it only does npm, and its own docs say a filter "is a presentation rule, not an access control". So I went with git-pkgs/proxy: one Go binary that caches npm, PyPI, pub, Maven, Go and a pile of other ecosystems, with the cooldown and the blocklist built in. It's young (v0.10, about nine months old) and GPL-3, and it has no login of its own, so a small token service and our Google login sit in front of it.

We hold the proxy itself to the same rule now: no upgrade to a release younger than a week. (We already broke that once: we moved to v0.10.0 two days after it came out. Rules are rules, except when they aren't ^^)

Twelve Days of Numbers

From 20 October to 1 November:

  • 81,710 package downloads through the gate, from 14 accounts (people, service tokens and our CI runners). npm is most of it with 69,824, then pub with 5,650, Maven with 5,012, PyPI with 675 and Go with 549.
  • 6,116 versions hidden from version lists because they were too new: 6,049 on npm and 67 on pub. The package manager simply picks the newest version that's allowed, so there's nothing to see.
  • 71 downloads refused because a lockfile already pinned a version that was too new.
  • 0 entries on the blocklist.

The 71 refusals are the price. The usual suspects were vite 8.3.5, rollup 4.65.1, @swc/core 1.16.20, caniuse-lite and electron-to-chromium.

Everything It Broke

A hand-drawn terminal running npm ci. It prints npm error code E404 and 404 Not Found for electron-to-chromium-1.5.472.tgz, version not found. A yellow sticky note on the corner says: too fresh, try again next Saturday.
Real output, from a lockfile pinning a version published the day before.

The 404 is what those 71 refusals look like. The lockfile pins a version the gate won't serve yet, because somebody resolved it without going through the gate. Set up the client and re-resolve, or wait.

The rest:

  • Not every lockfile follows the registry setting. npm and pnpm locks are fine. Yarn 1 fetches the exact URL written in yarn.lock, so it skips the gate until you regenerate the lock. The old binary bun.lockb hardcodes npmjs URLs. uv sync --frozen downloads straight from PyPI's file host. pub get re-resolves pubspec.lock the first time it sees the gate, so versions can move up. And Poetry has no registry variable at all; it needs a source in pyproject.toml and a re-lock.
  • pnpm 11 reads PNPM_CONFIG_REGISTRY, not NPM_CONFIG_REGISTRY.
  • Dependabot and Renovate have to wait at least as long as the gate, or an update PR for a version younger than a week fails CI. Dependabot's own default is three days, so it needs cooldown: default-days: 7 in dependabot.yml. Renovate's setting is in the table at the end.
  • npm audit passes through the gate to npmjs.org untouched, so it knows nothing about the wait.
  • Prereleases can sneak in. When the wait hides a package's current latest, the proxy hands out the newest allowed version by publish date, and that can be a prerelease. Ranges in package.json never pick prereleases, but a bare npx something can.
  • If the gate is down, CI can't install anything, because our self-hosted runners aren't allowed to go around it. That's on purpose, and it makes the gate as critical as the runners.

What a Week Doesn't Do

A cooldown only helps if somebody notices within the week. Usually somebody does: one blogger's count of 1,336 hijacked npm packages, using OSV advisory dates and his own definition of "hijacked", found that a seven-day cooldown would have blocked about 85% of them. Two that stayed quiet longer:

  • In March, a hijacked npm account published malicious versions of react-native-international-phone-number. When I checked the registry on 1 November, 0.12.3 was still tagged latest, 228 days after it went up, and the package still had about 134,000 downloads in the previous 30 days (Endor Labs). Its payload dependency has since been taken down, so it probably does nothing anymore.
  • The xz-utils backdoor shipped in 5.6.0 on 24 February 2024 and was found on 29 March. That's 34 days. It wasn't an npm or PyPI release, but a seven-day wait would have run out 27 days before anyone looked.

It also never looks inside a package, only at publish dates, so a poisoned version that nobody catches for a week gets in on day eight.

Try It: The Cooldown Setting for Each Package Manager

You don't need a proxy to start. Most tools have the setting built in now:

ToolSeven daysWhereSince
npmmin-release-age=7.npmrc11.10
pnpmminimumReleaseAge: 10080 (minutes)pnpm-workspace.yaml10.16
YarnnpmMinimalAgeGate: "1w".yarnrc.yml4.10
BunminimumReleaseAge = 604800 (seconds, under [install])bunfig.toml1.3
uvexclude-newer = "7 days"uv.toml0.9.17
pip--uploaded-prior-to=P7Dcommand line or pip.conf26.1
Dependabotcooldown: default-days: 7dependabot.yml2025
Renovate"minimumReleaseAge": "7 days"renovate.json2023

If you want the whole company covered, plus the blocklist and the download log, git-pkgs/proxy is the piece doing the work. The policy part of our config is this:

cooldown:
  default: "7d"
  ecosystems:
    npm: "7d"
    pypi: "7d"
    pub: "3d"
  packages: {}   # exceptions: "pkg:npm/<name>": "0", with a reason and an end date

denylist:
  packages: []   # e.g. "pkg:npm/chalk@5.6.1": gone from metadata, 403 everywhere

On our version (v0.10.0), Maven and Go come through no matter what default says. I checked: a two-day-old Maven release and a five-day-old Go module both got served.

A wait only buys time for somebody else to notice. Once they do, the blocklist takes the version out and the download log says who pulled it first.

vite 8.3.5 got in eventually, by the way. It was published on a Wednesday and the gate let it through the Wednesday after. The bouncer doesn't care how popular you are, only how old.

Until next time <3

End of transmission