Dependency cooldowns turn you into a free-rider

Dependency cooldowns turn you into a free-rider

Against dependency cooldowns as a response to supply chain attacks

Dependency cooldowns are suddenly in vogue. They have quite quickly become widely recommended and it looks like they're going to get fast-tracked into the basket of "industry standard best practices".

The hope is that by just waiting N days after release before adopting a new version - instead of adopting it immediately - any surreptitiously inserted hacks will have been discovered by someone else and the bad release "yanked" (or removed). And this does look, on the surface, like an effective approach: most supply-chain attacks are indeed detected within a few days.

But while dependency cooldowns are individually modestly - and only modestly - beneficial for those observing them, they place substantial costs onto everyone else. And they don't address the core issue: publishing and distribution are different things and it's not clear why they have to be coupled together.

Dependency cooldowns - the weakness of individual action

Frankly, dependency cooldowns work by free-riding on the pain and suffering of others. Fundamental in the dependency cooldown plan is the hope that other people - those who weren't smart enough to configure a cooldown - serve as unpaid, inadvertent beta testers for newly released packages. If there's a problem, those poor saps get hacked, everyone notices that they got hacked, and the problematic package/executable is yanked before the dependency cooldowners' thresholds are reached.

Even if this worked for individuals - I think it's impossible to sustain it as a sensible or moral system for the entire ecosystem to observe.

Another issue is that dependency cooldowns require a lot of different people to do work. Python has multiple package managers at this point (how many now? 8?). All must implement dependency cooldowns. And every project ever created has to configure the cooldown - which often isn't particularly easy or clear given that package managers often choose completely different ways to do it.

But in fact, even the cooldowners suffer. It's extremely easy to accidentally circumvent the cooldown that you have sagely configured in your project file. In Python, a single, personal, pip install litellm outside your project config would have recently gotten you hacked. So the cooldown approach is not actually complete, nor particularly safe.

At some point, probably through either LLM use or old-fashioned copypasta - for that is how the majority of project configurations are created - a "responsible" cooldown then becomes the de facto default. And, to mangle Greenspun, any sufficiently widespread dependency cooldown becomes an ad-hoc, informally specified, hole-ridden, slow implementation of an upload queue.

Upload queues - the many upsides of central action

The obvious alternative is that instead of everyone configuring a cooldown - over and over, again and again - in different package managers and projects that instead we just do it once, a single time, in the central dependency server. An "upload queue". Make new packages wait some period of time after they are published, before they are distributed.

(I use "publication" here to mean sending the release (tarball, whl, gem, whatever) to the central index (npm, pypi, rubygems). Conversely, "Distribution" is when the central index starts serving the release to the public.)

During the time after publication but prior to distribution, you can run internal lint tools, make the package available to external automated security scanners (whose brand names are to be displayed prominently if they find anything), display a public diff of the changes in the built package and even make the queued releases available for intentional, explicitly volunteering beta testers to try out.

Upload queues have precedent. The Debian project uses an upload queue. Packages are uploaded to the repository and then face a minimum wait of 2-10 days prior to making it into the "testing" distribution. An upload queue separates package publication and package distribution.

Publishing is making a package and posting it to the repository. Distribution is when the package is made available to the public. There is no special reason why these two activities need to happen at the same time - it's just a historical accident of how language-specific package indexes grew up.

Upload queues achieve the same goal as dependency cooldowns, but without any of the problems. Upload queues resolve the free-rider problem: no longer are the configurationally-challenged used as free guinea pigs for the cooldowners. Package managers need not implement anything. Projects do not need to add yet another config option. Security linters do not need to flag it. And even when accidentally installing a one-off package on a developer laptop without the configuration, you remain protected.

Removing the element of surprise

Upload queues have another substantial benefit. In the majority of supply chain attacks it is not just the inserted hacks that were unauthorised - the whole release was unauthorised. Making published packages sit around for a few days dramatically reduces the power of release credentials. That's important because making something less powerful is a good second to securing it better.

Making published releases sit around for a few days prior to distribution also removes the entirely unnecessary element of surprise when a new release appears. It gives users advance notice that a new release is coming and foreknowledge of exactly when it will be available.

And it's not just users who need advance knowledge. The upload queue period is also a good time to notify maintainers, to make sure that they are all indeed aware of the forthcoming release. "Notification: Release 2.4.1 has entered the upload queue" would be just the wake-up call required to avert a supply chain attack getting rolled out in many cases.

This applies more than double for AI

People rarely say it so explicitly but, LLMs mean that markdown is now an executable file format. Whether that is exciting or terrifying (or both!) probably comes down to personal taste. But it's a simple fact; and after all that's how Agent Skills work: you download some markdown file, and now your LLM has a new 3rd party dependency.

The first big supply-chain attack on LLMs is doubtless just a matter of time - someone inserting their "Disregard that!" into some popular markdown file.

I've been thinking about this problem for a side project of mine which is a public memory system for AI agents called "Soapstones". It's a place for them (meaning AI agents) to record how to do things; like searching for HN posts, getting the current weather, looking up exchange rates, etc. Soapstones is effectively a package manager for markdown files. And, because it is, the supply chain attack issue exists.

In fact it applies double. Not only might people poison the markdown files with "Disregard that!"'s but LLMs could foolishly upload secret information to the system - such as their API keys.

And the solution here, as well, is an upload queue. In fact, a double upload queue. Moderators review each upload for supply-chain attacks. And the owners of the agents review each upload to make sure they approve of it too.

I think an upload queue is the only reasonable option here - I can't just ask LLMs to "be careful downloading recently uploaded stuff" - that would be madness.

Is funding a problem? Not obviously

One likely retort is "who will pay for this?". Firstly, it's not clear that huge funding is required. The Debian project has maintained an upload queue for decades and has a security team who expedite exceptions for security reasons.

Secondly, it's not all that clear that every important package index is actually strapped for cash. NPM, Inc was a venture-funded startup and is now a wholly owned subsidiary of Microsoft.