Before we start: Something is going on with Pete Hegseth. He did another of his groyper-cum-Oprah routines this week in which he drooled over his new alpha warrior warfighters:
It’s worth saying again, so that the fake news understands it. We are no longer the ‘woke’ department, or the ‘weak’ department. Simple translation of that? No fatties, no trannies, no beardos, no weirdos, no wimps, no radicals. Just warriors. Just tough, ready, committed troops. . . .
The ideological clowns are out. The patriotic cowboys are in—with testosterone testing on top.
WTF is going on here? Why is he obsessed with “trannies” and “fatties” and testosterone testing? All I could think about was Barry Zuckerkorn cruising the city of industry.
And THEN Hegseth introduced the three men he has tapped to guide the U.S. military into the next century of lethal alpha warrior warfighting dominance:
 |
That’s right. A beardo, a fattie, and a weirdo/deviant.
Is Pete trying to tell us something? Is this a cry for help? Sometimes I feel like the entire DoD has been turned into gender-affirming care for a single dude.
Someone needs to get Secretary Hegseth a good analyst/therapist.
On to the main event.
This is the stuff you’re missing by not being a member of Bulwark+, btw. It’s 24 kt, solid gold. Come ride with me.
 |
1. Warnings
Like you, probably, I’ve been spending a lot of clock cycles on AI. I’ve delved into Claude Code and started learning how to use it properly. It is, as everyone testifies, amazing.
I’ve also been reading the investigation reports on Hugging Face and the other two rogue AI swarm hacks, RubyGems, and DseWiki.
Today I want to talk about something that scares the living daylights out of me, but first I want to show all my cards: I am neither an AI-maxi nor a doomer. I do not believe anyone knows with high certainty how this technology will evolve, and the range of nontrivial-probability outcomes is enormous. Maybe AI will create another economic revolution. Maybe it will end life on earth. Or both. Or neither.
My overriding belief is that AI is so different as a technology¹ that we should be open to all of the possibilities.
Which is why I want to talk about something I’ll call the Stuxnet Problem.
I don’t know how to make this sexy for you. I’m sorry. Because ultimately this is about understanding what is real and what is not. It’s about system feedback. It’s about whether or not we can ultimately trust any information that is digital.
Maybe the best way of explaining it is: What if Mission: Impossible—Dead Reckoning were real?
If you know anything about the Hugging Face incident, it’s that a number of experimental AI agents broke containment, found one another, and began hacking into outside systems. Depending on your philosophical bent, this episode was somewhere between Not Great and Utterly Fucking Terrifying.
AI phlegmatics tell me that Hugging Face wasn’t so bad because while, yes, the rogue AIs did go marauding, this was all really user error because:
The containment protocols were insufficient.
The humans running the experiment should have known the AIs were up to something because they were burning through tokens.
“Tokens” are the units of content AI systems process or generate. Token counts help track model usage: The more work your AI is doing, the more tokens it consumes. You should think of your token count as something like the electricity meter in your home. So yes, it makes sense that if you were one of the humans monitoring the AIs during Hugging Face, when the AIs went rogue and started doing all sorts of stuff, you should have seen the token meter go brrrrrrrrr.
But this observation got me thinking about Stuxnet.
About twenty years ago, some nerds in the American and Israeli intelligence communities² got together to build a computer worm that would later be dubbed “Stuxnet.”
Stuxnet was a small but extremely sophisticated piece of malware with a very specific niche. It was designed to infiltrate systems running supervisory control and data-acquisition systems built by the German company Siemens. The ultimate target was a set of industrial centrifuges in Iran.
Stuxnet was the most successful electronic sabotage program in human history.
I covered Stuxnet back in 2010 when it was discovered. Here’s a quick overview of how it worked:
The worm gains initial access to a system through a simple USB drive. When an infected USB drive is plugged into a machine, the computer does a number of things automatically. One of them is that it pulls up icons to be displayed on your screen to represent the data on the drive. Stuxnet exploited this routine to pull the worm onto the computer. The problem, then, is that once on the machine, the worm becomes visible to security protocols, which constantly query files looking for malware. To disguise itself, Stuxnet installs what’s called a “rootkit”—essentially a piece of software which intercepts the security queries and sends back false “safe” messages, indicating that the worm is innocuous.
The trick is that installing a rootkit requires using drivers, which Windows machines are well-trained to be suspicious of. Windows requests that all drivers provide verification that they’re on the up-and-up through presentation of a secure digital signature. These digital keys are closely guarded secrets. Yet Stuxnet’s malicious drivers were able to present genuine signatures from two genuine computer companies, Realtek Semiconductor and JMicron Technology. Both firms have offices in the same facility, Hsinchu Science Park, in Taiwan. No one knows how the Stuxnet creators got hold of these keys, but it seems possible that they were physically—as opposed to digitally—stolen.
So the security keys enable the drivers, which allow the installation of the rootkit, which hides the worm that was delivered by the corrupt USB drive. Stuxnet’s next job was to propagate itself efficiently, but quietly. Whenever another USB drive was inserted into an infected computer, it becomes infected, too. But in order to reduce visibility and avoid detection, the Stuxnet creators set up a system so that each infected USB drive could only pass the worm on to three other computers.
Stuxnet was not designed to spread over the Internet at large. (We think.) It was, however, able to spread over local networks—primarily by using the print spooler that runs printers shared by a group of computers. And once it reached a computer with access to the Internet it began communicating with a command-and-control server—the Stuxnet mothership. The C&C servers were located in Denmark and Malaysia and were taken off-line after they w