The collateral damage to human experience in an AI-driven Internet

Recently there was yet another reported hack from the supposed “rogue” AI agents (cough poor sandbox cough). This time the victim was RubyGems. RubyGems is a community-run service to host Ruby’s gems, a.k.a. libraries.

There were several interesting findings about the hack, how it was discovered, how it happened, etc. I won’t get into the details; RubyHack has in-depth coverage of the findings.

From RubyHack’s postmortem, one particular paragraph caught my attention.

This incident primarily occurred in May. In response to this, RubyGems temporarily disabled new user signups from May 12 to May 16. Afterwards, RubyGems introduced additional security measures, such as requiring non-disposable, verified emails, as well as adding rate limits to new signups.

During their hacking session, AI agents were creating a large number of RubyGems account with disposable, free email addresses in order to access and publish their own libraries. They didn’t have access to these inboxes and exploited another vulnerability in RubyGems to bypass email verification, yet the fix that landed on users was a ban on disposable emails.

Many people, actual humans, use disposable email addresses for various purposes ranging from privacy to managing spam. But now there is one less website where you can use those email addresses, thanks to the behaviour of the bots.

This isn’t the first time, and certainly won’t be the last, that the behaviours and capabilities of AI agents have impacted the user experience for actual human users. CAPTCHAs were the first tax we paid for bots, then came SEO fluff. Email restrictions are the next, and the bills keep getting bigger.

I can already see a near future where one catastrophic event from the AI agent hacks is used by governments to put stricter access controls, real ID verification, snooping, weaker encryption, and more. Governments across the world have been trying this for years, all they’ve been missing is one clear-cut justification that benefits the website owners as well. Every private platform that tightens verification makes the idea feel more normal.

The 9/11 attacks permanently changed air travel for everyone. You used to be able to walk a friend to the gate; now you arrive two hours early to get through security. And those same security measures have since been used to justify searching your phone, your bags, even your body. Governments will use AI-agent attacks the same way.

Now these “scripts”, “robots”, “AI agents” didn’t write themselves. They were written by real humans and released into the world, making the experience shitty for other actual humans. There should be consequences, but there probably won’t be. At the very least, they should be treated with the same disdain as someone who takes hundreds of cartons of drinking water during an emergency relief distribution, or who queues up at a food bank, takes food, sells it outside and calls it “hacking the system.” Instead, they’re revered in the tech/startup ecosystem for pushing the frontier and challenging the “status quo.”

Slowly but surely, the Internet, once a connecting force unlike anything humanity had ever had, is being taken over by an endless stream of scripts and machines, and it’s getting worse for the people it was built for.