98. How to Lose A Billion Dollars in AI
A lack of security planning allowed the work product of America's top AI developer to be rapidly transferred to a competitor. How is this possible? What are the implications?
As this is time sensitive data, I apologize for not working on this topic earlier. I was very involved in my Demographic Collapse series, 92A to 97F.
As a game economist, my role in industry started (as early as 2000) with developing new methods to create game economies, and simultaneously creating methods to defend them from attack. Creating a new economy, potentially more efficient and powerful than the “real” economy is very exciting work. But if you make an economy that you cannot defend, it’s like opening up a free buffet. “Take all my food and don’t pay me! Please!”
So it turns out the security side is more important, and also much more complex. It is the side of game economics that top industry leaders told me for years was impossible to do. They had tried with active countermeasures and failed, where I ultimately succeeded using passive design technologies that removed any economic incentives for attacking my economies. I just built better economies instead of punishing people for helping themselves to my poorly designed free buffet.
I don’t know why this is such a weird concept to companies, and why they will drop huge budgets into punitive/reactive techs like active countermeasures instead of just building smarter. So when I see the same pattern repeating in some of the world’s most advanced technology companies, I am really surprised even though I probably should not be.
So let’s talk about what happened with perhaps the world’s hottest technology: AI development. As I’ve disclosed before, my partner who I have lived with for the last five years spent the first four of those years as a producer in a top Large Language Model company that helped all the big names in AI develop their systems. While I never looked at her screens, her basic explanations of what she was doing allowed me to reverse engineer in my head most of what was going on.
This work was so top secret that it often involved working in locked and sealed rooms where no radio or microwaves could get in or out. This developer spent a lot of money to make sure the tech they were developing was as secure as possible. Humans were hired from over 100 countries to read and correct various voice samples, to improve the translations. Hundreds of thousand of people worked for years full time to get these Large Language Models started. This was certainly a Megaproject, and was treated that way.
Once this technology was transferred to the clients (all the big names that all of you know), it was up to them to safeguard the tech on their end. You would expect high tech companies to take security very seriously, especially if some of their clients were government military departments.
Of course securing a technology that you will be selling (or even just showing) to consumers in order to impress your investors can make things very complicated. Just like things can go very wrong with my game economies if I don’t properly plan for all the things that consumers/rivals could do with that access.
The problem here is that an AI chatbot, once it’s pretty advanced, can communicate just as well as a human. In most cases, better. More importantly, an AI chatbot can talk faster. Perhaps thousands of times faster if output is in text. And it is designed to handle millions of simultaneous queries.
That means your chatbot can do all the work that this other developer did in 4 plus years to train your AI, in just a few weeks. Maybe faster. If you allow a competitor to ask your chatbot to train their chatbot, it can do this rapidly. Which… is exactly what happened with DeepSeek. This transfer method is now being called “distillation”. That’s basically a fancy word for “You left your laptop unlocked and I put a thumbnail drive in it and transferred the contents of your hard drive into my thumbnail”. But instead of it being a direct 1 for 1 transfer, the transfer involves millions of questions being ask of one chatbot by another chatbot. Humans might call this “tutelage”. But machines can do things much faster than humans, and you can have 1000 computers doing this non stop 24 hours a day, asking hundreds of questions per second.
New Economies
I often refer to economists as “conventional economists”, because they are taught the designs created by Adam Smith which he published in 1775 in The Wealth of Nations. Small changes have been made to conventional economics since 1775, but you might be surprised by how little it has changed. I make this distinction because when I talk about “New Realities” and new economies, I see myself as a “new economist”. A person who creates what comes after conventional economics. When I went back to school to learn economics at the age of 41, I made it clear to all my professors that I wasn’t there to learn how economics works. I was there to learn why economics doesn’t work, so that I would not repeat those mistakes in my new economies.
When humans create AI, with the purpose of supplementing or replacing human labor, they are creating a New Economy. Ideally they have some plan for this New Economy, maybe even a design! If they don’t, it will be designed by all the opportunists who will hack that new economy. Just like what happened with all those early game economies that were designed defenceless.
What I think happened here is that a bunch of techies created a new technology used to build a New Economy, without having a single economist on staff. A simple oversight? Simple yes. But also a huge oversight. There might have been a few people with some conventional economics education, but no training in how to build a New Economy. Because from what I saw when I went to study Economics at a top university is that this technology does not exist within conventional economics.
When I proposed the creation of a field of New Economics, I was blocked from academia for “non academic” reasons despite having the top GPA of all applicants at all universities I applied to. So this lack of New Economic technology in society isn’t accidental. It’s not an oversight. It was planned. Perhaps a form of protectionism to slow the obsolescence of conventional economics.
So this New Economy was released into the wild with zero protections. Not even a plan. It’s like it was just rushed out in order to satisfy investors that the tech bros were brilliant and were going to solve the world’s problems. Without any plan, or even knowing what those problems were.
I wouldn’t be surprised if some of the same people I used to battle to defend game economies (these people were called “Gold Farmers” in the early days of this century) evolved to hack AI chatbots. It amazes me how many of the early Gold Farmers that I knew became pioneers in the crypto space. These people were all expert in adapting to new technology and exploiting it. And Gold Farming wasn’t a minor thing. It was so potentially lucrative that Goldman Sachs took over the primary Farmer exchange in 2007 and sent their representative (a guy named Steve Bannon) to run it.
I would imagine Goldman Sachs is all up inside AI by now, so this feels like history repeating itself.
Implications
So the developers of AI in the USA opened up their free buffet and got pillaged. Which of course they did. They didn’t do all the work to build those models, those hundreds of thousands of people all over the world did. So they didn’t appreciate the value of the data they were working with. They just were excited about being famous. Just like the guys from Cambridge that created the psychometric modelling tech that they admitted “would be dangerous in the wrong hands” in one of their papers, before the tech was transferred to “the wrong hands”.
I honestly didn’t notice that little detail until now. History is repeating itself, and with the same actors in new roles. Mr. Bannon is not some political hack. He’s basically the Anti-Ramin, trying to break New Economies as they emerge. He has an excellent track record and powerful allies. And as long as major USA companies keep side-lining me for helping regulators and consumers (and non commercial reasons), they leave themselves defenceless by throwing out the baby with the bathwater.
So now this technology has been transferred to DeepSeek. That tech was then modified to return misinformation if you ask it about something that touches CCP ideology. So not only was ChatGPT tech stolen/transferred, but then it was made open source so that the tech is essentially worthless (infinite supply = zero value) which is exactly what Gold Farmers do. They sell their harvested assets to as many gullible consumers as possible making it impossible to punish the people with the “hot” assets or to do a server roll back.
Weaponizing the new chatbot with an ideology makeover is just the cherry on the top.
AI is not just an expensive technology. It’s potentially a very dangerous technology. I’m not the first scientist or writer to point out that AI has the potential to cause another extinction event. Musk has pointed that out (and he’s a big investor in the tech) and I really enjoyed the Terminator movies staring my former boss Arnold Schwarzenegger (back at the Governor’s Council on Fitness and Sport).
Can you imagine if the Americans creating the first nuclear bomb had decided “oh hey, let’s let the public try this out at home!” ? What could go wrong? While it’s difficult/abstract to try to compare one potential source of extinction to another, AI is arguably a more serious threat than nuclear weapons if it is misused. Who would possibly misuse AI? That sounds like the punchline to a bad joke. But the answer could potentially be “AI” as you can see it has the ability to teach itself. It taught DeepSeek after all. And it did it quickly.
I’ve said before that AI is not being developed in the USA for consumer use. Showing it to consumers, that was just to appease investors and to provide cover for the important uses of AI, which are all military. China isn’t worried about the USA cornering the market for consumer applications of AI. They just open sourced it after all. They are seeking to drop the cost of consumer AI to zero, like they are in multiple other industries. This causes private industries without government backing to go out of business, which is why China does this in as many industries as possible.
But AI companies in the USA won’t go out of business because their primary customer is the USA military, which is a branch of the government. It’s likely the most heavily subsidized industry in the world. ChatGPT was just a song and dance to influence investors, and cash out a bit early.
One implication is that it made me look a bit stupid when I tried to explain that users of AI are not customers, they are the product because they are being farmed to train AI. But now that AI is training AI, we don’t even need humans for that anymore. I failed to anticipate that the creators of ChatGPT (OpenAI) would be silly enough to allow unlimited access to their tech. But… of course they did.
And a Billion Dollars was flushed down the toilet.
That’s not the worst of it though. Now that we don’t need humans to train AI anymore, what do we need humans for? To build SMR and Thorium reactors for AI? Okay, for at least 10 years. But most humans are not useful for that. If AI is going to be using humans for labor, and the humans only know how to ask AI to do their job for them, wouldn’t it make sense to cut out the middle man (humans) as fast as possible?
This is obvious, right?
This is why I started my 6 part series on Demographic Collapse with 92A, explaining why the USA is unconcerned about Demographic Collapse. They are seeking to reduce their “labor pool” as fast as possible. The military applications, which are already being field tested, are designed to facilitate this selectively: some demographics can be reduced while sparing “preferred demographics”. Somehow I suspect I’m not in the preferred demographic.
Military applications come in 3 primary flavors as best I can tell:
Demographic targeting/culling. Not science fiction, it’s been deployed for over a year on live (now dead) targets.
Code Breaking. Dr. Alan Turing’s work breaking the German encryption code was so critical that without it the Allies might have lost WW2. His torture and death after they were done with him leaves me no illusions as to the benevolence or gratitude of these sorts of people. AI can break almost any code nearly instantly once it is combined effectively with Quantum Computing. This is where the real race is. This could theoretically be used to usurp control of an enemy’s entire computer infrastructure, rendering their military useless or even under opposition control.
Drone management. Wars in the near future are going to be determined by who has more drones. But a million drones doesn’t help unless you have a million qualified operators. Thus AI is essential for this purpose. Because the only thing better than 1 million drones is 100 million drones. And the “Loyal Wingman” program to develop a fleet of unmanned aircraft has been in development since 2019 in the USA.
It’s fair to say that all three of these synergistic military tech paths have unlimited budgets, unlike the civilian applications of AI which are just there to distract them. I would expect some more serious effort will be made to prevent the transfer of these technologies.
This foreknowledge isn’t very helpful unless you are in the right demographic. Maybe I should look up Steve Bannon and start kissing the ring. “Look Steve, I didn’t mean to vilify you. I was young and stupid. I’ve been following your career for 18 years. In a way, I’m your biggest fan!” Do you think it will work? If you think that flamboyant playboy Musk is the power behind the throne, you’d be half right. But reportedly him and Bannon have a rivalry. I wouldn’t be surprised if Bannon comes out on top. I suspect he’s the smarter of the two. And he’s way better at keeping secrets.