A longish holiday does provide some Perspective of the Gimlety View. What blew up during the last big summer holiday, as I was building my Vitamin D for the long winter while at the Jersey Shore, was the news made by one Jacob Coxon, a former researcher at Anthropic (developer of Claude), that advanced AI within 10 years has a more than 10% chance to have the ability to hack into enough systems, initiate bioweapons, nukes, and otherwise kill off Humanity. The humans who originally designed it would be unable to stop rogue (and roguish) frontier AI. This started with Coxon’s well-timed posting on X on 8 September and percolated through tech media right before the 25th anniversary of the mass murders on 9/11/01. Mass media, already overheated because of the anniversary, exploded like a pressure cooker.
Axios, the techie bulletin board and for generalist media types a source for pithy quotes, on 9 September quoted a MIT 2025 study of “experts” giving a 21.5% chance of “AI possessing dangerous capabilities”, with a 21% chance of AI initiating “Cyberattacks, weapon development or use, and mass harm”. It was a reasonably analytic article full of nuggets for a writer to run with.
This past week introduced us to the latest pop tech slang, P(doom), a cute term for the percent chance of AI Doomsday. P(doom) has now surfaced everywhere from Elon Musk to Anthropic’s CEO Dario Amodei. (Impress your friends and relatives at the next cocktail party or tailgating/cookout.)
The mass media in the following week ran with quotes by Coxon, old tech stalwarts, and politicians. Plenty of P(doom) predictions. Emblematic of the coverage was the Los Angeles Times recap published on, ironically, 11 September, a day of real, not theoretical, doom.
Readers know from my running coverage that this Editor is skeptical of AI both in a business sense (a financial time bomb) and of its utility in its current models. Automating many functions within healthcare? Sure. Providing deeper insights into imaging to speed diagnosis? Faster drug development? Bravo. Speeding market and marketing analysis, sussing out needs? Hey, hey! But equally I have been scathing of companies that buy AI services and believe vendor promises, without cross-checking for accuracy, dumping the skilled humans doing these jobs in mass layoffs. These people are the boots on the ground who should be doing the cross-checks and upgrading the models that come from developers usually unfamiliar with medicine and healthcare–and certainly without deep and specific knowledge the boots have. It turns out that the mass layoffs in many companies, from team managers, data analytics, customer services, marketers to operations, have been to fund so-called investments and token spending in AI tools. ROI has gone into a Missing Man Formation in the finance department, while services to customers and in healthcare, patients, go sideways–as does market share and revenue. CFOs, funders and Mr. Market are just waking up to this surprise that they forgot to teach at HBS.
I’ve also been scathing in coverage of certain companies such as Open AI, Oracle, and somewhat so of Anthropic, Microsoft, Meta, and Google. These companies are pushing one to two year bricks-and-mortar data center development without responsibility for local effects on water, power, noise, and land use–something that a GM, Ford, Toyota, or Worldwide Widgets building a factory would face and mitigate. This thoughtless pushing is proving to be politically evenhanded in opposition; these companies have received epic wedgies and a few pitchforks in their metaphorical derrieres as a deserved result. But this is manageable; because of pushback, it’s diminishing as a ‘reason why’ except as a political club for unscrupulous politicians.
What’s a lot more devastating is that apparently all the forecasting is inflated beyond belief. There is no decent idea of metrics, of matching up capacity to future demand. Not by OpenAI, not by Anthropic as they prep for IPOs. This is an ancient problem that always, reliably, cyclically bites developing sectors in their nether regions. Not a hype cycle like we saw circa 2006-15 with telehealth, not bad management, but in other industries within recent memory, with ‘sky’s the limit’ booms followed by disaster:
TNW discusses in a deeper dive the debt structure and why PIMCO could make this bet where banks could not [reference is to Oracle’s debt funding]. The question it raises is whether the furious pace of data center building is another cycle of overbuilding–and if it is, will it be absorbed in time? The ominous parallels: the 2000s building boom in an earlier iteration of data centers, the fiberoptic boom of the early 2000s that broke WorldCom, Global Crossing, Winstar, Corning, and 360Networks, cloud overbuilding that left Amazon Web Services with years of excess capacity (it helps to have a deep-pocketed and not all that transparent parent), and others. This Editor would also liken it to the early years of 1980s-90s airline deregulation (too many airlines, too much debt, too many seats) and about a decade in the cruise ship industry where too many cabins were chasing too few people. These took decades and multiple bankruptcies to settle. TTA 7 May
Writer Ed Zitron, who has few parallels in Gimlety-ness, analyzed how OpenAI could easily meet the Devil of Demise sooner rather than later in his article What Happens If OpenAI Dies?, one of our Must Reads of 19 August. OpenAI’s losses are terrifying, contrary to their PR spin, and will not change in the immediate future. It has to become the most successful company since Caesar Crossed The Rubicon–or it croaks and may well pull down an economy with it. I will add from other reading that Anthropic, ahead of OpenAI, is painted in a similar corner.
All the above, save P(doom), are hoofbeats from horses, not zebras. What’s happening is a boom and billions in concentrated funding that has hit multiple headwinds. Not the end of the world but not a cheerful earful for a stressed economy. In the short term, the two major AI companies have more of a challenge from F(doom)–financial doom.
Then suddenly, we have two things happening in the past 10 days:
- A short-term, low-level former Anthropic employee, Jacob Coxon, with few followers on X, suddenly bursting out and ‘going wide’ with a doomsday prediction. Quoted and covered everywhere with the things that at least some of his elders like Steve Wozniak and Geoffrey Hinton have been saying for over three years. (The Woz I’ve traced back to 2015!)
- Dario Amodei, CEO of Anthropic, Sam Altman, still CEO of OpenAI, and Elon Musk (Grok/X) then calling for government safety regulation. Not guardrails. Regulation. CNBC, OpenAI blog
Private companies demanding government regulation of their business truly boggles the mind. This should raise red flags as to why, and why now.
P(doom) supposedly has a ten-year horizon. F(doom) works far faster, may be more likely, and once it starts, there may not be an Anthropic or OpenAI to worry about.
Some Gimlety Speculation, IMO only:
- Both Anthropic and OpenAI float on a sea of red ink and Other People’s Money (OPM). Their lenders want to monetize and exit their investment with an IPO. Both companies were racing to IPOs. Except neither company is logically, reasonably, in any financial sense, ready. Yet Anthropic was due to IPO before end of 2026 with OpenAI racing to beat them.
- Investors are also pushing for another OpenAI funding round to cash out secondary shares held by employees, according to CNBC today (16 Sept). Which, in a case of an IPO, means more shares for the investors.
- Both the Coxon media pickup and the subsequent Amodei/Altman calls for regulation effectively stop the IPO clock. One can speculate about the timing and origins, but it is fortuitous for both companies.
It is also being used as a political club in the upcoming midterms. The usual Senate suspects are demanding complete regulation of AI because “we are losing control”. President Trump, somewhat on the back foot on this, calls their fears a hoax and a scam that will prevent US leadership in AI development. Yet Sam Altman along with Nvidia’s Jensen Huang will be attending the state dinner for Chinese President Xi Jinping’s visit to Washington next week, China is building a lot of data centers, well away of course from any international observation; Uyghurs don’t get to complain about data centers and water usage to Beijing. One doesn’t even know if they are real or like their empty hinterland cities. Yet Chinese-engineered AI models may take the lower end of the market, much like steel.
There’s an old saying in medicine: when you hear hoofbeats, think first of horses, not zebras. It dates back to Dr. Theodore Woodward of the University of Maryland medical school and the 1940s in teaching his students to first rule in or out common diseases rather than searching first for exotic, rare diseases because they’re more interesting. (Apologies to those with rare diseases, and may AI be a savior in this.) Taking the analogy to business, the ‘horses’ are the shaky business models around AI development, capacity versus demand, and data centers. Throw in the Chinese capacity and models. The ‘zebras’ are Humanity’s Extinction By AI.
What’s running in that herd out there, who’s painting the horses black and white, and why? The consequences can be severe.
Leave a Reply