Why are so many people so sure that the big AI providers are losing money on inference? It reminds me of the comments about how Uber can never make money. Their unit economics were fine and they were only losing money because they chose to do so on customer acquisition.

In this case, I don't think it's even obvious that inference should be money losing today. API cost for frontier models has gone way up and the people claiming some cost for inference are relying on made up assumptions.

@danluu I think it’s people hoping that the price increases are just a lower subsidy and not at cost or profitable. I’m not sure the evidence available necessarily disproves that either.

@danluu see also https://infosec.exchange/@codinghorror/115243205026725683

If you read widely cited folks saying AI economics are bad today (e.g., Ed Zitron from the quote), they're not credible. Sure, Ed writes rants, swears a lot, and throws the word "scam" around, all of which gets a lot of clicks, but the reasoning seems wrong.

If I were going to be skeptical about the economics of AI, it would be some kind of reasoning about how cheap models keep improving, so the frontier labs need to stay ahead (I don't follow it closely enough to know if that's plausible).

@danluu my impression from friends at frontier labs is that inference isn't anywhere close to losing money right now. Training is a huge money pit of course but inference is pretty profitable just measured on a cost per token basis, even at the subsidised monthly subscription rates.

(Obviously this is hearsay and you shouldn't treat it as particularly credible, but I personally think it's very likely to be true)

@danluu the pro models are drastically better than the free models, so this is important

@danluu i don't think the guy is a messiah or anything, but beyond the swearing, Ed Zitron's analysis is a hell of a lot more credible than saying it's not credible with the only counterargument being that it sounds wrong. what about is sounds wrong? why is he not credible? why are those that say that these companies who have not demonstrated returns and are racing to ratchet up cash in while limiting usage on the subscription models (one of his main criticism being that subscription models make no economic sense because they decouple the driver of cost from the revenue mechanism) are profitable?

@DRMacIver It seems like inference could be massively profitable. Whether or not training costs offset is a tuning knob like Uber had.

Given how much OpenAI and Anthropic have jacked up prices recently (if you want their latest models), and how there's no real competitor (I try Gemini every once in a while and it's never been in the same league for coding, though it is better at some things), it doesn't seem like OAI and Antropic are competing on price, which leaves room for a lot of profit.

@danluu part of the argument is that not just that it might not be profitable now, but that the amount of profitable that it would need to be to justify the amount of capital expenditure that has already been made and is promised is numerically impossible. JPMorgan estimated 1.2 trillion in AI debt back in december 2025, goldman sachs estimates another 500 billion in 2026. Where is the evidence that inference is profitable enough to pay off 1.7 trillion? If it was really profitable, all the publicly traded AI companies would be screaming this at the top of their quarterly reports.

@jonny @danluu These are unfathomably large numbers so to get a grip I looked up Shell's yearly profits (x*10^10 USD where 0<x<4), and with a wild estimate it'd take ~50 years for them to pay off this kind of money (while not investing in anything else)

@jonny @danluu Ed is one of the few people who bother to read the financial statements put out by all these companies and also researches how many of the promised data centres actually materialised (not that many), so if there ever was a credible source doing proper journalism, that's him. He could of course still be wrong, but he's not just hand waving made up figures.

@jonny @danluu I do agree that Ed is kind of a prick, can be fixated on dubious claims and extrapolates a lot, but overall his reporting is the one with the most precise data. Maybe inference is or will be profitable? We have no reliable sources on either side of the claim. The leading companies would probably shout it everywhere if it was, and they usually lie quite a bit to embellish the picture so the fact that they do not really talk about it is telling.

Maybe the industry can be sustained by every company deciding to pay $1000 seats apiece per engineer in perpetuity, but that sounds like a bad idea in aggregate.

@mathieui @jonny @danluu You are off by at least one order of magnitude. Salesforce.com expects to spend $300M on LLMs this year for 15,000 engineers, thus $20,000/engineer, so they are clearly seeing the value, despite the limitations of current models.

@fazalmajid @mathieui @danluu so the argument that inference is profitable is that companies who have invested directly in the companies they are buying tokens from and also use those tokens in their products (so the division of total spend on tokens / engineers is nonsenical) spend a lot on tokens?

@danluu what's the point of considering inference costs in isolation anyway?

If most compute is spent on training, the real question is about overall profitability.

If money comes to be lacking, they could spend less on training, while still making about as much money on inference.

@danluu In this example are you comparing Uber to Nvidia, or Uber to an LLM provider?

Either way, I don't think the economic models are at all similar. What makes you think differently?