I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing.
I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train.
Now there's monitoring, reporting, alerting not just on overall cost but on "over-use" of best/priciest models based on total-or-percent tokens/dollars, etc. All of this comes with direct developer engagement & standardized management escalation for holding it wrong.
To me this customer behavior does not smell like a product you can 10x the pricing on to get profitable. We have exited the exploration phase and now ROI matters.
steveBK123
It's not an affordability crisis, it's a financial crisis. The models get cheaper super fast. By this time next year Fable 5 will cost less than Sonnet does today. That's not the problem. The problem is that many companies are going to realize that they don't get any ROI from AI. Generating code faster != more profit. Most of the Fortune 500 will likely realize this and then the token budgets will come crashing down. Most of their ideas are _bad_ ideas. Implementing bad ideas faster, won't lead to more profit.
Sure, you can use AI to potentially replace software engineers, but the F500 are also terrified of not having accountability or making mistakes. They won't be firing any engineers. In that scenario, there's just no room for AI usage. If you have to be responsible for all the code, then... AI has to either manage it completely autonomously (which even Fable can't) or... humans have to be in the loop which means they still have to understand the code. The best way to understand the code is to write the code yourself. So there's no productivity gain to be had.
I'm pro-AI, but I think we're due for a big crash next year.
woeirua
> Zitron's numbers don't tell us the real cost of generating tokens but, subject to the assumption that the platforms are not subsidizing the token price, that means Anthropic is subsidizing their enterprise customers by up to 40 times, and OpenAI up to 70 times
Neither Anthropic nor OpenAI are subsidizing enterprise customers. Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high value $200/mo plan. Both organizations have moved to a "cheaper plan per user + API Pricing after that" (e.g. $20/mo + usage). The $100/$200/mo plans are for individuals only (of course, many individuals use these plans at work, but that's beside the point; they aren't selling this plan to enterprises).
> SemiAnalysis also analyzed the platform's gross margins, implausibly assuming that tokens were priced at 4 times the cost of generating them and: With the current subsidies, all it takes for a user to have a gross margin of at best negative 25% is for them to use as little as 25% of their rate limit.
The article's source for this claim is not SemiAnalysis; its Zitron. But once you dig through his article, Zitron links to a SemiAnalysis tweet [1] where they, as the paragraph states, implausibly assume gross margins of 75% to come up with their weird analysis of the subscription plans. Citing this for anything is weird, because afaik that 75% number is a total shot in the dark. We have no clue what their margins are. My take is that the only reason that 75% number is implausible is because it may underestimate the inference margins of Ant/OAI's API pricing.
The over investment by VC means that yeah, they are offering all of this below market rate. It's like Enron where they have to keep the scheme going, and dumping on retail investors is the only thing they can do now.
So we are going to go through a big IPO period. Everything will fall apart because VCs already extracted the growth value, and that will show up after the bag has been passed. Things will implode. What survives afterwards is what we will have.
androiddrew
My take is that Anthropic and OpenAI simply are NOT competing on price. 2 big players are often not enough to create tension on price.
Chinese models and open model providers are, indeed, competing on price, and the difference shows.
tacone
The unit economics might be just fine. We'll know more after IPO.
The drug dealer analogy has a darker side to it, however.
Once your dependent, they can drive up the price just because. It doesn't need to be for existential reasons.
fny
Lol I feel like no one has any attention span here. Tech shit is expensive in the beginning when it's new. It gets cheaper with time. This is a tech forum, don't we know this? Of course people overreact in both directions on both sides of the issue. It's a very fast technology, wait for things to settle before making grand declarations.
chermi
Shouldn't we know a better answer to these questions once Anthropic's IPO materials surface publicly? I understand, and maybe even expect, SpaceX's materials to be all over the place and skate on by any discussion of unit economics, but the nerds over at Anthropic might just be forthright enough to just tell us what their margin is on tokens as part of their IPO.
knuckleheads
The estimate that AI companies need to replace 27% of jobs to service their debt is interesting. But at least Anthropic and Meta seem to have their eyes on replacing software engineers.
There are ~1.6M software engineers on the US [0], earning a bit under 150k/year on average [1]. If AI companies captured all of that spend, that amounts to about 250B/year. The article assumed that they need around 300B/year to keep up with their debt.
At least based on Meta's recent behavior, forcing 30-50% of developers to switch to data labeling, it looks like that is actually their game plan.
comments (10)
I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train.
Now there's monitoring, reporting, alerting not just on overall cost but on "over-use" of best/priciest models based on total-or-percent tokens/dollars, etc. All of this comes with direct developer engagement & standardized management escalation for holding it wrong.
To me this customer behavior does not smell like a product you can 10x the pricing on to get profitable. We have exited the exploration phase and now ROI matters.
steveBK123
Sure, you can use AI to potentially replace software engineers, but the F500 are also terrified of not having accountability or making mistakes. They won't be firing any engineers. In that scenario, there's just no room for AI usage. If you have to be responsible for all the code, then... AI has to either manage it completely autonomously (which even Fable can't) or... humans have to be in the loop which means they still have to understand the code. The best way to understand the code is to write the code yourself. So there's no productivity gain to be had.
I'm pro-AI, but I think we're due for a big crash next year.
woeirua
Neither Anthropic nor OpenAI are subsidizing enterprise customers. Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high value $200/mo plan. Both organizations have moved to a "cheaper plan per user + API Pricing after that" (e.g. $20/mo + usage). The $100/$200/mo plans are for individuals only (of course, many individuals use these plans at work, but that's beside the point; they aren't selling this plan to enterprises).
> SemiAnalysis also analyzed the platform's gross margins, implausibly assuming that tokens were priced at 4 times the cost of generating them and: With the current subsidies, all it takes for a user to have a gross margin of at best negative 25% is for them to use as little as 25% of their rate limit.
The article's source for this claim is not SemiAnalysis; its Zitron. But once you dig through his article, Zitron links to a SemiAnalysis tweet [1] where they, as the paragraph states, implausibly assume gross margins of 75% to come up with their weird analysis of the subscription plans. Citing this for anything is weird, because afaik that 75% number is a total shot in the dark. We have no clue what their margins are. My take is that the only reason that 75% number is implausible is because it may underestimate the inference margins of Ant/OAI's API pricing.
[1] https://x.com/SemiAnalysis_/status/2064815045767213400?ref=w...
827a
So we are going to go through a big IPO period. Everything will fall apart because VCs already extracted the growth value, and that will show up after the bag has been passed. Things will implode. What survives afterwards is what we will have.
androiddrew
Chinese models and open model providers are, indeed, competing on price, and the difference shows.
tacone
The drug dealer analogy has a darker side to it, however.
Once your dependent, they can drive up the price just because. It doesn't need to be for existential reasons.
fny
chermi
knuckleheads
There are ~1.6M software engineers on the US [0], earning a bit under 150k/year on average [1]. If AI companies captured all of that spend, that amounts to about 250B/year. The article assumed that they need around 300B/year to keep up with their debt.
At least based on Meta's recent behavior, forcing 30-50% of developers to switch to data labeling, it looks like that is actually their game plan.
[0] https://en.wikipedia.org/wiki/Software_engineering_demograph...
[1] https://www.indeed.com/career/software-engineer/salaries
qnleigh
Anyone know what they are spending this on? Can't remember seeing one OpenAI ad.. Is it just pr and influencers? Ads in the US?
gizzlon