GLM 5.2 and the coming AI margin collapse

LoganDark

I hope cheaper inference eventually means faster speeds at the lower tiers. I don't want to settle for 100 t/s, but I don't want to pay $10 per prompt either

KronisLV

They have a vision MCP to make up for the model itself not having the capability natively: https://docs.z.ai/devpack/mcp/vision-mcp-server I also found their web search to be mostly okay. Furthermore, in case this is of interest to anyone, if you use their ZCode harness then you get bigger Coding Plan quotas: https://zcode.z.ai/en Used it for a bit, it sits somewhere between OpenCode Desktop (still new but nice) and Claude Desktop (recent versions are good). As for GLM 5.2 as a model - with max thinking it’s generally satisfactory, somewhere between Sonnet 5 and Opus 4.8, better than DeepSeek V4 Pro for sure. Pricing wise, the subscription doesn’t seem as good as expected. I spent like 60% of the weekly limits of the Pro (50 USD) plan in one day, only because each 5 hour limit only gave me 20% to spend, otherwise it’d be 80-100%. Not even doing anything crazy, just parallel long form work on 2 projects with about 96% cache rate and at most 3 parallel code review sub-agents. Their Max (100 USD) subscription would last me the whole week, but so does Anthropic for the same money and so would OpenAI. Off-peak is more palatable but I can’t just twiddle my thumbs at 9 AM to 1 PM local time. Proper savings would show up with the Max plan and yearly billing, but that’s more of a tough sell.

budsniffer952

> the least understood upcoming shift in AI economics. Then proceeds to talk about something in the AI news every day. Hey, did you guys hear? Open source models are cheaper and their quality is increasing! So, first, by no measure is GLM5.2 as good as Opus. Second, yes, open source models will put pressure on margins...eventually. Everyone knows that. But do you think today's AI business model is the same as tomorrow's?

fny

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure components like Redis and Elastic Search have Apache equivalents, but they still command healthy margins. I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue. It's nobody gets fired for buying IBM all over again.

softwaredoug

I also think we’ll approach a point where increasing intelligence is not really going to suddenly improve most work tasks. I bet that’s already happened actually. We’re oohing and aahing about models, when the ones a few versions ago did a good enough job for most of the dumb coding, etc we do

throwdbaaway

Seems like a pretty pointless post that still centers around output tokens. In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.

montroser

This article only promises to get into "the coming AI margin collapse" in a yet to be published "part two". This part only makes the point that GLM 5.2 is pretty good (no shit).

benjiro29

> I'd be very surprised if it wasn't more than 50% cheaper for nearly all workflows, for a very similar level of quality. If your using pure API ... providers like neuralwatt cut that cost down even more by using energy as the actual cost. So GLM 5.2 is more expensive then GLM 5.1 on their service (those thinking tokens), compared to API costs, its dirt cheap. And way more tokens then the zai subscription delivers. We are seeing a move towards more realistic pricing on actual consumption based usage. Be it DeepSeek, Xiaomi (MiMo), or zai's GLM via neuralwatt. The main issue facing subscriptions a-la-carte usage, is that a lot of the heavy hitters really drain the resources. And that as a business model can not survive without ... a) increasing the prices. b) everything goes to actual token/energy usage based billing but with more realistic pricing, and not the bloated API prices that are focused on companies. We shall see what the future holds but things will change.

felixfurtak

> It turns out that nearly every agentic session does a lot of web searching for looking up items This is why Google will win the race over most of its competitors. They own search.

0xbadcafebee

Yes, margin on model inference is high with some providers. If you just wanted inference (at cost), you'd buy a GPU, or rent one from AWS or Microsoft. But you're not paying OpenAI/Anthropic for inference. You're paying them for a platform. Every feature OpenAI/Anthropic bake into their applications, models, online services, etc - anything that isn't pure LLM text generation - is a custom integrated add-on service that LLM weights do not include. Even if open weights became cheaper and better than OpenAI/Anthropic, most people would still pay for OpenAI/Anthropic, because they give you things the weights alone don't give you. Comparing Z.ai GLM 5.2 to Claude Code w/Opus 4.8 is like comparing Linux Kernel 7.0 to Microsoft Windows 11. If you don't know much about computers, you'd say these are the same things. If you know a lot about computers, you know the latter has a thousand extra things that make a huge difference in what it does out of the box. Which one you use speaks to what kind of customer you are. Sure, GLM 5.2 doesn't have vision; but an AI power user can plumb together any VLM with the text generation of GLM 5.2 in most AI harnesses, just like a Linux power user can combine the Linux kernel with KDE Desktop. Most people don't use Linux and KDE, because it's unpopular, difficult to use, hard to get support for. Instead they pay for Windows or Mac, because there's lots of support, with a giant company pouring money and effort into filling all the usability gaps, making it seamless. Most people don't pay for the cheapest possible thing. They pay for the thing they can afford that improves their life while making it easier. An open weight alone is almost completely unusable by itself (like the Linux kernel), compared to an AI platform (a completely usable system). If you're constantly wondering about when open weights will reach parity with OpenAI/Anthropic, you're a Linux person. If you just pay $20/$50/$100 for OpenAI/Anthropic without thinking about it, you're a Windows/Mac person. There is nothing wrong with either of these groups, but they are fundamentally different, and always will be. An LLM weight is simply a different category of thing than an entire AI platform/provider.

sailfast

How long will that $4.40 rate persist? Until we know more about the real unit economics it will be damn near impossible to rely on steady inference costs or make them predictable at the enterprise level. Gonna be a wild ride for awhile.

zuzululu

i would use glm 5.2 if the servers weren't in china i mean i guess my employers wouldn't know the difference but i'd like to play it safe and keep everything in america

_pdp_

IMHO, cheaper inference means higher costs overall :) because everyone will use more thus driving up the investment required to stay current or to compete. Switching models is also kind of easy but not plug-and-play. Most harnesses out there do very poor job with the open weight models. Unlike Opus, GLM 5.2 ends up in loops and hallucinates a lot more. If your harness is built on the expectation that the LLM will perform well, then switching to GLM 5.2 will be an uphill struggle. We had to refactor our harness and introduce more defences because of GLM. The cost savings are substantial. Obviously it really depends on your workloads but it is noticeable cheaper for agentic work. Coding - I don't know. We do have some coding agents on GLM 5.2 and what I noticed with some landing page experiments that the results between GLM and Opus are identical - they might be using the same training data? Obviously Opus is still substantially better model. I don't think there is an argument to be made here but GLM 5.2 is cost effective and really good too. Overall, we switched all of our internal agents to GLM 5.2 and because it is Open Weight we are in talks to get the model from certain geo locations giving us more freedom as well as extra protection. Overall I think this industry will be in much better place because of GLM 5.2 and whatever open-weight models come next.

spyckie2

It’s important that none of these entities can collude to price fix. Having China be the competitor ensures that. Basic microeconomics is still the easiest way to understand token economies. How is it not a competitive market (where profits go to zero?). Anything A or O does to keep more margin, any competitor can copy or choose to undercut, and undercutting has the benefit of collecting training data. So what is going to stop gross profit of tokens going to zero except for collusion/price fixing?

redrix

The fact that these Chinese models are getting close to “Opus-grade” despite costing 6x-8x less is huge. As the token bills start to come in, those economics will be harder to ignore (regardless of the origin of the LLM); especially as there will be many CIOs sweating over their quick and costly AI initiatives showing little ROI. My hope is that the EU also steps up their own competition in the frontier model space so that it’s not just China v USA.

samuelknight

Inference has been decreasing in cost by about 10x per year since 2023.

maxglute

How fast is glm 5.2 in western hosts? It's doing everything I want it to, but going through PRC host it takes like 5-10 times longer. Not sure if that is nature of modest or PRC computer infra/routing.

dbalatero

> Of course, this was a hugely poor read of where the costs actually lie in AI. Training - while no doubt capex intensive - is a fixed, up-front cost. You spend hundreds of millions to train a model, then you are "done". I don't understand this point that people make. If you're consistently needing[0] to train new models and the cost of training relative to the % improvement seems to go higher, isn't this just a constant cost that you continue to bear? The footnote seems to allude to this, but then sort of waves it away anyways. Also are there continuing incremental training costs to keep models relevant? Or do they only have knowledge of events up to the day they were trained? [0] needing, because you have competitors and people expect more and more.

gnarbarian

the economics of this are a little counterintuitive. is there a market saturation point for intelligence? how about for software? it seems like the more you have the more you want because you're trying to do more things. as the models get smarter I get busier because I'm doing more things...

yalogin

I don’t understand the argument here. The article doesn’t describe a collapse or the breadcrumbs for it. The only argument I can put together is companies hosting the open source models in house or use some service like Amazon that could potentially host them and so replace the frontier models. Data center and specifically infra to host llms is still the main sticking point given the security concerns about data going to china. The article doesn’t make these arguments coherently