r/stockmarket
Hyperscalers are implementing techniques that could compress memory usage by up to 40x
Hyperscalers' new memory compression tech like NVDA's KVTC threatens the 'no more cycles' narrative for memory stocks as ASPs peak.
- DRAM/NAND spot prices appear to have peaked and will likely decline due to growing competition from Chinese memory manufacturers.
- Memory stock prices historically peak 5-8 months before actual DRAM prices peak.
- Hyperscalers are developing techniques, such as NVDA's KVTC, that can compress memory usage by 20-40x, fundamentally challenging the 'cyclicality is dead' narrative.
Everyone is continuing to pile into DRAM etf, MU, and SNDK right now after a monster earnings report by MU. The primary driver of the revenue, earnings, and margin growth has been the drastic increase in DRAM/ NAND prices amidst increasing demand for memory as part of the AI buildout. As a result, memory companies have been a direct beneficiary of the insane CapEx spending being done by the hyperscalers.
The new narrative has been that the historically cyclical subsector will no longer be cyclical moving forward, as there will now be a constant, continuous demand for memory as chips/ data-centers continue to evolve.
Again, an interesting part about Micron’s earnings was that shipment growth did not drastically drive revenue growth (shipments even decreased in some business units), but the massive increase in ASPs is what drove the blowout sales #s. Now DRAM/ NAND flash spot prices have already seemed to top out, and they’ll likely continue to move downward as several Chinese memory players have been growing their presence in the market. Additionally, in past DRAM cycles, memory stock prices peaked about 5-8 months before actual DRAM prices peaked.
But what’s not discussed AS MUCH in the “cyclicality is dead” debates is the measures that the big memory buyers have been working on and planning to implement/ build upon to optimize their memory usage moving forward.
The attached table shows techniques developed by US big tech companies that could compress memory usage by 20-40x.
A brief description of NVDA’s KV Cache Transform Coding (KVTC) method:
KVTC is a method that borrows from classical media compression to dramatically reduce the size of the Key-Value (KV) cache in Large Language Models (LLMs). By applying PCA and entropy coding, it shrinks memory demands up to 20× (and up to 40× for certain use cases) without modifying model weights
HBM space saved by KVTC compression is going to be immediately consumed by hyperscalers deploying next-generation trillion-parameter LLM weight. This KVTC will fuel the market even more
This is what I thought. But your number is off - 1T params is not very big these days, Mythos/Fable is estimated to be 10T params model. GPT-4 from early 2023 was estimated to be 1.8T model.
I think we'll see more 10T+ models soon especially with the new memory improvements.
Jevons Paradox says fuck your puts
This message brought to you by SK Hynix (wink)
Right but when you bottleneck narrative disappears, your stock will no longer be elevated. The market currently only likes bottlenecks. They don't actually care how much you're growing or what your margin is. Otherwise SaaS and Mag 7 wouldn't have been sold off.
Compression you say? What’s the Weissman score with this KVTC method??
Their new middle-out technique will break the Weissman score.
5.1 it’s a new record!
Same articles came out last earnings. Did the same with google and Nvidia. Just manipulation of retail
Efficiency improvements often expand the market because they lower the cost of doing more work. That’s especially true for agentic AI.
Suppose Google cuts KV cache memory by 6× using TurboQuant. Instead of running one agent, they may now run six agents simultaneously, support much longer context windows, or serve many more users. The saved memory is often immediately reinvested into capability.
The chart also tells you where the industry believes the bottlenecks are. None of them eliminate the need for memory.
Like adding another lane to the highway. Traffic problem doesn't get solved, but it does enable people to live farther from work.
Wait till your port gets compressed by 95%
I'm in chips since 2011.
Been hearing cyclicality is dead every upcycle. Good for me, I get richer. But the downcycle always comes after.
A.L.W.A.Y.S
This RAM shortage in particular.... It's days are SO counted. China is ramping up production right now. And we all know what happens when the Chinese government decides to corner a market. They'll overproduce like crazy, and the price of RAM will go back to where it was before 2020, perhaps even less
This is what I never understood.
There is a practical case for A.I production and its disinflationary impact but it was always about a couple of key questions that one - trick pony bulls are violently opposed to:
- what is the economics like after the CapEx spending?
- if memory cyclicity no longer holds true, where will the additional liquidity, almost all of which are currently from bond offerings, free cash flow, debt market and private credit market, that funds these memory and data centers buildout come from?
- hyperscalers are used to dominating the market: theyay be forced to accept exorbitant prices now but behind closed doors they are strategising means to break memory's maker grip on the memory market.
Didn't Nvidia's scare from Deepseek, Google's TPU and AMD taught anyone anything?