NVIDIA AI quietly released something worth paying attention to: a technique called Star Elastic that lets you run three differently-sized models from a single checkpoint file. No separate downloads, no duplicated cache state, no storage bloat. One file, three models. If inference engines adopt this properly, it could meaningfully change how local model deployments handle variable workloads.
What Star Elastic Actually Is
Star Elastic is applied to Nemotron Nano v3, NVIDIA's hybrid Mamba-Transformer-MoE architecture. The base model sits at 30B total parameters with 3.6B active parameters. Through elastic model slicing, two smaller nested variants are derived from that same checkpoint: a 23B model with 2.8B active parameters, and a 12B model with 2.0B active parameters.
The key word is active. These aren't three separate models with different total parameter counts. They're subnetworks within the larger model, selectively activated depending on which tier you're using. Think of it less like three different models and more like three different operating modes of the same model.
The best analogy from the LocalLLaMA thread captures it well:
Think of this as like scalable video coding, you have a UHD stream, but strip some layers and you have a HD, or SD stream, it's all a single file stream, not multiple ones.
UHD, HD, SD. One file.
The Three Tiers at a Glance
- 30B / 3.6B active: Full quality, higher compute cost
- 23B / 2.8B active: Middle ground for balanced speed and output quality
- 12B / 2.0B active: Fastest tier, best for high-volume draft generation
All three live in a single checkpoint with zero storage overhead from maintaining three separate deployments. Critically, the nested models share the same KV cache infrastructure, so you can switch between tiers during inference without duplicating cache state. That's where the real VRAM savings come in.
KV Cache Sharing and Why It Matters for VRAM
Running three separate models of different sizes normally means three separate KV caches, which adds up fast in memory. Star Elastic sidesteps this entirely. Because the smaller models are subnetworks of the larger one, they share cache state. One commenter in the thread put it plainly:
If inference engines can dynamically scale compute per request without duplicating cache state it'll save a ton of VRAM overhead.
This is particularly relevant for local deployments where VRAM is the binding constraint. If you're currently running a single model at a fixed size because you can't afford the memory footprint of keeping multiple sizes warm, Star Elastic offers a way out.
Speed vs. Quality: The Real-Time Tradeoff
The practical pitch here is a draft-then-verify workflow. The 12B tier handles high-volume reasoning generation quickly, while the 30B tier reviews and filters for quality. According to figures referenced in the thread (sourced from the Hugging Face model page, though exact hardware specs were not confirmed), the 12B model can reportedly push out around 70,000 tokens of reasoning in roughly 10 seconds. Treat that number as directional until hardware context is confirmed.
One commenter described the intended workflow:
The 12b can chug out 70,000K of reasoning in literally 10 seconds at it's speeds, then the 30b can look at it and filter what's reasonable.
This is functionally similar to speculative decoding, where a smaller draft model proposes tokens and a larger verifier model accepts or rejects them. The difference is that Star Elastic makes the size relationship explicit and trained into the checkpoint, rather than coordinating two entirely separate models after the fact.
How It Compares to Existing Approaches
Speculative decoding is the closest analog in spirit. A fast draft model generates candidates, a larger model validates. Star Elastic does something similar but keeps everything within one checkpoint and one shared cache, reducing coordination complexity.
Mixture of Experts is the closest structural analog. The thread framed it this way:
It's like an MOE where there's the big expert, medium expert and small expert, and they've trained the router to select between different sized experts.
Traditional MoE routes between expert subnetworks of similar size. Star Elastic routes between expert subnetworks of different scales, which is a meaningful distinction for latency-sensitive tasks.
Separate model deployments are the obvious alternative without this. You'd download a 30B, a 23B, and a 12B separately, manage three KV caches, and coordinate between them in your inference pipeline. Star Elastic collapses all of that into one file and one cache.
One caveat worth flagging: a comparison to Qwen QwQ-32B came up in the thread, with at least one user suggesting QwQ-32B produces better results than Star Elastic on Nemotron. That claim is anecdotal and hasn't been independently verified. Wait for a proper side-by-side benchmark before drawing conclusions.
What Needs More Clarity
A few things about Star Elastic aren't fully settled from available information. The exact release date is unclear (the Reddit post is from approximately mid-January 2025, described as 11 days old at time of posting, but this is unconfirmed). The 70,000-token-in-10-seconds figure lacks confirmed hardware specs. It's also not entirely clear whether the slicing approach is fully zero-shot or required some retraining to produce well-behaved nested subnetworks. These are worth watching as more independent testing surfaces.
Bottom Line
Star Elastic is a practical idea with real implications for local model deployment. If you're running Nemotron Nano v3 locally and toggling between speed and quality on different tasks, this is worth exploring. The single-checkpoint, shared-cache design solves a genuine operational headache. The open question is how it holds up against competing reasoning models in rigorous benchmark conditions, and that data isn't in yet.
Sources
- LocalLLaMA: NVIDIA AI Releases Star Elastic, One Checkpoint Three Models