The Case for an Open Source GPU Box Snapshot Tool

A rented GPU box dies the same way every time. One Vast.ai reviewer put it about as plainly as it gets: the "machine simply went offline after 10 days and $1,000 spent generating data," and the environment they had built went offline with it. The reflex answer the industry offers is a save button, a snapshot, a stop-and-resume feature you get from whichever provider you happen to be renting from that week. I want to argue the opposite. The tool that snapshots and restores a whole GPU box is exactly the wrong thing to rent from your provider, and the case for an open source GPU box snapshot tool is stronger than it looks. That is the thing we have decided to build in the open, so this post lays out the reasoning rather than a product.

TL;DR: The primitives to snapshot and restore a GPU box already live in open source, mainly CRIU and NVIDIA's cuda-checkpoint. The fast, convenient versions built on top of them (Modal, NVIDIA Dynamo, Thunder Compute) are each locked to a single platform. The missing piece is an open, box-level tool that captures the whole machine and restores it on any provider. We are building that primitive in the open instead of renting an edge on someone else's.

What is a GPU box snapshot tool, exactly?

A GPU box snapshot tool captures the state of a running GPU machine so you can bring it back later without rebuilding it. In practice the phrase covers three things: a GPU memory snapshot for cold starts, a single-provider disk snapshot of a volume, and a full-box snapshot of the whole machine. Only the last one lets you move.

That distinction is the entire reason this category is confusing. When a provider says "snapshot," they usually mean the second kind, a disk image pinned to their datacenter, or the first kind, a memory-state trick that speeds up their own scheduler. Neither one gives you back the thing you actually lost when the box died: the filesystem, the repo checked out at the right commit, the virtualenv, the custom nodes, the models sitting at their original paths. A working environment is the whole box, not a volume you bolt onto it. A snapshot tool worth the name captures the box.

The stakes here are measured in real hours and dollars, which is why the fix keeps getting reinvented. Rebuilding a non-trivial GPU environment from scratch runs 30 to 60 minutes per cold start, all of it billed at GPU-hour rates while the card does nothing useful. A network volume that persists your data instead bills you around $0.10 per GB per month even while the GPU is off, and it cannot leave the datacenter it was created in. So the two mainstream answers are "pay to rebuild every time" or "pay to store, and stay put." A portable box snapshot is the third option, and it is the one nobody sells you outright.

The open-source primitives already exist, they are just not assembled yet

Here is the part that surprised me when I dug into it. The hard, low-level machinery for checkpointing a GPU process is already open source and already good.

CRIU, Checkpoint/Restore In Userspace, has been freezing running Linux processes to disk for years, and it is baked into Docker, Podman, and LXD. On its own it cannot manage a GPU, because "NVIDIA GPUs provide functionality beyond that of a standard Linux kernel." NVIDIA closed that gap with cuda-checkpoint, an open utility that toggles a process's CUDA state so the GPU side can be dumped and restored, and which is explicitly designed to be "combined with CRIU to fully checkpoint CUDA applications." NVIDIA even published a full walkthrough of the suspend, dump, restore, resume cycle.

Everyone in this space builds on those same two open pieces. When a founder building a competing cold-start product was asked on Hacker News how their approach worked, the answer was blunt:

"both approaches rely on cuda-checkpoint, since cuda-checkpoint is currently the main primitive NVIDIA exposes for interacting with GPU memory during checkpoint/restore."

Source: Cerebrium, on Hacker News

So the raw primitive is a shared, open foundation. What is missing is not the ability to checkpoint a GPU process. It is a tool that wraps the whole box, not just the CUDA state, and restores it somewhere else. That slot is open, and it is the interesting one.

Why does the convenient version keep shipping closed?

The convenient version ships closed because the incentives point that way. A provider's fast snapshot mostly exists to cut their own cold-start cost and to keep your workload on their cloud. Portability off their platform is directly against that interest, so the good, productized versions are built to restore only back into the same place they came from.

Look at what the best-in-class options actually let you do. Modal's GPU memory snapshots deliver real gains, "10x reductions in cold start time," but they require code changes (you annotate your app and opt in) and only run inside Modal. NVIDIA's Dynamo Snapshot turns a roughly one-minute cold start into a ten-second warm restore, a claimed 21x reduction in start time for a 120B model, but it is Kubernetes-native, experimental, and needs a privileged DaemonSet. Thunder Compute will snapshot an instance, but restore only lands as a Thunder template inside Thunder, snapshots are capped at 50, restore can take up to 8 minutes per 100GB, the docs warn "we do not provide explicit guarantees about snapshot durability," and they tell you to also copy your data to GitHub or cloud storage yourself. Even the open primitive underneath has hard edges: cuda-checkpoint needs driver 550 or newer, is x64 only, and waits for in-flight CUDA work to finish before it can freeze.

None of those is a bad product. Each one is a good product with a leash. The leash is the point. A snapshot that can only restore where it was born is a convenience feature for the vendor's cloud, not a portable artifact you own. And the orchestration frameworks that are open, like SkyPilot and dstack, solve a different problem entirely: they move your job across clouds, but you still assemble the environment from scratch on the other side. They provision the box. They do not carry its state.

The case for building it open, in public

If the primitive is shared and open, and the productized versions are all leashed to one cloud, the honest move is to build the box-level tool as open source and compete on the layer above it. That is the open-core bet, and it is not charity. As Open Core Ventures puts it, "open source is the distribution and R&D strategy, while proprietary features are the monetization strategy." You open the primitive that only has value if people adopt it, and you charge for the managed layer that runs on top.

There is a sharper version of this argument for anything that wants to become a standard. A recent essay on open source as go-to-market made the case directly: "if you need to establish a standard before a competitor does, open source is often the only credible path." A snapshot format for GPU boxes is exactly that kind of thing. It is only useful if it is trusted, inspectable, and not something that can be taken away when a provider changes its pricing or its mind. You cannot ask people to bet their environment on a format they cannot read, from a company that also wants to keep them on one cloud.

That is why we are building this in public rather than as a quiet internal feature. The primitive, the snapshot and restore of the whole box, is the part we think should be open and owned by whoever uses it. The managed layer, pausing an idle box to zero and resuming it on whoever is cheapest, is where a real business can live without holding your environment hostage. Owning an open standard is a better long-term position than renting a feature edge on top of someone else's closed one.

The ecosystem is already reaching for this

The demand is not hypothetical, and it is not coming from us. Inside vLLM, the most widely used open inference engine, there is an open feature request for exactly this, from a user who tried the closed versions and wanted the open one:

"I noticed that Modal and InferX have implemented the CUDA checkpoint/restore API to drastically reduce cold start. I tried both and it seems to work extremely well. It would also allow hosting platforms to host more models and make users only pay for working GPU time."

Source: vLLM issue #33930

That is the pattern across every thread I read: people have used the fast closed version, they liked it, and now they want a portable open one they can run anywhere. Some are not waiting. There are already early open-source projects claiming S3-portable snapshots and 17x faster cold starts for large models, self-reported numbers, but a clear signal that an open, portable format is forming whether or not any single vendor blesses it. When practitioners keep hand-rolling the same missing tool, it is usually a sign the tool should exist as a shared standard rather than a per-vendor feature. The whole genre of rclone-to-S3 scripts and auto-checkpoint-every-15-minutes cron jobs that fills the gap today is a product-shaped hole, and it is the same hole we wrote about in our breakdown of persistent cross-provider environments.

To be straight about where we are, because this is a build-in-public post and not a launch: the tool we are building, ogre, snapshots a full GPU box, the filesystem, the environment, the models, and restores it on a different provider. We have validated one full round-trip end to end, a pause and resume that brought back a complete environment byte-for-byte on a new box. The bigger claims, cross-provider reliability across many restore cycles and a head-to-head on completeness against single-cloud snapshots, are still being proven in public rather than asserted. That is the honest state today, and it is why this is an argument for a category and not a "go install it" pitch.

What an open source GPU box snapshot tool should actually do

If you are evaluating anything in this space, or just want to know what to hold out for, here is the short checklist the argument above reduces to.

• It captures the box, not a volume. The full filesystem, the repo at its commit, the virtualenv, the installed packages, the model files at their original paths. If it only saves a data disk, you will still rebuild the environment.
• Restore is provider-agnostic. A snapshot you take on one provider should restore on another. If restore only lands back inside the same cloud, it is a convenience feature, not portable state, and you can read more on that split in our writeup on recovering models across providers.
• The format is open and inspectable. You should be able to read what a snapshot contains and restore it without asking permission. That is the whole point of not renting the primitive.
• Idle state costs nothing to keep. A snapshot sitting in your own storage should not bill you like a running volume. The reason to snapshot at all is to stop paying for a box you are not using.

Price is table stakes and always will be. The thing that actually keeps you from moving to a cheaper box is the fear of rebuilding your environment on the other side. An open, portable, box-level snapshot tool removes that fear, and it should belong to the people using it, not to the provider they happen to be renting from this week.

About the author

I am on the team at Aquanode. I am writing this as someone who has spent a lot of time reading provider docs, CRIU and cuda-checkpoint issues, and the community threads where people keep asking for an open version of the closed thing they just tried. What I can speak to firsthand is the building side: we are developing ogre, an open snapshot-and-restore tool for GPU boxes, in public, and validating it round-trip before we make bigger claims about it. The sources below are where the quotes and numbers come from.

Sources

• CRIU and GPU checkpointing: the open process checkpoint/restore foundation and its GPU limitation.
• NVIDIA/cuda-checkpoint: the open CUDA checkpoint utility, designed to combine with CRIU.
• NVIDIA, "Checkpointing CUDA Applications with CRIU": the suspend, dump, restore, resume walkthrough.
• NVIDIA Dynamo Snapshot: K8s-native checkpoint/restore on CRIU + cuda-checkpoint.
• Modal, GPU Memory Snapshots: "10x reductions in cold start time," platform-bound, requires code changes.
• Thunder Compute snapshots docs: restore-as-template, single-cloud, "no explicit guarantees about snapshot durability."
• SkyPilot and dstack: open cross-cloud orchestration that provisions boxes but does not carry their state.
• vLLM issue #33930: the open community request for portable GPU snapshotting.
• Hacker News thread on GPU cold-start snapshotting: everyone depends on the same open cuda-checkpoint primitive.
• Open Core Ventures, "Building venture-scale open core": open source as distribution, proprietary features as monetization.
• Vermilion Cliffs, "Why Open Source as GTM Is Having a Moment": open source as the path to establishing a standard.
• Vast.ai Trustpilot review: the "machine simply went offline after 10 days and $1,000 spent" reclaim story.