Know what interruption means on the product
AWS documents interruption notices for EC2 Spot; Vast.ai documents interruptible instances that can be paused based on rental priority. The exact signal and resume behavior differ. Build recovery around the product you choose.
On-demand does not mean hardware cannot fail. It changes the purchasing and interruption model, while backups and application recovery remain necessary.
| Condition | Interruptible | On-demand |
|---|---|---|
| Flexible overnight batch | Good trial candidate with checkpoints | Useful if recovery is costly |
| Interactive service | Needs a suitable baseline and failover | Simpler starting capacity model |
| Hard deadline | Price delays and retries explicitly | Confirm capacity for the full window |
| No tested restore | Fix recovery before committing | Still needs backups |
References: EC2 Spot interruption notices · Vast.ai instance types
A cheaper rate can lose on repeated work
Consider an illustrative eight-hour job. At $2 per GPU-hour it costs $16 in compute. At $1 per GPU-hour, a run that consumes ten paid hours after retries costs $10. If repeated failures expand it to eighteen paid hours, it costs $18.
The example ignores storage and engineering time to keep the arithmetic visible. Include them in a purchasing estimate, and keep waiting time separate from billable time. A job can be cheap but finish too late.
Reduce the amount of work at risk
Split independent batches into smaller tasks and save progress outside the worker. For training, choose a checkpoint interval from the cost of saving and the cost of repeating work. Test a restore before a long run.
Use a stable job identifier when publishing results so a retry does not create duplicate outputs. A queue should distinguish pending, running and durably completed work.
Use a deadline-aware policy
For a flexible job, start on interruptible capacity and define when to switch to another allocation if progress falls behind. Verify that fallback capacity and data access actually exist before depending on that plan.
For a customer-facing service, assess the entire failover path under load. A lower rate should not be purchased at the expense of a latency promise the application cannot meet.
Questions readers ask
Does an interruption warning guarantee time to save?
No. The checkpoint may take longer than the warning, and failure paths differ. Save during ordinary operation too.
Is on-demand failure-proof?
No. It still requires durable state, monitoring and a recovery plan.
Sources and editorial notes
Primary documentation reviewed October 9, 2026. Provider descriptions reflect published documentation. Recommendations are our analysis; worked figures are illustrative unless explicitly stated otherwise. No independent provider benchmark or uptime ranking is claimed.
Read the editorial policy. NeuraMesh is a prototype, not live verified GPU inventory.