How to rent a cloud GPU for AI training and inference

By the end of this guide you will have rented a cloud GPU, run a training or inference job on it, saved your results somewhere safe, and shut the machine down so it stops costing money. The service that does this is a GPU cloud: a provider that lets you rent NVIDIA hardware by the second, minute or hour instead of buying it.

Choosing a cloud GPU for AI is mostly a matter of matching the job to the way you want to pay. This guide explains the main types of provider, walks through a first rental step by step, and compares seven options from our Developer platforms category.

Key takeaways

• Work out your memory needs and how long the job runs before you look at prices, then rent the smallest GPU that fits.
• The main decision is the rental style: a dedicated instance, a serverless service that scales to zero, a marketplace of interruptible machines, or a reserved cluster.
• Billing is usually prepaid or metered, and instances keep costing money until you stop them. Some providers stop or delete your machine when the balance hits zero.
• Save checkpoints and results to storage that outlives the instance, especially on spot or interruptible capacity.

What you need before you start

• A job you can describe. For example, fine-tuning a model, serving an open model, or running a batch of inference. Know which framework it uses and roughly how much GPU memory it needs.
• Your code and data in portable form. A container image or a repository you can pull, plus datasets in object storage or on your machine.
• A payment method or prepaid credit. Runpod, Vast.ai and Nebius work from prepaid credit, and Modal requires a payment method on file.
• Basic command-line skills. Many options are reached over SSH, JupyterLab, a CLI or an API.
• Time. A first rental can be running in under an hour. Training itself takes as long as it takes.
• Difficulty. Moderate. Serverless platforms are easier; clusters and Kubernetes are harder.

Which kind of AI tool fits this job

Pick by how steady your workload is and how much you want to manage.

On-demand GPU instances for training and experiments. Lambda rents Linux virtual machines with one, two, four or eight NVIDIA GPUs, preinstalled with a software bundle that includes CUDA, PyTorch and TensorFlow, and billed in one-minute increments. Runpod offers dedicated Pods billed per second, with data ingress and egress free. Nebius AI Cloud lets you launch up to 32 GPUs from a self-service console without a sales call.

Serverless GPUs for inference and bursty jobs. Modal runs Python code on GPUs and CPUs that scale up and down automatically, billed per second, with no charge once an app has scaled to zero. Runpod Serverless also has workers that scale to zero. This style suits inference endpoints where traffic is uneven.

A marketplace when price matters most. Vast.ai lists GPUs from many data centers at prices set by supply and demand, with On-Demand, Interruptible and Reserved instance types. The trade-off is that interruptible machines can be reclaimed.

Large-scale capacity and committed plans. CoreWeave publishes per-hour prices for NVIDIA instances and offers Reservations, Flex Reservations, On-Demand and Spot plans, aimed at large training and inference work. Lambda's 1-Click Clusters and Nebius's commitment discounts target the same need.

A serving layer on top of GPUs. BentoML is an open-source model-serving library with a managed platform that autoscales, scales to zero, and can run on its own cloud GPUs or in your own cloud account. It helps when the hard part is serving, not renting.

Step 1: Size the job

You finish this step with a minimum GPU memory figure, a rough run time and a budget ceiling.

1. Note the model size and the precision you will use, and whether you are training or only running inference.
2. Read the model's documentation for memory guidance, and add headroom for the batch size and context length you plan to use.
3. Estimate run time from a small trial on any machine you already have.
4. Multiply the hourly rate of a candidate GPU by the hours, and add storage and any other charges.

Check it worked by writing down "needs at least X GB of GPU memory, about Y hours, budget Z."

Step 2: Choose a rental style

Match the way you pay to the shape of the work.

1. For a one-off training run, rent an on-demand instance and stop it when finished.
2. For an inference API with uneven traffic, prefer serverless that scales to zero, such as Modal or Runpod Serverless.
3. For batch work that can restart from checkpoints, consider cheaper interruptible or spot capacity from Vast.ai, CoreWeave or Nebius.
4. For steady, long-running or multi-GPU work, ask about reservations and clusters, which several providers quote through sales.

Check it worked by being able to say why the style fits, in one sentence.

Step 3: Compare providers on price and limits

Hourly rates are only part of the picture, so compare the whole bill.

1. Use the table below to see each provider's billing unit and stated limits.
2. Check what is billed besides the GPU: storage, networking, stopped instances. Vast.ai says stopped instances keep accruing storage charges.
3. Check region coverage. Lambda ties each instance to a single geographic region.
4. Check the availability of the GPU type you need before you commit.

Check it worked by having a shortlist of two providers with an estimated total cost each.

Step 4: Create an account and add credit

Several of these providers are prepaid or require a payment method, so get this done before you need the machine.

1. Sign up for an account.
2. Add credit or a payment method. Vast.ai lists a $5 minimum deposit, and Nebius suggests adding $25 or more to start.
3. Set a spend limit or low-balance alert if the provider has one. Runpod lists low-balance alerts and a default spend limit, and Modal has spend budgets.
4. Create SSH keys or API tokens as the provider asks.

Check it worked by seeing your credit balance in the console.

Step 5: Launch an instance or endpoint

Start the smallest configuration that meets your Step 1 numbers.

1. Choose the GPU type and count, and a template or image. Vast.ai lists templates such as PyTorch, TensorFlow and ComfyUI, and Lambda instances come with a software stack and JupyterLab.
2. Attach persistent storage if you need your files to survive. On Runpod, Pods without a network volume are terminated with data lost when the balance reaches $0.
3. Launch and wait for the machine to become ready. Vast.ai warns that fresh image pulls can take a while to start.
4. Connect by SSH, JupyterLab, the CLI or the API.

Check it worked by running a command that prints the GPU's name and available memory.

Step 6: Run a short test, then the real job

Never spend a full budget on a first launch.

1. Run a few steps of the job to confirm that data loads, the GPU is used, and memory is enough.
2. Note the speed and extrapolate the cost for the full run.
3. Write checkpoints regularly to persistent storage or object storage, and make the job resumable.
4. Start the full run, and watch the first few minutes for errors.

Check it worked by resuming from a checkpoint after deliberately stopping the job.

Step 7: Save results and shut everything down

This is the step that protects your budget.

1. Copy outputs, logs and trained weights off the machine.
2. Stop or delete the instance. A stopped instance may still bill for storage, so delete it when you no longer need the disk.
3. For serverless endpoints, confirm they scale to zero when idle.
4. Check the provider's billing page the next day for unexpected charges.

Check it worked by seeing no running instances in the console and a flat balance afterwards.

Tools for this job compared

Tool	Good fit for	Pricing	Stated limits
Runpod	Pods and serverless endpoints; web and API	Paid	Pods stop at a $0 balance and those without a network volume are terminated; credits are non-refundable
Lambda	On-demand NVIDIA VMs with a preinstalled software stack; web, API, CLI	Paid, from $0.69, usage-based (checked 24 Oct 2026)	Each instance is tied to a single region; cluster pricing requires contacting sales
Vast.ai	A GPU marketplace with interruptible options; web, API, CLI	Paid	Credit is prepaid; interruptible instances can be reclaimed; disk size is fixed at creation
Modal	Serverless GPUs defined in Python; web, API, CLI	Freemium, paid from $250/mo (checked 24 Oct 2026)	Starter plan includes up to 3 seats and 100 containers; a payment method must be on file
Nebius AI Cloud	Self-service NVIDIA clusters with managed Kubernetes or Slurm	Paid, from $0.06, usage-based (checked 24 Oct 2026)	Self-service access is capped at 32 GPUs; spot prices change as often as every 15 minutes
CoreWeave	Large-scale training and inference with spot and reserved plans	Paid	Reservations pricing not listed publicly; spot capacity can be interrupted; On-Demand depends on available capacity
BentoML	Serving open and custom models, in its cloud or yours	Free trial	Pay As You Go has community Slack support only; requires engineering knowledge of model serving

Prices and plan limits change, so confirm the current terms on each vendor's site before you commit.

Common mistakes to avoid

• Leaving the instance running. GPU time is billed whether or not your code is busy. Fix it by stopping or deleting the machine as soon as the job ends, and by setting alerts.
• Keeping the only copy of your work on the instance. Disks can be lost when a balance runs out or an interruptible machine is reclaimed. Fix it by writing checkpoints and results to persistent storage.
• Renting more GPU than the job needs. Larger cards cost more per hour and may not speed up a small model. Fix it by measuring memory and speed in a short test first.
• Comparing only the hourly rate. Storage, stopped-instance charges, region choice and interruptions change the real cost. Fix it by estimating the whole run, as in Step 3.
• Using interruptible capacity for jobs that cannot restart. Spot and interruptible machines are cheaper for a reason. Fix it by making the job resumable or by choosing an on-demand plan.

How we put this together

This is an evidence-based guide built from each tool's listing in our directory, which is compiled from the vendor's own site and pricing page. It is not a review based on using the tools. We don't rank tools, and nobody paid to be included. Some outbound links may be affiliate links; see our disclosure.

Frequently asked questions

How much does it cost to rent a cloud GPU?

It depends on the GPU and the provider. Published examples include on-demand rates starting at $0.69 per GPU per hour on Lambda and an A100 PCIe Pod example at $1.79/hr on Runpod, while an 8-GPU H100 node on CoreWeave is listed at $49.24 per hour on demand. Check the live price for the exact GPU you need.

Is there a free cloud GPU for AI?

Not generally. Modal's Starter plan costs $0 plus compute and includes monthly compute, and its listing says startups and academic researchers can apply for free compute credits. BentoML lists a free trial with a one-time compute credit. The other providers here list paid usage only.

Should I use serverless or a dedicated GPU?

Use serverless when traffic is uneven and you want to pay nothing while idle. Use a dedicated instance for long training runs or steady inference. Serverless adds cold-start considerations, so test the first request after idle time.

Can I rent a GPU for a single hour?

Yes with several providers. Lambda bills in one-minute increments, and Runpod, Modal and Vast.ai bill per second. Some deployments need a minimum balance, for example Runpod needs at least one hour of credits to deploy an on-demand Pod.

Is my data private on a rented GPU?

That depends on the provider and the plan, and the listings here do not settle it. Read each provider's data and security terms before you upload anything confidential, and remove data from instances when you finish.

Compare more options in our Developer platforms category.