A GPU server for your own LLM

Which size you need: 8B models run on 8 GB cards and answer simple questions and summaries well. 14B models want 12 to 16 GB. 32B models are where self-hosted models start to feel close to the big hosted ones for coding and writing, and they need 24 GB. A 70B model needs 48 GB, on one card or split over two. Context costs memory too: Ollama’s documentation says the memory needed grows with the context length and the number of requests served at once, so leave a few gigabytes free. Change the GPU memory filter to match your model.

Per hour or per month: GPU clouds bill by the hour, and the bills here show what a server costs when it runs all month, with the IPv4 address and any setup fee added. If you only need the model a few hours a day, divide accordingly and check that the provider stops billing a stopped server; some keep charging for the disk.

When you don’t need a GPU at all: an assistant that calls a hosted model’s API, such as OpenClaw, runs on a small VPS. Small models of 1 to 3B can run on a CPU with enough memory, slowly. For anything bigger, the GPU is the server.