Hardware Buying Guides
Hardware Buying Guides

RAM Offloading for Local LLMs: A Practical Hardware Buying Guide

System RAM can make large local language models loadable, but capacity alone does not make them fast. This guide explains CPU-only inference, partial GPU offload, memory bandwidth, realistic performance, and what to buy at three budget levels.

John EditorNov 13, 2026 11m read
ShareWhatsAppXFacebook

By Kaito Tanaka

# RAM Offloading for Local LLMs: A Practical Hardware Buying Guide

Bottom line: Buy system memory to make the model fit, but buy memory bandwidth—or unified memory with an accelerated backend—if you also expect responsive generation. A 64GB two-DIMM configuration is the sensible target for a serious reusable desktop, while 128GB is an enthusiast capacity whose late-2026 price can overwhelm the rest of the build. If a model already fits completely in GPU VRAM, full offload is usually preferable to splitting it across VRAM and system RAM.

RAM-first inference is a capacity strategy before it is a speed strategy. It lets you run models that would otherwise fail to load, but a dual-channel DDR5 desktop should not be expected to perform like a model residing entirely in accelerator memory.

What RAM Offloading Actually Means

A local model needs space for its weights, runtime buffers, and key-value—or KV—cache. Quantization reduces the weight footprint, but larger models and longer contexts can still exceed available VRAM. A RAM-first configuration keeps all or part of the model in system memory and executes those layers on the CPU. This is the core trade-off described in practical CPU-and-RAM inference guidance↗: ordinary memory offers useful capacity, but decode performance remains constrained by the platform feeding the processor.

There are three basic operating modes:

  • CPU-only: All model layers remain in system RAM and execute on the CPU. This maximizes usable capacity without depending on GPU VRAM, but generation is generally limited by DDR memory bandwidth.
  • Partial CPU/GPU offload: Some layers run from VRAM while the remainder stay in system RAM. This can be faster than CPU-only execution, but the slower side and transfers between execution domains can constrain the complete pipeline.
  • Full GPU offload: Model weights and required runtime allocations fit in VRAM. This avoids the principal CPU/RAM bottleneck and is normally the performance-first choice.

The correct mode therefore depends on the objective. CPU-only operation makes sense when fitting a model is more important than latency, when reusing a machine with abundant RAM, or when a small quantized model is adequate. Partial offload is useful when a GPU can accelerate a meaningful fraction of a model but cannot hold everything. Full offload wins when the entire intended model, context, and cache fit reliably.

Capacity planning should include operational headroom rather than comparing only the model-file size with installed RAM. The OS and application also need memory, while context growth expands the KV cache. A model that barely loads may become unstable, page to slower storage, or fail when its context is increased. General local-LLM RAM guidance↗ is therefore best treated as a starting point, not a substitute for estimating the exact model and context.

Why More CPU Cores Do Not Fix Decode

Inference has two phases with different bottlenecks. Prompt processing, also called prefill, evaluates many input tokens in parallel. It benefits from compute resources, core count, batching, and optimized matrix operations. More cores can also improve concurrency when several requests are active.

Token generation, or decode, is sequential: each new token depends on the preceding state. The processor repeatedly streams model data while reading and updating the KV cache. On a normal desktop, this phase is usually memory-bandwidth-bound, so adding cores eventually produces little benefit once the two DDR5 channels are saturated.

L3 cache can help particular kernels and working sets, but it is a secondary, workload-dependent advantage. A 64MB L3 cache is not a replacement for DRAM bandwidth when model weights span many gigabytes. Likewise, increasing context length enlarges the KV cache and raises memory use; at long contexts, cache traffic competes with model weights for available bandwidth.

Software Controls: CPU-Only and Partial Offload

The open-source **llama.cpp** project↗ is the foundation for much of the GGUF ecosystem. For CPU-only operation, launch with:

`--n-gpu-layers 0`

or its short form:

`-ngl 0`

For partial offload, set `-ngl` to the desired number of layers. Increasing the value moves more layers to the GPU, subject to available VRAM. The ideal setting is not universal: model architecture, quantization, context, backend, and free memory all matter.

Ollama exposes the same concept through `num_gpu`, which refers to model layers rather than the number of physical GPUs. Setting `num_gpu: 0` forces CPU-only operation. If the option is omitted, Ollama automatically attempts to place the maximum safe number of layers in VRAM; when the complete model cannot fit, it can split execution between CPU RAM and GPU VRAM. The Ollama FAQ↗ also explains how to inspect whether a loaded model is running on the CPU, GPU, or a mixture of both.

LM Studio provides equivalent controls in its graphical model loader and through `lms load`. Its CLI accepts:

  • `--gpu off` for CPU-only loading.
  • A ratio such as `--gpu 0.5` for a partial layer offload.
  • `--gpu max` to attempt maximum GPU use.
  • `--estimate-only` to estimate resource allocation without loading the model.

These options are documented in the LM Studio local-model CLI guide↗. Use the estimate before committing to a large context, then verify actual memory use after loading.

On Apple Silicon, **MLX LM**↗ takes a different route. CPU and GPU use a unified physical memory pool, so a large allocation does not need to be divided into conventional system RAM and discrete VRAM. This makes high-capacity Apple configurations attractive for local inference, although performance still depends on the specific chip’s memory bandwidth, model format, and quantization.

Choosing RAM Capacity, Data Rate, and Topology

A RAM-first desktop should prioritize capacity first, stable channel population second, and data rate third. A fast kit that cannot hold the intended model is useless; a four-DIMM setup that forces a major frequency reduction may also decode more slowly than its advertised module rating suggests.

| RAM tier | Capacity | Preferred topology | Practical data-rate target | Best use | |---|---:|---|---|---| | Minimum viable | 32GB | 2×16GB | Platform-supported DDR5 | Small quantized models | | Recommended | 64GB | 2×32GB | DDR5-5200 to 6000 | Serious local experimentation | | Enthusiast | 128GB | Prefer two DIMMs where available; otherwise validated 4×32GB | Stability before headline speed | Larger models and contexts |

The table is intentionally conservative. Two DIMMs are preferable because they preserve one module per channel and reduce memory-controller stress. Four modules do not create four memory channels on a mainstream desktop: both processors considered below still have only two channels.

AMD Ryzen 9 9950X versus Intel Core Ultra 9 285K

The AMD Ryzen 9 9950X has 16 cores, 32 threads, 64MB of L3 cache, two DDR5 channels, and support for up to 256GB. Its official supported memory speeds are specifically DDR5-5600 for 2×1R or 2×2R, falling to DDR5-3600 for 4×1R or 4×2R. Those are the official specifications—not a promise that every overclocked kit or four-DIMM configuration will hold a higher profile.

The Intel Core Ultra 9 285K has 24 cores comprising 8 P-cores and 16 E-cores, 24 threads, 36MB Smart Cache, two memory channels, support for DDR5-6400, and a 256GB maximum. Its higher core count can be useful for prefill and concurrent workloads, but it does not eliminate the dual-channel decode ceiling.

Neither processor is automatically the universal local-LLM winner. Model, quantization, thread settings, memory configuration, and software backend must be held constant before drawing a conclusion. Broader local-AI hardware guidance↗ is useful for defining the workload, but buyers should benchmark their actual model instead of selecting on core count alone.

Both chips can sustain substantial CPU activity during inference. Cooling should therefore be selected for prolonged all-core workloads rather than brief desktop bursts. Use a cooler appropriate to the processor’s sustained power behavior, maintain unobstructed airflow, and validate temperatures and clock stability during a long prompt-processing run—not merely at idle.

Performance: What the Numbers Do and Do Not Prove

The following are standardized MLX benchmark results on an M4 Max, using 2,048 input tokens and 128 generated tokens. They are Apple Metal and unified-memory measurements, not DDR5 desktop CPU results.

Standardized MLX performance table

| Model | Precision | Generation (tok/s) | Memory (GB) | |---|---|---:|---:| | Qwen3-4B-Instruct-2507 | bf16 | 52.47 | 9.02 | | Qwen3-4B-Instruct-2507 | q4 | 134.52 | 3.35 | | Qwen3-30B-A3B-Instruct-2507 | q8 | 83.16 | 33.46 | | Qwen3-30B-A3B-Instruct-2507 | q4 | 113.33 | 18.20 |

These results show how lower precision can reduce memory footprint and raise decode throughput on that specific platform. The 30B-A3B model is a Mixture-of-Experts design, so its aggregate size should not be compared naively with a dense model that activates all parameters for every token.

They cannot be substituted for a dual-channel DDR5 desktop forecast. For a broad point of comparison, evidence across ordinary desktop setups places CPU-only 3B q4 generation in a representative 5–25 tok/s range. That range is non-standardized and is not a purchasing promise: missing details such as CPU, RAM speed, thread count, model build, context, and backend prevent it from becoming a measured comparison.

A benchmark without its model file, quantization, context, software version, thread count, and offload setting is a story—not a reproducible buying metric.

Use `llama-bench` to separate `pp` prompt-processing throughput from `tg` sequential-generation throughput. A CPU with strong `pp` results may still produce modest `tg` numbers because decode has reached the RAM-bandwidth limit. Test the exact GGUF, context length, and `-ngl` value you expect to use; the llama.cpp repository↗ is the appropriate starting point for current benchmark behavior.

Late-2026 DDR5 Pricing Changes the Budget

The following snapshot is explicitly from late-2026 listings observed in research and is highly volatile:

  • Team T-Force Delta RGB 64GB (2×32GB) DDR5-5200: $999.99 at Newegg.
  • Corsair Vengeance 64GB DDR5-6000: $1,039.99–$1,049.99.
  • A-Tech 128GB (4×32GB) DDR5-4800: $2,087.38.
  • A-Tech 128GB (4×32GB) DDR5-5600: $2,137.08.

These are retailer listings, not universal street prices. Other marketplace listings can vary, and none should be interpreted as a stable long-term average. They nevertheless make one fact unavoidable: a budget guide cannot responsibly treat new 64GB or 128GB DDR5 as inexpensive.

Three honest budget paths

  • Under $500: Treat this strictly as an upgrade or reuse path. Add compatible memory to an existing machine, use hardware already owned, and begin with a smaller quantized model. A newly built 64GB DDR5 tower is not credible within this budget under the cited listings.
  • $500–$1,500: Center the plan on reuse, used hardware, or an Apple Silicon configuration only when current local pricing genuinely fits. A new 64GB DDR5 desktop cannot be responsibly promised inside this tier using the late-2026 DDR5 listings.
  • $1,500 and above: Prioritize 64–128GB capacity plus a modern CPU, or consider an Apple unified-memory system whose total configuration meets the workload. At the cited prices, a 128GB consumer DDR5 build is likely to exceed the tier’s entry point substantially before the remaining system components are included.

Do not sacrifice capacity for decorative features or a marginal advertised memory-speed increase. For this workload, the first question is whether the model and context fit; the second is whether the selected topology runs stably at an acceptable data rate.

Why RAMDeck Is Not a Normal Buying Recommendation

The crowdfunded RAMDeck was promoted at $109 early-bird and $199 in later positioning. The TechRadar examination of RAMDeck↗ is important because the product does not literally merge physical memory into one hardware pool. It packages discovery and orchestration around distributed inference using the free llama.cpp RPC software.

That distinction matters. Distribution can make an oversized model load, but dense models must move extensive data across devices and the network. Demonstrations cited for dense models reached only 1.92 tok/s for 27B across four devices and 1.37–3.1 tok/s for 120B configurations. The reported 16–39 tok/s range applies to MoE workloads under carefully qualified accelerator and fast-network conditions; it is not a general result for arbitrary pooled devices.

RAMDeck may offer convenience for users unwilling to configure RPC nodes, but it remains crowdfunded and duplicates capability available through free software. It should be viewed as an experimental convenience layer, not an alternative to buying sufficient local memory or a normal recommendation for dependable inference.

Verdict

For most buyers, the best RAM-first configuration is 64GB in two DIMMs, paired with a modern CPU, adequate sustained-load cooling, and a quantized model chosen with context headroom. Choose 128GB only when a specific model requires it and the exceptionally high late-2026 memory cost is justified.

Use CPU-only mode for capacity and reuse, partial offload when a GPU can hold a meaningful share of the model, and full GPU offload whenever the complete working set fits. Apple Silicon with MLX is the cleaner high-capacity alternative when unified-memory pricing and the desired configuration fit the budget.

Above all, do not buy from an isolated token-per-second claim. Estimate memory first, test `pp` and `tg` separately, preserve the intended context length, and benchmark the exact model you plan to run.

#local LLM#DDR5 memory#CPU inference#GPU offloading#Apple Silicon

Comments

Open discussion — no account needed. Be respectful.

0/4000
Loading comments…