Building an AI Homelab with Proxmox, Kubernetes, 200Gb Networking and Ceph

A homelab shared on Reddit this August combines three Proxmox hosts, 36 physical CPU cores, 192 GB of DDR5 memory, four ASUS GX10 systems used as Kubernetes nodes, 512 GB of total VRAM, and a 200Gb high speed network fabric. The environment serves local large language models, image generation, identity services, GitOps workloads, and other projects. The owner also plans to add high speed NICs to the Proxmox nodes and build faster Ceph storage.

This is far beyond the usual single mini PC homelab, but it is useful because it exposes the same architecture questions that appear in production AI infrastructure. Compute is only one part of the system. GPU utilization, storage throughput, east west traffic, failure domains, virtualization overhead, orchestration, backup, and operational complexity all start to interact.

The Mr.PlanB Proxmox hub is a useful starting point for the virtualization layer. The more interesting question is how that layer should connect to Kubernetes and AI specific hardware without turning the lab into one giant dependency chain.

Proxmox is the control layer, not necessarily the GPU layer

Proxmox works well as a general virtualization foundation for management services, utility VMs, test systems, storage services, firewalls, and conventional application workloads. It gives a lab owner a consistent place to create VMs and containers, manage networks and storage, and experiment with clustering and automation.

GPU heavy AI workloads introduce different priorities. Direct hardware access, driver compatibility, memory capacity, PCIe topology, and network throughput can matter more than VM density. In the featured build, dedicated AI nodes participate in Kubernetes while Proxmox remains a separate cluster. That division can be sensible because it lets each layer do what it is best at.

A lab does not need to virtualize every accelerator simply because a hypervisor is available. Sometimes the cleanest architecture is to use Proxmox for shared infrastructure and Kubernetes management workloads while keeping GPU nodes closer to bare metal. The choice depends on whether the goal is isolation, portability, hardware utilization, or learning a specific production pattern.

Kubernetes turns the GPU machines into a shared compute pool

Once several accelerator nodes are available, Kubernetes becomes attractive because it can schedule workloads across them, expose services consistently, and provide a common deployment model for inference, model serving, APIs, and supporting applications.

The hard part is keeping expensive GPUs productive. The AI and GPU workloads on Kubernetes guide covers accelerator discovery, scheduling, node pools, drivers, storage throughput, networking, autoscaling, resilience, observability, and cost. Those concerns become visible quickly in a homelab with multiple models and services competing for GPU memory and data paths.

A local model may appear fast when tested alone, then slow down when image generation, embeddings, storage traffic, and background jobs share the same fabric. Kubernetes can schedule workloads, but it cannot create bandwidth or VRAM that does not exist. Good cluster design still requires understanding where the real bottleneck is.

A 200Gb fabric changes what storage can become

High speed networking is especially interesting because distributed storage often fails to meet expectations when the network is treated as an afterthought. Ceph replicates data across nodes and depends heavily on network behavior during normal writes, recovery, rebalance, and failure scenarios.

With sufficiently fast links, Ceph can become much more attractive for a Proxmox cluster, but line rate alone does not guarantee good storage. Drive performance, CPU capability, network latency, switch configuration, MTU consistency, NIC drivers, replication settings, and failure domain design still determine the result.

The Ceph guide explains the building blocks including monitors, managers, OSDs, placement groups, replication, erasure coding, self healing, and scaling. For a three node Proxmox cluster, the key lesson is to size Ceph around the slowest important component rather than the headline network speed. A fast fabric can remove one constraint and simply expose the next one.

Separate storage traffic from everything else

A lab that carries VM traffic, Kubernetes traffic, storage replication, backups, management, and AI model data on the same path can become unpredictable. This is where a more production minded network design starts to pay off.

Ceph replication and recovery can generate large east west flows. AI inference nodes may move model weights and datasets. Backup jobs can create sustained transfers. Kubernetes adds service and control traffic. Management access should remain reachable even when workloads are busy.

Logical separation through VLANs is a starting point, but high throughput environments may also need physical separation or carefully planned interfaces. The objective is not complexity for its own sake. It is to prevent one heavy workload from making the rest of the lab difficult to operate.

The same thinking applies to failure domains. If the same switch, power source, or storage path supports every layer, a sophisticated cluster may still have a very simple single point of failure.

Fast infrastructure needs equally serious backup

A lab with hundreds of gigabytes of VRAM and high speed storage can rebuild compute quickly, but the data may be harder to recreate. Model files can often be downloaded again, while Git repositories, application databases, identity configuration, fine tuning outputs, prompts, datasets, and service state may be unique.

That makes backup architecture important even in an experimental environment. The Proxmox Backup Server guide covers incremental deduplicated backups, verification, remote synchronization, encryption, pruning, and tape support. Kubernetes workloads need their own recovery design as well because cluster resources and persistent application data do not always live in the same place.

The practical approach is to classify data. Disposable model caches should not receive the same protection as irreplaceable databases. Reproducible Kubernetes manifests may need Git history more than image level backup. Stateful services need application consistent protection and tested restores.

The real value of this homelab is architectural learning

The impressive part of a 200Gb AI homelab is not the benchmark number. It is the opportunity to observe how infrastructure layers interact.

Proxmox can teach virtualization, clustering, storage, networking, backup, and automation. Kubernetes adds scheduling, service networking, desired state, secrets, operators, and GitOps. Ceph adds distributed storage behavior and failure recovery. GPU nodes add a scarce hardware resource with its own drivers, memory limits, and performance characteristics.

When these systems are connected, the lab becomes a place to test realistic questions. What happens when a Ceph node fails during heavy inference traffic? How long does a model service recover after a Kubernetes node disappears? Which traffic dominates the fabric during backup? Can management remain responsive during storage recovery? Which services should be virtualized and which should stay on bare metal?

Those questions are more valuable than simply building the fastest rack possible. A strong homelab gives you controlled failure, measurable bottlenecks, and a safe environment to change the architecture repeatedly. That is exactly why Proxmox, Kubernetes, Ceph, and AI infrastructure make such an interesting combination.

Originally published on the Mr.PlanB blog.