When I originally began running clusters for the College of Engineering I set them up with default network configurations in warewulf and perceus. This usually meant using 172.16.0.0/16 private address space, with addresses chosen anywhere within that range (and not sequentially). This essentially meant that every cluster was more or less running the same IP subnets on its private network.

The first flaw in this plan came when we started getting clusters with multiple network interfaces (gigabit ethernet plus AMSO RDMA ethernet, for example). I need to assign each compute node more than one IP address, and since the interfaces were running on different (non-chained) switches, those addresses needed to come from a unique subnet.

None of the clusters are large enough to use more than 200 addresses per subnet, so rather than use the full 10.0.0.0/8 address space (cluster number in second digit, last two digits partitioned into local subnet and host) I’ve opted to restrict myself to 10.0.0.0/16. The third digit is partitioned into a 5-bit cluster number and a 3-bit local subnet – that’s 32 clusters with 8 local subnets each. That’s plenty