10GbE for 88CHF

My HPE Proliant DL380p virtualization server is running out of storage space again. Unfortunately, only HDDs from HPE can be installed in this server, which are relatively expensive for hobby use. It must therefore be a simple and cost-effective solution.

Idea

On the internet I could find two cheap used 10Gb network cards, the ConnectX®-2 from Mellanox. These cards can be connected via Direct Attach Cable (DAC) for short distances. The following figure shows the network architecture. The virtualization server is connected directly to the file server with 10GbE. On both computers, the network interfaces has a fixed IP address.

10GbE setup

Realization

The network card is detected by ESXi 6.5.

ESXi NICs interfaces

Unfortunately, the virtualization server has become louder after installing the network card. The server doesn’t fully recognize this interface card, or at least not the expected temperature sensors. The fans on the side where the card is located rotate faster.

ESXi fan speed

ESXi 6.5 allows you to integrate two types of network storage, NFS and iSCSI. As I have already had good experience with NFS in other projects, I use this variation. On the file server, file sharing is adjusted under /etc/exports to allow full access from the virtualization server’s IP.

/home/shaolin/vmware_ssd 10.20.30.41(rw,async,no_wdelay,no_root_squash,insecure_locks,sec=sys,anonuid=1025,anongid=100)

Under ESXi a virtual switch is created for the 10GbE interface. In this way it was possible to set a fixed IP.

ESXi switch topology

Conclusion

The total hardware expenses are 88 CHF. VMs running over the 10GbE line do not show any big performance losses compared to locally stored VMs. When large files are copied in a VM, the copy speed decreases to less than 100MB/s. This is acceptable for VMs that do not constantly interact with an impatient user.