After waiting for about a week, the cards finally arrived. It’s a pair of Mellanox ConnectX-3 EN. 10GbE card with SFP+ port. They costed a bit over 10k yen total.
After a quick test using spare PC, I proceeded to install them to the respective systems.
The one in server (FreeBSD) as expected didn’t quite work right away because I added kernel module loader setting to the wrong system. And then it still didn’t work because I forgot to set the card online and only added it to the bridge.
On the other hand, the desktop system, I set it up for Hyper-V switch and after solving problems server side, I ran some iperf tests. First it shows less than 1Gbps, and then the system crashed. Thankfully it crashed only once but subsequent iperf tests ground the system to a halt, barely responding to anything. Music playing in the background also heavily stuttered.
A bit curious, I tested from inside a VM, and it was fine. No stutter or anything.
But I only got about 5Gbps – half the speed I got from initial testing. Messing around a bit I figured skipping bridge on server side and enabling jumbo frame helped. I got over 8Gbps both in VM and host. Testing on the desktop still heavily stuttered, including plain file copy.
After a few hours of trying various settings, the conclusion I got was sharing Hyper-V virtual switch with host system is bad. Now I think about it, I’ve already been having similar occasional stutter problem even on gigabit. This could be the problem?
Well, as the main reason for 10GbE is for dev VM, I ended up dedicating the link for VMs with the host, desktop, itself back to gigabit. At least it’s now dedicated gigabit. The performance inside VM is also pretty good so everything is all right now.
With this, I used the final slot of my server motherboard. Which means I need to buy two dual port 10GbE cards if I want to have fast link for both VM and desktop. I guess if I can find SFP+ 10GbE card in PCI-e x1 that can work as well…
There are exactly two of such cards for rather cheap in YJA at the moment but the sellers don’t say if the cards work at all.
Ebay shopping part 2?