MPIO/Load balancing on the Windows/StarWind server side

[joshabts]

Forgive me if this has been asked and answered, but searching around I haven't been able to find a direct answer....

What is the best way to properly load balance and fully utilize MPIO on the target/server side of StarWind. I know in VMware for example I need to create VMkernel ports with unique IP addresses, associate those to specific NICs and then manage paths and choose to use Round Robin load balancing.

But on the StarWind side, do I simply use NIC teaming/LACP to bond the interfaces and let the switch distribute amongst them? I can see this in the long run being "balanced" but in the near term with say a single host the potential for not being completely balanced is there.

Or do I need to likewise give each interface on the StarWind server it's own IP address and then create a target to each from in VMware rather than using a single target IP? If they are all the same subnet then it will detect multiple paths correct even though that may be misleading? I.e. if we have 4 NIC's in the VMware host and 4 in the Starwind/Windows server, we would end up with 16 paths, one per nic to each of the 4 IP addresses on the Starwind side?

Hopefully this is making sense. I am trying to figure out the best way to achieve my full throughput but without having to burden with creating a separate VLAN/subnet per NIC interface.

[Anatoly (staff)]

Forgive me if this has been asked and answered, but searching around I haven't been able to find a direct answer....

What is the best way to properly load balance and fully utilize MPIO on the target/server side of StarWind. I know in VMware for example I need to create VMkernel ports with unique IP addresses, associate those to specific NICs and then manage paths and choose to use Round Robin load balancing.

But on the StarWind side, do I simply use NIC teaming/LACP to bond the interfaces and let the switch distribute amongst them? I can see this in the long run being "balanced" but in the near term with say a single host the potential for not being completely balanced is there.

Or do I need to likewise give each interface on the StarWind server it's own IP address and then create a target to each from in VMware rather than using a single target IP? If they are all the same subnet then it will detect multiple paths correct even though that may be misleading? I.e. if we have 4 NIC's in the VMware host and 4 in the Starwind/Windows server, we would end up with 16 paths, one per nic to each of the 4 IP addresses on the Starwind side?

Hopefully this is making sense. I am trying to figure out the best way to achieve my full throughput but without having to burden with creating a separate VLAN/subnet per NIC interface.

Usually we prefer running the MPIO instead of NIC bonding. Don`t want to disappoint you with such short answer, but that is mostly it:) Usually you just need to follow our guides and that squeezes maximum out of the system. I`d like to share one hint with you and community now: one StarWind devices can give ~60000 IOPS as maimum. so if you expect higher workload, you need to create mode devices.

[joshabts]

Ok -- so I believe I have MPIO setup, but I am clearly missing something...

When I run CrystalDiskMark for example directly against my RAID10 of 24 spindles (7200RPM SATA 2TB), I can achieve ~815MB/s of throughput. When I run it on a VM whose disk resides in StarWind, and I have a single iSCSI VMkernel port (10.1.20.10) configured on the VMware host and a single iSCSI port configured in Windows (10.1.20.2) I get ~104MB/s of throughput. So ok, this makes sense, I have a single 1Gbps connection, so we have maxed that out.

But, when I add the second VMkernel port in its own vSwitch (10.1.20.11) and add a second port in Windows (10.1.20.3) I still get ~104MB/s of throughput. Monitoring the NIC usage I see it split it almost exactly 50/50 between the NICs on both sides (VMware and Windows). I would have though I should see roughly 208MB/s of throughput, since the disks can obviously serve it (815MB/s when direct).

Is this expectation unreasonable/false? Or am I missing something critical in my setup that would be causing this?

I have set using esxcli the IOPS limit for round robin path selection to 1 and 0 (tried both) and neither gives me any improvement. Further, I went all the way to 4 connections on both ends and I get exactly 25% on all links and still maintain roughly 104MB/s. So it seems to be round robin to the extent bandwidth is split 1/n where n is the number of iSCSI nic's I have allocated. But my understanding was I should get full use out of them.

If anyone has any suggestions or would need to see certain parts of my configuration, I'd be more than happy to share, just tell me where/what is the most helpful.

Appreciate any insight here!

[Anatoly (staff)]

I`d expect to see full utilization, unless the storage cannot give you speed more then 1Gbps.
To be honest I`m not sure if Crystal Disk is somehow reliable, so I~d ask you to switch to IOmeter and check 4k and 64k block sizes with different workload and share with us MB/s and IOPS after.