This is driving me nuts so I'm hoping that someone here may have encountered this.
I have a three node setup and the Starwind clusters are replicating perfectly between all three with no issues. The problem is trying to connect the iSCSI targets with MPIO between two of the servers. I have 4 10Gig ports in each server dedicated to the sync and iSCSI data but they are interconnected to each other directly with cables instead of going though a switch.
This is my very current simplified configuration:
Server 1: iSCSI Data IP addresses 10.2.3.1 and 10.2.6.1
Server 3: iSCSI Data IP addresses 10.2.3.3 and 10.2.6.3
Server 2 can be ignored as it doesn't factor into the issue and all the sync IP addresses for Starwind are in a different subnet from the iSCSI.
Both server 1 and 3 have discovery target portals setup to look at each other as far as I can tell (ignore the 10.2.2.x and 10.2.4.x subnets as they are for server 2).

- Capture.JPG (34.83 KiB) Viewed 16579 times
When I'm on server 3 and connecting to the iSCSI target located on server 1, I can assign 10.2.3.3 and 10.2.6.3 as the initator IPs and 10.2.3.1 and 10.2.6.1 as the target portal IPs like I'm supposed to.

- Capture1.JPG (24.72 KiB) Viewed 16579 times
However, when I'm on server 1 and connecting to the iSCSI target located on server 3, I can only assign 10.2.3.1 as the initator IP and 10.2.3.3 as the target portal IP. I am unable to select the 10.2.6.3 as the target for 10.2.6.1.

- Capture2.JPG (31.64 KiB) Viewed 16579 times
This doesn't seem to make any sense why it works one way and not the other. I have tried dumping the target portals and recreating them as well as ensuring the firewalls, etc are off. There's no CHAP permissions I'm using nor are there any deny access rights in Starwind on any server. If anyone has any ideas it would be greatly appreciated. I've been pulling my hair out for hours trying to figure this out.