Starwind vSAN Free multiple NICs for SyncInterface

tig_jeff · Fri Jan 05, 2018 10:07 pm

I'm trying to setup a test (about a year in the making) that will use 2 Hyper-V systems.
Each system has 6 NICs.
On the both systems I had bonded three of the NICs together for data sync, but was told that I should not do this as it could break the sync between systems in vSAN free.

The section of the powershell script I'm referencing is:
$firstNode.SyncInterface = "#p2=10.3.0.2:3260"
And
$secondNode.SyncInterface = "#p1=10.3.0.1:3260"

Whereas I have figured that #p2 indicates the destination address of the sync for the $firstNode.
And #p1 indicates the destination of the sync for the $secondNode.

...right?

So, with that, can we do this? It didn't error out when I tried, but it never looked like it used the second interface on each system used for sync.
$firstNode.SyncInterface = "#p2=10.3.0.2:3260;#p2=10.4.0.2:3260"
And
$secondNode.SyncInterface = "#p1=10.3.0.1:3260;#p1=10.4.0.1:3260"

Please advise.

Sat Jan 06, 2018 7:15 am

Additional IPs need to be indicated like below:

Code: Select all

$firstNode.SyncInterface = "#p2=10.3.0.2:3260,10.4.0.2:3260"

tig_jeff · Wed Jan 17, 2018 9:07 pm

Thanks Boris!!

Wed Jan 17, 2018 9:24 pm

You are welcome.

zendzipr · Tue Jan 23, 2018 3:08 am

I am attempting the same thing and not seeing successful results.

$firstNode.SyncInterface = "#p2=172.16.2.3,172.16.3.3:3260"

$secondNode.SyncInterface = "#p1=172.16.2.2,172.16.3.2:3260"

I am able to build the cluster, however sync only happens over the first IP addresses for each node, second interface does not sync.

I can change the IP addresses around and sync happens on the other interface.

I did test this with the console before and sync happens as expected.

Any help is appreciated, thank you.

Tue Jan 23, 2018 5:10 am

$firstNode.SyncInterface = "#p2=172.16.2.3,172.16.3.3:3260"

The port value is missing for the first IP in the array. It should be like this:
$firstNode.SyncInterface = "#p2=172.16.2.3:3260,172.16.3.3:3260"

zendzipr · Tue Jan 23, 2018 4:20 pm

Thank you.. Sorry for the confusion, I sent different tries at making this work.

Code: Select all

$firstNode.SyncInterface = "#p2=172.16.3.3:3260,172.16.2.3:3260"
$secondNode.SyncInterface = "#p1=172.16.3.2:3260,172.16.2.2:3260"

Still does not work.

I can do a single interface, I can swap them, the first one in each SyncInterface field is the one that works.

My xml in all cases only has one of each IP, even if I put two in the SyncInterface field

Code: Select all

          <transport type="iSCSI">
            <links>
              <link id="1" type="data" priority="1" connections="1">
                <peer ip="172.16.3.3" port="3260"/>
              </link>
              <link id="2" type="control" priority="1" connections="1">
                <peer ip="192.168.86.41" port="3260"/>
              </link>
            </links>
          </transport>

I have even tried stopping starwind service and manually editing the _HA. file but this resulted in breaking things.

Wed Jan 24, 2018 11:33 am

Code: Select all

#Import-Module StarWindX

try
{
    $server = New-SWServer -host 127.0.0.1 -port 3261 -user root -password starwind

    $server.Connect()

    $firstNode = new-Object Node

    $firstNode.ImagePath = "My computer\D"
    $firstNode.ImageName = "masterImg123"
    $firstNode.Size = 256
    $firstNode.CreateImage = $true
    $firstNode.TargetAlias = "targetha123"
    $firstNode.AutoSynch = $true
    $firstNode.SyncInterface = "#p2=172.16.20.22:3260,172.16.21.2:3260"
    $firstNode.HBInterface = "#p2=172.16.10.22:3260,172.16.11.2:3260"
    $firstNode.CacheSize = 64
    $firstNode.CacheMode = "wb"
    $firstNode.PoolName = "pool1"
    $firstNode.SyncSessionCount = 1
    $firstNode.ALUAOptimized = $true
    
    #
    # device sector size. Possible values: 512 or 4096(May be incompatible with some clients!) bytes. 
    #
    $firstNode.SectorSize = 4096
	
	#
	# 'SerialID' should be between 16 and 31 symbols. If it not specified StarWind Service will generate it. 
	# Note: Second node always has the same serial ID. You do not need to specify it for second node
	#
	#$firstNode.SerialID = "050176c0b535403ba3ce02102e33eab" 
    
    $secondNode = new-Object Node

    $secondNode.HostName = "192.168.0.85"
    $secondNode.HostPort = "3261"
    $secondNode.Login = "root"
    $secondNode.Password = "starwind"
    $secondNode.ImagePath = "My computer\D"
    $secondNode.ImageName = "partnerImg123"
    $secondNode.Size = 256
    $secondNode.CreateImage = $true
    $secondNode.TargetAlias = "partnerha123"
    $secondNode.AutoSynch = $true
    $secondNode.SyncInterface = "#p1=172.16.20.21:3260,172.16.21.1:3260"
    $secondNode.HBInterface = "#p1=172.16.10.21:3260,172.16.11.1:3260"
    $secondNode.ALUAOptimized = $true
        
    $device = Add-HADevice -server $server -firstNode $firstNode -secondNode $secondNode -initMethod "Clear"
    
    $syncState = $device.GetPropertyValue("ha_synch_status")

    while ($syncState -ne "1")
    {
        #
        # Refresh device info
        #
        $device.Refresh()

        $syncState = $device.GetPropertyValue("ha_synch_status")
        $syncPercent = $device.GetPropertyValue("ha_synch_percent")

        Start-Sleep -m 2000

        Write-Host "Synchronizing: $($syncPercent)%" -foreground yellow
    }
}
catch
{
    Write-Host "Exception $($_.Exception.Message)" -foreground red 
}

$server.Disconnect()

This script worked like a charm in my test lab. It adds multiple interfaces for both Sync and iSCSI channels. Look into your script and see what might be wrong there.

tig_jeff · Wed Jan 24, 2018 2:23 pm

In my initial testing it did not appear to work either.
It may only use it if it finds that the bandwidth on the first adapter is saturated and then starts to use the 2nd, 3rd and 4th.

I would like to say that I HAVE gotten this working with only PowerShell.
I'll be posting a HOW-TO on this in the near future... in another thread.

zendzipr · Wed Jan 24, 2018 3:35 pm

Boris (staff) wrote: This script worked like a charm in my test lab. It adds multiple interfaces for both Sync and iSCSI channels. Look into your script and see what might be wrong there.

Boris, thank you for the additional assistance. Unfortunately, this did not work either.

Below is the powershell script followed by masterImg123_HA and masterImg123

As you can see from the XML, only one of each interface type is created. Unless there are other locations for the configuration, I don't see anywhere the second interfaces exist.

Code: Select all

#Import-Module StarWindX

try
{
    $server = New-SWServer -host 127.0.0.1 -port 3261 -user root -password starwind

    $server.Connect()

    $firstNode = new-Object Node

    $firstNode.ImagePath = "My computer\E"
    $firstNode.ImageName = "masterImg123"
    $firstNode.Size = 256000
    $firstNode.CreateImage = $true
    $firstNode.TargetAlias = "targetha123"
    $firstNode.AutoSynch = $true
    $firstNode.SyncInterface = "#p2=172.16.2.3:3260,172.16.3.3:3260"
    $firstNode.HBInterface = "#p2=192.168.86.41:3260,172.16.1.3:3260"
    $firstNode.CacheSize = 64
    $firstNode.CacheMode = "wb"
    $firstNode.PoolName = "pool1"
    $firstNode.SyncSessionCount = 1
    $firstNode.ALUAOptimized = $true
    
    #
    # device sector size. Possible values: 512 or 4096(May be incompatible with some clients!) bytes. 
    #
    $firstNode.SectorSize = 4096
   
   #
   # 'SerialID' should be between 16 and 31 symbols. If it not specified StarWind Service will generate it. 
   # Note: Second node always has the same serial ID. You do not need to specify it for second node
   #
   #$firstNode.SerialID = "050176c0b535403ba3ce02102e33eab" 
    
    $secondNode = new-Object Node

    $secondNode.HostName = "192.168.86.41"
    $secondNode.HostPort = "3261"
    $secondNode.Login = "root"
    $secondNode.Password = "starwind"
    $secondNode.ImagePath = "My computer\E"
    $secondNode.ImageName = "partnerImg123"
    $secondNode.Size = 256000
    $secondNode.CreateImage = $true
    $secondNode.TargetAlias = "partnerha123"
    $secondNode.AutoSynch = $true
    $secondNode.SyncInterface = "#p1=172.16.2.2:3260,172.16.3.2:3260"
    $secondNode.HBInterface = "#p1=192.168.86.42:3260,172.16.1.2:3260"
    $secondNode.ALUAOptimized = $true
        
    $device = Add-HADevice -server $server -firstNode $firstNode -secondNode $secondNode -initMethod "Clear"
    
    $syncState = $device.GetPropertyValue("ha_synch_status")

    while ($syncState -ne "1")
    {
        #
        # Refresh device info
        #
        $device.Refresh()

        $syncState = $device.GetPropertyValue("ha_synch_status")
        $syncPercent = $device.GetPropertyValue("ha_synch_percent")

        Start-Sleep -m 2000

        Write-Host "Synchronizing: $($syncPercent)%" -foreground yellow
    }
}
catch
{
    Write-Host "Exception $($_.Exception.Message)" -foreground red 
}

$server.Disconnect()

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<header signature="StarWind" version="1.0">
  <device active="true" plugin="HAImage" name="HAImage">
    <storages>
      <storage id="1" type="device" name="HAImage" lun="0x0">
        <interval size="250" units="GB"/>
        <inquiry>
          <serial_id>16c560e6c2614679b23f644d50b54a</serial_id>
          <vendor id="STARWIND"/>
          <product id="STARWIND" revision="0001"/>
          <eui_64>77DE3EC244A6E9D7</eui_64>
        </inquiry>
        <geometry>
          <sector size="512" psize="4096"/>
          <track sectors="16"/>
          <cylinder tracks="32" count="65535"/>
        </geometry>
        <caching>
          <cache type="write-back" size="64" units="MB" level="1">
            <storages>
              <storage_ref id="1"/>
            </storages>
          </cache>
        </caching>
      </storage>
    </storages>
  </device>
  <system>
    <resources>
      <storages>
        <storage id="1" name="RAM" type="RAM">
          <interval size="64" units="MB"/>
        </storage>
        <storage id="2" name="imagefile1" type="device" lun="0x0">
          <interval size="250" units="GB"/>
        </storage>
        <storage id="3" name="iqn.2008-08.com.starwindsoftware:192.168.86.41-partnerha123" type="remote" lun="0x0">
          <transport type="iSCSI">
            <links>
              <link id="1" type="data" priority="1" connections="1">
                <peer ip="172.16.2.3" port="3260"/>
              </link>
              <link id="2" type="control" priority="1" connections="1">
                <peer ip="192.168.86.41" port="3260"/>
              </link>
            </links>
          </transport>
        </storage>
      </storages>
      <network/>
    </resources>
  </system>
  <node id="1" name="HAImage" shut="false" active="true" flags="0">
    <storages>
      <storage_ref id="2"/>
    </storages>
    <parameters>
      <type>1</type>
      <priority>0</priority>
      <auto_sync>true</auto_sync>
      <sync_status>0</sync_status>
      <sync_delay>50</sync_delay>
      <sync_traffic_share>50</sync_traffic_share>
      <failover_conf>0</failover_conf>
      <last_sync_snap_id>0</last_sync_snap_id>
      <maintenance_mode/>
    </parameters>
  </node>
  <node id="2" name="iqn.2008-08.com.starwindsoftware:192.168.86.41-partnerha123" shut="false" active="true">
    <storages>
      <storage_ref id="3"/>
    </storages>
    <parameters>
      <type>1</type>
      <priority>1</priority>
      <sync_status>0</sync_status>
    </parameters>
  </node>
</header>

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<header signature="StarWind" version="1.0">
  <device active="true" plugin="imagefile" name="imagefile">
    <storages>
      <storage id="1" type="device" name="imagefile" lun="0x0">
        <interval size="250" units="GB"/>
        <inquiry>
          <serial_id>77DE3EC244A6E9D7</serial_id>
          <vendor id="STARWIND"/>
          <product id="STARWIND        " revision="0001"/>
          <eui_64>77DE3EC244A6E9D7</eui_64>
        </inquiry>
        <geometry>
          <sector size="512" psize="4096"/>
          <track sectors="16"/>
          <cylinder tracks="32" count="65535"/>
        </geometry>
        <caching>
          <cache type="write-back" size="64" units="MB" level="1">
            <storages>
              <storage_ref id="1"/>
            </storages>
          </cache>
        </caching>
      </storage>
    </storages>
  </device>
  <system>
    <resources>
      <storages>
        <storage id="1" name="RAM" type="RAM">
          <interval size="64" units="MB"/>
        </storage>
        <storage id="2" name="My computer\E\masterImg123.img" type="file">
          <interval size="250" units="GB"/>
        </storage>
      </storages>
      <network/>
    </resources>
  </system>
  <node id="1" name="imagefile" shut="false" active="true">
    <storages>
      <storage_ref id="2"/>
    </storages>
  </node>
</header>

And adding on, I have completed some additional testing.

New server install, Server 2016
StarWind with Management Console.

Configure using Management console and I get expected results.

Code: Select all

              <link id="1" type="data" priority="1" connections="1">
                <peer ip="172.16.3.2" port="3260"/>
              </link>
              <link id="2" type="data" priority="1" connections="1">
                <peer ip="172.16.2.2" port="3260"/>
              </link>
              <link id="3" type="control" priority="1" connections="1">
                <peer ip="192.168.86.42" port="3260"/>
              </link>
              <link id="4" type="control" priority="1" connections="1">
                <peer ip="172.16.1.2" port="3260"/>
              </link>

Configure using powershell, only get one of each connection.

I saved log files from both types if that helps.

zendzipr · Wed Jan 24, 2018 3:37 pm

tig_jeff wrote:In my initial testing it did not appear to work either.
It may only use it if it finds that the bandwidth on the first adapter is saturated and then starts to use the 2nd, 3rd and 4th.

I would like to say that I HAVE gotten this working with only PowerShell.
I'll be posting a HOW-TO on this in the near future... in another thread.

I look forward to seeing your new how-to. I have been struggling with this and do not see it working. based on the XML created, there is only one interface. Not sure how a second one can be used.

When testing with the GUI, I am able to successfully create with multiple interfaces and sync happens over both equally, as if load balanced.

tig_jeff · Wed Jan 24, 2018 4:51 pm

Zendzipr-
I'll hustle and see if I can get one going.

I just got my 10Gbe adapters (which will replace my quad port adapters).
So, I'll see if I can get my how-to done by the end of next week... which will show multi link syncing.

Thu Jan 25, 2018 9:28 am

zendzipr,

Please submit a support case here.