CreateHAPartnerWitness.ps1 problem

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
AlexeyK
Posts: 7
Joined: Thu Mar 20, 2025 2:45 pm

Wed Mar 26, 2025 4:33 pm

I'm trying to setup 2 node synchronous replication + partner witness using Windows server with StarWind vSAN Free application (StarWind Virtual SAN v8.0.0 (Build 19551, [SwSAN], Win64)) using powershell.

My lab setup is:
3 servers, 2 for data and 1 for witness role:
STOR1 (192.168.123.41, 10.10.123.41, 10.11.123.41)
STOR2 (192.168.123.42, 10.10.123.42, 10.11.123.42)
STOR2 (192.168.123.43, 10.10.123.43, 10.11.123.43)

3 networks:
LAN (192.168.123.0/24)
iSCSI/SYNC (10.10.123.0/24)
iSCSI/HB (10.11.123.0/24)

Firewall is disabled

I'm using CreateHAPartnerWitness.ps1 sample and just using the correct IP addresses and paths
It looks like this:

Code: Select all

param($addr="192.168.123.41", $port=3261, $user="root", $password="starwind",
	$addr2="192.168.123.42", $port2=$port, $user2=$user, $password2=$password,
	$addrW="192.168.123.43", $portW=$port, $userW=$user, $passwordW=$password,
#common
	$initMethod="Clear",
	$size=128,
	$sectorSize=512,
	$failover=1,
	$bmpType=1,
	$bmpStrategy=0,
#primary node
	$imagePath="My computer\C\sw",
	$imageName="masterImg31",
	$createImage=$true,
	$storageName="",
	$targetAlias="targetha31",
	$poolName="pool1",
	$syncSessionCount=1,
	$aluaOptimized=$true,
	$cacheMode="none",
	$cacheSize=0,
	$syncInterface="#p2=10.10.123.42:3260;#p3=10.10.123.43:3260",
	$hbInterface="",
	$createTarget=$true,
	$bmpFolderPath="",
#secondary node
	$imagePath2="My computer\C\sw",
	$imageName2="partnerImg32",
	$createImage2=$true,
	$storageName2="",
	$targetAlias2="partnerha32",
	$poolName2="pool1",
	$syncSessionCount2=1,
	$aluaOptimized2=$false,
	$cacheMode2=$cacheMode,
	$cacheSize2=$cacheSize,
	$syncInterface2="#p1=10.10.123.41:3260;#p3=10.10.123.43:3260",
	$hbInterface2="",
	$createTarget2=$true,
	$bmpFolderPath2="",
#third node
	$imagePathW="My computer\C\sw",
	$imageNameW="witness33",
	$targetAliasW="witness33",
	$syncInterfaceW="#p1=10.10.123.41:3260;#p2=10.10.123.42:3260",
	$hbInterfaceW="",
	$nodeTypeW=8
	)

Import-Module StarWindX

try
{
	Enable-SWXLog

	$server = New-SWServer -host $addr -port $port -user $user -password $password

	$server.Connect()
    
	$firstNode = new-Object Node

	$firstNode.HostName = $addr
	$firstNode.HostPort = $port
	$firstNode.ImagePath = $imagePath
	$firstNode.ImageName = $imageName
	$firstNode.Size = $size
	$firstNode.CreateImage = $createImage
	$firstNode.StorageName = $storageName
	$firstNode.TargetAlias = $targetAlias
	$firstNode.SyncInterface = $syncInterface
	$firstNode.HBInterface = $hbInterface
	$firstNode.PoolName = $poolName
	$firstNode.SyncSessionCount = $syncSessionCount
	$firstNode.ALUAOptimized = $aluaOptimized
	$firstNode.SectorSize = $sectorSize
	$firstNode.CacheMode = $cacheMode
	$firstNode.CacheSize = $cacheSize
	$firstNode.FailoverStrategy = $failover
	$firstNode.CreateTarget = $createTarget
	$firstNode.BitmapStoreType = $bmpType
	$firstNode.BitmapStrategy = $bmpStrategy
	$firstNode.BitmapFolderPath = $bmpFolderPath
                                                   
	$secondNode = new-Object Node

	$secondNode.HostName = $addr2
	$secondNode.HostPort = $port2
	$secondNode.Login = $user2
	$secondNode.Password = $password2
	$secondNode.ImagePath = $imagePath2
	$secondNode.ImageName = $imageName2
	$secondNode.CreateImage = $createImage2
	$secondNode.StorageName = $storageName2
	$secondNode.TargetAlias = $targetAlias2
	$secondNode.SyncInterface = $syncInterface2
	$secondNode.HBInterface = $hbInterface2
	$secondNode.SyncSessionCount = $syncSessionCount2
	$secondNode.ALUAOptimized = $aluaOptimized2
	$secondNode.CacheMode = $cacheMode2
	$secondNode.CacheSize = $cacheSize2
	$secondNode.FailoverStrategy = $failover
	$secondNode.CreateTarget = $createTarget2
	$secondNode.BitmapFolderPath = $bmpFolderPath2

	$thirdNode = new-Object Node

	$thirdNode.HostName = $addrW
	$thirdNode.HostPort = $portW
	$thirdNode.Login = $userW
	$thirdNode.Password = $passwordW
	$thirdNode.ImagePath = $imagePathW
	$thirdNode.ImageName = $imageNameW
	$thirdNode.TargetAlias = $targetAliasW
	$thirdNode.SyncInterface = $syncInterfaceW
	$thirdNode.HBInterface = $hbInterfaceW
	$thirdNode.FailoverStrategy = $failover
	$thirdNode.Type = $nodeTypeW

	$device = Add-HADevice -server $server -firstNode $firstNode -secondNode $secondNode -thirdNode $thirdNode -initMethod $initMethod

	$syncState = $device.GetPropertyValue("ha_synch_status")

	while ($syncState -ne "1")
	{
		#
		# Refresh device info
		#
		$device.Refresh()

		$syncState = $device.GetPropertyValue("ha_synch_status")
		$syncPercent = $device.GetPropertyValue("ha_synch_percent")

		Start-Sleep -m 2000

		Write-Host "Synchronizing: $($syncPercent)%" -foreground yellow
	}
}
catch
{
	Write-Host $_ -foreground red
}
finally
{
	$server.Disconnect()
}


Script works only partly.
Storage files, HA devices and targets are created but synchronization never goes past 1%. It's just stuck.
The only error in Management Console is that partners are not synchronized.

I can create 2 node HA device without witness or with SMB witness without any problem with the same setup.

So. This setup seems pretty clean and pristine. Is it possible at all to create 2 node HA device with partner witness using powershell only?
yaroslav (staff)
Staff
Posts: 3598
Joined: Mon Nov 18, 2019 11:11 am

Wed Mar 26, 2025 6:11 pm

Hi,

Check this thread https://forums.starwindsoftware.com/vie ... NT8#p37208. Try skipping full synchronization.
If the files are created and synchronization starts, the script is working.
Please also make sure to use a dedicated replication link for data nodes' replication (i.e., it should not be combined with management).
AlexeyK
Posts: 7
Joined: Thu Mar 20, 2025 2:45 pm

Thu Mar 27, 2025 10:18 am

Same script.
Yes, replication interfaces are dedicated
Skipping initial synchronization with $initMethod="NotSynchronize"
No luck

Code: Select all

param($addr="192.168.123.41", $port=3261, $user="root", $password="starwind",
	$addr2="192.168.123.42", $port2=$port, $user2=$user, $password2=$password,
	$addrW="192.168.123.43", $portW=$port, $userW=$user, $passwordW=$password,
#common
	$initMethod="NotSynchronize",
	$size=128,
	$sectorSize=512,
	$failover=1,
	$bmpType=1,
	$bmpStrategy=0,
#primary node
	$imagePath="My computer\C\sw",
	$imageName="masterImg31",
	$createImage=$true,
	$storageName="",
	$targetAlias="targetha31",
	$poolName="pool1",
	$syncSessionCount=1,
	$aluaOptimized=$true,
	$cacheMode="none",
	$cacheSize=0,
	$syncInterface="#p2=10.10.123.42:3260;#p3=10.10.123.43:3260",
	$hbInterface="",
	$createTarget=$true,
	$bmpFolderPath="",
#secondary node
	$imagePath2="My computer\C\sw",
	$imageName2="partnerImg32",
	$createImage2=$true,
	$storageName2="",
	$targetAlias2="partnerha32",
	$poolName2="pool1",
	$syncSessionCount2=1,
	$aluaOptimized2=$false,
	$cacheMode2=$cacheMode,
	$cacheSize2=$cacheSize,
	$syncInterface2="#p1=10.10.123.41:3260;#p3=10.10.123.43:3260",
	$hbInterface2="",
	$createTarget2=$true,
	$bmpFolderPath2="",
#third node
	$imagePathW="My computer\C\sw",
	$imageNameW="witness33",
	$targetAliasW="witness33",
	$syncInterfaceW="#p1=10.10.123.41:3260;#p2=10.10.123.42:3260",
	$hbInterfaceW="",
	$nodeTypeW=8
	)

Import-Module StarWindX

try
{
	Enable-SWXLog

	$server = New-SWServer -host $addr -port $port -user $user -password $password

	$server.Connect()
    
	$firstNode = new-Object Node

	$firstNode.HostName = $addr
	$firstNode.HostPort = $port
	$firstNode.ImagePath = $imagePath
	$firstNode.ImageName = $imageName
	$firstNode.Size = $size
	$firstNode.CreateImage = $createImage
	$firstNode.StorageName = $storageName
	$firstNode.TargetAlias = $targetAlias
	$firstNode.SyncInterface = $syncInterface
	$firstNode.HBInterface = $hbInterface
	$firstNode.PoolName = $poolName
	$firstNode.SyncSessionCount = $syncSessionCount
	$firstNode.ALUAOptimized = $aluaOptimized
	$firstNode.SectorSize = $sectorSize
	$firstNode.CacheMode = $cacheMode
	$firstNode.CacheSize = $cacheSize
	$firstNode.FailoverStrategy = $failover
	$firstNode.CreateTarget = $createTarget
	$firstNode.BitmapStoreType = $bmpType
	$firstNode.BitmapStrategy = $bmpStrategy
	$firstNode.BitmapFolderPath = $bmpFolderPath
                                                   
	$secondNode = new-Object Node

	$secondNode.HostName = $addr2
	$secondNode.HostPort = $port2
	$secondNode.Login = $user2
	$secondNode.Password = $password2
	$secondNode.ImagePath = $imagePath2
	$secondNode.ImageName = $imageName2
	$secondNode.CreateImage = $createImage2
	$secondNode.StorageName = $storageName2
	$secondNode.TargetAlias = $targetAlias2
	$secondNode.SyncInterface = $syncInterface2
	$secondNode.HBInterface = $hbInterface2
	$secondNode.SyncSessionCount = $syncSessionCount2
	$secondNode.ALUAOptimized = $aluaOptimized2
	$secondNode.CacheMode = $cacheMode2
	$secondNode.CacheSize = $cacheSize2
	$secondNode.FailoverStrategy = $failover
	$secondNode.CreateTarget = $createTarget2
	$secondNode.BitmapFolderPath = $bmpFolderPath2

	$thirdNode = new-Object Node

	$thirdNode.HostName = $addrW
	$thirdNode.HostPort = $portW
	$thirdNode.Login = $userW
	$thirdNode.Password = $passwordW
	$thirdNode.ImagePath = $imagePathW
	$thirdNode.ImageName = $imageNameW
	$thirdNode.TargetAlias = $targetAliasW
	$thirdNode.SyncInterface = $syncInterfaceW
	$thirdNode.HBInterface = $hbInterfaceW
	$thirdNode.FailoverStrategy = $failover
	$thirdNode.Type = $nodeTypeW

	$device = Add-HADevice -server $server -firstNode $firstNode -secondNode $secondNode -thirdNode $thirdNode -initMethod $initMethod

	$syncState = $device.GetPropertyValue("ha_synch_status")

	while ($syncState -ne "1")
	{
		#
		# Refresh device info
		#
		$device.Refresh()

		$syncState = $device.GetPropertyValue("ha_synch_status")
		$syncPercent = $device.GetPropertyValue("ha_synch_percent")

		Start-Sleep -m 2000

		Write-Host "Synchronizing: $($syncPercent)%" -foreground yellow
	}
}
catch
{
	Write-Host $_ -foreground red
}
finally
{
	$server.Disconnect()
}

result:
Request to STOR1 ( 192.168.123.41 ) : 3261
-
control 0x000001CD60DB9F80 -InitSynchronize:"" -SynchronizationType:"4"
-
200 Failed: error occurred while initial synchronization process was running..
yaroslav (staff)
Staff
Posts: 3598
Joined: Mon Nov 18, 2019 11:11 am

Thu Mar 27, 2025 10:33 am

Try different Target and image file names.
AlexeyK
Posts: 7
Joined: Thu Mar 20, 2025 2:45 pm

Thu Mar 27, 2025 10:57 am

I'm clearing up headers directory (C:\Program Files\StarWind Software\StarWind\headers), image file location, and all targets and devices that are left in app after each unsuccesful run.

Different names for targets and image files

Code: Select all

param($addr="192.168.123.41", $port=3261, $user="root", $password="starwind",
	$addr2="192.168.123.42", $port2=$port, $user2=$user, $password2=$password,
	$addrW="192.168.123.43", $portW=$port, $userW=$user, $passwordW=$password,
#common
	$initMethod="NotSynchronize",
	$size=128,
	$sectorSize=512,
	$failover=1,
	$bmpType=1,
	$bmpStrategy=0,
#primary node
	$imagePath="My computer\C\sw",
	$imageName="masterImg31-different",
	$createImage=$true,
	$storageName="",
	$targetAlias="targetha31-different",
	$poolName="pool1",
	$syncSessionCount=1,
	$aluaOptimized=$true,
	$cacheMode="none",
	$cacheSize=0,
	$syncInterface="#p2=10.10.123.42:3260;#p3=10.10.123.43:3260",
	$hbInterface="",
	$createTarget=$true,
	$bmpFolderPath="",
#secondary node
	$imagePath2="My computer\C\sw",
	$imageName2="partnerImg32-different",
	$createImage2=$true,
	$storageName2="",
	$targetAlias2="partnerha32-different",
	$poolName2="pool1",
	$syncSessionCount2=1,
	$aluaOptimized2=$false,
	$cacheMode2=$cacheMode,
	$cacheSize2=$cacheSize,
	$syncInterface2="#p1=10.10.123.41:3260;#p3=10.10.123.43:3260",
	$hbInterface2="",
	$createTarget2=$true,
	$bmpFolderPath2="",
#third node
	$imagePathW="My computer\C\sw",
	$imageNameW="witness33-different",
	$targetAliasW="witness33-different",
	$syncInterfaceW="#p1=10.10.123.41:3260;#p2=10.10.123.42:3260",
	$hbInterfaceW="",
	$nodeTypeW=8
	)

Import-Module StarWindX

try
{
	Enable-SWXLog

	$server = New-SWServer -host $addr -port $port -user $user -password $password

	$server.Connect()
    
	$firstNode = new-Object Node

	$firstNode.HostName = $addr
	$firstNode.HostPort = $port
	$firstNode.ImagePath = $imagePath
	$firstNode.ImageName = $imageName
	$firstNode.Size = $size
	$firstNode.CreateImage = $createImage
	$firstNode.StorageName = $storageName
	$firstNode.TargetAlias = $targetAlias
	$firstNode.SyncInterface = $syncInterface
	$firstNode.HBInterface = $hbInterface
	$firstNode.PoolName = $poolName
	$firstNode.SyncSessionCount = $syncSessionCount
	$firstNode.ALUAOptimized = $aluaOptimized
	$firstNode.SectorSize = $sectorSize
	$firstNode.CacheMode = $cacheMode
	$firstNode.CacheSize = $cacheSize
	$firstNode.FailoverStrategy = $failover
	$firstNode.CreateTarget = $createTarget
	$firstNode.BitmapStoreType = $bmpType
	$firstNode.BitmapStrategy = $bmpStrategy
	$firstNode.BitmapFolderPath = $bmpFolderPath
                                                   
	$secondNode = new-Object Node

	$secondNode.HostName = $addr2
	$secondNode.HostPort = $port2
	$secondNode.Login = $user2
	$secondNode.Password = $password2
	$secondNode.ImagePath = $imagePath2
	$secondNode.ImageName = $imageName2
	$secondNode.CreateImage = $createImage2
	$secondNode.StorageName = $storageName2
	$secondNode.TargetAlias = $targetAlias2
	$secondNode.SyncInterface = $syncInterface2
	$secondNode.HBInterface = $hbInterface2
	$secondNode.SyncSessionCount = $syncSessionCount2
	$secondNode.ALUAOptimized = $aluaOptimized2
	$secondNode.CacheMode = $cacheMode2
	$secondNode.CacheSize = $cacheSize2
	$secondNode.FailoverStrategy = $failover
	$secondNode.CreateTarget = $createTarget2
	$secondNode.BitmapFolderPath = $bmpFolderPath2

	$thirdNode = new-Object Node

	$thirdNode.HostName = $addrW
	$thirdNode.HostPort = $portW
	$thirdNode.Login = $userW
	$thirdNode.Password = $passwordW
	$thirdNode.ImagePath = $imagePathW
	$thirdNode.ImageName = $imageNameW
	$thirdNode.TargetAlias = $targetAliasW
	$thirdNode.SyncInterface = $syncInterfaceW
	$thirdNode.HBInterface = $hbInterfaceW
	$thirdNode.FailoverStrategy = $failover
	$thirdNode.Type = $nodeTypeW

	$device = Add-HADevice -server $server -firstNode $firstNode -secondNode $secondNode -thirdNode $thirdNode -initMethod $initMethod

	$syncState = $device.GetPropertyValue("ha_synch_status")

	while ($syncState -ne "1")
	{
		#
		# Refresh device info
		#
		$device.Refresh()

		$syncState = $device.GetPropertyValue("ha_synch_status")
		$syncPercent = $device.GetPropertyValue("ha_synch_percent")

		Start-Sleep -m 2000

		Write-Host "Synchronizing: $($syncPercent)%" -foreground yellow
	}
}
catch
{
	Write-Host $_ -foreground red
}
finally
{
	$server.Disconnect()
}

Still no luck. Same error:
Request to STOR1 ( 192.168.123.41 ) : 3261
-
control 0x000001CD60E18440 -InitSynchronize:"" -SynchronizationType:"4"
-
200 Failed: error occurred while initial synchronization process was running..
yaroslav (staff)
Staff
Posts: 3598
Joined: Mon Nov 18, 2019 11:11 am

Thu Mar 27, 2025 11:40 am

Try a bigger device size, please.
AlexeyK
Posts: 7
Joined: Thu Mar 20, 2025 2:45 pm

Thu Mar 27, 2025 12:01 pm

Tried $size=1024

With $initMethod="Clear" it is still a stuck sync at 1%

With $initMethod="NotSynchronize" it is the same error as before:
Request to STOR1 ( 192.168.123.41 ) : 3261
-
control 0x000001CD60DF4500 -InitSynchronize:"" -SynchronizationType:"4"
-
200 Failed: error occurred while initial synchronization process was running..
yaroslav (staff)
Staff
Posts: 3598
Joined: Mon Nov 18, 2019 11:11 am

Thu Mar 27, 2025 12:56 pm

Please check the WIndows-based Management Console. Do you see the synchronization going or do both nodes remain not synchronized?
AlexeyK
Posts: 7
Joined: Thu Mar 20, 2025 2:45 pm

Thu Mar 27, 2025 1:04 pm

Both nodes remain not synchronized and there is no synchronization going on.

Image

But in powershell $device.GetPropertyValue("ha_synch_percent") returns 1%
yaroslav (staff)
Staff
Posts: 3598
Joined: Mon Nov 18, 2019 11:11 am

Thu Mar 27, 2025 1:40 pm

It looks like comms are interrupted and nodes enter the isolated state.
Please see the MTU values and set them to 1514.
Also, to force synchronization, try using manual synchronization. Use SyncHaDevice.ps1, comment $device.Synchronize([SwHaSyncType]::SW_HA_SYNC_FULL, "") and uncomment #$device.MarkAsSynchronized()
AlexeyK
Posts: 7
Joined: Thu Mar 20, 2025 2:45 pm

Thu Mar 27, 2025 6:21 pm

Changed MTU on iSCSI interfaces back to 1514 - no effect. Same errors.
I don't think that it is a communication problem because I'm able to sucessfully create HA device with or without SMB witness with the same config.

$device.Synchronize([SwHaSyncType]::SW_HA_SYNC_FULL, "") - produces error:
Failed to perform synchronization (1) from
-
control 0x000001CD60DE8F00 -Synchronize: -SynchronizationType:"1"
-
200 Failed: can't find available or valid partner for synchronization..

So I think $device.Synchronize(...) could be omitted

And after all $device.MarkAsSynchronized() did its job and HA device now looks fine.
Obviously it is not initialized properly but I can perform full format of the volume later I guess.

Reproduced result with goal setup and everything works.

Note to myself - always do $server.Disconnect()

Thanks for support! =)
yaroslav (staff)
Staff
Posts: 3598
Joined: Mon Nov 18, 2019 11:11 am

Thu Mar 27, 2025 7:29 pm

You are welcome. I am glad to read that forcing synchronization worked in the end! :)
Post Reply