dedup doesnt work

cellardoor · Wed Mar 28, 2012 3:03 am

I connect to Starwind iscsi target with MS iscsi initiator from localhost, configured 'Deduplicated disk device", mount to windows as a 4 TB volume.

the dedup feature doesnt work, starwind data file is the same size as my mounted iscsi volume.

I use windows 2008 r2 64 bit and Starwind 5.8.

Please advice. Thanks

Wed Mar 28, 2012 9:25 am

Dear cellador,

Could you please provide us with two screenshots:
1.Disk manager where we will see the size of mounted DDDsik
2.The windows explorer window where we will see the size of all DeDup related files.

Thank you

cellardoor · Wed Mar 28, 2012 7:43 pm

When we increase dedup block size to 16kb or greater, dedup stops working(sometimes the starwind file would grow to as much as double the actual data size). We are now noticing wildly different results when using different NTFS Allocation Unit sizes on the iscsi target volumes. It seems when we match the NTFS Allocation Unit Size(of target volume) to the Starwind Block Size we get desirable results.

We are testing by backing up multiple identical copies of large DB files. Currently we are using 64k Starwind Block Sizes and 64k NTFS Allocation Unit Sizes(on the iSCSI target volume) with very good results.

Is there a best practice guide on partitioning the Starwind file disks and the Starwind iscsi target volumes?

Wed Mar 28, 2012 10:02 pm

Use 4KB blocks for deduplication. StarWind is not memory pig so you should have the same RAM usage as if you'd be using ZFS with 128-256KB blocks. But with better dedupe ratio of course.

cellardoor wrote:When we increase dedup block size to 16kb or greater, dedup stops working(sometimes the starwind file would grow to as much as double the actual data size). We are now noticing wildly different results when using different NTFS Allocation Unit sizes on the iscsi target volumes. It seems when we match the NTFS Allocation Unit Size(of target volume) to the Starwind Block Size we get desirable results.

We are testing by backing up multiple identical copies of large DB files. Currently we are using 64k Starwind Block Sizes and 64k NTFS Allocation Unit Sizes(on the iSCSI target volume) with very good results.

Is there a best practice guide on partitioning the Starwind file disks and the Starwind iscsi target volumes?

cellardoor · Fri Mar 30, 2012 7:02 pm

after delete all the files from one starwind iscsi volume, Starwind data file size reminds unchanged, see attached screenshot.

Sun Apr 01, 2012 6:28 pm

Dear cellardoor,

Me and my honorable colleague were asking and suggested you something. Can you please comment this somehow?

About your question - yes, it shouldn`t shrink after data was erased by design for now.

olivierolejniczak · Tue Jun 05, 2012 11:17 am

Hi

I also noticed that that dedup.spdata, dedup.spmetadata & dedup.spbitmap didn't shrink when data is removed the deduplicated iscsi disk.
This is normal as the impact on the host performance would be tremendous.
Althought , isn't there a way to "force" a block database cleanup?

Tue Jun 05, 2012 2:48 pm

As I noticed in my previous message here: this is how it designed to work for now.
In future release we will have "Data Re-Use", which means that if some data was erased from iSCSI vloume and the blocks are no longer used with it all the new data will be writen over those blocks.