LSFS overhead & defragmentation
Posted: Wed Sep 24, 2014 3:41 pm
Doing some tests with LSFS - bet you hadn't guessed that 
Created new 500GB LSFS storage on two-node HA and started copying some test files onto it using robocopy via Windows 2012 server/iSCSI MPIO. 228GB of data has been copied:
And there are 275GB of SPSPX files on the server:
So the SAN is using 17% more space that the amount of data copied. NOTE: no files have been deleted/overwritten - this is a completely fresh copy.
Is this kind of overhead expected? Slightly higher than I expected but I'm aware that there are many more writes when copying files like this than just the data itself. 363,926 files have been copied which will have resulted in many of additional writes to the directory. Even when the server is idle, there are writes going onto $LogFile, $Bitmap & $Mft.
Thinking about the master file table alone, the same block will have been re-written many times as individual new directory entries are written into the same directory block. In effect, the same block will have been overwritten many times. Will defragmentation pick this up and eventually free that space or does defragmentation only work at the 5GB SPSPX level? Or does it work at a more granular level, i.e. if a block has been flagged as deleted/overwritten, are blocks from higher up moved down so that eventually an entire 5GB SPSPX file will become free and can be deleted. Is that a way to see defragmentation in action, you'll end up with gaps in the SPSPX numbering system?
So many questions
Cheers, Rob.
Created new 500GB LSFS storage on two-node HA and started copying some test files onto it using robocopy via Windows 2012 server/iSCSI MPIO. 228GB of data has been copied:
- sshot-58.png (28.39 KiB) Viewed 7778 times
- sshot-59.png (23.6 KiB) Viewed 7778 times
Is this kind of overhead expected? Slightly higher than I expected but I'm aware that there are many more writes when copying files like this than just the data itself. 363,926 files have been copied which will have resulted in many of additional writes to the directory. Even when the server is idle, there are writes going onto $LogFile, $Bitmap & $Mft.
Thinking about the master file table alone, the same block will have been re-written many times as individual new directory entries are written into the same directory block. In effect, the same block will have been overwritten many times. Will defragmentation pick this up and eventually free that space or does defragmentation only work at the 5GB SPSPX level? Or does it work at a more granular level, i.e. if a block has been flagged as deleted/overwritten, are blocks from higher up moved down so that eventually an entire 5GB SPSPX file will become free and can be deleted. Is that a way to see defragmentation in action, you'll end up with gaps in the SPSPX numbering system?
So many questions
Cheers, Rob.
