Hard Disk Monitoring

Good day.

I am trying to setup sleep times for my disks, trying to see what is preventing them from sleeping. One thing I have discovered is that HDS is keeping my USB drives awake. They are all Western Digital Passport drives, and it is easy to see when they sleep because the LED on the drives starts flashing. They power down on their own I think, I am not sure that the Windows Power Setting for turning off hard disks after a time has any effect. Anyway, when I use smartctl -i --nocheck=standby, smartctl tells me CHECK POWER MODE: incomplete response, ATA output registers missing. So if HDS is using the same mechanism for detecting sleeping drives, then it must be failing for those USB drives.

I see ExcludeDetect= in the INI file; can I use this to exclude those drives from HDS? Ideally, I would like to see them in the UI when I run it manually, but to exclude them from reading SMART data every 30 minutes. For now I have disabled HDS, but this is not an ideal solution.

I will try running HDSentinel.exe /ENABLETEST /POWER /REPORT, but I'd like a temporary solution now.

Thank you.
Best Regards,

> I have discovered is that HDS is keeping my USB drives awake.

Yes of course, this is completely normal and expected.
The periodic detection acts like any read command, causing that the idle timer of the drive resets when the status detected.
So if status detection happens (for example as it is the default setting) once per every 5 minutes and the hard disk should enter sleeping after 10 minutes, then it is completely normal and expected that it can't enter to sleep mode.

If you adjust the detection frequency on Configuration -> Advanced options page (drag the slider to right, to perform less frequent detections) then the hard disk can enter sleeping mode due to inactivity.
Then Hard Disk Sentinel automatically detects its power state (idle or sleeping) to verify the situation, to prevent wake up if the drive is sleeping.

There are three problems:
- USB external drives often have internal power management functions which may work independently from Windows settings and attempt to put the disk drive to sleep mode even if it should not
- the functions to detect the power state may either not work at all (as you verified in other tool too) or may even provide incorrect status (for example suggesting "awake" status when the above mentioned internal power management put the drive to sleep mode.
- sometimes the functions to detect the power state may even wake up the drive

The best solution is to enable the Automatic detection based on disk utilization option on Configuration -> Advanced options page in addition to moving the slider to right to perform less frequent detections.

By this option, when the periodic detection should happen, Hard Disk Sentinel verifies first if the amount of read/written data changed since last detection (which displayed on the Performance page, if Windows Performance Counters are working). If no data transfer happened - then the drive is sleeping, so it is not accessed, the status (even power state) is not detected at all.

BTW, this topic already covered on the forum:
https://www.hdsentinel.com/forum/viewto ... 8&start=20

Good day HDSentinel.

I appreciate the time you took to explain, but everything you said prior to "Automatic detection based on disk utilization" I knew already. I know that reading SMART data wakes the drive, and I know that my USB drives have their own power management, and that they do not even support the functions to detect power state, etc. And my HDS detection is every 30 minutes and I think this is higher than the self-sleep of the USB drives, because I have seen them sleep for some minutes before HDS wakes them up again. For my internal disks I have them sleep @ 25 minutes and HDS detection is set for 30 minutes - It should work, but I have not yet tested fully, I started with the USB drives because it is easy to see when they sleep.

I have tried "Automatic detection based on disk utilization" before, and I did not like it. However, the manual doesn't explain how it works (it has not been updated, there is no mention of this option at all in the manual,) so I could not make a precise decision on this. You should really say in the manual what you said to me just now: Every time HDS is scheduled to check the drives, it checks to see if some bytes have been read/written since the last detection, and if zero it does not query the drive (please confirm)? Also, how do you detect the amount of read/written data? Because from my experience, the "performance" TAB of HDS is not accurate - the total amount read/written is off a lot for example (HDS says total written is 2TB, Samsung Magician says 5TB, and Samsung program is correct) and the transfer speed graphs are like 30 seconds behind reality (you use average too much,) etc...

Can you also explain how to use ExcludeDetect= please? What do I put there, how do I enumerate the drives I want to exclude?

PS: I searched for topics with USB in the subject, and found one from 2015, too old. The thread you mentioned is from 2011, I think this is too old as well.

Thank you.
Best Regards,

> everything you said prior to "Automatic detection based on disk utilization" I knew already.

Yes, as this is true for long time, discussed already on the forum. This may be true for both internal and external hard disk drives too, so generally not really "new".


> I have tried "Automatic detection based on disk utilization" before, and I did not like it.

Sorry but this is the best solution I could make to overcome the limitations and possible problems we may have with these drives related to power state.


> However, the manual doesn't explain how it works (it has not been updated, there is no mention of this option at all in the manual,)

The manual clearly describes and yes, it recently updated, exactly to add new options/information
Please check online at: https://www.hdsentinel.com/help/en/30_c_adv.html
(see the description about Automatic detection based on disk utilization ).


> Because from my experience, the "performance" TAB of HDS is not accurate

Sorrry to say - but this is accurate, just counts and works completely differently than the "total lifetime data written" value measured and counted by SSDs (and some hard disks).

The Performance tab works based on Windows Performance Counter objects. These provide the current and total amount of data read/written by the OS and any other programs, applications.
As this counts possible CACHED bytes, usually this is not 100% same as the data REALLY written/read by the device. Plus SSDs may compress data, resulting in different amount of data written.

Also the Performance page (of course) counts only when Hard Disk Sentinel is active and when the Windows performance counters are working (maybe not always true for external disk drives).

In contrast, the SSD lifetime written data counter is always counting (even during Windows installation) - so it is completely normal and expected if that counter is different. But it does not mean any value is more accurate than the other.

This function of Windows may be used (without reading the self-monitoring counter) exactly to determine disk activity in general, several other tools use to indicate/show disk activity.


> the total amount read/written is off a lot for example (HDS says total written is 2TB, Samsung
> Magician says 5TB, and Samsung program is correct)

Hard Disk Sentinel also displays the total amount of written data on the Overview page, if the SSD supports that counter.
This should show the total amount *really* written to the SSD, according the internal counter of the SSD.
Yes, this can be different than the Performance page. Ideally the two numbers would match only
- if Hard Disk Sentinel would be always active from the very first installation of the SSD (even during Windows installation)
- if we'd always write high amount of uncompressible data (so there would be no caching and the SSD could not compress internally)
As you can imagine, this is impossible, so it is normal that the values are different. But this is not problem.


> and the transfer speed graphs are like 30 seconds behind reality (you use average too much,)

If you check the Performance page, you can see both the current and the averaged values.
Of course to determine the actual usage, not the averaged graph used....


> Can you also explain how to use ExcludeDetect= please? What do I put there, how do I enumerate the drives I want to exclude?

Sorry, but that is a completely different thing.
You do not need to manually edit/add any information there.
The ExcludeDetect lists drives where Hard Disk Sentinel should not detect the capacity: the used/free space.
If you open Configuration -> Disk Control page, there is an option "Do not examine space information of the following disks"
What you specify there - will be saved in the INI file at the ExcludeDetect section.
For example if you specify CDE , then Hard Disk Sentinel will not check the used/free space periodically to reflect disk usage.
You may try this for the USB drive, just to check if there is any difference.


> PS: I searched for topics with USB in the subject, and found one from 2015,
> too old. The thread you mentioned is from 2011, I think this is too old as well.

Not really, as things are same generally, the power state issues (as you can see) not fixed by USB chips.

Hi HDSentinel (you should really sign your messages with at least your first name, so I can say "Hi John" lol).

Good news - Automatic detection based on disk utilization works perfectly! Now that I know what it does, I think your solution is brilliant! Nice one, thanks a lot!

> The manual clearly describes and yes, it recently updated, exactly to add new options/information
> Please check online at: https://www.hdsentinel.com/help/en/30_c_adv.html
> (see the description about Automatic detection based on disk utilization ).

Ahh, but NOT the manual that I get when I press F1 inside the program! Can you generate a new CHM file please, and include it when the program next updates? The one in my HDS folder is from 2014.

> Sorrry to say - but this is accurate, just counts and works completely differently than the "total lifetime
> data written" value measured and counted by SSDs (and some hard disks).

I know it uses Windows counters, but how can it possibly be off by 3 TerraBytes! I do not think I have written 3TB outside of Windows while HDS was not running; I did install HDS pretty soon after I installed Windows. This is why I say it is not very accurate/useful.

> This function of Windows may be used (without reading the self-monitoring counter) exactly
> to determine disk activity in general, several other tools use to indicate/show disk activity.

Ah yes, I do have programs like that (Disk Activity Lights, Resource Monitor, etc) and they do work correctly - Ok, sorry, my mistake, HDS likely works just fine.

>> Can you also explain how to use ExcludeDetect= please?
>
> Sorry, but that is a completely different thing.
> You do not need to manually edit/add any information there.
> The ExcludeDetect lists drives where Hard Disk Sentinel should not detect the capacity

Ahhh, THAT's what it's for! You should really think about adding a setting to simply exclude some drives totally from HDS, I know this has been asked for before.

Thanks for your great support.
Best Regards,

Hi,

Thanks, good to hear tha the Automatic detection based on disk utilization option helped


> Ahh, but NOT the manual that I get when I press F1 inside the program!
> Can you generate a new CHM file please, and include it when the program next updates? The one in my HDS folder is from 2014.

Sorry for that. The installer contains the new, updated CHM file. If you may installed an older version and used the automatic update, it is possible that the help CHM file not updated automatically. If you prefer, you can download the latest one directly from www.hdsentinel.com/files/en.chm and place in the folder of the software, so that it will open and show the updated version.


> I know it uses Windows counters, but how can it possibly be off by 3 TerraBytes!
> I do not think I have written 3TB outside of Windows while HDS was not running;
> I did install HDS pretty soon after I installed Windows. This is why I say it is not very accurate/useful.

Yes, sounds interesting. Windows may perform updates and/or perform background task (plus we can count the initial OS install) but yes, this should be less than 3 TBytes.

Ideally the counters should move together when large amount of data written which can't be compressed. If you prefer, you may use Report menu -> Send test report to developer option two times: before and after writing some (at least 10 GB) data and waiting some time to make sure that the SSD will record and provide the updated numbers.
This way it is possible to check how the counters (both of the SSD itself and the ones provided by Windows Performance Objects) change/increase, verify if they reflect (approximately) the same amount of written data.


> You should really think about adding a setting to simply exclude some drives totally from HDS, I know this has been asked for before.

Sorry, but to be honest, I do not really prefer that .... This is why there is no such option available now. Maybe in a future version.

Have a nice day,

Janos Mathe

Good day Janos!

I am still VERY HAPPY about Automatic detection based on disk utilization!

> Sorry for that. The installer contains the new, updated CHM file. If you may installed an older
> version and used the automatic update, it is possible that the help CHM file not updated automatically.
> If you prefer, you can download the latest one directly from www.hdsentinel.com/files/en.chm and
> place in the folder of the software, so that it will open and show the updated version.

I downloaded the portable version and used that to update my file, thanks. But the Auto-Update is not updating the file, you might want to look into that.

> Yes, sounds interesting. Windows may perform updates and/or perform background task (plus
> we can count the initial OS install) but yes, this should be less than 3 TBytes.

It is indeed a big difference, since total written according to SSD data is 5TB (and HDS says only 2TB). I do sometimes re-image the Windows partition (outside of Windows,) but I would be surprised if I re-imaged for 3TB of data out of 5TB total.

> Ideally the counters should move together when large amount of data written which can't be compressed.
> If you prefer, you may use Report menu -> Send test report to developer option two times: before and after
> writing some (at least 10 GB) data and waiting some time to make sure that the SSD will record and provide
> the updated numbers. This way it is possible to check how the counters (both of the SSD itself and the ones
> provided by Windows Performance Objects) change/increase, verify if they reflect (approximately) the same
> amount of written data.

That's a good idea, I will do some tests when I get the time and see how many bytes HDS says were written. If it's off, I will redo and Send Test Report...

>> You should really think about adding a setting to simply exclude some drives totally from HDS, I know this has been asked for before.
> Sorry, but to be honest, I do not really prefer that .... This is why there is no such option available now. Maybe in a future version.

Hahaha, I *know* you don't want to do that, because it would be there by now if you wanted, but I do not understand why not, it's up to the user to decide which drives he wants to watch

Thanks again for letting me know how Automatic detection based on disk utilization works! It is perfect for USB drives (they are fragile and do not have many Rated Hours, I think it is important that they sleep when they can. My real hard drives, I do not think it is so important that they sleep.)

Best Regards,

> Hahaha, I *know* you don't want to do that, because it would be there by now if you wanted, but I do not understand why not,
> it's up to the user to decide which drives he wants to watch

I completely understand and agree. The user can select which drives he wants to watch: by right click on the upper left area (the list of physical drives) it is possible to un-select drives to hide them and focus on other disks only.

This designed exactly to focus on important disks. But this does NOT mean that the hidden disks are not monitored.

In numerous situations (for example upon RAID configuration, multi-drive docks, RAID enclosures and even some motherboard chipsets and so) some drives need to be detected together. So un-selecting a drive (to exclude from detection) would be not possible in many such situations.

Also if a drive could be un-selected, then users may assume that Hard Disk Sentinel would never access that particular drive at all. While this could be managed during normal operation, on startup and on different situations (for example change of confifuration when drive(s) inserted/removed and the updated configuration should be detected) we NEED to access all disks, at least to determine its identification data to know "oh yes, this is the one I need to avoid".

With other words, even if you could select XY drive to be excluded, there is no warranty that the particular drive would be NEVER touched.
This is why it is possible to customize the ways (both displaying and also alerting to define specific thresholds) but not possible to configure drive to be excluded completely.

Hope this helps

Good day Janos.

> With other words, even if you could select XY drive to be excluded, there is no warranty
> that the particular drive would be NEVER touched. This is why it is possible to customize the
> ways (both displaying and also alerting to define specific thresholds) but not possible to
> configure drive to be excluded completely.

Ah, yes, I understand now. Every time I think I have something I think you forgot, you prove me wrong

Don't forget to look into why the help file is not updated by auto-update! It really should.

Thanks again.
Best Regards,