Hard Disk Monitoring

I have several external hard drives, and they are not connected. and I want to prevent bit rot and check the health of the drives.

1- is the Extended Self-Test just full SMART Scan?

2- for an HDD, I know I have to Refresh the Data on the HDD every year. so does running the Extended Self-Test enough? Or should I run the Surface Test (Read) as well? And after that should I also run Chkdsk?

So basically, I am not sure what is the best practice for my cold archive data which is stored in external HDD. should I annually run just extended self test, or all three?

> 1- is the Extended Self-Test just full SMART Scan?

Not really sure what you mean "just full SMART Scan"

The Disk menu -> Extended Self-Test function (as the Help explains) verifies major components of the hard disk (read/write heads, servo, electronics, internal memory, etc) and also performs a full surface scan.
Usually this test completes only if all sectors (the complete surface area) readable - but stops upon (any minor) problem (as explained in the Help too).
So if this test completes - then usually we can assume the drive is working correctly.

The only problem is (even if the test shows no errors) we have no details about possible disk transfer rate. Dropping the performance can indicate that the data may be harder to read back (and can fail later).

This is also explained in the Help, please check: https://www.hdsentinel.com/help/en/62_testfaq.html

About chkdsk - the problem is the same. Even if it completes, chkdsk does not show any details about possible retries and/or slower areas.
The use of chkdsk is absolutely not recommended in this case - as it never helps. Even if sounds surprising, chkdsk can show errors on a perfect drive - and can show perfect status on a failed drive, because it checks only the logical drive (partition) not the entire physical disk drive.


> 2- for an HDD, I know I have to Refresh the Data on the HDD every year. so does running the Extended Self-Test enough?
> Or should I run the Surface Test (Read) as well? And after that should I also run Chkdsk?

If you prefer, you can easily perform a complete refresh: the Disk menu -> Surface test -> Read+Write+Read test (refresh data area) designed exactly for this purpose. By default, the test performs an additional write cycle (before re-writing the data) with different write pattern exactly to cycle the bits in the sector. You can disable this (to simple re-write and verify the data just to be sure) if you uncheck the option: for this, select Disk menu -> Surface test -> Read+Write+Read test (refresh data area) and before starting the test, select the Configuration tab in this window and uncheck the Extensive read-write-read test with additional write operation option at the bottom.

Personally I'd use
- Disk menu -> Short self test for a generic checkup (or you can use the Extended self test if you prefer)
- Disk menu -> Surface test -> Read test
as it performs a complete surface test, verifies and reports not only possible errors but degraded performance (with darker green blocks), retries or anything similar. Plus it checks/reports possible changes in disk status.

The above all ignored by chkdsk - so use of chkdsk is absolutely not required as it is not sensitive enough to report real disk problems.

First, I apologize, I have mental retardation, and recently I started facing memory loss on top of it, hence my recent worry and need to preserve my memories stored in HDDs. Thus, my comprehension is affected. please forgive the annoyance that it is caused by me.

1- Does All Disk -> Surface test read the file-system data as well, like the NTFS file table etc., or in other word EVERYTHING stored on the HDD?

2- In case of Read+Write+Read test (refresh data area) does it rewrite the file-system data as well?

3- Does the Read+Write+Read test (refresh data area) use Standard hash algorithm to verify the data rewritten back to the disk?

4- In the Read+Write+Read test (refresh data area) does HDsentienal Hold the first Read data in RAM, until it's verified?

5- If HDsentienal holds the First read data in RAM, then if the Second/final Read operation gave a hash verification error, then does HDsentienal try to rewrite the data and check again?

6- If only a hash of the first read data is held in RAM, does that means after a failed verification, the failed sector will be shown in RED, or will the Read+Write+Read test (refresh data area) stop at that sector?

7- Sorry I am just curious , what checksum/hash algorithm is used in Read+Write+Read test (refresh data area) ?


8- So in case of External HDDs stored on a shelf for years there is no need to run chkdsk as long as surface test (read) or (read-write-read) is ran am I right?

9- How frequently and at which situations do you recommend running chkdsk?

10 - I value your opinion more than any other online as you are well informed about HDDs, so I need your confirmation (sorry for being redundant)

From other forum posts, it seems that HDDs don't require data refresh because magnetic bits stored on HDD's don't fade and thus there will be no bit rot, of course, as long as it is stored properly. knowing that, what do you recommend the consumer to do to prevent data loss in situations like mine:

A- I have two copies of my data first stored on HTL blurays and another copy in external HDD drives that sit without being turned on for YEARS.

B- For HDDs I am planning to connect connect them to my laptop every 3 Years and run extended self-test, and then Disk menu -> Surface test -> Read test (because you said it is enough and Read-Write-Read is unnecessary for magnetic storage many years ago**) And then at the end check the Hash of all files stored in the HDD, if all is fine then repeat it again after 3 years. Additionally after 10 years from the date of HDD purchase, I will replace HDD with a new one.

C- For blurays, I plan to check file hashes every 5 years. and replace the disc every 20 years.

If that is not a good plan, then what is the BEST and most paranoid or safest practice to keep my data safe? or If you were in my situation what would you do?


unrelated:
I am worried of bitrot in Microsd cards in my phone, as the data fades with time as you mentioned here: https://www.hdsentinel.com/forum/viewtopic.php?t=10331)

11-How frequently should I remove the card and run Read+Write+Read test (refresh data area)? And If the microsd card is connected to Android phone does that mean the OS refresh the data?

**https://www.hdsentinel.com/forum/viewtopic.php?t=10331

Wow, really lots of questions

> 1- Does All Disk -> Surface test read the file-system data as well, like the NTFS file table etc., or in other word EVERYTHING stored on the HDD?

YES. The Disk menu -> Surface test functions process (read/write) all sectors on the drive.
This is why they can work even on unformatted drive and/or drive which is not readable under Windows (eg. Linux/MAC formatted).
The Disk menu -> Surface test tests all sectors with MBR, system data, NTFS, real files, unused (empty) sectors and sectors outside any partitions too.


> 2- In case of Read+Write+Read test (refresh data area) does it rewrite the file-system data as well?

Yes.


> 3- Does the Read+Write+Read test (refresh data area) use Standard hash algorithm to verify the data rewritten back to the disk?

Of course not. No hash used to verify / confirm that the data is 100% same.


> 4- In the Read+Write+Read test (refresh data area) does HDsentienal Hold the first Read data in RAM, until it's verified?

Yes: the data kept in the RAM, the sectors refreshed/updated and then read back and compared with the original data.


> 5- If HDsentienal holds the First read data in RAM, then if the Second/final Read operation gave a
> hash verification error, then does HDsentienal try to rewrite the data and check again?

There is no hash verification, but complete verification of the data.
No, in this case it does not re-write, just if the drive reports error during the actual write (or read) command.


> 6- If only a hash of the first read data is held in RAM, does that means after a failed verification,
> the failed sector will be shown in RED, or will the Read+Write+Read test (refresh data area) stop at that sector?

No hash used.
If the verification confirms that the sector is NOT what we written, then it showed as red, the bottom shows "Verify error sector: ..." and the following sector is processed. It does not stop at that sector of course.


> 7- Sorry I am just curious , what checksum/hash algorithm is used in Read+Write+Read test (refresh data area) ?

No hash used


> 8- So in case of External HDDs stored on a shelf for years there is no need to run chkdsk as long as
> surface test (read) or (read-write-read) is ran am I right?

Chkdsk is completely useless in this situation.
It would NOT report any delays, slower sectors and would check only the actual partition etc.
It is much better to use Disk menu -> Surface test -> Read test, as it would scan the complete drive (even areas ignored by chkdsk) and would report possible retries, degradations in performance, plus monitors status changes, reports any deviation, so generally much more sensitive to problems.

If you prefer, you can use the refresh (read-write-read) test too, just to ensure that all sectors are rewritten.


> 9- How frequently and at which situations do you recommend running chkdsk?

Only when the file system may be damaged. For example if (during an intensive write) we experience a sudden power loss, reset or similar. Then the file system may be in inconsistent state which needs to be checked and fixed. This is the only purpose of chkdsk: to check (and if required) fix the file system descriptors (MFT or FAT based on partition type), USN Journal, volume bitmap etc. all related to the LOGICAL drive (partition), not the physical disk drive.


> 10 - I value your opinion more than any other online as you are well informed about HDDs,
> so I need your confirmation (sorry for being redundant)

> From other forum posts, it seems that HDDs don't require data refresh because magnetic bits stored on HDD's don't fade
> and thus there will be no bit rot, of course, as long as it is stored properly.

Generally yes, if the drive stored properly AND the data recorded properly - then we can assume that the data will not "fade" away.
"Recorded properly" means that the drive received enough power for proper operation and was operating at proper temperatures.

A typical, very common problem/issue is that an external 2.5" USB hard disk (or SSD) advertised as "USB 2.0 compatible". Users connect to any USB slot and read/write the data - but they do not know that USB 2.0 connection gives only up to 500 mA power. These hard disks (and especially SSDs) require much more power for proper operation, so in this case everything seems fine - but the drive may had no opportunity to properly record the data. Weeks/months later unreadable files, weak/pending (or bad) sectors can appear - or in worst case, the USB hard disk may fail to start up at all.

If we are sure that the data recorded properly - we can assume that it can be read back for very long time.
Personally I use Read test to verify if the data can be read back - with the expected performance. If some areas seems slower than expected (showed with darker green blocks) then these may fail later, so in this case I'd use the Disk menu -> Surface test -> Read-write-read data exactly to "refresh" the sectors.


> knowing that, what do you recommend the consumer to do to prevent data loss in situations like mine:
> A - I have two copies of my data first stored on HTL blurays and another copy in external HDD drives that sit without being turned on for YEARS.

To be honest, I never use blurays - so I can't say anything about them. I only have long-time storage on HDDs.


> B- For HDDs I am planning to connect connect them to my laptop every 3 Years and run extended self-test, and
> then Disk menu -> Surface test -> Read test (because you said it is enough and Read-Write-Read is
> unnecessary for magnetic storage many years ago**)

Yes, this is perfect. I'd connect every 1-3 years for a read test - and maybe if the performance seems slower than expected, I'd use the Read-write-read test to refresh.


> Additionally after 10 years from the date of HDD purchase, I will replace HDD with a new one.

Yes, this is also correct - but it depends on the actual use.
I mean if you power on only for short time (so even in 10 years the total power on time remains low) then you may keep using them.
Personally I also have 10+ year OLD drives (with some 100 days total power on time) backups.


> C- For blurays, I plan to check file hashes every 5 years. and replace the disc every 20 years.

Sorry, I can't say anything about blurays. Considering how older CDs/DVDs (even good quality disks) fade so quickly (many of my old CDs/DVDs are no longer readable after 10-15 years) personally I do not really prefer this kind of storage.

> If that is not a good plan, then what is the BEST and most paranoid or safest practice to
> keep my data safe? or If you were in my situation what would you do?

I agree you - just personally I still prefer HDDs than optical storage.


> unrelated:
> I am worried of bitrot in Microsd cards in my phone, as the data fades with time as
> you mentioned here: https://www.hdsentinel.com/forum/viewtopic.php?t=10331)

Yes, I completely understand and agree. If I can say, originally the Read-Write-Read test designed exactly for such situations: to refresh data of flash storage (pendrives, memory cards).

> 11-How frequently should I remove the card and run Read+Write+Read test (refresh data area)?
> And If the microsd card is connected to Android phone does that mean the OS refresh the data?

Good question...
Generally modern SD cards (like SSDs) have some kind auto refresh function when powered.
Maybe similarly as for the hard disk drives: I'd start a Read test first - and if this Read test shows slower performance - then a refresh can help.