HowTo do a quick deep hashing/fixity of subfolders

[peter_b]

You may find or search this under "md5deep", but the command - as well as its package - was renamed/superseeded by "hashdeep".
Where "md5sum" creates a textfile per folder, the command "hashdeep" can be used to create fixity information for a whole subfolder structure.

The basic command is simple:

Code: Select all

$ hashdeep -l -r -c md5 .

Parameters:

• -l: Relative file paths: Makes the file more "portable" (easier to check even when moved to completely different location)
• -r: recursive. Include subfolders.
• -c md5: compute MD5 hashcodes.

I chose to do MD5 only, because I it's sufficient for data integrity checks, and I'm not using it for security.
Since hashdeep's default is MD5+SHA256, I dropped SHA256 to save computing power and speed up the process

The output looks something like this:

Code: Select all

%%%% HASHDEEP-1.0
%%%% size,md5,filename
## Invoked from: /media/pb/CAM_SD
## $ hashdeep -l -r -c md5 .
##
1328742227,62c8a3a5d58316d0758413bc414406ad,./DCIM/101XDPHH/S1010001.MP4
1327486509,34eec99491abe839db31c130320a2e04,./DCIM/101XDPHH/S1010002.MP4
1327514894,11d0e549e9832bbe8970e81e1bcdc494,./DCIM/101XDPHH/S1010003.MP4
1327837525,7f1c5e06a3c1fa7a179473c780fb6a7d,./DCIM/101XDPHH/S1010004.MP4
1328092473,d2f9bf6d35e4f763adb76edc608f8c92,./DCIM/101XDPHH/S1010005.MP4
260114642,3edd2bd0908e0accb66e2af72f00116e,./DCIM/102XDPHH/S1020004.MP4
1328250451,13e610710e2f9d323f4dd2c25620a540,./DCIM/101XDPHH/S1010006.MP4
613737399,677d7c2c6b21ad11969fdd8a840a938b,./DCIM/102XDPHH/S1020002.MP4
1327236635,e53122187f81e5da11b9643fad3e8b1f,./DCIM/101XDPHH/S1010007.MP4
1322809572,1652fb231da8c7a7daca0816e84cdef9,./DCIM/102XDPHH/S1020001.MP4
1328600259,59d28e4dfbc6972536f594031012e2ca,./DCIM/102XDPHH/S1020003.MP4

Links:
https://linhost.info/2010/05/using-hash ... integrity/

EDIT (May 17th, 2019):
Changed to relative paths (added "-l"), because I think it's the more regular use case that the files will be moved sooner or later.
Makes validation easier and less confusing.

[peter_b]

Here's one way to check the current folder, with the data in the fixity textfile "MD5DEEP.md5":

Code: Select all

$ hashdeep -l -e -vvv -r -a -k MD5DEEP.md5 .

Parameters:

• -l: Relative paths mode. Otherwise you'll only get "file moved" messages
• -e: Estimate time. Shows a progress indicator. Must-have for video handling.
• -vvv: Tell us what you're doing. With less verbosity, you'll only see errors - and therefore "dark screen" while its computing. Better with "vvv"
• -a: Audit mode = compare and verify/validate.
• -k MD5DEEP.md5: The textfile that contains fixity data from hashdeep (MD5, etc). I've arbitrarily named it ""MD5DEEP.md5".